arXiv subjects

Smit Kadvani

Publications and source records attributed to Smit Kadvani.

4 recordsLinked to original sources

arXiv2026

Twice as Fast, Until It Isn't: Quantized Serving of Mid-Size Open LLMs

A team serving a 9B-20B open-weight language model on vLLM or SGLang must choose a quantization recipe, and needs two numbers: the speedup over BF16 and the accuracy given up. We assemble every public measurement of these numbers that we could verify and find that the record supplies them only in pieces. Only two checkpoints, both INT4 builds of Gemma 3 12B, have both a vLLM timing and an accuracy score, and that score covers a single MMLU subject, one that the AWQ build was calibrated on. The batch-1 timings predate current engine releases, AMD and Intel Gaudi have no timing in this band at all, and FP4 has none against BF16. The timings that do exist follow one rule. Decoding a small batch is limited by memory bandwidth, so 4-bit weights roughly double speed (2.12x for Qwen2.5-14B on an A100), but the gain vanishes once a large batch makes the GPU compute-bound (1.02x for Gemma 3 12B at batch size 256), where FP8 is the recipe that can still help. Many published accuracy deltas, including every HumanEval delta, lie within benchmark sampling noise. We close with a seven-item reporting checklist for future measurements.

cs.AI↗

arXiv2026

How Far Can You Get Without a GPU? A Systematic Benchmark of Lightweight Hallucination Detection Across Question Answering, Dialogue, and Summarisation

Hallucination detection has become a pressing requirement for trustworthy AI deployment at scale. The most accurate detection methods depend on GPU-intensive inference, proprietary API calls, or white-box access to the generating model, putting them out of reach for resource-constrained researchers and practitioners. We explore a practical alternative: how well can hallucination detection perform using only lightweight, CPU-feasible methods built on public models? We benchmark four such detectors, ROUGE-L, semantic similarity, BERTScore, and a Natural Language Inference (NLI) detector based on a FEVER-trained DeBERTa model, together with a score-level ensemble of similarity and NLI. We evaluate them across all three tasks of the HaluEval benchmark: question answering (QA), dialogue, and summarisation. We calibrate on a held-out validation split, evaluate on 2,000 test instances per task, and report bootstrap confidence intervals. The similarity-NLI ensemble is the most consistent method, but absolute performance is highly task-dependent. It ranks best on QA (F1 = 0.792, AUC-ROC = 0.873) and on dialogue (F1 = 0.694, AUC-ROC = 0.749), where NLI is the strongest standalone method; on summarisation every method performs near chance (AUC-ROC between 0.469 and 0.574). We then ask whether that failure is intrinsic to lightweight detection or an artifact of our single-pass design, and find it is largely the latter. Raising the premise budget from 800 to 1600 characters lifts summarisation AUC-ROC from 0.567 to 0.629, and replacing single-pass scoring with sentence-level chunk aggregation reaches 0.683, still on CPU with the same model, though at roughly twenty times the NLI inference. Summarisation remains by far the hardest task, but our results do not support treating lightweight detection as intrinsically unsuited to it.

cs.CL↗

arXiv2026

PillarDETR: YOLO-Backbone and RT-DETR Head for Real-Time 3D Object Detection

Real-time 3D object detection is a critical component for the safe operation of autonomous driving systems and robotics. While LiDAR point clouds provide accurate spatial information, processing them efficiently remains a significant challenge. Traditional methods rely on complex 3D convolutions or anchor-based paradigms that struggle to balance detection accuracy with inference speed. In this paper, we propose PillarDETR, a novel end-to-end 3D object detection architecture that combines the efficiency of pillar-based LiDAR encoding with the representational power of modern 2D vision models. Specifically, PillarDETR replaces standard convolutional backbones with a Cross Stage Partial (CSP) network derived from YOLOv8, enabling richer feature extraction from pseudoimages. Furthermore, we discard conventional anchor-based or center-based detection heads in favor of a Real-Time Detection Transformer (RT-DETR) decoder. This hybrid design allows the network to capture global context and directly predict 3D bounding boxes without relying on non-maximum suppression (NMS). Extensive experiments on the KITTI and nuScenes benchmarks demonstrate that PillarDETR achieves a compelling trade-off between mean Average Precision (mAP) and inference latency. Our ablation studies confirm that integrating the YOLOv8 backbone and RT-DETR head yields substantial improvements over the PointPillars baseline, establishing PillarDETR as a highly effective solution for real-time 3D perception.

cs.CV↗

arXiv2026

Going beyond density functional theory accuracy: Leveraging experimental data to refine pre-trained machine learning interatomic potentials

Machine learning interatomic potentials (MLIPs) are inherently limited by the accuracy of the training data, usually consisting of energies and forces obtained from quantum mechanical calculations, such as density functional theory (DFT). Since DFT itself is based on several approximations, MLIPs may inherit systematic errors that lead to discrepancies with experimental data. In this paper, we use a trajectory re-weighting technique to refine DFT pre-trained MLIPs to match the target experimental Extended X-ray Absorption Fine Structure (EXAFS) spectra. EXAFS spectra are sensitive to the local structural environment around an absorbing atom. Thus, refining an MLIP to improve agreement with experimental EXAFS spectra also improves the MLIP prediction of other structural properties that are not directly involved in the refinement process. We combine this re-weighting technique with transfer learning and a minimal number of training epochs to avoid overfitting to the limited experimental data. The refinement approach demonstrates significant improvement for two MLIPs reported in previous work, one for an established nuclear fuel: uranium dioxide (UO2) and second one for a nuclear fuel candidate: uranium mononitride (UN). We validate the effectiveness of our approach by comparing the results obtained from the original (unrefined) DFT-based MLIP and the EXAFS-refined MLIP across various properties, such as lattice parameters, bulk modulus, heat capacity, point defect energies, elastic constants, phonon dispersion spectra, and diffusion coefficients. An accurate MLIP for nuclear fuels is extremely beneficial as it enables reliable atomistic simulation, which greatly reduces the need for large number of expensive and inherently dangerous experimental nuclear integral tests, traditionally required for the qualification of efficient and resilient fuel candidates.

cond-mat.mtrl-sci↗