Vridhi Jain | Searcharxiv

arXiv2026

Benchmark-Based Comparative Assessment of Publicly Benchmarked Indian Foundation Models: A Capability and Evaluation-Maturity Framework

Purpose: Governments increasingly fund indigenous foundation models to strengthen national AI capability, digital sovereignty, and multilingual computing. This paper assesses India's foundation-model ecosystem and examines whether apparent capability gaps in public benchmark evidence may also reflect gaps in evaluation maturity. Approach: The paper presents a structured, benchmark-based comparative assessment of Indian foundation models against global frontier and comparable-scale models across eight capability domains: general-purpose reasoning, coding and software engineering, agentic AI and computer use, cybersecurity, vision and image understanding, video and multimodal understanding, scientific research, and Indic language capability. Using only publicly reported results, it proposes an exploratory four-dimension Benchmark Maturity Index (BMI), scoring each domain on standardization, participation, independent verification, and national Findings: Indian models achieve strong scores on established benchmarks such as MMLU and MATH-500. However, these are now widely regarded as saturated, and frontier developers no longer report them. Indian models participate far less frequently in newer, agentic, and domain-specialized evaluations, and participation is highly uneven across organizations. Sarvam AI reports the broadest coverage by a substantial margin. The BMI refines, and in some cases revises, the maturity judgments a purely descriptive review would produce. Practical implications: Many apparent capability gaps cannot be distinguished, on available evidence, from evaluation-ecosystem gaps, with direct implications for how national AI programs should design monitoring and funding criteria. Originality: The paper proposes BMI as a reusable instrument for scoring evaluation-ecosystem maturity at the domain level and demonstrates its application to the Indian foundation-model ecosystem.

cs.CY↗

arXiv2026

Shadow-Based Noise Fingerprinting of Simulated Quantum Noise Models

Accurate noise classification is essential for operating near-term quantum processors, yet existing approaches, such as quantum process tomography, scale exponentially with system size, limiting their practicality for routine calibration. We propose a measurement-efficient noise fingerprinting pipeline that combines structured classical shadow tomography with physics-informed feature engineering to identify noise channels from a fixed set of 3-qubit probe circuits. Each sample is represented by a feature vector constructed from randomized Pauli measurements and derived observables designed to resolve physically similar noise channels that produce overlapping signatures under generic measurement sets. We evaluate random forest, extra trees, and a multilayer perceptron on 10,000 labeled samples spanning ten noise models. The three classifiers achieve comparable performance. In the reported runs, random forest and extra trees perform similarly, achieving approximately 0.736 test accuracy and 0.729-0.730 macro F1, compared with 0.715 accuracy and 0.699 macro F1 for the multilayer perceptron. We further analyze the effect of the noise-strength sampling range and conduct a limited sensitivity check using analogous 2- and 4-qubit probes. Confusion analysis shows that readout error, phase flip, thermal relaxation, and bit flip are classified with high reliability, while most remaining errors occur among channels with similar physical effects.

cs.SE↗