arXiv subjects

Srinath Srinivasan

Publications and source records attributed to Srinath Srinivasan.

7 recordsLinked to original sources

arXiv2026

Better Together, in the Right Order: Classical-then-LLM Optimization for SE

A growing body of work combines large language models (LLMs) with classical optimizers for software engineering (SE) configuration tasks. Often, the classical optimizer is in charge: it owns the search loop and calls the LLM only to assist in subroutines (e.g. to warm-start the first generation, propose a mutation, or stand in as a surrogate). We report that there is much value in the reverse approach: seeding an LLM with the results from a cheap classical learner. We call this method SNAP2. Applied to over 100 SE tasks, it is the single best of all methods studied, reaching the top tier on 85% of tasks, ahead of the same LLM run alone (75%) and ahead of every method in which the classical optimizer retains control. It is also less expensive: relative to the LLM-alone method, it uses roughly 30% fewer tokens and runs 1.4x faster, since the classical setup performs the inexpensive work, and the LLM is invoked only to finish. We conclude that it is unwise to study classical learners or LLMs in isolation: there is much value in combining the two, and in the order that combination is applied.

cs.SE↗

arXiv2026

Can AI be Easy? Lessons Learned from the EZR.py Toolkit

Much recent press claims that developers no longer need to read code. We disagree, at least within the domain of tabular software-engineering (SE) optimization tasks: rows of $x$ and $y$ values where the $y$ values are expensive to obtain. As evidence we present 400 lines of EZR.py, a Python toolkit (no heavy dependencies) that implements Naive Bayes, $k$-means clustering, classification and regression trees, simulated annealing, local search, active learning, and complementary-Bayes text-mining relevance filtering for tabular SE data. EZR was built by repeatedly reading and refactoring AI tools to simplify and unify them. The result demonstrates that many seemingly different learning algorithms are nearly the same once stripped back to their core: classical algorithms collapse to a few lines each, and a state-of-the-art active learner fits in roughly 80 lines. Tested on the 120+ tabular SE optimization tasks in the MOOT repository, these tiny tools perform as well as or better than state-of-the-art explanation tools (SHAP, LIME), the SMAC3 optimizer, and SVM-based text-mining filters (FASTREAD), while running 500$\times$ faster than SMAC3, using orders of magnitude less labelled data, and building trees from fewer than ten variables even when thousands are available. We conclude that, within the scope of tabular SE optimization, reading and refactoring code is a useful method of generating insight, and small unified toolkits can rival large libraries. EZR is available under an open-source license. Install via \textsf{pip install ezr}; example data at \textsf{github.com/timm/moot}.

cs.SE↗

arXiv2026

Beyond the Prompt: Assessing Domain Knowledge Strategies for High-Dimensional LLM Optimization in Software Engineering

Background/Context: Large Language Models (LLMs) demonstrate strong performance on low-dimensional software engineering optimization tasks ($\le$11 features) but consistently underperform on high-dimensional problems where Bayesian methods dominate. A fundamental gap exists in understanding how systematic integration of domain knowledge (whether from humans or automated reasoning) can bridge this divide. Objective/Aim: We compare human versus artificial intelligence strategies for generating domain knowledge. We systematically evaluate four distinct architectures to determine if structured knowledge integration enables LLMs to generate effective warm starts for high-dimensional optimization. Method: We evaluate four approaches on MOOT datasets stratified by dimensionality: (1) Human-in-the-Loop Domain Knowledge Prompting (H-DKP), utilizing asynchronous expert feedback loops; (2) Adaptive Multi-Stage Prompting (AMP), implementing sequential constraint identification and validation; (3) Dimension-Aware Progressive Refinement (DAPR), conducting optimization in progressively expanding feature subspaces; and (4) Hybrid Knowledge-Model Approach (HKMA), synthesizing statistical scouting (TPE) with RAG-enhanced prompting. Performance is quantified via Chebyshev distance to optimal solutions and ranked using Scott-Knott clustering against an established baseline for LLM generated warm starts. Note that all human studies conducted as part of this study will comply with the policies of our local Institutional Review Board.

cs.SE↗

arXiv2026

SmartOracle -- An Agentic Approach to Mitigate Noise in Differential Oracles

Differential fuzzers detect bugs by executing identical inputs across distinct implementations of the same specification, such as JavaScript interpreters. Validating the outputs requires an oracle and for differential testing of JavaScript, these are constructed manually, making them expensive, time-consuming, and prone to false positives. Worse, when the specification evolves, this manual effort must be repeated. Inspired by the success of agentic systems in other SE domains, this paper introduces SmartOracle. SmartOracle decomposes the manual triage workflow into specialized Large Language Model (LLM) sub-agents. These agents synthesize independently gathered evidence from terminal runs and targeted specification queries to reach a final verdict. For historical benchmarks, SmartOracle achieves 0.84 recall with an 18% false positive rate. Compared to a sequential Gemini 2.5 Pro baseline, it improves triage accuracy while reducing analysis time by 4$\times$ and API costs by 10$\times$. In active fuzzing campaigns, SmartOracle successfully identified and reported previously unknown specification-level issues across major engines, including bugs in V8, JavaScriptCore, and GraalJS. The success of SmartOracle's agentic architecture on Javascript suggests it might be useful other software systems- a research direction we will explore in future work.

cs.SE↗

arXiv2025

MOOT: a Repository of Many Multi-Objective Optimization Tasks

Software engineers must make decisions that trade off competing goals (faster vs. cheaper, secure vs. usable, accurate vs. interpretable, etc.). Despite MSR's proven techniques for exploring such goals, researchers still struggle with these trade-offs. Similarly, industrial practitioners deliver sub-optimal products since they lack the tools needed to explore these trade-offs. To address this, MOOT (http://tiny.cc/moot) is a repository of many SE multi-objective optimization tasks. MOOT's 120+ tasks cover software configuration, cloud tuning, project health, process modeling, hyperparameter optimization, and more. Sample scripts for reading MOOT and generating baseline results are available -- just clone the repository and run the sample rqx.sh files (from tiny.cc/moot0). To the best of our knowledge, MOOT is the largest and most varied collection of real multi-objective optimization tasks in SE. We note that MOOT's novelty is infrastructural, not algorithmic-we contribute curated data and research enablement, not new optimization methods. MOOT enables harder and more credible research. MOOT lets us replace studies on toy problems (or just half a dozen hand-picked examples) with case studies on 120+ examples. Such studies could focus on stability, sample efficiency, failure modes, cross-domain generality, or many other questions (see list in this document).

cs.SE↗

arXiv2024

Extracting Usable Predictions from Quantized Networks through Uncertainty Quantification for OOD Detection

OOD detection has become more pertinent with advances in network design and increased task complexity. Identifying which parts of the data a given network is misclassifying has become as valuable as the network's overall performance. We can compress the model with quantization, but it suffers minor performance loss. The loss of performance further necessitates the need to derive the confidence estimate of the network's predictions. In line with this thinking, we introduce an Uncertainty Quantification(UQ) technique to quantify the uncertainty in the predictions from a pre-trained vision model. We subsequently leverage this information to extract valuable predictions while ignoring the non-confident predictions. We observe that our technique saves up to 80% of ignored samples from being misclassified. The code for the same is available here.

cs.CV↗

arXiv2023

The Cost of Compression: Investigating the Impact of Compression on Parametric Knowledge in Language Models

Compressing large language models (LLMs), often consisting of billions of parameters, provides faster inference, smaller memory footprints, and enables local deployment. Two standard compression techniques are pruning and quantization, with the former eliminating redundant connections in model layers and the latter representing model parameters with fewer bits. The key tradeoff is between the degree of compression and the impact on the quality of the compressed model. Existing research on LLM compression primarily focuses on performance in terms of general metrics like perplexity or downstream task accuracy. More fine-grained metrics, such as those measuring parametric knowledge, remain significantly underexplored. To help bridge this gap, we present a comprehensive analysis across multiple model families (ENCODER, ENCODER-DECODER, and DECODER) using the LAMA and LM-HARNESS benchmarks in order to systematically quantify the effect of commonly employed compression techniques on model performance. A particular focus is on tradeoffs involving parametric knowledge, with the goal of providing practitioners with practical insights to help make informed decisions on compression. We release our codebase1 to enable further research.

cs.CL↗