arXiv subjects

Abhishek Gulati

Publications and source records attributed to Abhishek Gulati.

4 recordsLinked to original sources

arXiv2026

Reducing Hallucination in Enterprise AI Workflows via Hybrid Utility Minimum Bayes Risk (HUMBR)

Although LLMs drive automation, it is critical to ensure immense consideration for high-stakes enterprise workflows such as those involving legal matters, risk management, and privacy compliance. For Meta, and other organizations like ours, a single hallucinated clause in such high stakes workflows risks material consequences. We show that by framing hallucination mitigation as a Minimum Bayes Risk (MBR) problem, we can dramatically reduce this risk. Specifically, we introduce a Hybrid Utility MBR (HUMBR) framework that synthesizes semantic embedding similarity with lexical precision to identify consensus without ground-truth references, for which we derive rigorous error bounds. We complement this theoretical analysis with a comprehensive empirical evaluation on widely-used public benchmark suites (TruthfulQA and LegalBench) and also real world data from Meta production deployment. The results from our empirical study show that MBR significantly outperforms standard Universal Self-Consistency. Notably, 81% of the pipeline's suggestions were preferred over human-crafted ground truth, and critical recall failures were virtually eliminated.

cs.LG↗

arXiv2026

Best Practices on QSP Model Reporting for Regulatory Use: perspectives from ISoP QSP SIG Working Group

Quantitative systems pharmacology (QSP) models are increasingly applied to inform decision making across drug development and to support regulatory interactions within model informed drug development (MIDD). QSP supports a broad range of applications across drug development and can be tailored to specific therapeutic areas, mechanisms of action, and contexts of use (CoU). While this diversity is a core strength of QSP, it also presents challenges for reporting for regulatory use. Despite the growing impact of QSP models, there is currently no established guidance on how QSP analyses should be documented and reported for regulatory purposes. This white paper, developed by the International Society of Pharmacometrics (ISoP) QSP Special Interest Group Working Group on Credibility Assessment of QSP for Regulatory Use, seeks to address this gap by proposing best practices for QSP model reporting in regulatory settings. The recommendations are grounded in collective real world experience from regulatory interactions and are aligned with reporting guidance established for physiologically based pharmacokinetic (PBPK) modeling and reporting principles outlined in ICH M15. Rather than prescribing a rigid, one size fits all template, this work proposes a flexible, tiered reporting framework that accounts for development phase and model impact. The proposed framework is intended to facilitate regulatory review and enhance transparency while accommodating the inherent diversity of QSP modeling.

math.DS↗

arXiv2026

Just-in-Time Catching Test Generation at Meta

We report on Just-in-Time catching test generation at Meta, designed to prevent bugs in large scale backend systems of hundreds of millions of line of code. Unlike traditional hardening tests, which pass at generation time, catching tests are meant to fail, surfacing bugs before code lands. The primary challenge is to reduce development drag from false positive test failures. Analyzing 22,126 generated tests, we show code-change-aware methods improve candidate catch generation 4x over hardening tests and 20x over coincidentally failing tests. To address false positives, we use rule-based and LLM-based assessors. These assessors reduce human review load by 70%. Inferential statistical analysis showed that human-accepted code changes are assessed to have significantly more false positives, while human-rejected changes have significantly more true positives. We reported 41 candidate catches to engineers; 8 were confirmed to be true positives, 4 of which would have led to serious failures had they remained uncaught. Overall, our results show that Just-in-Time catching is scalable, industrially applicable, and that it prevents serious failures from reaching production.

cs.SE↗

arXiv2025

Mutation-Guided LLM-based Test Generation at Meta

This paper describes Meta's ACH system for mutation-guided LLM-based test generation. ACH generates relatively few mutants (aka simulated faults), compared to traditional mutation testing. Instead, it focuses on generating currently undetected faults that are specific to an issue of concern. From these currently uncaught faults, ACH generates tests that can catch them, thereby `killing' the mutants and consequently hardening the platform against regressions. We use privacy concerns to illustrate our approach, but ACH can harden code against {\em any} type of regression. In total, ACH was applied to 10,795 Android Kotlin classes in 7 software platforms deployed by Meta, from which it generated 9,095 mutants and 571 privacy-hardening test cases. ACH also deploys an LLM-based equivalent mutant detection agent that achieves a precision of 0.79 and a recall of 0.47 (rising to 0.95 and 0.96 with simple pre-processing). ACH was used by Messenger and WhatsApp test-a-thons where engineers accepted 73% of its tests, judging 36% to privacy relevant. We conclude that ACH hardens code against specific concerns and that, even when its tests do not directly tackle the specific concern, engineers find them useful for their other benefits.

cs.SE↗