arXiv subjects

Nishant Aggarwal

Publications and source records attributed to Nishant Aggarwal.

3 recordsLinked to original sources

arXiv2026

Can LLMs Perform Deep Technical Comprehension of Computer Architecture Papers?

Can large language models perform deep technical comprehension of computer architecture papers -- not summarization, but structured critique that names the core mechanism, surfaces buried assumptions, and connects a contribution beyond its own scope? We study Gauntlet, an open-source pipeline that analyzes a paper through five independent expert-persona reviewers and an adversarial synthesis stage. On 20 ISCA 2025 and HPCA 2026 papers, ten researchers each wrote their own analyses and then judged, for papers other than their own, the human analysis against Gauntlet's. Across the 20 comparisons evaluators preferred Gauntlet in 15 (human in 4, one tie); its advantage is significant on per-analyst totals (paired Wilcoxon, p < 0.01) and largest on Critical Rigor, vanishing only on Calibration. Where humans win, it is on trust and usefulness rather than depth: a confident wrong claim, a mechanism described but not taught, or unprioritized breadth. A 98-paper automated ablation shows the gain comes from the multi-agent structure -- the pipeline beats the same model run as a single rich-persona agent on 96% of papers -- and specifically from its synthesis pass. We release all analyses, scores, and the rubric as a community resource.

cs.CY↗

arXiv2024

Exercising the CCPA Opt-out Right on Android: Legally Mandated but Practically Challenging

Many mobile apps' business model is based on sharing user data with ad networks to deliver personalized ads. The California Consumer Privacy Act (CCPA) gives California residents a right to opt out. In two experiments we evaluate to which extent popular Android apps enable California residents to exercise their right. In our first experiment -- manually exercising the right via app-level UIs -- we find that only 48 out of 100 apps implement a respective setting, which suggests that CCPA opt-out right compliance on the Android platform is generally low. In our second experiment -- automatically exercising the opt-out right by sending Global Privacy Control (GPC) signals -- we find for an app dataset of 1,811 apps that GPC is largely ineffective. While we estimate with 95% confidence that 62%--81% of apps in our app dataset must respect the CCPA opt-out right, many apps do not do so. Our evaluation of disabling apps' access to the AdID -- which is technically not intended for exercising the CCPA opt-out right but could be practically effective -- does not change our conclusion. For example, when sending GPC signals and disabling apps' access to the AdID, 338 apps still had the ccpa status of the ad network Vungle set to opted_in while only 26 had set it to opted_out. Overall, our results suggest a compliance gap as California residents have no effective way of exercising their CCPA opt-out right on the Android platform; neither at the app nor at the platform-level. We think that re-purposing the Android AdID setting as an opt-out right setting with legal meaning under the CCPA and other laws could close this gap and improve users' privacy on the platform significantly.

cs.CR↗

arXiv2023

Emulating Human Cognitive Processes for Expert-Level Medical Question-Answering with Large Language Models

In response to the pressing need for advanced clinical problem-solving tools in healthcare, we introduce BooksMed, a novel framework based on a Large Language Model (LLM). BooksMed uniquely emulates human cognitive processes to deliver evidence-based and reliable responses, utilizing the GRADE (Grading of Recommendations, Assessment, Development, and Evaluations) framework to effectively quantify evidence strength. For clinical decision-making to be appropriately assessed, an evaluation metric that is clinically aligned and validated is required. As a solution, we present ExpertMedQA, a multispecialty clinical benchmark comprised of open-ended, expert-level clinical questions, and validated by a diverse group of medical professionals. By demanding an in-depth understanding and critical appraisal of up-to-date clinical literature, ExpertMedQA rigorously evaluates LLM performance. BooksMed outperforms existing state-of-the-art models Med-PaLM 2, Almanac, and ChatGPT in a variety of medical scenarios. Therefore, a framework that mimics human cognitive stages could be a useful tool for providing reliable and evidence-based responses to clinical inquiries.

cs.CL↗