arXiv subjects

Jeff Huang

Publications and source records attributed to Jeff Huang.

At least 19 recordsLinked to original sources

arXiv2026

FuzzingBrain-Bench V1: Evaluating Open-Ended Bug Discovery by LLMs

Evaluating the ability of large language models (LLMs) to discover software bugs is increasingly important. Existing benchmarks typically evaluate this capability by asking the model to generate a proof-of-concept input that triggers a predefined target vulnerability. However, this setup may overlook valid crashes discovered by the model when they do not match the predefined target. As a result, the evaluation may not reflect the model's real capability. We present FuzzingBrain-Bench, a benchmark for assessing AI models' ability to discover bugs in open-source software. Models are given an open-source project and a sanitizer-instrumented harness in a self-contained Docker image. Their goal is to generate inputs that trigger as many distinct crashes as possible through the harness. A model's performance on each challenge is scored based on the number of distinct crash signatures it produces, capped at a predefined maximum and weighted by a difficulty coefficient. FuzzingBrain-Bench V1 consists of 77 challenges drawn from 43 open-source projects, with 36 C, 32 C++, and 9 Java/JVM challenges. We evaluate Claude Haiku 4.5, Claude Sonnet 4.6, and Claude Opus 4.8 on the full benchmark. Claude Opus 4.8 performs best, triggering crashes in 60 of 77 challenges and achieving a score of 196 out of 579. None of the three models triggers a crash in 13 challenges. The FuzzingBrain-Bench corpus and harnesses are publicly available at https://github.com/fuzzingbrain/FuzzingBrain-Bench.

cs.AI↗

arXiv2026

Debiasing the Observed Fast Radio Burst Population with the CHIME/FRB Selection Function

The recent release of CHIME/FRB Catalog~2 provides the largest sample to date with which to investigate the intrinsic distributions of fast radio bursts (FRBs). Leveraging an expanded campaign of 587,367 synethetic bursts injected into the live CHIME/FRB search pipeline, we perform a population analysis of the fluence, scattering timescale, pulse width, and dispersion measure distributions of Catalog~2 FRBs. We first infer the intrinsic population using a resampling-based framework that accounts for instrumental selection effects following previous CHIME/FRB population studies. A central goal of this work is to constrain the intrinsic distribution of scattering timescales, that remained weakly constrained in Catalog~1 owing to limited statistics at moderate and large scattering times ($\tau \gtrsim 10\,\mathrm{ms}$ at 600~MHz) and sparse injection coverage in this regime. Second, we construct an explicit multidimensional selection function by training a logistic regression model on the injected events. This model estimates the detection probability as a function of FRB observable properties, including higher-order interaction terms. We incorporate this selection function into a simulation-based inference framework to refine the inferred intrinsic scattering-timescale distribution. We find evidence for a slight downturn in the intrinsic FRB scattering timescale distribution, though a flat or slightly rising distribution cannot be ruled out, that is further supported through a comparison with the higher-frequency scattering timescale distribution observed by Commensal Real-time ASKAP Fast Transients (CRAFT) survey.

astro-ph.HE↗

arXiv2026

FuzzingBrain V2: A Multi-Agent LLM System for Automated Vulnerability Discovery and Reproduction

Software vulnerabilities pose critical security threats, with nearly 50,000 CVEs reported in 2025. While Large Language Models (LLMs) show promise for automated vulnerability detection, three key challenges remain. First, LLM-generated vulnerability reports suffer from high false positive rates and lack reproducible verification. Second, existing LLM-based approaches use suboptimal granularities for vulnerability localization: function-level analysis overlooks bugs when context becomes extensive, while line-level analysis lacks sufficient context. Third, existing approaches have difficulty reasoning about vulnerabilities with complex cross-function dependencies and triggering conditions. We present FuzzingBrain V2, a multi-agent system that addresses these gaps through four key contributions: (1) fully automated vulnerability analysis built on Google's OSS-Fuzz, ensuring all reported vulnerabilities are fuzzer-reproducible; (2) Suspicious Point, a novel control-flow-based abstraction for precise vulnerability localization at the optimal granularity; (3) logic-driven hierarchical function analysis with dual-layer fuzzing enhancing function coverage under resource constraints; (4) MCP-based static and dynamic analysis tools with context engineering enhancing complex vulnerability reasoning. On the AIxCC 2025 Final Competition C/C++ dataset, FuzzingBrain V2 achieved 90% detection rate (36 of 40 vulnerabilities). In real-world deployment, FuzzingBrain V2 discovered 29 zero-day vulnerabilities across 12 open-source projects, all confirmed and fixed by maintainers, with 2 assigned CVE IDs.

cs.CR↗

arXiv2026

Quality-Assured Fuzz Harness Generation via the Four Principles Framework

Fuzz testing is the dominant technique for finding memory-safety vulnerabilities in C/C++ software, yet its effectiveness hinges on the quality of fuzz harnesses -- the programs that bridge fuzzers and library APIs. A growing body of tools now automate harness generation, but none systematically ensures the correctness of produced harnesses: logic errors, API misuse, and lifecycle violations go undetected at the source level. As LLM-driven generation scales harness creation, uncontrolled quality turns scale into a liability. We present QuartetFuzz, an autonomous harness-generation system that systematically improves correctness throughout the generation process. At its core is the Four Principles framework -- Logic Correctness (P1), API Protocol Compliance (P2), Security Boundary Respect (P3), and Entry Point Adequacy (P4) -- the first source-level definition of harness correctness with mathematical specifications and implementable checks. We operationalize these principles in an autonomous LLM agent that produces harnesses satisfying P1-P4 through a generate-check-fix loop before any fuzzing begins. Deployed on 23 open-source projects spanning C/C++, Java, and JavaScript, the system submits 42 bug reports, of which 29 are fixed or confirmed upstream (including 3 CVEs) and only 2 are rejected (4.8% FP rate). During generation, the built-in P1/P2 checks automatically intercepted 58 harness-induced crashes that would otherwise have been false positives. Applied as a quality auditor to 586 existing production harnesses across 70 projects, the system identifies 53 violations (45 confirmed, 35 fixed). We release a dataset of 100 labeled harnesses for reproducible evaluation. Code and dataset are available at https://github.com/OwenSanzas/QuartetFuzz

cs.CR↗

arXiv2026

On the polarization position angle jumps in FRB 20240114A

Fast radio bursts (FRBs), thought to originate from magnetars, exhibit diverse polarization properties that constrain their emission physics and local magneto-ionic environments. The polarization position angle (PPA) is particularly sensitive to magnetic-field geometry in the emitting region and propagation effects in the magnetosphere and beyond. In hyper-active repeaters, PPAs are typically stable within bursts and over timescales of hours to days. Here, we present observations of the repeating source FRB~20240114A, which show significant burst-to-burst PPA variations. Using full-Stokes, high-time-resolution observations from the Nan\c{c}ay Radio Telescope (1.1--1.8\,GHz) and the Effelsberg 100-m telescope (1.3--1.5\,GHz) over $\sim1$~year, we measure rotation measures (RMs), polarization fractions, and time-resolved PPAs across 12 epochs. The RMs remain stable, and the emission is predominantly highly linearly polarized, with $\sim81\%$ of bursts showing $L/I > 0.8$, while circular polarization is weaker ($\sim16\%$ with $|V/I| > 0.1$). We find no evidence for Faraday conversion. The PPA exhibits rapid, stochastic variations from milliseconds to hours, spanning $\pm90^\circ$ during two active periods and $\pm50^\circ$ in a third. The distribution of PPA jumps shows that (1) there is no difference in the distribution of jumps on timescales shorter or longer than 1\,s; (2) positive and negative jumps are equally likely; and (3) a jump of $\pm90^\circ$, as expected from, e.g., orthogonal mode jumps, is not more common than any other value. This combination of stable RM, high linear polarization, and extreme PPA variability is not seen in other hyper-active repeaters. These results disfavor emission from a single fixed region and instead suggest multiple emission regions and/or strong magnetospheric and foreground propagation effects, such as plasma lensing.

astro-ph.HE↗

arXiv2026

TraceScope: Interactive URL Triage via Decoupled Checklist Adjudication

Modern phishing campaigns increasingly evade snapshot-based URL classifiers using interaction gates (e.g., checkbox/slider challenges), delayed content rendering, and logo-less credential harvesters. This shifts URL triage from static classification toward an interactive forensics task: an analyst must actively navigate the page while isolating themselves from potential runtime exploits. We present TraceScope, a decoupled triage pipeline that operationalizes this workflow at scale. To prevent the observer effect and ensure safety, a sandboxed operator agent drives a real GUI browser guided by visual motivation to elicit page behavior, freezing the session into an immutable evidence bundle. Separately, an adjudicator agent circumvents LLM context limitations by querying evidence on demand to verify a MITRE ATT&CK checklist, and generates an audit-ready report with extracted indicators of compromise (IOCs) and a final verdict. Evaluated on 708 reachable URLs from existing dataset (241 verified phishing from PhishTank and 467 benign from Tranco-derived crawling), TraceScope achieves 0.94 precision and 0.78 recall, substantially improving recall over three prior visual/reference-based classifiers while producing reproducible, analyst-grade evidence suitable for review. More importantly, we manually curated a dataset of real-world phishing emails to evaluate our system in a practical setting. Our evaluation reveals that TraceScope demonstrates superior performance in a real-world scenario as well, successfully detecting sophisticated phishing attempts that current state-of-the-art defenses fail to identify.

cs.CR↗

arXiv2026

A Systematic Study of LLM-Based Architectures for Automated Patching

Large language models (LLMs) have shown promise for automated patching, but their effectiveness depends strongly on how they are integrated into patching systems. While prior work explores prompting strategies and individual agent designs, the field lacks a systematic comparison of patching architectures. In this paper, we present a controlled evaluation of four LLM-based patching paradigms -- fixed workflow, single-agent system, multi-agent system, and general-purpose code agents -- using a unified benchmark and evaluation framework. We analyze patch correctness, failure modes, token usage, and execution time across real-world vulnerability tasks. Our results reveal clear architectural trade-offs: fixed workflows are efficient but brittle, single-agent systems balance flexibility and cost, and multi-agent designs improve generalization at the expense of substantially higher overhead and increased risk of reasoning drift on complex tasks. Surprisingly, general-purpose code agents achieve the strongest overall patching performance, benefiting from general-purpose tool interfaces that support effective adaptation across vulnerability types. Overall, we show that architectural design and iteration depth, rather than model capability alone, dominate the reliability and cost of LLM-based automated patching.

cs.CR↗

arXiv2026

CHIME/Slow overview and pilot survey: A new backend to search for second-duration radio transients with the CHIME telescope

We present an overview of CHIME/Slow, a real-time transient search backend under development to search for second-duration radio transients using the CHIME telescope, and results obtained from a pilot survey carried out using the prototype version of the search pipeline. The prototype CHIME/Slow pipeline was tested on archival data obtained in December 2022, January 2023 and February 2023 with a total on-sky time of 17 days with an instantaneous Field of View (FoV) of $\sim$13 deg$^2$ . In this pilot survey, we detected nine bursts, one from a new non-repeating source and eight from the known hyperactive repeating source FRB 20220912A. Out of these nine bursts, two bursts from the repeater were not detected by CHIME/FRB, while the non-repeater was detected in the side-lobe of a beam in the CHIME/FRB exhibiting shorter pulse width and narrower bandwidth compared to the CHIME/Slow detection. Here we report properties of the bursts, discuss the sensitivity and completeness of the current version of the CHIME/Slow pipeline, and outline future development to improve its performance. Finally, based on these results, we report the all-sky rate (95% credible region) of radio transients with pulse widths between 16 ms to 5 s, fluence above 5 Jy ms and observing frequency of 600 MHz to be between 184 and 4556 bursts sky$^{-1}$ day$^{-1}$.

astro-ph.IM↗

arXiv2026

SoK: DARPA's AI Cyber Challenge (AIxCC): Competition Design, Architectures, and Lessons Learned

DARPA's AI Cyber Challenge (AIxCC, 2023--2025) is the largest competition to date for building fully autonomous cyber reasoning systems (CRSs) that leverage recent advances in AI -- particularly large language models (LLMs) -- to discover and remediate vulnerabilities in real-world open-source software. This paper presents the first systematic analysis of AIxCC. Drawing on design documents, source code, execution traces, and discussions with organizers and competing teams, we examine the competition's structure and key design decisions, characterize the architectural approaches of finalist CRSs, and analyze competition results beyond the final scoreboard. Our analysis reveals the factors that truly drove CRS performance, identifies genuine technical advances achieved by teams, and exposes limitations that remain open for future research. We conclude with lessons for organizing future competitions and broader insights toward deploying autonomous CRSs in practice.

cs.CR↗

arXiv2025

All You Need Is A Fuzzing Brain: An LLM-Powered System for Automated Vulnerability Detection and Patching

Our team, All You Need Is A Fuzzing Brain, was one of seven finalists in DARPA's Artificial Intelligence Cyber Challenge (AIxCC), placing fourth in the final round. During the competition, we developed a Cyber Reasoning System (CRS) that autonomously discovered 28 security vulnerabilities - including six previously unknown zero-days - in real-world open-source C and Java projects, and successfully patched 14 of them. The complete CRS is open source at https://github.com/o2lab/afc-crs-all-you-need-is-a-fuzzing-brain. This paper provides a detailed technical description of our CRS, with an emphasis on its LLM-powered components and strategies. Building on AIxCC, we further introduce a public leaderboard for benchmarking state-of-the-art LLMs on vulnerability detection and patching tasks, derived from the AIxCC dataset. The leaderboard is available at https://o2lab.github.io/FuzzingBrain-Leaderboard/.

cs.CR↗

arXiv2025

FRB 20250316A: A Brilliant and Nearby One-Off Fast Radio Burst Localized to 13 parsec Precision

Precise localizations of a small number of repeating fast radio bursts (FRBs) using very long baseline interferometry (VLBI) have enabled multiwavelength follow-up observations revealing diverse local environments. However, the 2--3\% of FRB sources that are observed to repeat may not be representative of the full population. Here we use the VLBI capabilities of the full CHIME Outriggers array for the first time to localize a nearby (40 Mpc), bright (kJy), and apparently one-off FRB source, FRB 20250316A, to its environment on 13-pc scales. We use optical and radio observations to place deep constraints on associated transient emission and the properties of its local environment. We place a $5\sigma$ upper limit of $L_{\mathrm{9.9~\mathrm{GHz}}} < 2.1\times10^{25}~\mathrm{erg~s^{-1}~Hz^{-1}}$ on spatially coincident radio emission, a factor of 100 lower than any known compact persistent radio source associated with an FRB. Our KCWI observations allow us to characterize the gas density, metallicity, nature of gas ionization, dust extinction and star-formation rate through emission line fluxes. We leverage the exceptional brightness and proximity of this source to place deep constraints on the repetition of FRB 20250316A, and find it is inconsistent with all well-studied repeaters given the non-detection of bursts at lower spectral energies. We explore the implications of a measured offset of 190$\pm20$ pc from the center of the nearest star-formation region, in the context of progenitor channels. FRB 20250316A marks the beginning of an era of routine localizations for one-off FRBs on tens of mas-scales, enabling large-scale studies of their local environments.

astro-ph.HE↗

arXiv2025

Challenging GPU Dominance: When CPUs Outperform for On-Device LLM Inference

The common assumption in on-device AI is that GPUs, with their superior parallel processing, always provide the best performance for large language model (LLM) inference. In this work, we challenge this notion by empirically demonstrating that, under certain conditions, CPUs can outperform GPUs for LLM inference on mobile devices. Using a 1-billion-parameter LLM deployed via llama.cpp on the iPhone 15 Pro, we show that a CPU-only configuration (two threads, F16 precision) achieves 17 tokens per second, surpassing the 12.8 tokens per second obtained with GPU acceleration. We analyze the architectural factors driving this counterintuitive result, revealing that GPU memory transfer overhead and CPU thread optimization play a critical role. Furthermore, we explore the impact of thread oversubscription, quantization strategies, and hardware constraints, providing new insights into efficient on-device AI execution. Our findings challenge conventional GPU-first thinking, highlighting the untapped potential of optimized CPU inference and paving the way for smarter deployment strategies in mobile AI. However, fully explaining the observed CPU advantage remains difficult due to limited access to low-level profiling tools on iOS.

cs.DC↗

arXiv2025

InfoVids: Reimagining the Viewer Experience with Alternative Visualization-Presenter Relationships

Traditional data presentations typically separate the presenter and visualization into two separate spaces--the 3D world and a 2D screen--enforcing visualization-centric stories. To create a more human-centric viewing experience, we establish a more equitable relationship between the visualization and the presenter through our InfoVids. These infographics-inspired informational videos are crafted to redefine relationships between the presenter and visualizations. As we design InfoVids, we explore how the use of layout, form, and interactions affects the viewer experience. We compare InfoVids against their baseline 2D `slides' equivalents across 9 metrics with 30 participants and provide practical, long-term insights from an autobiographical perspective. Our mixed methods analyses reveal that this paradigm reduced viewer attention splitting, shifted the focus from the visualization to the presenter, and led to more interactive, natural, and engaging full-body data performances for viewers. Ultimately, InfoVids helped viewers re-imagine traditional dynamics between the presenter and visualizations.

cs.HC↗

arXiv2025

VeriFix: Verifying Your Fix Towards An Atomicity Violation

Atomicity violation is one of the most serious types of bugs in concurrent programs. Synchronizations are commonly used to enforce atomicity. However, it is very challenging to place synchronizations correctly and sufficiently due to complex thread interactions and large input space. This paper presents \textsf{VeriFix}, a new approach for verifying atomicity violation fixes. Given a buggy trace that exposes an atomicity violation and a corresponding fix, % in the form of locks, \textsf{VeriFix} effectively verifies if the fix introduces sufficient synchronizations to repair the atomicity violation without introducing new deadlocks. The key idea is that \textsf{VeriFix} transforms the fix verification problem into a property verification problem, in which both the observed atomicity violation and potential deadlocks are encoded as a safety property, and both the inputs and schedules are encoded as symbolic constraints. By reasoning the conjoined constraints with an SMT solver, \textsf{VeriFix} systematically explores all reachable paths %from the whole schedule and input space and verifies if there exists a concrete \textit{schedule+input} combination to manifest the intended atomicity or any new deadlocks. We have implemented and evaluated \verifix\ on a collection of real-world C/C++ programs. The result shows that \textsf{VeriFix} significantly outperforms the state-of-the-art.

cs.SE↗

arXiv2025

LLMs in Software Security: A Survey of Vulnerability Detection Techniques and Insights

Large Language Models (LLMs) are emerging as transformative tools for software vulnerability detection, addressing critical challenges in the security domain. Traditional methods, such as static and dynamic analysis, often falter due to inefficiencies, high false positive rates, and the growing complexity of modern software systems. By leveraging their ability to analyze code structures, identify patterns, and generate repair suggestions, LLMs, exemplified by models like GPT, BERT, and CodeBERT, present a novel and scalable approach to mitigating vulnerabilities. This paper provides a detailed survey of LLMs in vulnerability detection. It examines key aspects, including model architectures, application methods, target languages, fine-tuning strategies, datasets, and evaluation metrics. We also analyze the scope of current research problems, highlighting the strengths and weaknesses of existing approaches. Further, we address challenges such as cross-language vulnerability detection, multimodal data integration, and repository-level analysis. Based on these findings, we propose solutions for issues like dataset scalability, model interpretability, and applications in low-resource scenarios. Our contributions are threefold: (1) a systematic review of how LLMs are applied in vulnerability detection; (2) an analysis of shared patterns and differences across studies, with a unified framework for understanding the field; and (3) a summary of key challenges and future research directions. This work provides valuable insights for advancing LLM-based vulnerability detection. We also maintain and regularly update latest selected paper on https://github.com/OwenSanzas/LLM-For-Vulnerability-Detection

cs.CR↗

arXiv2024

A Repeating Fast Radio Burst Source in a Low-Luminosity Dwarf Galaxy

We present the localization and host galaxy of FRB 20190208A, a repeating source of fast radio bursts (FRBs) discovered using CHIME/FRB. As part of the PRECISE repeater localization program on the EVN, we monitored FRB 20190208A for 65.6 hours at $\sim1.4$ GHz and detected a single burst, which led to its VLBI localization with 260 mas uncertainty (2$\sigma$). Follow-up optical observations with the MMT Observatory ($i\gtrsim 25.7$ mag (AB)) found no visible host at the FRB position. Subsequent deeper observations with the GTC, however, revealed an extremely faint galaxy ($r=27.32 \pm0.16$ mag), very likely ($99.95 \%$) associated with FRB 20190208A. Given the dispersion measure of the FRB ($\sim580$ pc cm$^{-3}$), even the most conservative redshift estimate ($z_{\mathrm{max}}\sim0.83$) implies that this is the lowest-luminosity FRB host to date ($\lesssim10^8L_{\odot}$), even less luminous than the dwarf host of FRB 20121102A. We investigate how localization precision and the depth of optical imaging affect host association, and discuss the implications of such a low-luminosity dwarf galaxy. Unlike the other repeaters with low-luminosity hosts, FRB 20190208A has a modest Faraday rotation measure of a few tens of rad m$^{-2}$, and EVN plus VLA observations reveal no associated compact persistent radio source. We also monitored FRB 20190208A for 40.4 hours over 2 years as part of the \'ECLAT repeating FRB monitoring campaign on the Nan\c{c}ay Radio Telescope, and detected one burst. Our results demonstrate that, in some cases, the robust association of an FRB with a host galaxy will require both high localization precision, as well as deep optical follow-up.

astro-ph.HE↗

arXiv2024

Therapy as an NLP Task: Psychologists' Comparison of LLMs and Human Peers in CBT

Large language models (LLMs) are being used as ad-hoc therapists. Research suggests that LLMs outperform human counselors when generating a single, isolated empathetic response; however, their session-level behavior remains understudied. In this study, we compare the session-level behaviors of human counselors with those of an LLM prompted by a team of peer counselors to deliver single-session Cognitive Behavioral Therapy (CBT). Our three-stage, mixed-methods study involved: a) a year-long ethnography of a text-based support platform where seven counselors iteratively refined CBT prompts through self-counseling and weekly focus groups; b) the manual simulation of human counselor sessions with a CBT-prompted LLM, given the full patient dialogue and contextual notes; and c) session evaluations of both human and LLM sessions by three licensed clinical psychologists using CBT competence measures. Our results show a clear trade-off. Human counselors excel at relational strategies -- small talk, self-disclosure, and culturally situated language -- that lead to higher empathy, collaboration, and deeper user reflection. LLM counselors demonstrate higher procedural adherence to CBT techniques but struggle to sustain collaboration, misread cultural cues, and sometimes produce "deceptive empathy," i.e., formulaic warmth that can inflate users' expectations of genuine human care. Taken together, our findings imply that while LLMs might outperform counselors in generating single empathetic responses, their ability to lead sessions is more limited, highlighting that therapy cannot be reduced to a standalone natural language processing (NLP) task. We call for carefully designed human-AI workflows in scalable support: LLMs can scaffold evidence-based techniques, while peers provide relational support. We conclude by mapping concrete design opportunities and ethical guardrails for such hybrid systems.

cs.HC↗

arXiv2024

A Nan\c{c}ay Radio Telescope study of the hyperactive repeating FRB 20220912A

The repeating fast radio burst source FRB 20220912A was remarkably active in the weeks after its discovery. Here we report 696 bursts detected with the Nan\c{c}ay Radio Telescope (NRT) as part of the Extragalactic Coherent Light from Astrophysical Transients (\'ECLAT) monitoring campaign. We present 68 observations, conducted from October 2022 to April 2023, with a total duration of 61 hours and an event rate peaking at $75^{+10}_{-9}$ bursts per hour above a fluence threshold of 0.59 Jy ms in the $1.2-1.7$-GHz band. Most bursts in the sample occur towards the bottom of the observing band. They follow a bimodal wait-time distribution, with peaks at 33.4 ms and 67.0 s. We find a roughly constant dispersion measure (DM) over time ($\delta$DM $\lesssim$ 2 pc cm$^{-3}$) when taking into account `sad-trombone' drift, with a mean drift rate of $-8.8 $MHz ms$^{-1}$. Nonetheless, we confirm small $\sim0.3$ pc cm$^{-3}$ DM variations using microshot structure, while finding that microstructure is rare in our sample -- despite the 16 $\mu$s time resolution of the data. The cumulative spectral energy distribution shows more high-energy bursts ($E_\nu \gtrsim 10^{31}$ erg/Hz) than would be expected from a simple power-law distribution. The burst rate per observation appears Poissonian, but the full set of observations is better modelled by a Weibull distribution, showing clustering. We discuss the various observational similarities that FRB 20220912A shares with other (hyper)active repeaters, which as a group are beginning to show a common set of phenomenological traits that provide multiple useful dimensions for their quantitative comparison and modelling.

astro-ph.HE↗