Searcharxiv⌕ Search

arXiv subjects

Guram Gogia

Publications and source records attributed to Guram Gogia.

7 recordsLinked to original sources

CWE-bench: Refocusing Agentic Security from Exploitation to Defense

Cybersecurity evaluations of AI agents have focused largely on offense, yet a working exploit does not show that an agent can make software safe. An attacker needs to exploit only one vulnerability; a defender must find and fix all of them. We introduce CWE-bench, a benchmark of 250 repository audit-and-repair tasks spanning 185 CWE types and eight languages, grouped into ten weakness classes. We release 100 tasks publicly and keep 150 held out. Agents are given a repository to audit and a description of its threat model, but are not told the weakness, where it occurs, or how many vulnerable sites there are. Each patch must block the exploit while preserving original functionality, as checked by a deterministic programmatic verifier. Verifiers were curated with panels of agentic judges and reviewed by hand, so that they reject incomplete repairs without penalizing correct ones. The best agent reaches 67% pass@1 and 79% pass@4. We further conduct analyses showing that (i) agents with similar pass@1 differ widely in retry gains and cost, and (ii) no single agent is best across languages or weakness classes. As the offensive capabilities of agents improve, CWE-bench provides a broad and rigorous measure of whether their defensive capabilities keep pace.

cs.CR↗

Rubricon: Automated Rubric Generation and Refinement for LLM Evaluation and Training

Rubrics enable reward assignment on tasks without verifiable answers, but are expensive to manually produce from scratch. We present Rubricon, a pipeline that generates discriminative rubrics without requiring human preference labels, rubric annotation, or generator-model training. Using any LLM, Rubricon samples multiple rubrics, selects the most discriminative candidate, adds criteria to capture overlooked requirements or response flaws, and prunes criteria that are misaligned or shallow. Alongside LLM self-critique and test-time scaling, Rubricon uses observed criterion pass rates on example model responses to guide refinement. Across five professional, consumer, clinical, and general-use LLM benchmarks, Rubricon produces rubrics whose scores of frontier LLM responses exhibit 9% greater alignment with labor-intensive expert-authored rubrics, compared to the best alternative synthetic rubric generation method among RRD, QSRG, OpenRubrics, and agentic generation via harnesses like Codex. On HealthBench, post-training a Qwen3.5-9B model with Rubricon rubrics yields 4.5% better performance than training with rubrics from the strongest alternative synthetic rubric generator, RRD.

cs.AI↗

Divergent Paths: Learning an Outcome-Predictive Representation of LLM Agent Trajectories

In this work, we reframe agentic trajectories as paths through a shared embedding space of actions and environmental observations. We show that successful and failed rollouts of the same task separate in this space, with simple per-task discriminators predicting held-out rollout outcomes at up to 0.95 AUROC across three agent benchmarks and three policy models. To exploit this signal across tasks, we introduce Precog-RM, a transformer-based model that meta-learns a representation in which outcome prediction can be performed in a few-shot setting. We show that Precog-RM generalizes to unseen tasks and models, and gives confident predictions of trajectory success long before termination. Deployed as an early stopping controller for Group Relative Policy Optimization (GRPO) on Tau2-Bench, Precog-RM cuts training FLOPs by up to 47.3\% while maintaining performance comparable to baseline GRPO.

cs.LG↗

BankerToolBench: Evaluating AI Agents in End-to-End Investment Banking Workflows

Existing AI benchmarks lack the fidelity to assess economically meaningful progress on professional workflows. To evaluate frontier AI agents in a high-value, labor-intensive profession, we introduce BankerToolBench (BTB): an open-source benchmark of end-to-end analytical workflows routinely performed by junior investment bankers. To develop an ecologically valid benchmark grounded in representative work environments, we collaborated with 502 investment bankers from leading firms. BTB requires agents to execute senior banker requests by navigating data rooms, using industry tools (market data platform, SEC filings database), and generating multi-file deliverables--including Excel financial models, PowerPoint pitch decks, and PDF/Word reports. Completing a BTB task takes bankers up to 21 hours, underscoring the economic stakes of successfully delegating this work to AI. BTB enables automated evaluation of any LLM or agent, scoring deliverables against 100+ rubric criteria defined by veteran investment bankers to capture stakeholder utility. Testing 9 frontier models, we find that even the best-performing model (GPT-5.4) fails nearly half of the rubric criteria and bankers rate 0% of its outputs as client-ready. Our failure analysis reveals key obstacles (such as breakdowns in cross-artifact consistency) and improvement directions for agentic AI in high-stakes professional workflows.

cs.AI↗

The origin of large amplitude oscillations of dust particles in a plasma sheath

Micron-size charged particles can be easily levitated in low-density plasma environments. At low pressures, suspended particles have been observed to spontaneously oscillate around an equilibrium position. In systems of many particles, these oscillations can catalyze a variety of nonequilibrium, collective behaviors. Here, we report spontaneous oscillations of single particles that remain stable for minutes with striking regularity in amplitude and frequency. The oscillation amplitude can also exceed 1 cm, nearly an order of magnitude larger than previously observed. Using an integrated experimental and numerical approach, we show how the motion of an individual particle can be used to extract the electrostatic force and equilibrium charge variation in the plasma sheath. Additionally, using a delayed-charging model, we are able to accurately capture the nonlinear dynamics of the particle motion, and estimate the particle's equilibrium charging time in the plasma environment.

physics.plasm-ph↗

Intermittent "Turbulence" in a Many-body System

In natural settings, intermittent dynamics are ubiquitous and often arise from a coupling between external driving and spatial heterogeneities. A well-known example is the generation of transient, turbulent puffs of fluid through a pipe with rough walls. Here we show how similar dynamics can emerge in a discrete, crystalline system of particles driven by noise. Polydispersity in particle masses leads to localized vibrational modes that effectuate a transition to a gas-like phase. A minimal model for the evolution of the system's mechanical energies exhibits quasi-cyclic oscillations, and a single, dimensionless number captures the essential features of the intermittent dynamics, analogous to the Reynolds number for pipe flow.

nlin.AO↗

Emergent Bistability and Switching in a Nonequilibrium Crystal

Multistability is an inseparable feature of many physical, chemical and biological systems which are driven far from equilibrium. In these nonequilibrium systems, stochastic dynamics often induces switching between distinct states on emergent timescales, for example, bistable switching is a natural feature of noisy, spatially-extended systems that consist of bistable elements. Nevertheless, here we present experimental evidence that bistable elements are not required for the global bistability of a system. We observe temporal switching between a crystalline, condensed state and a gas-like, excited state in a spatially-extended, quasi-two-dimensional system of charged microparticles. Accompanying numerical simulations show that conservative forces, damping, and stochastic noise are sufficient to prevent steady-state equilibrium, leading to switching between the two states over a range of time scales, from seconds to hours.

nlin.AO↗