Searcharxiv⌕ Search

arXiv subjects

David Kebo Houngninou

Publications and source records attributed to David Kebo Houngninou.

7 recordsLinked to original sources

Repeating Work Can Lower Energy per Successful Goal in LLM Agents

When a temporary call failure interrupts a tool-using LLM agent, the framework must choose how much completed work to repeat. Repeating less completed work intuitively saves computation, but recovery mechanisms also change subsequent execution and the chance of success. We compare recovery mechanisms by energy per successful goal (EpG), which divides the energy of all task runs (including failed runs) by the number that finish correctly. In controlled vLLM experiments on one node, we measure board energy above idle for eight recovery options on reliably solved multi-turn tasks, with selected comparisons across three model families. For the model tested across all eight options, with the prefix cache retained, starting over has 22 percent lower EpG than continuing after early failures. Under the same cache condition, continuing has 24 percent lower EpG than starting over after late failures. Among all eight recovery options, repeating only the interrupted turn has the lowest observed EpG after early and late tool failures on this model. Holding recovery fixed, retaining the prefix cache reduces prompt tokens needing fresh computation by 86 percent but reduces recovery energy by only 8 percent at contexts up to 7,333 tokens. These measurements and a condition relating recovery energy and success show when repeating work lowers EpG, helping framework developers choose how their agents recover on their intended models and tasks.

cs.PF↗

Understanding the Energy Cost of Quantized Agents on Shared GPUs

Quantization can lower the energy of language-model calls and free memory for serving more agents, but its benefit per completed task remains unclear when agents share a GPU. Measurements of fixed requests or isolated agents leave open how the agents' generated work interact with competition for compute and cache. We present a systematic study of precision and admission limits for two agent models on A100 deployments with 40 or 80 GB of memory, using coding and data-analysis tasks with independently verified outcomes. We replay recorded calls across precisions to isolate execution energy and cap the cache to test the value of freed memory. For both models with four admitted agents where requests fit in the cache, 4-bit weights with 16-bit activations execute the same calls with 10-25% less energy than 8-bit weights and activations. Despite this saving, extra work and fewer completions in live runs raise their energy per verified completion by 49%. Where requests compete for cache on the 40 GB deployment, 4-bit weights cost less, and capping their cache raises energy per verified completion by 34% with 16 agents. These results show how generated work and cache demand change the energy benefit of quantization, providing a practical basis for choosing precision and admission limits together.

cs.PF↗

Measuring the Energy Cost of Completing Tasks with Quantized Language-Model Agents

Quantization reduces the memory and energy needed to serve large language models (LLMs) and AI agents for coding and data analysis can now run on locally hosted LLMs. In practice, this benefit is commonly benchmarked per output token. Yet an agent calls its model repeatedly, runs tools, and revises its work, and quantization can change both how much the agent generates and whether it succeeds. Existing energy studies either compare quantized models on single requests or measure agents at one fixed precision, overlooking how a change in precision plays out across a complete agent task. To address this gap, this paper presents the first characterization of the task-level energy trade-offs of model quantization in locally hosted agents. We develop 29 agent tasks with automatic pass/fail checks in six families, ranging from single-answer SQL analytics to multi-step repository bug fixing, and measure CPU and GPU energy per output token and per completed task for 4-bit and 8-bit versions of two open-weight models on bare-metal NVIDIA A100 nodes with llama$.$cpp. Our findings reveal that 4-bit precision generally saves energy per token, though whether it saves energy per completed task is task dependent and changes with the model and the execution budget (i.e., limits on turns, tokens, and time). We trace this discrepancy to the extra tokens needed to complete a task in full, which translates into using up to 3.5 times as much energy per completed task. These findings offer actionable guidance for developers and operators choosing model precision for agents, showing how tokens per completed task, set against the per-token saving, reveal whether lower precision completes the work with less energy.

cs.PF↗

Energy per Successful Goal: Goal-Level Energy Accounting for Agentic AI Systems

Current AI energy benchmarks measure consumption per model invocation. For agentic systems, where a single user goal may trigger multistep orchestration, tool calls, retries, and failure recovery, the invocation count is an implementation artifact rather than a task property, and per inference normalization misrepresents the energy cost of completing a goal. We present A-LEMS (Agentic LLM Energy Measurement System), a cross layer framework that redefines the unit of AI energy accounting as Energy per Successful Goal (EpG): total workflow energy across all attempts, including failures and retries, divided by successfully completed goals. A-LEMS combines a temporal boundary model, a five layer attribution pipeline mapping hardware signals (RAPL on x86; SPBM and DCGM on ARM) to workflow energy, and a reproducibility protocol binding every measurement to hardware, environment, and runtime configuration. Building on EpG, we define the Orchestration Overhead Index (OOI), the ratio of agentic to linear EpG on matched goals under identical success criteria. Across nine task families on Intel x86 and NVIDIA Grace GB10 ARM, CPU only OOI ranges 0.35x to 11.07x. Including GPU dynamic energy, which accounts for 83.8% of agentic EpG in local inference, widens this to 0.4x to 26.48x, so CPU only measurement understates overhead about twofold for reasoning tasks. For remote inference, network wait energy shifts OOI from 2.5x to 2.35x. The overhead arises from orchestration structure (planning, retries, recovery) that induces additional inference calls, not from costlier individual calls. The metric is not biased upward: single tool calculator dispatch inverts to OOI = 0.61x (95% CI [0.35, 0.97]) on x86. Goal level accounting via EpG and OOI provides a measurement foundation for benchmarking agentic AI.

cs.AI↗

Formal-Verification-Guided Bayesian Optimization for Non-Intrusive Last-Mile Coverage Closure

Last-mile functional coverage closure for modern chip designs remains costly because a mature UVM environment exposes many controls while direct hits on the remaining bins are sparse. We present a non-intrusive workflow that closes pre-defined bins in an existing, human-authored functional coverage model; it does not synthesize verification objectives from the design specification. The workflow first uses abstract-syntax-tree (AST) parsing and cone-of-influence (COI) tracing to map each residual target to existing testbench knobs. Under a matched 20-simulation budget, Bayesian optimization (BO) is compared with continued constrained-random verification (CRV) and random search over the same AST/COI-reduced space. Formal reachability then removes only targets supported by complete unreachability proofs, after which retained targets are grouped by formal difficulty and processed by group-specific BO under a second matched 20-simulation budget. Cleanup and grouping never add numerator bins; only simulator coverage artifacts do. On OpenTitan AES, first-pass reduced-space BO reaches 241/271 bins; formal cleanup removes two denominator bins; the integrated post-cleanup stage - difficulty grouping, policy routing, and group-specific BO - adds 20 simulator-confirmed hits, yielding 261/271 raw coverage (96.31%) and 261/269 effective coverage (97.03%). On Entropy Source, first-pass reduced-space BO reaches 2117/2277 bins; formal cleanup removes ten denominator bins; the integrated post-cleanup stage - difficulty grouping, policy routing, and group-specific BO - adds 93 simulator-confirmed hits, yielding 2210/2277 raw coverage (97.06%) and 2210/2267 effective coverage (97.49%). These results characterize the integrated diagnosis-and-search workflow; the present evaluation does not isolate the causal contributions of difficulty classification, policy routing, and post-cleanup BO.

cs.AR↗

BugGen: A Self-Correcting Multi-Agent LLM Pipeline for Realistic RTL Bug Synthesis

Hardware complexity continues to strain verification resources, motivating the adoption of machine learning (ML) methods to improve debug efficiency. However, ML-assisted debugging critically depends on diverse and scalable bug datasets, which existing manual or automated bug insertion methods fail to reliably produce. We introduce BugGen, a first of its kind, fully autonomous, multi-agent pipeline leveraging Large Language Models (LLMs) to systematically generate, insert, and validate realistic functional bugs in RTL. BugGen partitions modules, selects mutation targets via a closed-loop agentic architecture, and employs iterative refinement and rollback mechanisms to ensure syntactic correctness and functional detectability. Evaluated across five OpenTitan IP blocks, BugGen produced 500 unique bugs with 94% functional accuracy and achieved a throughput of 17.7 validated bugs per hour-over five times faster than typical manual expert insertion. Additionally, BugGen identified 104 previously undetected bugs in OpenTitan regressions, highlighting its utility in exposing verification coverage gaps. Compared against Certitude, BugGen demonstrated over twice the syntactic accuracy, deeper exposure of testbench blind spots, and more functionally meaningful and complex bug scenarios. Furthermore, when these BugGen-generated datasets were employed to train ML-based failure triage models, we achieved high classification accuracy (88.1%-93.2%) across different IP blocks, confirming the practical utility and realism of generated bugs. BugGen thus provides a scalable solution for generating high-quality bug datasets, significantly enhancing verification efficiency and ML-assisted debugging.

cs.SE↗

Spec2Assertion: Automatic Pre-RTL Assertion Generation using Large Language Models with Progressive Regularization

SystemVerilog Assertions (SVAs) play a critical role in detecting and debugging functional bugs in digital chip design. However, generating SVAs has traditionally been a manual, labor-intensive, and error-prone process. Recent advances in automatic assertion generation, particularly those using machine learning and large language models (LLMs), have shown promising potential, though most approaches remain in the early stages of development. In this work, we introduce Spec2Assertion, a new technique for automatically generating assertions from design specifications prior to RTL implementation. It leverages LLMs with progressive regularization and incorporates Chain-of-Thought (CoT) prompting to guide assertion synthesis. Additionally, we propose a new evaluation methodology that assesses assertion quality across a broad range of scenarios. Experiments on multiple benchmark designs show that Spec2Assertion generates 70% more syntax-correct assertions with 2X quality improvement on average compared to a recent state-of-the-art approach.

cs.AR↗