Searcharxiv⌕ Search
Home/Archive/arXiv

arXiv · 2610.20394

What Should a Software Team Spend on Agents? Simulation-Based Cost Optimization of Mixed AI-Human Teams

Steffen Herbold
View original source ↗Read source document ↗Export metadata ↓

Abstract

Context: AI agents are rapidly adopted in software development, but their token costs are rising and it is unclear how much a team should spend on agents, and on which tasks. Objective: We determine how teams should divide spending between human developers and AI agents to deliver the features a project demands at the lowest total cost. Method: We model mixed teams of human developers and AI agents with eight specialized roles as a discrete-time stochastic process covering bug injection and detection, technical debt, human supervision of agents, and release cycles with feature freezes. Using Monte Carlo simulation and evolutionary search, we determine cost-optimal teams for 432 scenarios varying developer capability, project velocity, debt dynamics, agent capability and price, and bug costs, and compare them with a human-only baseline. Results: Escaped bugs are the dominant cost driver, and 61% of the agent budget goes to finding bugs. Agent capability matters more than token price, since higher-quality outputs reduce downstream quality assurance costs. In teams with capable agents, human developers mostly fix bugs and leave the implementation of new features to the agents. Teams with high-capability agents are typically cheaper than human-only teams, but low-capability agents offer no consistent advantage. At realistic velocities, the optimal teams still spend more on developers than on agents. Conclusions: There is neither a universal token budget per developer nor a universal agentic team structure. As in traditional development, this depends on many factors and teams need to be carefully tailored to the project context to achieve the optimal cost-quality trade-off.

Explore related subjects

cs.SE

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Steffen Herbold. 2026-10-14. What Should a Software Team Spend on Agents? Simulation-Based Cost Optimization of Mixed AI-Human Teams. https://arxiv.org/abs/2610.20394

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Securing High-Concurrency Ticket Sales: A Framework Based on Microservice

The railway ticketing system is one of the most important public service infrastructure. In peak periods such as holidays, it is often faced with the challenge of high concurrency scenarios because of a large number of users accessing at the same time. The traditional aggregation architecture can not meet the peak user requirements because of its insufficient fault tolerance and low ability. Therefore, the system needs to use microservice architecture for development, and add multiple security methods to ensure that the system can have good stability and data consistency under high concurrency scenarios, and can respond quickly to user requests. This paper introduces the use of B/S architecture and Spring Cloud to design and develop a railway ticket purchase system that can maintain stability and reliability under high concurrency scenarios, and formulate multiple security design methods for the system. This system integrates a range of functions, such as real-time train inquiries, dynamic seat updates, online seat selection, and ticket purchasing, effectively addressing common problems associated with offline ticket purchasing, such as long queues and delayed information. It enables a complete online process from inquiry and booking to payment and refunds. Furthermore, the "add passenger" function allows users to purchase tickets for others, extending the convenience of online ticketing to people with limited internet access. The system design prioritizes security and stability, while also focusing on high performance, and achieves these goals through a carefully designed architecture and the integration of multiple middleware components. After the completion of the system development, the core interface of the system is tested, and then the results are analyzed. The test data proves that the system has good ability and stability under high concurrency.

cs.SE↗
arXiv2026

Terminal Agents Suffice for Enterprise Automation

There has been growing interest in building agents that can interact with digital platforms to execute meaningful enterprise tasks autonomously. Among the approaches explored are tool-augmented agents built on abstractions such as Model Context Protocol (MCP) and web agents that operate through graphical interfaces. Yet, it remains unclear whether such complex agentic systems are necessary given their cost and operational overhead. We argue that a coding agent equipped only with a terminal and a filesystem can solve many enterprise tasks more effectively by interacting directly with platform APIs. We evaluate this hypothesis across diverse real-world systems and show that these low-level terminal agents match or outperform more complex agent architectures at a fraction of the cost. Our findings suggest that simple, flexible programmatic interfaces combined with strong foundation models should be the backbone of enterprise automation.

cs.SE↗
arXiv2026

T2J-BENCH: Benchmarking Coding Agents on Execution-Grounded PyTorch-to-JAX Repository Migration

Migrating a machine-learning codebase across frameworks--for example from PyTorch to JAX--is a routine but costly engineering task that is fundamentally cross-paradigm: eager, stateful, object-oriented modules must be rewritten as pure, jittable functions over an explicit parameter pytree. We ask whether coding agents can perform such whole-repository migrations correctly, and argue that existing code benchmarks, which grade unit-test pass rates or surface-level similarity to a reference translation, cannot answer this: they neither require the migrated system to \emph{behave} the same nor prevent a JAX reimplementation from faking equivalence by secretly importing PyTorch itself. We introduce T2J-BENCH, a benchmark of 63 whole-repository PyTorch-to-JAX migration tasks spanning 18 model families across both training and serving, where torch-free checkpoint conversion is itself part of the deliverable. T2J-BENCH grades execution-grounded functional equivalence: each candidate is run inside a separate, torch-free, offline verifier and scored against references frozen from the PyTorch original, under a hard independence gate and a continuous, threshold-free reward. Across twenty off-the-shelf agent--model configurations, the strongest recovers only 0.676 of the reward against a 0.980 oracle, with substantial numeric and behavioral divergence despite high spec pass rates. Finally, we contribute TWINHARNESS, a reference-grounded agent harness that turns the agent's executable PyTorch source into legitimate self-verification feedback; applied to an opus-5 agent it raises the reward from 0.649 to 0.723--the highest of any configuration--using only task-visible resources.

cs.SE↗