Searcharxiv⌕ Search

arXiv · 2611.16867

ControlHarbor: A Benchmark for Control Design by Coding Agents

Abstract

LLM agents, i.e., language models run by a harness, can now write, run and debug software autonomously. This raises the question of whether they can also carry out control system design. In this paper, we consider control system design as a task for LLM agents and grade the designed controller as control engineers do: the controller is given in a single Python file mapping measurements to actions, evaluated on held-out seeds against an ordered set of requirements, under specified physical perturbations and structural certificates, without using neural networks. We formulate the task as a chance-constrained feasibility problem and instantiate it in ControlHarbor, a benchmark of 44 tasks over 13 plants, which includes network consensus, vehicle platooning, the MuJoCo hopper, walker and humanoid. The tasks are packaged in the Harbor format, so that any agent harness can be run against them, and the same environments also serve learning algorithms other than LLMs, such as reinforcement learning (RL), under the same evaluation. With ControlHarbor, we observe that the controllers designed by three frontier coding agents, Claude Opus 5.5, Fable 5.1 and Sonnet 5.5, reach the RL return on every parity task, but only in a minority of sessions. We also observe that the controllers certified for return alone usually fail under perturbations. When robustness is part of the specification, a structured design meets it at the RL return on the humanoid task, where domain-randomised RL does not; under large pushes, domain-randomised RL succeeds and no designed controller does. The benchmark, the controllers, and their grades are released.

Explore related subjects

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Jieqiang Wei, Fahad Ali, Jonny Reichwald, Silun Zhang. 2026-11-12. ControlHarbor: A Benchmark for Control Design by Coding Agents. https://searcharxiv.com/abs/2611.16867

Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite Searcharxiv by Indiciae: BibTeX · RIS.

KEEP EXPLORING

Related papers

Scattering in the planar restricted $(N+1)$-body problem

In the restricted $(N+1)$-body problem, as time goes to positive or negative infinity the massless body has four possible final motions: hyperbolic, parabolic, oscillatory and bounded. In the hyperbolic case, the massless body comes from or goes to infinity with a non-zero limiting energy and a limiting direction, and in the parabolic case, it comes from or goes to infinity with a zero limiting energy and possibly a limiting direction. In this paper we study scattering orbits that come from infinity and then go to infinity. It will be proven that for the planar restricted $(N+1)$-body problem with the motions of the primaries being collision-free and periodic, for any two given asymptotic directions, there exists at least one hyperbolic scattering orbit for any positive limiting energy and one parabolic scattering orbit. The orbits are either always collision-free or regularizable with one collision.

math.DS↗

Exponential mixing for equilibrium states of time changes of hyperbolic geodesic flows

Let $M$ be a closed hyperbolic manifold of dimension $n\ge2$, let $ψ$ be a smooth positive function on its unit tangent bundle $SM$, and let $f^t_ψ$ be the time change of the geodesic flow with speed $1/ψ$. We prove that $f^t_ψ$ is exponentially mixing, for Hölder observables, with respect to the equilibrium state of every Hölder potential, as asked by Bowen and Ruelle. This includes the smooth invariant measure and the measure of maximal entropy. The proof runs Dolgopyat's cancellation argument at a variable \emph{local frequency} $Λ_b=|b|(1+\|\bY_b\|)$ (Proposition~\ref{prop:localfreq}): the scale $1/|b|$ is replaced by a scale that depends on the point, as in the work of Tsujii and Zhang in dimension three, and the argument runs without a smooth contact structure and without $C^1$ strong foliations. Many of the techniques, in particular the reduction to an expanding semiflow by quotienting along the stable foliation, are inspired by the earlier work of Butterley and War \cite{BW20}.

math.DS↗

Structural functional identifiability of unknown differential equation observation maps

In many differential equation models of real-world systems, the mechanistic structure of the dynamics is known, but the map from the dynamics to the observations is not. This is potentially problematic, as it can prevent us from e.g. learning model parameter values from data. Examples where this occurs include measuring a fluorescent marker as a proxy for the dynamics of a cellular circuit, and using wastewater measurements to follow the development of an epidemic. The nascent field of hybrid modelling, in which machine learning components (like neural networks) are embedded within mechanistic models, provides the tools to learn such maps. Here, the system's dynamics keep their mechanistic form, while a neural network represents the unknown observation map and is trained alongside any unknown core system parameters. This raises a question of identifiability: when can the true observation map be learnt, and when does one simply learn some map that transforms the dynamics into the available data? Building on previous work, we show how structural functional identifiability analysis can establish whether a model's unknown observation maps and core system parameters are identifiable, i.e. can be uniquely learnt from data. We both prove general results on when models with unknown observation maps are, and are not, identifiable, and demonstrate a general methodology (based on differential algebra) for assessing the identifiability of such models. This paves the way for applying classic differential equation workflows to the many real-world systems that contain such unknown maps.

math.DS↗