Adaptive Agent Design

Raj Kiriti Velicheti; Subhonmesh Bose; Tamer Başar

Home/Archive/arXiv

arXiv · 2609.12486

Adaptive Agent Design

Raj Kiriti Velicheti·Subhonmesh Bose·Tamer Başar

View original source ↗Read source document ↗Export metadata ↓

Abstract

We consider an agent acting against a general non-Markovian environment. The agent maintains its agent states, but is free to choose a transition kernel across those states and optimize its state-feedback control policies. We study the bi-level agent design problem that optimizes the transition kernel and the policy it induces, given said kernel with offline data of observations and actions obtained via a behavioral policy. For general environments, we show that a soft $Q$-learning algorithm converges almost surely to the fixed point of a soft Bellman equation defined by the stationary averages that the behavioral policy and the chosen kernel induce, and we delineate what separates the resulting policy from an optimal one. In partially observed Markov decision problems, we analyze convergence properties of parametrized transition kernel design via zero-th order and Bayesian optimization techniques.

Explore related subjects

eess.SY cs.AI cs.GT cs.SY math.OC

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Raj Kiriti Velicheti, Subhonmesh Bose, Tamer Başar. 2026-09-11. Adaptive Agent Design. https://arxiv.org/abs/2609.12486

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Mixed Bernstein-Fourier Approximants for Optimal Trajectory Generation with Periodic Behavior

Efficient trajectory generation is crucial for autonomous systems; however, current numerical methods often struggle to handle periodic behaviors effectively, particularly when the onboard sensors require equidistant temporal sampling. This paper introduces a novel mixed Bernstein-Fourier approximation framework tailored explicitly for optimal motion planning. Our proposed methodology leverages the uniform convergence properties of Bernstein polynomials for nonperiodic behaviors while effectively capturing periodic dynamics through the Fourier series. Theoretical results are established, including uniform convergence proofs for approximations of functions, derivatives, and integrals, as well as detailed error bound analyses. We further introduce a regulated least squares approach for determining approximation coefficients, enhancing numerical stability and practical applicability. Within an optimal control context, we establish the feasibility and consistency of approximated solutions to their continuous counterparts. We also extend the covector mapping theorem, providing theoretical guarantees for approximating dual variables crucial in verifying the necessary optimality conditions from Pontryagin's Maximum Principle. Numerical examples illustrate the method's superior performance, demonstrating substantial improvements in computational efficiency and precision in scenarios with complex periodic constraints and dynamics. Our mixed Bernstein-Fourier methodology thus presents a robust, theoretically grounded, and computationally efficient approach for advanced optimal trajectory planning in autonomous systems.

eess.SY↗

arXiv2026

Emulation-based Neuromorphic Control for the Stabilization of LTI Systems

Neuromorphic engineering aims at designing computing and control systems inspired by the neurons and the brain. For the control community, neuromorphic control is an emerging topic that focuses on designing event-based spiking controllers in the form of spiking neural networks (SNNs). At present, systematic methods for designing and analyzing such controllers are lacking. Therefore in this paper we present a systematic approach for stabilizing linear time-invariant (LTI) systems using SNN-based controllers, in the form of a network of integrate-and-fire neurons, whose input is the measured output from the plant, and which generate spiking control signals. The new approach consists of a two-step emulation-based design procedure. In the first step, we establish conditions on the neuron parameters to ensure that the spiky signal generated by a pair of neurons emulates any continuous-time signal input to the neurons with arbitrary accuracy in terms of a special metric for spiky signals. In the second step, we propose a novel stability notion, called spiky-Input-to-State Stability (sISS) building on this metric, and prove that an asymptotically stable LTI system has this sISS property. By combining these steps, a certifiable practical stability property of the closed-loop system can be established. The approach is illustrated in a numerical case study.

eess.SY↗

arXiv2026

Overlapping Covariance Intersection: Fusion with Partial Structural Knowledge of Correlation from Multiple Sources

Emerging large-scale engineering systems rely on distributed fusion for situational awareness, where agents combine noisy local sensor measurements with exchanged information to obtain fused estimates. However, at the sheer scale of these systems, tracking cross-correlations becomes infeasible, preventing the use of optimal filters. Covariance intersection (CI) methods address fusion problems with unknown correlations by minimizing worst-case uncertainty based on available information. Existing CI extensions exploit limited correlation knowledge but cannot incorporate structural knowledge of correlation from multiple sources, which naturally arises in distributed fusion problems. This paper introduces Overlapping Covariance Intersection (OCI), a generalized CI framework that accommodates this novel information structure. We formalize the OCI problem and establish necessary and sufficient conditions for feasibility. We show that a family-optimal solution can be computed efficiently via semidefinite programming, enabling real-time implementation. The proposed tools enable improved fusion performance for large-scale systems while retaining robustness to unknown correlations.

eess.SY↗