Searcharxiv⌕ Search

arXiv · 2610.07167

Interleaved Projected Gradient Descent for Safe Imitation Learning

Abstract

We propose an imitation-learning design for neural-network control policies under state and input constraints. Training alternates a standard imitation gradient step with a block of $k$ safety steps that pull the network's actions toward their projection onto the safe set; at run time, the controller is the trained network alone, with no safety filter. We analyze this scheme as inexact projected gradient descent in the space of policy actions. When the projected actions are recomputed at every safety step and each step moves the actions consistently toward the safe set, letting $k$ grow logarithmically yields asymptotic constraint satisfaction on the training states and bounds the distance to the constrained optimum of the imitation loss; with the projected actions held fixed, the same holds only if they are exactly representable by the network. On a nonlinear autonomous racing task, we compare our method with adding a weighted constraint-violation penalty to the imitation loss. With a sufficiently large weight, our method matches the lap time of unconstrained imitation while reducing the fraction of violating episodes from $15\%$ to $1\%$, about six times fewer than the penalty approach at its best weight. Its lap times are less sensitive to the weight, which instead sets how quickly violations vanish during training. In racing, the safety corrections are sparse and the conditions of the analysis do not hold; the gain arises instead through the data collected during training. These gains come at the cost of additional training computation.

Explore related subjects

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Shengfan Cao, Francesco Borrelli. 2026-10-05. Interleaved Projected Gradient Descent for Safe Imitation Learning. https://searcharxiv.com/abs/2610.07167

Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite Searcharxiv by Indiciae: BibTeX · RIS.

KEEP EXPLORING

Related papers

Mean-Field Learning for Storage Aggregation

Distributed energy storage devices can be aggregated to provide operational flexibility for power systems. Their efficient integration into system-level optimization calls for an accurate and computationally tractable surrogate that jointly represents aggregate flexibility and operating costs. However, constructing such a surrogate is challenging due to device heterogeneity, nonconvex operating constraints, and high-dimensional power trajectories. To address these challenges, this paper develops a mean-field learning framework for storage aggregation. Using random-set expectations, we show that the joint representation of aggregate flexibility and operating costs converges almost surely to a deterministic convex mean-field limit. This limit provides a learning benchmark for storage aggregation, against which we bound the approximation error of any convex surrogate. These theoretical results motivate a convex virtual-battery surrogate with physically interpretable parameters. We learn these parameters through a bilevel optimization problem that matches the optimal values of the surrogate and the storage fleet when both respond to a set of price scenarios. We derive closed-form gradients to solve this problem efficiently. Case studies illustrate the theoretical results and demonstrate the framework's approximation accuracy, computational efficiency, and robustness to device heterogeneity.

eess.SY↗

HyperKKL: Enabling Non-Autonomous State Estimation through Dynamic Weight Conditioning

This paper proposes HyperKKL, a novel learning approach for designing Kazantzis-Kravaris/Luenberger (KKL) observers for non-autonomous nonlinear systems. While KKL observers offer a rigorous theoretical framework by immersing nonlinear dynamics into a stable linear latent space, its practical realization relies on solving Partial Differential Equations (PDE) that are analytically intractable. Current existing learning-based approximations of the KKL observer are mostly designed for autonomous systems, failing to generalize to driven dynamics without expensive retraining or online gradient updates. HyperKKL addresses this by employing a hypernetwork architecture that encodes the exogenous input signal to instantaneously generate the parameters of the KKL observer, effectively learning a family of immersion maps parameterized by the external drive. We rigorously evaluate this approach against a curriculum learning strategy that attempts to generalize from autonomous regimes via training heuristics alone. The novel approach is illustrated on four numerical simulations in benchmark examples including the Duffing, Van der Pol, Lorenz, and Rössler systems.

eess.SY↗

Dynamic Flexibility Requests in Local Flexibility Markets: Quantifying the DSO Willingness to Pay

Local Flexibility Markets (LFMs) require Distribution System Operators (DSOs) to determine both the quantity of flexibility to procure and the corresponding willingness to pay during market clearing. Existing approaches typically rely on centralized AC-OPF clearing algorithms with limited practical scalability or strictly localized, static flexibility requests driven primarily by congestion management, while the economic value of flexibility is largely neglected. This paper proposes a dynamic flexibility-request methodology in which the DSO's willingness to pay is embedded directly into the market-clearing objective by monetizing transformer and cable aging, network losses, and voltage congestion. To enable computationally efficient clearing, exact convex piecewise-linear epigraph reformulations of the IEEE C57.91 transformer aging model and an Arrhenius-based cable aging model are developed and formally proven to preserve exactness. The proposed framework is validated on a modified CIGRE MV benchmark and compared with a recent state-of-the-art flexibility-request methodology. The results demonstrate significantly higher market liquidity, more efficient flexibility procurement, and improved network operations while preserving the market's transparency and non-discrimination principles.

eess.SY↗