Searcharxiv⌕ Search

arXiv · 2611.15133

Sim-to-Real Powertrain Vibration Control via Model-Based Assisted Domain-Randomized Reinforcement Learning

Abstract

Controllers designed from nominal models often degrade on mechanical systems with nonlinearities and parametric variations, such as automotive powertrains with backlash. Deep reinforcement learning (DRL) with domain randomization can acquire robustness to such model discrepancies, but wide randomization destabilizes training and yields conservative policies, and experimental transfer of such policies has rarely been demonstrated in vibration control. This paper presents model-based controller assisted domain randomization (MBCADR), a framework that integrates model-based control into domain-randomized DRL. An $H_{2}$ controller designed from the nominal model supplies a baseline action, while a residual policy trained with twin delayed deep deterministic policy gradient under randomized dynamics compensates for nonlinearities and uncertainties. This model-based assistance relieves the agent of synthesizing the entire control input, thereby stabilizing training under extensive randomization. For each realization of the uncertain parameters, the augmented plant--controller state is shown to be a Markov state of the closed loop, so that domain-randomized training is rigorously formulated as policy optimization over a parameterized family of Markov decision processes. A Lyapunov-based stability analysis further derives a forward-invariant set and an explicit ultimate bound, in which the learned action enters only through its saturation level. Monte Carlo simulations with multiple training seeds show that MBCADR reduces performance variability compared with either component alone. Experiments on a powertrain test bench with backlash, using the simulation-trained policy on an embedded processor without further tuning, confirm successful sim-to-real transfer.

Explore related subjects

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Heisei Yonezawa, Ansei Yonezawa, Itsuro Kajiwara. 2026-11-11. Sim-to-Real Powertrain Vibration Control via Model-Based Assisted Domain-Randomized Reinforcement Learning. https://searcharxiv.com/abs/2611.15133

Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite Searcharxiv by Indiciae: BibTeX · RIS.

KEEP EXPLORING

Related papers

Mean-Field Learning for Storage Aggregation

Distributed energy storage devices can be aggregated to provide operational flexibility for power systems. Their efficient integration into system-level optimization calls for an accurate and computationally tractable surrogate that jointly represents aggregate flexibility and operating costs. However, constructing such a surrogate is challenging due to device heterogeneity, nonconvex operating constraints, and high-dimensional power trajectories. To address these challenges, this paper develops a mean-field learning framework for storage aggregation. Using random-set expectations, we show that the joint representation of aggregate flexibility and operating costs converges almost surely to a deterministic convex mean-field limit. This limit provides a learning benchmark for storage aggregation, against which we bound the approximation error of any convex surrogate. These theoretical results motivate a convex virtual-battery surrogate with physically interpretable parameters. We learn these parameters through a bilevel optimization problem that matches the optimal values of the surrogate and the storage fleet when both respond to a set of price scenarios. We derive closed-form gradients to solve this problem efficiently. Case studies illustrate the theoretical results and demonstrate the framework's approximation accuracy, computational efficiency, and robustness to device heterogeneity.

eess.SY↗

HyperKKL: Enabling Non-Autonomous State Estimation through Dynamic Weight Conditioning

This paper proposes HyperKKL, a novel learning approach for designing Kazantzis-Kravaris/Luenberger (KKL) observers for non-autonomous nonlinear systems. While KKL observers offer a rigorous theoretical framework by immersing nonlinear dynamics into a stable linear latent space, its practical realization relies on solving Partial Differential Equations (PDE) that are analytically intractable. Current existing learning-based approximations of the KKL observer are mostly designed for autonomous systems, failing to generalize to driven dynamics without expensive retraining or online gradient updates. HyperKKL addresses this by employing a hypernetwork architecture that encodes the exogenous input signal to instantaneously generate the parameters of the KKL observer, effectively learning a family of immersion maps parameterized by the external drive. We rigorously evaluate this approach against a curriculum learning strategy that attempts to generalize from autonomous regimes via training heuristics alone. The novel approach is illustrated on four numerical simulations in benchmark examples including the Duffing, Van der Pol, Lorenz, and Rössler systems.

eess.SY↗

Dynamic Flexibility Requests in Local Flexibility Markets: Quantifying the DSO Willingness to Pay

Local Flexibility Markets (LFMs) require Distribution System Operators (DSOs) to determine both the quantity of flexibility to procure and the corresponding willingness to pay during market clearing. Existing approaches typically rely on centralized AC-OPF clearing algorithms with limited practical scalability or strictly localized, static flexibility requests driven primarily by congestion management, while the economic value of flexibility is largely neglected. This paper proposes a dynamic flexibility-request methodology in which the DSO's willingness to pay is embedded directly into the market-clearing objective by monetizing transformer and cable aging, network losses, and voltage congestion. To enable computationally efficient clearing, exact convex piecewise-linear epigraph reformulations of the IEEE C57.91 transformer aging model and an Arrhenius-based cable aging model are developed and formally proven to preserve exactness. The proposed framework is validated on a modified CIGRE MV benchmark and compared with a recent state-of-the-art flexibility-request methodology. The results demonstrate significantly higher market liquidity, more efficient flexibility procurement, and improved network operations while preserving the market's transparency and non-discrimination principles.

eess.SY↗