Searcharxiv⌕ Search

arXiv · 2611.10712

Deep Reinforcement Learning with Massively Parallel Simulation for Low-Thrust Multi-Revolution Rendezvous

Abstract

The growing demand for on-orbit servicing and constellation management in low Earth orbit requires autonomous and reliable low-thrust rendezvous. Traditional numerical methods struggle with the long-horizon nature of multi-revolution maneuvers under perturbations, while standard deep reinforcement learning (DRL) methods are limited by low simulation throughput and poor sample efficiency. To address these challenges, this paper proposes a massively parallel DRL guidance framework. A fully tensorized, GPU-resident orbital propagator is developed to integrate $J_2$-perturbed orbital dynamics and policy optimization within a zero-copy pipeline, achieving up to a 67-fold increase in simulation throughput over conventional CPU baselines. In addition, the rendezvous problem is formulated in modified equinoctial elements and trained with a progressive curriculum learning strategy to improve convergence and generalization across increasingly challenging rendezvous scenarios. Simulations show that the proposed framework can learn robust rendezvous trajectories for complex multi-revolution rendezvous over a broad range of orbital configurations. Additional evaluations under a high-fidelity model and eclipse-induced thrust interruptions further demonstrate the robustness of the learned policy under practical mission constraints. Moreover, the trajectories generated by the proposed framework can be used as effective initial guesses for optimization-based trajectory solvers, improving the convergence of high-precision trajectory optimization. The method offers a scalable approach for real-time low-thrust guidance and trajectory optimization.

Explore related subjects

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Chenglong Wu, Yi Zhou, Bin Cheng, Nan Zhang, Hexi Baoyin. 2026-11-07. Deep Reinforcement Learning with Massively Parallel Simulation for Low-Thrust Multi-Revolution Rendezvous. https://searcharxiv.com/abs/2611.10712

Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite Searcharxiv by Indiciae: BibTeX · RIS.

KEEP EXPLORING

Related papers

Mean-Field Learning for Storage Aggregation

Distributed energy storage devices can be aggregated to provide operational flexibility for power systems. Their efficient integration into system-level optimization calls for an accurate and computationally tractable surrogate that jointly represents aggregate flexibility and operating costs. However, constructing such a surrogate is challenging due to device heterogeneity, nonconvex operating constraints, and high-dimensional power trajectories. To address these challenges, this paper develops a mean-field learning framework for storage aggregation. Using random-set expectations, we show that the joint representation of aggregate flexibility and operating costs converges almost surely to a deterministic convex mean-field limit. This limit provides a learning benchmark for storage aggregation, against which we bound the approximation error of any convex surrogate. These theoretical results motivate a convex virtual-battery surrogate with physically interpretable parameters. We learn these parameters through a bilevel optimization problem that matches the optimal values of the surrogate and the storage fleet when both respond to a set of price scenarios. We derive closed-form gradients to solve this problem efficiently. Case studies illustrate the theoretical results and demonstrate the framework's approximation accuracy, computational efficiency, and robustness to device heterogeneity.

eess.SY↗

HyperKKL: Enabling Non-Autonomous State Estimation through Dynamic Weight Conditioning

This paper proposes HyperKKL, a novel learning approach for designing Kazantzis-Kravaris/Luenberger (KKL) observers for non-autonomous nonlinear systems. While KKL observers offer a rigorous theoretical framework by immersing nonlinear dynamics into a stable linear latent space, its practical realization relies on solving Partial Differential Equations (PDE) that are analytically intractable. Current existing learning-based approximations of the KKL observer are mostly designed for autonomous systems, failing to generalize to driven dynamics without expensive retraining or online gradient updates. HyperKKL addresses this by employing a hypernetwork architecture that encodes the exogenous input signal to instantaneously generate the parameters of the KKL observer, effectively learning a family of immersion maps parameterized by the external drive. We rigorously evaluate this approach against a curriculum learning strategy that attempts to generalize from autonomous regimes via training heuristics alone. The novel approach is illustrated on four numerical simulations in benchmark examples including the Duffing, Van der Pol, Lorenz, and Rössler systems.

eess.SY↗

Dynamic Flexibility Requests in Local Flexibility Markets: Quantifying the DSO Willingness to Pay

Local Flexibility Markets (LFMs) require Distribution System Operators (DSOs) to determine both the quantity of flexibility to procure and the corresponding willingness to pay during market clearing. Existing approaches typically rely on centralized AC-OPF clearing algorithms with limited practical scalability or strictly localized, static flexibility requests driven primarily by congestion management, while the economic value of flexibility is largely neglected. This paper proposes a dynamic flexibility-request methodology in which the DSO's willingness to pay is embedded directly into the market-clearing objective by monetizing transformer and cable aging, network losses, and voltage congestion. To enable computationally efficient clearing, exact convex piecewise-linear epigraph reformulations of the IEEE C57.91 transformer aging model and an Arrhenius-based cable aging model are developed and formally proven to preserve exactness. The proposed framework is validated on a modified CIGRE MV benchmark and compared with a recent state-of-the-art flexibility-request methodology. The results demonstrate significantly higher market liquidity, more efficient flexibility procurement, and improved network operations while preserving the market's transparency and non-discrimination principles.

eess.SY↗