Fenglan Wang | Searcharxiv

arXiv2025

Incremental Policy Iteration for Unknown Nonlinear Systems with Stability and Performance Guarantees

This paper proposes a general incremental policy iteration adaptive dynamic programming (ADP) algorithm for model-free robust optimal control of unknown nonlinear systems. The approach integrates recursive least squares estimation with linear ADP principles, which greatly simplifies the implementation while preserving adaptive learning capabilities. In particular, we develop a sufficient condition for selecting a discount factor such that it allows learning the optimal policy starting with an initial policy that is not necessarily stabilizing. Moreover, we characterize the robust stability of the closed-loop system and the near-optimality of iterative policies. Finally, we perform numerical simulations to demonstrate the effectiveness of the proposed method.

math.OC↗

arXiv2025

Safe Multi-Agent Navigation via Constrained HJB-Informed Learning

Multi-agent navigation in unknown and cluttered environments has broad applications, yet remains fundamentally challenging. In particular, dense agent-agent and agent-obstacle reactive interactions can exacerbate the inherent competition between collision-avoidance constraints and goal-reaching objectives. Most existing approaches mitigate this by applying per-step safety filtering on top of a predefined goal-reaching controller or by designing heuristic loss functions that penalizes safety constraints violation gradient. While effective in sparse environments, these methods still suffer from overly-conservative behaviors when interactions become dense. To overcome these limitations, we propose HJB-GNN, a Hamilton-Jacobi-Bellman (HJB)-based learning framework that jointly learns a graph neural network (GNN)-parameterized control barrier function for explicit safety enforcement, a distributed GNN-based navigation policy, and a value function that induces goal-reaching behavior. By exploiting the analytical solution of the constrained HJB equation, the proposed method derives graph-dependent Lagrange multipliers that adaptively balance collision-avoidance and goal-reaching across diverse multi-agent navigation scenarios. Moreover, HJB-GNN supports centralized training with distributed deployment. Extensive simulations and real-world experiments with Crazyflie drone swarms demonstrate its superior safety and goal-reaching performance, as well as strong scalability and generalizability to large-scale teams operating in previously unseen, dense environments.

eess.SY↗