Online Approximate Dynamic Programming within Linear Model Validity Regions Exploiting Impulsive Braking Inputs
Persistent excitation required for online approximate dynamic programming (ADP) can drive a nonlinear system beyond the region where its local linear model remains valid. For a class of second-order nonlinear systems , this paper develops an impulse-supervised exploration and learning framework that retains the state within a prescribed linear model validity region (LMVR) while permitting any desired persistent excitation for critic learning. The online learning dynamics are formulated as a hybrid system, and positive invariance of the prescribed exploration set within LMVR is established by resetting the state into its interior through impulsive braking, followed by recovery before learning resumes. Simulations on a nonlinear system demonstrate exploration within the LMVR despite high-amplitude probing excitation and convergence of the learned weights to that given by algebraic Riccati equation.