Explicit Convergence Regions of PID-Damped Accelerated Gradient Methods in Nonconvex Optimization
Momentum-based accelerated gradient methods are widely adopted to expedite convergence in nonconvex optimization, but are prone to overshooting and oscillatory behavior. A class of PID-damped accelerated gradient methods mitigates this issue by augmenting classical momentum methods with a discrete-time derivative damping term. However, the coupling among the step size, momentum, and derivative gain renders the explicit characterization of their convergence regions analytically intractable, leaving explicit theoretical convergence boundaries unexplored. In this paper, we model this class of algorithms as a third-order nonlinear feedback dynamical system and establish explicit three-dimensional convergence regions for the step size, momentum, and derivative gain via a robust control-theoretic analysis based on the Kalman-Yakubovich-Popov (KYP) lemma, formally guaranteeing linear convergence under the Regularity Condition. Furthermore, we reveal a strict geometric upper bound on the derivative gain dictated by the nonconvex curvature, beyond which over-damping severely contracts the feasible step-size region, providing a rigorous theoretical explanation for the overdamped stagnation phenomenon. Numerical experiments corroborate the theoretical boundaries and illustrate practical parameter selection guidelines for the derivative gain.