Optimal Nonergodic Primal-Dual Complexity of Efficient Inexact Parameter-Free Augmented Lagrangian Methods

Abstract

Augmented Lagrangian (AL) methods are a classical framework for constrained optimization, but for directly verifiable approximate KKT points, known first-order complexity bounds for standard inexact AL methods are suboptimal, while the best known proximal augmented Lagrangian (PAL) bounds retain an additional logarithmic factor. We consider linearly constrained convex composite problems with a smooth convex term and a possibly nonsmooth closed proper convex term with compact domain. We develop three inexact AL schemes that preserve the standard AL subproblem structure and attain the optimal primal-dual complexity $\mathcal O(\epsilon^{-1})$ in the convex setting, improving prior AL bounds of $\mathcal O(\epsilon^{-4/3})$, $\mathcal O(\epsilon^{-7/4})$, and $\mathcal O(\epsilon^{-2})$, and removing the logarithmic factor from PAL guarantees. Two variants are parameter-free, and all three admit nonergodic guarantees, including a stronger last-iterate guarantee for one variant. These results show that proximal regularization, ergodic averaging, and prior knowledge of problem-dependent constants are not intrinsic requirements for attaining optimal verifiable primal-dual complexity within the standard AL framework. A key ingredient is a parameter-free accelerated method that computes verifiable stationarity certificates for the standard, unregularized AL subproblems with optimal complexity. In the strongly convex setting, our methods attain near-optimal complexity $\mathcal O(\epsilon^{-1/2}\log(\epsilon^{-1}))$, with two parameter-free variants. Numerical experiments on six problem classes, including elastic-net least-squares regression, group-sparse Huberized support vector machines, and a quantum semidefinite program (SDP), demonstrate substantial computational advantages over a representative PAL method, with speedups frequently ranging from $5$ to $50$ times.