A Single-Loop Penalty-based Algorithm for Stochastic Minimax Optimization with Nonlinear Coupled Constraints

Abstract

We study stochastic nonconvex-concave minimax optimization with nonlinear coupled constraints that are convex in the maximization variable. To address the nonsmoothness arising from such constraints, we develop a penalty-based smooth approximation that combines quadratic penalization of the coupled constraints with quadratic regularization of the inner maximization problem. Based on this approximation, we propose SPACO, a single-loop stochastic gradient algorithm that tracks the inner maximizer by one stochastic ascent step, updates the outer variable using an inexact stochastic descent direction, and adaptively updates the penalty and regularization parameters over the iterations. For the penalty-based smooth approximation, we establish convergence guarantees from both minimizer and stationarity perspectives. In particular, we introduce enhanced KKT conditions and show that stationary points of the smooth approximations can converge to points satisfying these conditions. An example illustrates that the enhanced KKT conditions can help exclude KKT points that are not local minimizers. For SPACO, we prove non-asymptotic complexity bounds for stationarity and feasibility, as well as asymptotic subsequential convergence to enhanced KKT points. Numerical experiments on synthetic examples, fairness-aware classification, and constrained generative adversarial network training demonstrate the effectiveness of the proposed method.