arXiv subjects

Qichao Cao

Publications and source records attributed to Qichao Cao.

3 recordsLinked to original sources

arXiv2026

Single-Loop Gradient Algorithms for Pessimistic Bilevel Optimization Problems

Bilevel optimization has recently attracted growing attention, particularly in the development of efficient numerical methods. Despite substantial progress on optimistic bilevel optimization, pessimistic bilevel optimization (PBO) remains much less explored, especially the design of fully first-order, single-loop gradient-based methods. To address this gap, we propose a smooth approximation of PBO through reformulation, penalization and regularization, and establish convergence guarantees in terms of both minimizers and stationarity. Building on this framework, we then develop two single-loop algorithms for deterministic and stochastic PBOs, respectively. Both use only first-order gradient information and avoid second-order derivatives and inner-loop subproblem solves. Non-asymptotic convergence rates for the proposed algorithms are established to provide theoretical guarantees. Through a systematic empirical study of both synthetic and practical problem instances, we demonstrate that our algorithms are highly effective and efficient. In particular, our results on spam classification and Smart Predict-then-Optimize further illustrate PBO's advantages over its classical optimistic bilevel counterpart, highlighting its strong potential for practical modeling and the delivery of robust solutions.

math.OC↗

arXiv2026

A Single-Loop Penalty-based Algorithm for Stochastic Minimax Optimization with Nonlinear Coupled Constraints

We study stochastic nonconvex-concave minimax optimization with nonlinear coupled constraints that are convex in the maximization variable. To address the nonsmoothness arising from such constraints, we develop a penalty-based smooth approximation that combines quadratic penalization of the coupled constraints with quadratic regularization of the inner maximization problem. Based on this approximation, we propose SPACO, a single-loop stochastic gradient algorithm that tracks the inner maximizer by one stochastic ascent step, updates the outer variable using an inexact stochastic descent direction, and adaptively updates the penalty and regularization parameters over the iterations. For the penalty-based smooth approximation, we establish convergence guarantees from both minimizer and stationarity perspectives. In particular, we introduce enhanced KKT conditions and show that stationary points of the smooth approximations can converge to points satisfying these conditions. An example illustrates that the enhanced KKT conditions can help exclude KKT points that are not local minimizers. For SPACO, we prove non-asymptotic complexity bounds for stationarity and feasibility, as well as asymptotic subsequential convergence to enhanced KKT points. Numerical experiments on synthetic examples, fairness-aware classification, and constrained generative adversarial network training demonstrate the effectiveness of the proposed method.

math.OC↗

arXiv2026

A Single-Loop Gradient Algorithm for Pessimistic Bilevel Optimization via Smooth Approximation

Bilevel optimization has garnered significant attention in the machine learning community recently, particularly regarding the development of efficient numerical methods. While substantial progress has been made in developing efficient algorithms for optimistic bilevel optimization, the study of methods for solving Pessimistic Bilevel Optimization (PBO) remains relatively less explored, especially the design of fully first-order, single-loop gradient-based algorithms. This paper aims to bridge this research gap. We first propose a novel smooth approximation to the PBO problem, using penalization and regularization techniques. Building upon this approximation, we then propose SiPBA (Single-loop Pessimistic Bilevel Algorithm), a new gradient-based method specifically designed for PBO which avoids second-order derivative information or inner-loop iterations for subproblem solving. We provide theoretical validation for the proposed smooth approximation scheme and establish theoretical convergence for the algorithm SiPBA. Numerical experiments on synthetic examples and practical applications demonstrate the effectiveness and efficiency of SiPBA.

math.OC↗