arXiv subjects

Xiaoning Bai

Publications and source records attributed to Xiaoning Bai.

4 recordsLinked to original sources

arXiv2026

A Proximal Newton Framework with Local Superlinear Convergence for Simple Bilevel Optimization

In this paper, we study a penalty method for simple bilevel optimization in which the lower-level objective has a convex composite structure. Our approach is based on a vanishing-penalty reformulation, which gives rise to a sequence of penalized problems as the penalty parameter decreases. A key difficulty is that metric subregularity of the lower-level optimality mapping does not, in general, yield a uniform error bound for the penalized problems as the penalty parameter vanishes, preventing a direct application of standard proximal Newton theory. To address this difficulty, we first analyze a regularized proximal Newton step by exploiting the specific structure of simple bilevel problems, and establish the local estimates that underpin superlinear convergence. Building on this local analysis, we propose SiB-Newton, which incorporates the proximal Newton step into a globalization strategy. We establish convergence guarantees for SiB-Newton and show that, under suitable local conditions, Newton steps are eventually accepted and the iterates converge at a superlinear rate. Numerical experiments on synthetic examples verify the predicted local superlinear convergence rate.

math.OC↗

arXiv2026

Limiting Stationarity of Regularized Gap-Function Reformulations for Bilevel Optimization with Unbounded Multipliers

Value-function-type reformulations have generated a broad class of methods for bilevel optimization. However, the corresponding value-function-type constraints are inherently degenerate and generally fail to satisfy standard constraint qualifications, so the associated multiplier sequences may be unbounded and bounded-multiplier convergence analyses become inapplicable. We study this issue for the regularized gap-function reformulation of bilevel problems with constrained convex lower-level programs. We prove that accumulation points of approximate stationary sequences are C-stationary for the corresponding Karush-Kuhn-Tucker-based mathematical program with complementarity constraints (MPCC), even when the multiplier sequence associated with the regularized gap-function constraint is unbounded. The result holds under Mangasarian-Fromovitz constraint qualification (MFCQ) for the upper- and lower-level constraint systems and MPCC-MFCQ at the limiting MPCC point, without any constraint qualification on the regularized gap-function constraint itself. We further provide an example showing that approximate stationary points of the standard regularized gap-function reformulation may converge to a point that is C-stationary but not M-stationary. To guarantee M-stationarity, we introduce a slack-based two-parameter penalty formulation preserving exact multiplier-slack complementarity and establish M-stationarity under a domination condition on the penalty parameters. We develop an inexact slack-penalty method with adaptive penalty updates and feasibility correction, whose accumulation points are M-stationary under the stated assumptions.

math.OC↗

arXiv2026

Alternating Gradient-Type Algorithm for Bilevel Optimization with Inexact Lower-Level Solutions via Moreau Envelope-based Reformulation

In this paper, we study a class of bilevel optimization problems where the lower-level problem is a convex composite optimization model, which arises in various applications, including bilevel hyperparameter selection for regularized regression models. To solve these problems, we propose an Alternating Gradient-type algorithm with Inexact Lower-level Solutions (AGILS) based on a Moreau envelope-based reformulation of the bilevel optimization problem. The proposed algorithm does not require exact solutions of the lower-level problem at each iteration, improving computational efficiency. We prove the convergence of AGILS to stationary points and, under the Kurdyka-Łojasiewicz (KL) property, establish its sequential convergence. Numerical experiments, including a toy example and a bilevel hyperparameter selection problem for the sparse group Lasso model, demonstrate the effectiveness of the proposed AGILS.

math.OC↗

arXiv2024

A Highly Efficient Adaptive-Sieving-Based Algorithm for the High-Dimensional Rank Lasso Problem

The high-dimensional rank lasso (hdr lasso) model is an efficient approach to deal with high-dimensional data analysis. It was proposed as a tuning-free robust approach for the high-dimensional regression and was demonstrated to enjoy several statistical advantages over other approaches. The hdr lasso problem is essentially an $L_1$-regularized optimization problem whose loss function is Jaeckel's dispersion function with Wilcoxon scores. Due to the nondifferentiability of the above loss function, many classical algorithms for lasso-type problems are unable to solve this model. In this paper, inspired by the adaptive sieving strategy for the exclusive lasso problem [1], we propose an adaptive-sieving-based algorithm to solve the hdr lasso problem. The proposed algorithm makes full use of the sparsity of the solution. In each iteration, a subproblem with the same form as the original model is solved, but in a much smaller size. We apply the proximal point algorithm to solve the subproblem, which fully takes advantage of the two nonsmooth terms. Extensive numerical results demonstrate that the proposed algorithm (AS-PPA) is robust for different types of noises, which verifies the attractive statistical property as shown in [2]. Moreover, AS-PPA is also highly efficient, especially for the case of high-dimensional features, compared with other methods.

math.OC↗