arXiv subjects

Jiulin Wang

Publications and source records attributed to Jiulin Wang.

5 recordsLinked to original sources

arXiv2024

Penalty-based Methods for Simple Bilevel Optimization under Hölderian Error Bounds

This paper investigates simple bilevel optimization problems where we minimize an upper-level objective over the optimal solution set of a convex lower-level objective. Existing methods for such problems either only guarantee asymptotic convergence, have slow sublinear rates, or require strong assumptions. To address these challenges, we propose a penalization framework that delineates the relationship between approximate solutions of the original problem and its reformulated counterparts. This framework accommodates varying assumptions regarding smoothness and convexity, enabling the application of specific methods with different complexity results. Specifically, when both upper- and lower-level objectives are composite convex functions, under an $α$-H{ö}lderian error bound condition and certain mild assumptions, our algorithm attains an $(ε,ε^β)$-optimal solution of the original problem for any $β> 0$ within $\mathcal{O}\left(\sqrt{{1}/{ε^{\max\{α,β\}}}}\right)$ iterations. The result can be improved further if the smooth part of the upper-level objective is strongly convex. We also establish complexity results when the upper- and lower-level objectives are general nonsmooth functions. Numerical experiments demonstrate the effectiveness of our algorithms.

math.OC↗

arXiv2024

Value-Function Root-Finding Algorithms for Composite Convex Simple Bilevel Optimization

This paper considers composite convex simple bilevel optimization (SBO), which minimizes a composite convex function over the solution set of another composite convex minimization problem. We use a scalar value-function reformulation under which the bilevel optimal value is characterized as the leftmost root of a scalar equation. To evaluate this value function without level-set proximal access, we construct a first-order oracle based on Lagrangian duality. The resulting feasibility and value-error bounds, together with multiplier information, enable a bisection method and a safeguarded Newton-type method to obtain an $(\epsilon,\epsilon)$-solution. Under the stated assumptions and for fixed problem data and initialization tolerance, both methods achieve an operation complexity of $\widetilde{\mathcal{O}}(\epsilon^{-1/2})$, matching, up to logarithmic factors, the known near-optimal dependence for smooth convex SBO and the accelerated first-order rate for unconstrained composite convex optimization.

math.OC↗

arXiv2024

Near-Optimal Convex Simple Bilevel Optimization with a Bisection Method

This paper studies a class of simple bilevel optimization problems where we minimize a composite convex function at the upper-level subject to a composite convex lower-level problem. Existing methods either provide asymptotic guarantees for the upper-level objective or attain slow sublinear convergence rates. We propose a bisection algorithm to find a solution that is $ε_f$-optimal for the upper-level objective and $ε_g$-optimal for the lower-level objective. In each iteration, the binary search narrows the interval by assessing inequality system feasibility. Under mild conditions, the total operation complexity of our method is ${\tilde {\mathcal{O}}}\left(\max\{\sqrt{L_{f_1}/ε_f},\sqrt{L_{g_1}/ε_g} \} \right)$. Here, a unit operation can be a function evaluation, gradient evaluation, or the invocation of the proximal mapping, $L_{f_1}$ and $L_{g_1}$ are the Lipschitz constants of the upper- and lower-level objectives' smooth components, and ${\tilde {\mathcal{O}}}$ hides logarithmic terms. Our approach achieves a near-optimal rate, matching the optimal rate in unconstrained smooth or composite convex optimization when disregarding logarithmic terms. Numerical experiments demonstrate the effectiveness of our method.

math.OC↗

arXiv2021

On local minimizers of generalized trust-region subproblem

Generalized trust-region subproblem (GT) is a nonconvex quadratic optimization with a single quadratic constraint. It reduces to the classical trust-region subproblem (T) if the constraint set is a Euclidean ball. (GT) is polynomially solvable based on its inherent hidden convexity. In this paper, we study local minimizers of (GT). Unlike (T) with at most one local nonglobal minimizer, we can prove that two-dimensional (GT) has at most two local nonglobal minimizers, which are shown by example to be attainable. The main contribution of this paper is to prove that, at any local nonglobal minimizer of (GT), not only the strict complementarity condition holds, but also the standard second-order sufficient optimality condition remains necessary. As a corollary, finding all local nonglobal minimizers of (GT) or proving the nonexistence can be done in polynomial time. Finally, for (GT) in complex domain, we prove that there is no local nonglobal minimizer, which demonstrates that real-valued optimization problem may be more difficult to solve than its complex version.

math.OC↗

arXiv2021

Trust-region and $p$-regularized subproblems: local nonglobal minimum is the second smallest objective function value among all first-order stationary points

The local nonglobal minimizer of trust-region subproblem, if it exists, is shown to have the second smallest objective function value among all KKT points. This new property is extended to $p$-regularized subproblem. As a corollary, we show for the first time that finding the local nonglobal minimizer of Nesterov-Polyak subproblem corresponds to a generalized eigenvalue problem.

math.OC↗