arXiv subjects

Yule Zhang

Publications and source records attributed to Yule Zhang.

7 recordsLinked to original sources

arXiv2026

A Quadratic-Approximation-Based Stochastic Approximation Method for Weakly Convex Stochastic Programming

We propose a novel stochastic approximation algorithm, termed PMQSopt, for solving weakly convex stochastic optimization problems involving expectation-valued functions. The algorithm is constructed by integrating the proximal method of multipliers with quadratic approximations of the original stochastic problem. We analyze the sample complexity of PMQSopt in terms of the total number of stochastic gradient evaluations required. The convergence of the algorithm is characterized by three metrics associated with the $\epsilon$-KKT conditions: the average squared norm of the gradient of the Moreau envelope of the Lagrangian, the average constraint violation, and the average complementarity violation. For each of these metrics, we establish an expected convergence rate of $\mathcal{O}(T^{-1/4})$ after $T$ iterations. Furthermore, we show that with probability at least $1-1/T^{2/3}$, the gradient of the Lagrangian satisfies an $\mathcal{O}(T^{-1/8})$ bound; with probability at least $1-2/T^{2/3}$, the constraint violation achieves an $\mathcal{O}(T^{-1/4})$ bound; and with probability at least $1-3/T^{2/3}$, the complementarity violation attains an $\mathcal{O}(T^{-1/4})$ bound. All results are established under two mild conditions: (i) weak convexity of all problem functions, and (ii) the existence of a strictly feasible point. The proposed PMQSopt algorithm is a sequentially strongly convex programming method that is readily implementable. Numerical experiments illustrate its practical performance.

math.OC↗

arXiv2026

A Proximal Augmented Lagrangian Method Based on Quadratic Approximations for Weakly Convex Optimization

This paper proposes QPALM, a proximal augmented Lagrangian method based on quadratic approximations, for solving nonlinear programming problems with weakly convex objective and constraint functions. The algorithm is constructed by incorporating quadratic approximations of both the objective and constraint functions into a proximal Lagrangian framework. We establish its non-asymptotic convergence rate in terms of the total number of subproblems solved. The convergence of QPALM is characterized by three metrics associated with the $\varepsilon$-KKT conditions: the squared norm of the gradient of the Moreau envelope of the Lagrangian, the average constraint violation, and the average complementarity violation. All three metrics are shown to converge at a rate of $O(T^{-1/3})$ after $T$ iterations. Preliminary numerical results demonstrate the practical efficiency of the proposed method. These results are established under two mild conditions: (i) weak convexity of all problem functions, and (ii) the existence of a strictly feasible point. The proposed QPALM is a sequentially strongly convex programming method that is readily implementable.

math.OC↗

arXiv2023

Variational Analysis of the Orthogonally Invariant Norm Cone of Symmetric Matrices

A large number matrix optimization problems are described by orthogonally invariant norms. This paper is devoted to the study of variational analysis of the orthogonally invariant norm cone of symmetric matrices. For a general orthogonally invariant norm cone of symmetric matrices, formulas for the tangent cone, normal cone and second-order tangent set are established. The differentiability properties of the projection operator onto the orthogonally invariant norm cone are developed, including formulas for the directional derivative and the B-subdifferential. Importantly, the directional derivative is characterized by the second-order derivative of the corresponding symmetric function, which is convenient for computation. All these results are specified to the Schatten $p$-norm cone, especially to the second-order cone of symmetric matrices.

math.OC↗

arXiv2022

Solving Stochastic Optimization with Expectation Constraints Efficiently by a Stochastic Augmented Lagrangian-Type Algorithm

This paper considers the problem of minimizing a convex expectation function with a set of inequality convex expectation constraints. We present a computable stochastic approximation type algorithm, namely the stochastic linearized proximal method of multipliers, to solve this convex stochastic optimization problem. This algorithm can be roughly viewed as a hybrid of stochastic approximation and the traditional proximal method of multipliers. Under mild conditions, we show that this algorithm exhibits $O(K^{-1/2})$ expected convergence rates for both objective reduction and constraint violation if parameters in the algorithm are properly chosen, where $K$ denotes the number of iterations. Moreover, we show that, with high probability, the algorithm has $O(\log(K)K^{-1/2})$ constraint violation bound and $O(\log^{3/2}(K)K^{-1/2})$ objective bound. Some preliminary numerical results demonstrate the performance of the proposed algorithm.

math.OC↗

arXiv2020

Stochastic Approximation Proximal Method of Multipliers for Convex Stochastic Programming

This paper considers the problem of minimizing a convex expectation function over a closed convex set, coupled with a set of inequality convex expectation constraints. We present a new stochastic approximation type algorithm, namely the stochastic approximation proximal method of multipliers (PMMSopt) to solve this convex stochastic optimization problem. We analyze regrets of a stochastic approximation proximal method of multipliers for solving convex stochastic optimization problems. Under mild conditions, we show that this algorithm exhibits ${\rm O}(T^{-1/2})$ rate of convergence, in terms of both optimality gap and constraint violation if parameters in the algorithm are properly chosen, when the objective and constraint functions are generally convex, where $T$ denotes the number of iterations. Moreover, we show that, with at least $1-e^{-T^{1/4}}$ probability, the algorithm has no more than ${\rm O}(T^{-1/4})$ objective regret and no more than ${\rm O}(T^{-1/8})$ constraint violation regret. To the best of our knowledge, this is the first time that such a proximal method for solving expectation constrained stochastic optimization is presented in the literature.

math.OC↗

arXiv2019

Online Alternating Direction Method of Multipliers for Online Composite Optimization

In this paper, we investigate regrets of an online semi-proximal alternating direction method of multiplier (Online-spADMM) for solving online linearly constrained convex composite optimization problems. Under mild conditions, we establish ${\rm O}(\sqrt{N})$ objective regret and ${\rm O}(\sqrt{N})$ constraint violation regret at round $N$ when the dual step-length is taken in $(0,(1 +\sqrt{5})/2)$ and penalty parameter $\sigma$ is taken as $\sqrt{N}$. We explain that the optimal value of parameter $\sigma$ is of order ${\rm O}(\sqrt{N})$. Like the semi-proximal alternating direction method of multiplier (spADMM), Online-spADMM has the advantage to resolve the potentially non-solvability issue of the subproblems efficiently. We show the usefulness of the obtained results when applied to different types of online optimization problems and verify the theoretical result by numerical experiments}. The inequalities established for Online-spADMM are also used to develop iteration complexity of the average update of spADMM for solving linearly constrained convex composite optimization problems.

math.OC↗

arXiv2017

The Rate of Convergence of the Augmented Lagrangian Method for a Nonlinear Semidefinite Nuclear Norm Composite Optimization Problem

We propose two basic assumptions, under which the rate of convergence of the augmented Lagrange method for a class of composite optimization problems is estimated. We analyze the rate of local convergence of the augmented Lagrangian method for a nonlinear semidefinite nuclear norm composite optimization problem by verifying these two basic assumptions. Without requiring strict complementarity, we prove that, under the constraint nondegeneracy condition and the strong second order sufficient condition, the rate of convergence is linear and the ratio constant is proportional to 1/c, where c is the penalty parameter that exceeds a threshold \bar c>0. The analysis is based on variational analysis about the proximal mapping of the nuclear norm and the projection operator onto the cone of positively semidefinite symmetric matrices.

math.OC↗