arXiv subjects

Guoyin Li

Publications and source records attributed to Guoyin Li.

At least 19 recordsLinked to original sources

arXiv2026

Radial-type error bounds for semidefinite feasibility problems without strict feasibility: qualitative estimates and asymptotic tightness

In this paper, we develop a systematic framework for deriving explicit error bounds for semidefinite feasibility problems without assuming strict feasibility (Slater's condition), a setting in which existing results are limited. Our main technical contribution is the introduction of radial-type H\"{o}lder error bounds, where the error bound constant depends explicitly on the norm of the reference matrix through radial modulus functions. By combining facial reduction with recently developed facial residual functions, we obtain explicit descriptions of these modulus functions, yielding qualitative radial-type H\"{o}lder error bounds \emph{without imposing any constraint qualifications}. Our results complement the classical work of Sturm by providing explicit estimates for the constants involved in Sturm's local H\"{o}lder error bounds over bounded sets with a given size. We further analyze the asymptotic behavior of these bounds as the dimension of the underlying matrix space grows, identifying regimes in which they can be asymptotically tight up to a dimension-free constant. As an application, we establish explicit error bounds for the optimality system of semidefinite programs by reformulating them as feasibility problems, a setting where Slater's condition typically fails. Under the generically satisfied strict complementarity condition, we derive radial-type error bounds without assuming the usual solution uniqueness requirement, and demonstrate their asymptotic tightness through an explicit example.

math.OC↗

arXiv2026

Interwoven SDP in Primal-Dual Proximal Splitting Methods for Adjustable Robust Convex Optimisation with SOS-Convex Polynomial Constraints

We propose a novel methodology for solving a two-stage adjustable robust convex optimisation problem with a general (proximable) convex objective function and constraints defined by sum-of-squares (SOS) convex polynomials. These problems appear in many decision-making applications. However, they are challenging to solve and typically cannot be reformulated as numerically tractable convex optimisation models, such as conic linear programs, that can be solved directly using existing software. We show that the robust problem admits an equivalent representation as a convex composite unconstrained optimisation model that preserves the same objective values, under quadratic decision rules on the adjustable decision variables. Building on this reformulation, we develop a tailored first-order primal-dual proximal splitting method. By leveraging semidefinite programming (SDP) techniques as well as tools from convex analysis and real algebraic geometry, we establish its theoretical properties, including computable SDP-based formulas for projections onto closed convex sets, specified by SOS-convex polynomial inequalities. Numerical experiments on a two-stage lot-sizing model with both linear as well as SOS-convex polynomial storage costs under demand uncertainty demonstrate the effectiveness and applicability of the proposed approach. Our approach enables the incorporation of SDP techniques into a primal-dual proximal splitting framework, thereby broadening the class of problems to which these methods can be effectively applied.

math.OC↗

arXiv2024

Piecewise SOS-Convex Moment Optimization and Applications via Exact Semi-Definite Programs

This paper presents exact Semi-Definite Program (SDP) reformulations for infinite-dimensional moment optimization problems involving a new class of piecewise Sum-of-Squares (SOS)-convex functions and projected spectrahedral support sets. These reformulations show that solving a single SDP finds the optimal value and an optimal probability measure of the original moment problem. This is done by establishing an SOS representation for the non-negativity of a piecewise SOS-convex function over a projected spectrahedron. Finally, as an application and a proof-of-concept illustration, the paper presents numerical results for the Newsvendor and revenue maximization problems with higher-order moments by solving their equivalent SDP reformulations. These reformulations promise a flexible and efficient approach to solving these models. The main novelty of the present work in relation to the recent research lies in finding the solution to moment problems, for the first time, with piecewise SOS-convex functions from their numerically tractable exact SDP reformulations.

math.OC↗

arXiv2024

Generalized Metric Subregularity with Applications to High-Order Regularized Newton Methods

This paper pursues a twofold goal. First, we introduce and study in detail a new notion of variational analysis called generalized metric subregularity, which is a far-going extension of the conventional metric subregularity conditions. Our primary focus is on examining this concept concerning first-order and second-order stationary points. We develop an extended convergence framework that enables us to derive superlinear and quadratic convergence under the generalized metric subregularity condition, broadening the widely used KL convergence analysis framework. We present verifiable sufficient conditions to ensure the proposed generalized metric subregularity condition and provide examples demonstrating that the derived convergence rates are sharp. Second, we design a new high-order regularized Newton method with momentum steps, and apply the generalized metric subregularity to establish its superlinear convergence. Quadratic convergence is obtained under additional assumptions. Specifically, when applying the proposed method to solve the (nonconvex) over-parameterized compressed sensing model, we achieve global convergence with a quadratic local convergence rate towards a global minimizer under a strict complementarity condition.

math.OC↗

arXiv2024

Frank-Wolfe-type methods for a class of nonconvex inequality-constrained problems

The Frank-Wolfe (FW) method, which implements efficient linear oracles that minimize linear approximations of the objective function over a fixed compact convex set, has recently received much attention in the optimization and machine learning literature. In this paper, we propose a new FW-type method for minimizing a smooth function over a compact set defined as the level set of a single difference-of-convex function, based on new generalized linear-optimization oracles (LO). We show that these LOs can be computed efficiently with closed-form solutions in some important optimization models that arise in compressed sensing and machine learning. In addition, under a mild strict feasibility condition, we establish the subsequential convergence of our nonconvex FW-type method. Since the feasible region of our generalized LO typically changes from iteration to iteration, our convergence analysis is completely different from those existing works in the literature on FW-type methods that deal with fixed feasible regions among subproblems. Finally, motivated by the away steps for accelerating FW-type methods for convex problems, we further design an away-step oracle to supplement our nonconvex FW-type method, and establish subsequential convergence of this variant. Numerical results on the matrix completion problem with standard datasets are presented to demonstrate the efficiency of the proposed FW-type method and its away-step variant.

math.OC↗

arXiv2023

A full splitting algorithm for fractional programs with structured numerators and denominators

In this paper, we consider a class of nonconvex and nonsmooth fractional programming problems, that involve the sum of a convex, possibly nonsmooth function composed with a linear operator and a differentiable, possibly nonconvex function in the numerator and a convex, possibly nonsmooth function composed with a linear operator in the denominator. These problems have applications in various fields. We propose an adaptive full-splitting proximal subgradient algorithm that addresses the challenge of decoupling the composition of the nonsmooth component with the linear operator in the numerator. We specifically evaluate the nonsmooth function in the numerator using its proximal operator of its conjugate function. Furthermore, the smooth component in the numerator is evaluated through its gradient, and the nonsmooth in the denominator is managed using its subgradient. We demonstrate subsequential convergence toward an approximate lifted stationary point and ensure global convergence under the Kurdyka-\L ojasiewicz property, all achieved without full-row rank assumptions on the linear operators. We provide further discussions on {\it the tightness of the convergence results of the proposed algorithm and its related variants, and the reasoning behind aiming for an approximate lifted stationary point}. We construct a series of counter-examples to show that the proposed algorithm and its variant might diverge when seeking exact solutions. A practical version incorporating a nonmonotone line search is also developed to enhance its performance significantly. Our theoretical findings are validated through simulations involving limited-angle CT reconstruction and the robust sharp-ratio-type minimization problem.

math.OC↗

arXiv2023

Inertial Proximal Block Coordinate Method for a Class of Nonsmooth Sum-of-Ratios Optimization Problems

In this paper, we consider a class of nonsmooth sum-of-ratios fractional optimization problems with block structure. This model class is ubiquitous and encompasses several important nonsmooth optimization problems in the literature. We first propose an inertial proximal block coordinate method for solving this class of problems by exploiting the underlying structure. The global convergence of our method is guaranteed under the Kurdyka--Lojasiewicz (KL) property and some mild assumptions. We then identify the explicit exponents of the KL property for three important structured fractional optimization problems. In particular, for the sparse generalized eigenvalue problem with either cardinality regularization or sparsity constraint, we show that the KL exponents are 1/2, and so, the proposed method exhibits linear convergence rate. Finally, we illustrate our theoretical results with both analytic and simulated numerical examples.

math.OC↗

arXiv2022

Retraction-based first-order feasible methods for difference-of-convex programs with smooth inequality and simple geometric constraints

In this paper, we propose first-order feasible methods for difference-of-convex (DC) programs with smooth inequality and simple geometric constraints. Our strategy for maintaining feasibility of the iterates is based on a "retraction" idea adapted from the literature of manifold optimization. When the constraints are convex, we establish the global subsequential convergence of the sequence generated by our algorithm under strict feasibility condition, and analyze its convergence rate when the objective is in addition convex according to the Kurdyka-Lojasiewicz (KL) exponent of the extended objective (i.e., sum of the objective and the indicator function of the constraint set). We also show that the extended objective of a large class of Euclidean norm (and more generally, group LASSO penalty) regularized convex optimization problems is a KL function with exponent $\frac12$; consequently, our algorithm is locally linearly convergent when applied to these problems. We then extend our method to solve DC programs with a single specially structured nonconvex constraint. Finally, we discuss how our algorithms can be applied to solve two concrete optimization problems, namely, group-structured compressed sensing problems with Gaussian measurement noise and compressed sensing problems with Cauchy measurement noise, and illustrate the empirical performance of our algorithms.

math.OC↗

arXiv2022

A proximal subgradient algorithm with extrapolation for structured nonconvex nonsmooth problems

In this paper, we consider a class of structured nonconvex nonsmooth optimization problems, in which the objective function is formed by the sum of a possibly nonsmooth nonconvex function and a differentiable function whose gradient is Lipschitz continuous, subtracted by a weakly convex function. This type of structured problems has many practical applications in machine learning and statistics such as compressed sensing, signal recovery, sparse dictionary learning, clustering, matrix factorization, and others. We develop a flexible extrapolated proximal subgradient algorithm for solving these problems with guaranteed subsequential convergence to a stationary point. The global convergence of the whole sequence generated by our algorithm is also established under the Kurdyka-Lojasiewicz property. To illustrate the promising numerical performance of the proposed algorithm, we conduct numerical experiments on two important nonconvex models. This includes a least squares problem with a nonconvex regularization and an optimal power flow problem with distributed energy resources.

math.OC↗

arXiv2021

Calculus of the exponent of Kurdyka-Łojasiewicz inequality and its applications to linear convergence of first-order methods

In this paper, we study the Kurdyka-Łojasiewicz (KL) exponent, an important quantity for analyzing the convergence rate of first-order methods. Specifically, we develop various calculus rules to deduce the KL exponent of new (possibly nonconvex and nonsmooth) functions formed from functions with known KL exponents. In addition, we show that the well-studied Luo-Tseng error bound together with a mild assumption on the separation of stationary values implies that the KL exponent is $\frac12$. The Luo-Tseng error bound is known to hold for a large class of concrete structured optimization problems, and thus we deduce the KL exponent of a large class of functions whose exponents were previously unknown. Building upon this and the calculus rules, we are then able to show that for many convex or nonconvex optimization models for applications such as sparse recovery, their objective function's KL exponent is $\frac12$. This includes the least squares problem with smoothly clipped absolute deviation (SCAD) regularization or minimax concave penalty (MCP) regularization and the logistic regression problem with $\ell_1$ regularization. Since many existing local convergence rate analysis for first-order methods in the nonconvex scenario relies on the KL exponent, our results enable us to obtain explicit convergence rate for various first-order methods when they are applied to a large variety of practical optimization models. Finally, we further illustrate how our results can be applied to establishing local linear convergence of the proximal gradient algorithm and the inertial proximal algorithm with constant step-sizes for some specific models that arise in sparse recovery.

math.OC↗

arXiv2021

Kurdyka-Łojasiewicz exponent via inf-projection

Kurdyka-Lojasiewicz (KL) exponent plays an important role in estimating the convergence rate of many contemporary first-order methods. In particular, a KL exponent of $\frac12$ for a suitable potential function is related to local linear convergence. Nevertheless, KL exponent is in general extremely hard to estimate. In this paper, we show under mild assumptions that KL exponent is preserved via inf-projection. Inf-projection is a fundamental operation that is ubiquitous when reformulating optimization problems via the lift-and-project approach. By studying its operation on KL exponent, we show that the KL exponent is $\frac12$ for several important convex optimization models, including some semidefinite-programming-representable functions and some functions that involve $C^2$-cone reducible structures, under conditions such as strict complementarity. Our results are applicable to concrete optimization models such as group fused Lasso and overlapping group Lasso. In addition, for nonconvex models, we show that the KL exponent of many difference-of-convex functions can be derived from that of their natural majorant functions, and the KL exponent of the Bregman envelope of a function is the same as that of the function itself. Finally, we estimate the KL exponent of the sum of the least squares function and the indicator function of the set of matrices of rank at most $k$.

math.OC↗

arXiv2020

Extrapolated Proximal Subgradient Algorithms for Nonconvex and Nonsmooth Fractional Programs

In this paper, we consider a broad class of nonsmooth and nonconvex fractional programs, where the numerator can be written as the sum of a continuously differentiable convex function whose gradient is Lipschitz continuous and a proper lower semicontinuous (possibly nonconvex) function, and the denominator is weakly convex over the constraint set. This model problem includes the composite optimization problems studied extensively lately, and encompasses many important modern fractional optimization problems arising from diverse areas such as the recently proposed scale invariant sparse signal reconstruction problem in signal processing. We propose a proximal subgradient algorithm with extrapolations for solving this optimization model and show that the iterated sequence generated by the algorithm is bounded and any of its limit points is a stationary point of the model problem. The choice of our extrapolation parameter is flexible and includes the popular extrapolation parameter adopted in the restarted Fast Iterative Shrinking-Threshold Algorithm (FISTA). By providing a unified analysis framework of descent methods, we establish the convergence of the full sequence under the assumption that a suitable merit function satisfies the Kurdyka--Łojasiewicz (KL) property. In particular, our algorithm exhibits linear convergence for the scale invariant sparse signal reconstruction problem and the Rayleigh quotient problem over spherical constraint. In the case where the denominator is the maximum of finitely many continuously differentiable weakly convex functions, we also propose an enhanced extrapolated proximal subgradient algorithm with guaranteed convergence to a stronger notion of stationary points of the model problem. Finally, we illustrate the proposed methods by both analytical and simulated numerical examples.

math.OC↗

arXiv2020

Calculating Radius of Robust Feasibility of Uncertain Linear Conic Programs via Semidefinite Programs

The radius of robust feasibility provides a numerical value for the largest possible uncertainty set that guarantees robust feasibility of an uncertain linear conic program. This determines when the robust feasible set is non-empty. Otherwise the robust counterpart of an uncertain program is not well-defined as a robust optimization problem. In this paper, we address a key fundamental question of robust optimization: How to compute the radius of robust feasibility of uncertain linear conic programs, including linear programs? We first provide computable lower and upper bounds for the radius of robust feasibility for general uncertain linear conic programs under the commonly used ball uncertainty set. We then provide important classes of linear conic programs where the bounds are calculated by finding the optimal values of related semidefinite linear programs (SDPs), among them uncertain SDPs, uncertain second-order cone programs and uncertain support vector machine problems. In the case of an uncertain linear program, the exact formula allows us to calculate the radius by finding the optimal value of an associated second-order cone program.

math.OC↗

arXiv2019

Efficiently Maximizing a Homogeneous Polynomial over Unit Sphere without Convex Relaxation

This work studies the problem of maximizing a higher degree real homogeneous multivariate polynomial over the unit sphere. This problem is equivalent to finding the leading eigenvalue of the associated symmetric tensor of higher order, which is nonconvex and NP-hard. Recent advances show that semidefinite relaxation is quite effective to find a global solution. However, the solution methods involve full/partial eigenvalue decomposition during the iterates, which heavily limits its efficiency and scalability. On the other hand, for odd degree (odd order) cases, the order has to be increased to even, which potentially reduces the efficiency. To find the global solutions, instead of convexifying the problem, we equivalently reformulate the problem as a nonconvex matrix program based on an equivalence property between symmetric rank-1 tensors and matrices of any order, which is a generalization of the existing results. The program is directly solved by a vanilla alternating direction method, which only involves the computation of leading eigenvalue/singular value of certain matrices, benefiting from the special structure of the program. Although being nonconvex, under certain hypotheses, it is proved that the algorithm converges to a leading eigenvalue of the associated tensor. Numerical experiments on different classes of tensors demonstrate that the proposed approach has a significant improvement in efficiency and scalability, while it can keep the effectiveness of semidefinite relaxation as much as possible. For instance, the proposed method finds the leading eigenpair of a third-order 500 dimensional Hilbert tensor in a personal computer within 100 seconds.

math.OC↗

arXiv2017

Peaceman-Rachford splitting for a class of nonconvex optimization problems

We study the applicability of the Peaceman-Rachford (PR) splitting method for solving nonconvex optimization problems. When applied to minimizing the sum of a strongly convex Lipschitz differentiable function and a proper closed function, we show that if the strongly convex function has a large enough strong convexity modulus and the step-size parameter is chosen below a threshold that is computable, then any cluster point of the sequence generated, if exists, will give a stationary point of the optimization problem. We also give sufficient conditions guaranteeing boundedness of the sequence generated. We then discuss one way to split the objective so that the proposed method can be suitably applied to solving optimization problems with a coercive objective that is the sum of a (not necessarily strongly) convex Lipschitz differentiable function and a proper closed function; this setting covers a large class of nonconvex feasibility problems and constrained least squares problems. Finally, we illustrate the proposed algorithm numerically.

math.OC↗

arXiv2016

Finding the maximum eigenvalue of a class of tensors with applications in copositivity test and hypergraphs

Finding the maximum eigenvalue of a symmetric tensor is an important topic in tensor computation and numerical multilinear algebra. This paper is devoted to a semi-definite program algorithm for computing the maximum $H$-eigenvalue of a class of tensors with sign structure called $W$-tensors. The class of $W$-tensors extends the well-studied nonnegative tensors and essentially nonnegative tensors, and covers some important tensors arising naturally from spectral hypergraph theory. Our algorithm is based on a new structured sums-of-squares (SOS) decomposition result for a nonnegative homogeneous polynomial induced by a $W$-tensor. This SOS decomposition enables us to show that computing the maximum $H$-eigenvalue of an even order symmetric $W$-tensor is equivalent to solving a semi-definite program, and hence can be accomplished in polynomial time. Numerical examples are given to illustrate that the proposed algorithm can be used to find maximum $H$-eigenvalue of an even order symmetric $W$-tensor with dimension up to $10,000$. We present two applications for our proposed algorithm: we first provide a polynomial time algorithm for computing the maximum $H$-eigenvalues of large size Laplacian tensors of hyper-stars and hyper-trees; second, we show that the proposed SOS algorithm can be used to test the copositivity of a multivariate form associated with symmetric extended $Z$-tensors, whose order may be even or odd. Numerical experiments illustrate that our structured semi-definite program algorithm is effective and promising.

math.SP↗

arXiv2016

New Classes of Positive Semi-Definite Hankel Tensors

A Hankel tensor is called a strong Hankel tensor if the Hankel matrix generated by its generating vector is positive semi-definite. It is known that an even order strong Hankel tensor is a sum-of-squares tensor, and thus a positive semi-definite tensor. The SOS decomposition of strong Hankel tensors has been well-studied by Ding, Qi and Wei \cite{DQW1}. On the other hand, very little is known for positive semi-definite Hankel tensors which are not strong Hankel tensors. In this paper, we study some classes of positive semi-definite Hankel tensors which are not strong Hankel tensors. These include truncated Hankel tensors and quasi-truncated Hankel tensors. Then we show that a strong Hankel tensor generated by an absoluate integrable function is always completely decomposable, and give a class of SOS Hankel tensors which are not completely decomposable.

math.SP↗

arXiv2016

Convergence rate analysis for averaged fixed point iterations in the presence of Hölder regularity

In this paper, we establish sublinear and linear convergence of fixed point iterations generated by averaged operators in a Hilbert space. Our results are achieved under a bounded Hölder regularity assumption which generalizes the well-known notion of bounded linear regularity. As an application of our results, we provide a convergence rate analysis for Krasnoselskii-Mann iterations, the cyclic projection algorithm, and the Douglas-Rachford feasibility algorithm along with some variants. In the important case in which the underlying sets are convex sets described by convex polynomials in a finite dimensional space, we show that the Hölder regularity properties are automatically satisfied, from which sublinear convergence follows.

math.OC↗