arXiv subjects

Qia Li

Publications and source records attributed to Qia Li.

13 recordsLinked to original sources

arXiv2026

An Infeasible Method with Feasibility Safeguard for Nonsmooth Composite Optimization Over Manifolds

In this paper, we consider nonsmooth composite optimization over compact embedded submanifolds defined by nonlinear equality constraints. We propose a feasibility-safeguarded inexact proximal linearized method (FSIPL), which allows infeasible iterates while keeping them within a prescribed bounded neighborhood of the manifold. Each iteration approximately solves a strongly convex proximal linearized subproblem, performs a correction step to reduce constraint violation, and uses a merit-function-based nonmonotone backtracking line search to select stepsizes and accept trial iterates. The feasibility safeguard, incorporated into both correction and line search, controls infeasibility and makes the boundedness needed in the analysis a consequence of the algorithmic design. We prove finite termination of backtracking, subsequential convergence to stationary points, and an $O(\varepsilon^{-2})$ outer iteration complexity bound. Under a Kurdyka--\L{}ojasiewicz assumption on a suitable auxiliary function, we further establish full-sequence convergence. Numerical results on sparse PCA and sparse spectral clustering illustrate its efficiency.

math.OC↗

arXiv2025

An efficient proximal algorithm for squared L1 over L2 regularized sparse recovery

In this paper, we consider a squared $L_1/L_2$ regularized model for sparse signal recovery from noisy measurements. We first establish the existence of optimal solutions to the model under mild conditions. Next, we propose a proximal method for solving a general fractional optimization problem which has the squared $L_1/L_2$ regularized model as a special case. We prove that any accumulation point of the solution sequence generated by the proposed method is a critical point of the fractional optimization problem. Under additional KL assumptions on some potential function, we establish the sequential convergence of the proposed method. When this method is specialized to the squared $L_1/L_2$ regularized model, the proximal operator involved in each iteration admits a simple closed form solution that can be computed with very low computational cost. Furthermore, for each of the three concrete models, the solution sequence generated by this specialized algorithm converges to a critical point. Numerical experiments demonstrate the superiority of the proposed algorithm for sparse recovery based on squared $L_1/L_2$ regularization.

math.OC↗

arXiv2025

Proximal Gradient Descent Ascent Methods for Nonsmooth Nonconvex-Concave Minimax Problems on Riemannian Manifolds

Nonsmooth nonconvex-concave minimax problems have attracted significant attention due to their wide applications in many fields. In this paper, we consider a class of nonsmooth nonconvex-concave minimax problems on Riemannian manifolds. Owing to the nonsmoothness of the objective function, existing minimax manifold optimization methods cannot be directly applied to solve this problem. We propose two manifold proximal gradient descent ascent (MPGDA) algorithms for solving the problem. The first algorithm alternatively performs one or multiple manifold proximal gradient descent steps and a proximal ascent step at each iteration, and we prove that it can find an $\varepsilon$-game-stationary point and an $\varepsilon$-optimization-stationary point within $\mathcal{O}(\varepsilon^{-3})$ outer iterations. The second algorithm alternatively performs one manifold proximal gradient descent step and a proximal gradient ascent step, and we show that it can reach an $\varepsilon$-game-stationary point and an $\varepsilon$-optimization-stationary point within $\mathcal{O}(\varepsilon^{-4})$ outer iterations. Numerical experiments on an analytic example, fair sparse PCA, and sparse spectral clustering are conducted to illustrate the advantages of the proposed algorithms.

math.OC↗

arXiv2025

A min-max reformulation and proximal algorithms for a class of structured nonsmooth fractional optimization problems

In this paper, we consider a class of structured nonsmooth fractional minimization, where the first part of the objective is the ratio of a nonnegative nonsmooth nonconvex function to a nonnegative nonsmooth convex function, while the second part is the difference of a smooth nonconvex function and a nonsmooth convex function. This model problem has many important applications, for example, the scale-invariant sparse signal recovery in signal processing. However, the existing methods for fractional programs are not suitable for solving this problem due to its special structure. We first present a novel nonfractional min-max reformulation for the original fractional program and show the connections between their global (local) optimal solutions and stationary points. Based on the reformulation, we propose an alternating maximization proximal descent algorithm and show its subsequential convergence towards a critical point of the original fractional program under a mild assumption. Moreover, we prove that the proposed algorithm can find an $\epsilon$-critical point of the considered problem within $\mathcal{O}(\epsilon^{-2})$ iterations. By further assuming the Kurdyka-{\L}ojasiewicz (KL) property of an auxiliary function, we also establish the convergence of the entire solution sequence generated by the proposed algorithm. Finally, some numerical experiments on the $L_1/L_2$ least squares problem and scale-invariant sparse signal recovery are conducted to demonstrate the efficiency of the proposed method.

math.OC↗

arXiv2024

Proximal methods for structured nonsmooth optimization over Riemannian submanifolds

In this paper, we consider a class of structured nonsmooth optimization problems over an embedded submanifold of a Euclidean space, where the first part of the objective is the sum of a difference-of-convex (DC) function and a smooth function, while the remaining part is a weakly convex function over a smooth function. This model problem has many important applications in machine learning and scientific computing, for example, the sparse Fisher discriminant analysis. We propose a manifold proximal-gradient-subgradient algorithm (MPGSA) and show that under mild conditions any accumulation point of the solution sequence generated by it is a critical point of the underlying problem. By assuming the Kurdyka-{\L}ojasiewicz property of an auxiliary function, we further establish the convergence of the full sequence generated by MPGSA under some suitable conditions. When the second component of the DC function involved is the maximum of finite continuously differentiable convex functions, we also propose an enhanced MPGSA with guaranteed subsequential convergence to a lifted B-stationary points of the optimization problem. Finally, some preliminary numerical experiments are conducted to illustrate the efficiency of the proposed algorithms.

math.OC↗

arXiv2023

An equivalent reformulation and multi-proximity gradient algorithms for a class of nonsmooth fractional programming

In this paper, we consider a class of structured fractional programs, where the numerator part is the sum of a block-separable (possibly nonsmooth nonconvex) function and a locally Lipschitz differentiable (possibly nonconvex) function, while the denominator is a convex (possibly nonsmooth) function. We first present a novel reformulation for the original problem and show the relationship between optimal solutions, critical points and KL exponents of these two problems. Inspired by the reformulation, we propose a flexible framework of multi-proximity gradient algorithms (MPGA), which computes the proximity operator with respect to the Fenchel conjugate associated with the convex denominator of the original problem rather than evaluating its subgradient as in the existing methods. Also, MPGA employs a nonmonotone linear-search scheme in its gradient descent step, since the smooth part in the numerator of the original problem is not globally Lipschitz differentiable. Based on the framework of MPGA, we develop two specific algorithms, namely, cyclic MPGA and randomized MPGA, and establish their subsequential convergence under mild conditions. Moreover, the sequential convergence of cyclic MPGA with the monotone line-search (CMPGA_ML) is guaranteed if the extended objective associated with the reformulated problem satisfies the Kurdyka-{\L}ojasiewicz (KL) property and some other mild assumptions. In particular, we prove that the corresponding KL exponents are 1/2 for several special cases of the fractional programs, and so, CMPGA_ML exhibits a linear convergence rate. Finally, some preliminary numerical experiments are performed to demonstrate the efficiency of our proposed algorithms.

math.OC↗

arXiv2023

A Successive Two-stage Method for Sparse Generalized Eigenvalue Problems

The Sparse Generalized Eigenvalue Problem (sGEP), a pervasive challenge in statistical learning methods including sparse principal component analysis, sparse Fisher's discriminant analysis, and sparse canonical correlation analysis, presents significant computational complexity due to its NP-hardness. The primary aim of sGEP is to derive a sparse vector approximation of the largest generalized eigenvector, effectively posing this as a sparse optimization problem. Conventional algorithms for sGEP, however, often succumb to local optima and exhibit significant dependency on initial points. This predicament necessitates a more refined approach to avoid local optima and achieve an improved solution in terms of sGEP's objective value, which we address in this paper through a novel successive two-stage method. The first stage of this method incorporates an algorithm for sGEP capable of yielding a stationary point from any initial point. The subsequent stage refines this stationary point by adjusting its support, resulting in a point with an enhanced objective value relative to the original stationary point. This support adjustment is achieved through a novel procedure we have named support alteration. The final point derived from the second stage then serves as the initial point for the algorithm in the first stage, creating a cyclical process that continues until a predetermined stopping criterion is satisfied. We also provide a comprehensive convergence analysis of this process. Through extensive experimentation under various settings, our method has demonstrated significant improvements in the objective value of sGEP compared to existing methodologies, underscoring its potential as a valuable tool in statistical learning and optimization.

math.OC↗

arXiv2023

Parameterized proximal-gradient algorithms for L1/L2 sparse signal recovery

The ratio of L1 and L2 norms (L1/L2), serving as a sparse promoting function, receives considerable attentions recently due to its effectiveness for sparse signal recovery. In this paper, we propose an L1/L2 based penalty model for recovering sparse signals from noiseless or noisy observations. It is proven that stationary points of the proposed problem tend to those of the elliptically constrained L1/L2 minimization problem as the smoothing parameter goes to zero. Moreover, inspired by the parametric approach for the fractional programming, we design a parameterized proximal-gradient algorithm (PPGA) as well as its line search counterpart (PPGA_L) for solving the proposed model. The closed-form solution of the involved proximity operator is derived, which enable the efficiency of the proposed algorithms. We establish the global convergence of the entire sequences generated by PPGA and PPGA_L with monotone objective values by taking advantage of the fact that the objective of the proposed model is a KL function. Numerical experiments show the efficiency of the proposed algorithms over the state-of-the-art methods in both noiseless and noisy sparse signal recovery problems.

math.OC↗

arXiv2023

Smoothing algorithms for nonsmooth and nonconvex minimization over the stiefel manifold

We consider a class of nonsmooth and nonconvex optimization problems over the Stiefel manifold where the objective function is the summation of a nonconvex smooth function and a nonsmooth Lipschitz continuous convex function composed with an linear mapping. We propose three numerical algorithms for solving this problem, by combining smoothing methods and some existing algorithms for smooth optimization over the Stiefel manifold. In particular, we approximate the aforementioned nonsmooth convex function by its Moreau envelope in our smoothing methods, and prove that the Moreau envelope has many favorable properties. Thanks to this and the scheme for updating the smoothing parameter, we show that any accumulation point of the solution sequence generated by the proposed algorithms is a stationary point of the original optimization problem. Numerical experiments on building graph Fourier basis are conducted to demonstrate the efficiency of the proposed algorithms.

math.OC↗

arXiv2020

First-order algorithms for a class of fractional optimization problems

We consider in this paper a class of single-ratio fractional minimization problems, in which the numerator part of the objective is the sum of a nonsmooth nonconvex function and a smooth nonconvex function while the denominator part is a nonsmooth convex function. In this work, we first derive its first-order necessary optimality condition, by using the first-order operators of the three functions involved. Then we develop first-order algorithms, namely, the proximity-gradient-subgradient algorithm (PGSA), PGSA with monotone line search (PGSA_ML) and PGSA with nonmonotone line search (PGSA_NL). It is shown that any accumulation point of the sequence generated by them is a critical point of the problem under mild assumptions. Moreover, we establish global convergence of the sequence generated by PGSA or PGSA_ML and analyze its convergence rate, by further assuming the local Lipschitz continuity of the nonsmooth function in the numerator part, the smoothness of the denominator part and the Kurdyka- Lojasiewicz property of the objective. The proposed algorithms are applied to the sparse generalized eigenvalue problem associated with a pair of symmetric positive semidefinite matrices and the corresponding convergence results are obtained according to their general convergence theorems. We perform some preliminary numerical experiments to demonstrate the efficiency of the proposed algorithms

math.OC↗

arXiv2017

Capped Lp approximations for the composite L0 regularization problem

The composite L0 function serves as a sparse regularizer in many applications. The algorithmic difficulty caused by the composite L0 regularization (the L0 norm composed with a linear mapping) is usually bypassed through approximating the L0 norm. We consider in this paper capped Lp approximations with $p>0$ for the composite L0 regularization problem. For each $p>0$, the capped Lp function converges to the L0 norm pointwisely as the approximation parameter tends to infinity. We point out that the capped Lp approximation problem is essentially a penalty method with an Lp penalty function for the composite L0 problem from the viewpoint of numerical optimization. Our theoretical results stated below may shed a new light on the penalty methods for solving the composite L0 problem and help the design of innovative numerical algorithms. We first establish the existence of optimal solutions to the composite L0 regularization problem and its capped Lp approximation problem under conditions that the data fitting function is asymptotically level stable and bounded below. Asymptotically level stable functions cover a rich class of data fitting functions encountered in practice. We then prove that the capped Lp problem asymptotically approximates the composite L0 problem if the data fitting function is a level bounded function composed with a linear mapping. We further show that if the data fitting function is the indicator function on an asymptotically linear set or the L0 norm composed with an affine mapping, then the composite L0 problem and its capped Lp approximation problem share the same optimal solution set provided that the approximation parameter is large enough.

math.OC↗

arXiv2016

On optimal solutions of the constrained $\ell_0$ regularization and its penalty problem

The constrained $\ell_0$ regularization plays an important role in sparse reconstruction. A widely used approach for solving this problem is the penalty method, of which the least square penalty problem is a special case. However, the connections between global minimizers of the constrained $\ell_0$ problem and its penalty problem have never been studied in a systematic way. This work provides a comprehensive investigation on optimal solutions of these two problems and their connections. We give detailed descriptions of optimal solutions of the two problems, including existence, stability with respect to the parameter, cardinality and strictness. In particular, we find that the optimal solution set of the penalty problem is piecewise constant with respect to the penalty parameter. Then we analyze in-depth the relationship between optimal solutions of the two problems. It is shown that, in the noisy case the least square penalty problem probably has no common optimal solutions with the constrained $\ell_0$ problem for any penalty parameter. Under a mild condition on the penalty function, we establish that the penalty problem has the same optimal solution set as the constrained $\ell_0$ problem when the penalty parameter is sufficiently large. Based on the conditions, we further propose exact penalty problems for the constrained $\ell_0$ problem. Finally, we present a numerical example to illustrate our main theoretical results.

math.OC↗

arXiv2016

Two-step Fixed-point proximity algorithms for multi-block separable convex problems

Multi-block separable convex problems recently received considerable attention. This class of optimization problems minimizes a separable convex objective function with linear constraints. The algorithmic challenges come from the fact that the classic alternating direction method of multipliers (ADMM) for the problem is not necessarily convergent. However, it is observed that ADMM outperforms numerically many of its variants with guaranteed theoretical convergence. The goal of this paper is to develop convergent and computationally efficient algorithms for solving multi-block separable convex problems. We first characterize the solutions of the optimization problems by proximity operators of the convex functions involved in their objective function. We then design a two-step fixed-point iterative scheme for solving these problems based on the characterization. We further prove convergence of the iterative scheme and show that it has O(1/k) convergence rate in the ergodic sense and the sense of the partial primal-dual gap, where k denotes the iteration number. Moreover, we derive specific two-step fixed-point proximity algorithms (2SFPPA) from the proposed iterative scheme and establish their global convergence. Numerical experiments for solving the sparse MRI problem demonstrate the numerical efficiency of the proposed 2SFPPA.

math.OC↗