arXiv subjects

Zexian Liu

Publications and source records attributed to Zexian Liu.

12 recordsLinked to original sources

arXiv2026

A New Primal-Dual Algorithm with Two Convex Combinations and Linesearch for General Convex-Concave Saddle-Point Problems

Convex-concave saddle-point problems are ubiquitous across diverse domains, including machine learning, image processing, economics, and equilibrium problems. Primal-dual algorithms provide a highly effective and powerful framework for convex-concave saddle-point problems. Convex combination has become a crucial acceleration technique for primal-dual algorithms, and the integration of this technique has recently made these algorithms a highly active research topic. The choice of the convex combination parameter often has a significant impact on both the theoretical analysis and the numerical performance of the corresponding algorithms. However, the requirements on this parameter imposed by theory are sometimes inconsistent with those suggested by numerical experiments. For instance, theoretical analysis often requires the parameter to be small, while numerical experiments tend sometimes to favor larger values. To address this inconsistency and further advance primal-dual algorithms with convex combination, we develop a novel strategy based on two convex combinations, integrate it into a primal-dual framework, and propose a new primal-dual algorithm with linesearch, termed NPDAL-n, for general convex-concave saddle-point problems. The proposed two convex combinations in NPDAL-n ensure that the permissible range of the convex combination parameters is mainly determined by theoretical considerations, with little regard for numerical performance. Through rigorous Lyapunov energy descent analysis, we establish the global convergence and a sublinear ergodic convergence rate of $\mathcal{O}(1/N)$ for NPDAL-n under standard assumptions. When the primal function is strongly convex, we develop an accelerated version of NPDAL-n that achieves an optimal $\mathcal{O}(1/N^2)$ rate.

math.OC↗

arXiv2026

A New Primal-Dual Algorithm with Convex Combination and Extrapolation for Convex-Concave Saddle Point Problems with Nonlinear Coupling Term

Convex-concave saddle point problems with nonlinear coupling term have wide applications in signal processing, machine learning, robust optimization, and generative models, among others. Primal-dual algorithms are widely used for convex-concave saddle point problems. However, when handling nonlinear coupling term in convex-concave saddle point problems, primal-dual algorithms usually encounter geometric mismatches between variable and mapping spaces, delayed gradient information, and strong dependence on linesearch, resulting in less stable performance and complex convergence analysis. To address these issues, we propose a new primal-dual algorithm named PDAce by combining convex combination and extrapolation strategies. More specifically, in the update of the primal variable, we construct a convex combination point to replace the current iterative point and compute the Jacobian matrix of the vector function in the nonlinear coupling term at the convex combination point. Besides, we use the latest information of convex combination points to extend extrapolation to the mapping space in the update of the dual variable. The core innovations lie in performing linearization of the vector function in the nonlinear coupling term at convex combination points and shifting extrapolation from the variable space to the nonlinear mapping space. This design completely eliminates nonlinear residual terms and allows for rigorous convergence analysis without linesearch. Under mild convex assumptions, we construct a new Lyapunov potential function to prove that PDAce is globally convergent with an ergodic convergence rate of $\mathcal{O}(1/N)$. Moreover, we develop an accelerated version of PDAce, termed aPDAce, which achieves $\mathcal{O}(1/N^2)$ rate under strong convexity of the primal function, and linear convergence when both the primal and dual functions are strongly convex.

math.OC↗

arXiv2026

A unified framework for inexact adaptive stepsizes in the gradient methods, the conjugate gradient methods and the quasi-Newton methods for strictly convex quadratic optimization

The inexact adaptive stepsizes for the conjugate gradient method and the quasi-Newton method are very rare. The exact stepsizes in the gradient method, the conjugate gradient method and the quasi-Newton method for strictly convex quadratic optimization have a unified framework, while the unified framework for inexact adaptive stepsizes in the gradient method, the conjugate gradient method and the quasi-Newton method for strictly convex quadratic optimization still remains unknown. Based on the above observations, we propose a unified framework for inexact adaptive stepsizes in the gradient method, the conjugate gradient method and the quasi-Newton method for strictly convex quadratic optimization, which is called approximately optimal stepsize. The global convergence and the convergence rate of the gradient method with the approximately optimal stepsize are established by exploring the relation between the approximately optimal stepsize and the famous Barzilai-Borwein (BB) stepsizes. Some numerical results are presented, which confirm the remarkable numerical advantage of the gradient method, the conjugate gradient method and the quasi-Newton method with the unified framework for inexact adaptive stepsizes. Some open problems about the gradient method, the conjugate gradient method and the quasi-Newton method with approximately optimal stepsize are raised.

math.OC↗

arXiv2025

New Primal-Dual Algorithm for Convex Problems

Primal-dual algorithm (PDA) is a classic and popular scheme for convex-concave saddle point problems. It is universally acknowledged that the proximal terms in the subproblems about the primal and dual variables are crucial to the convergence theory and numerical performance of primal-dual algorithms. By taking advantage of the information from the current and previous iterative points, we exploit two new proximal terms for the subproblems about the primal and dual variables. Based on two new proximal terms, we present a new primal-dual algorithm for convex-concave saddle point problems with bilinear coupling terms and establish its global convergence and O(1/N ) ergodic convergence rate. When either the primal function or the dual function is strongly convex, we accelerate the above proposed algorithm and show that the corresponding algorithm can achieve O(1/N^2) convergence rate. Since the conditions for the stepsizes of the proposed algorithm are related directly to the spectral norm of the linear transform, which is difficult to obtain in some applications, we also introduce a linesearch strategy for the above proposed primal-dual algorithm and establish its global convergence and O(1/N ) ergodic convergence rate . Some numerical experiments are conducted on matrix game and LASSO problems by comparing with other state-of-the-art algorithms, which demonstrate the effectiveness of the proposed three primal-dual algorithms.

math.OC↗

arXiv2025

3D Dental Model Segmentation with Geometrical Boundary Preserving

3D intraoral scan mesh is widely used in digital dentistry diagnosis, segmenting 3D intraoral scan mesh is a critical preliminary task. Numerous approaches have been devised for precise tooth segmentation. Currently, the deep learning-based methods are capable of the high accuracy segmentation of crown. However, the segmentation accuracy at the junction between the crown and the gum is still below average. Existing down-sampling methods are unable to effectively preserve the geometric details at the junction. To address these problems, we propose CrossTooth, a boundary-preserving segmentation method that combines 3D mesh selective downsampling to retain more vertices at the tooth-gingiva area, along with cross-modal discriminative boundary features extracted from multi-view rendered images, enhancing the geometric representation of the segmentation network. Using a point network as a backbone and incorporating image complementary features, CrossTooth significantly improves segmentation accuracy, as demonstrated by experiments on a public intraoral scan dataset.

cs.CV↗

arXiv2023

On the properties of the linear conjugate gradient method

The linear conjugate gradient method is an efficient iterative method for the convex quadratic minimization problems $ \mathop {\min }\limits_{x \in { \mathbb R^n}} f(x) =\dfrac{1}{2}x^TAx+b^Tx $, where $ A \in R^{n \times n} $ is symmetric and positive definite and $ b \in R^n $. It is generally agreed that the gradients $ g_k $ are not conjugate with respective to $ A $ in the linear conjugate gradient method (see page 111 in Numerical optimization (2nd, Springer, 2006) by Nocedal and Wright). In the paper we prove the conjugacy of the gradients $ g_k $ generated by the linear conjugate gradient method, namely, $$g_k^TAg_i=0, \; i=0,1,\cdots, k-2.$$ In addition,a new way is exploited to derive the linear conjugate gradient method based on the conjugacy of the search directions and the orthogonality of the gradients, rather than the conjugacy of the search directions and the exact stepsize.

math.OC↗

arXiv2023

A new subspace minimization conjugate gradient method for unconstrained minimization

Subspace minimization conjugate gradient (SMCG) methods have become a class of quite efficient iterative methods for unconstrained optimization and have attracted extensive attention recently. Usually, the search directions of SMCG methods are generated by minimizing approximate models with the approximation matrix $ B_k $ of the objective function at the current iterate over the subspace spanned by the current gradient $ g_k $ and the latest search direction. The $ g_k^TB_kg_k $ must be estimated properly in the calculation of the search directions, which is crucial to the theoretical properties and the numerical performance of SMCG methods. It is a great challenge to estimate it properly. The projection technique has been used successfully to generate conjugate gradient directions such as Dai-Kou conjugate gradient direction. Motivated by the above two observations, in the paper we present a new subspace minimization conjugate gradient methods by using a projection technique based on the memoryless quasi-Newton method. More specially, we project the search direction of the memoryless quasi-Newton method into the subspace spanned by the current gradient and the latest search direction and drive a new search direction, which is proved to be descent. Remarkably, the proposed method without any line search enjoys the finite termination property for two dimensional convex quadratic functions, which is helpful for designing algorithm. An adaptive scaling factor in the search direction is given based on the above finite termination property. The proposed method does not need to determine the parameter $ ρ_k $ and can be regarded as an extension of Dai-Kou conjugate gradient method. The global convergence of the proposed method is analyzed. Numerical comparisons indicate the proposed method is very promising.

math.OC↗

arXiv2023

A Regularized Limited Memory Subspace Minimization Conjugate Gradient Method for Unconstrained Optimization

In this paper, based on the limited memory techniques and subspace minimization conjugate gradient (SMCG) methods, a regularized limited memory subspace minimization conjugate gradient method is proposed, which contains two types of iterations. In SMCG iteration, we obtain the search direction by minimizing the approximate quadratic model or approximate regularization model. In RQN iteration, combined with regularization technique and BFGS method, a modified regularized quasi-Newton method is used in the subspace to improve the orthogonality. Moreover, some simple acceleration criteria and an improved tactic for selecting the initial stepsize to enhance the efficiency of the algorithm are designed. Additionally, an generalized nonmonotone line search is utilized and the global convergence of our proposed algorithm is established under mild conditions. Finally, numerical results show that, the proposed algorithm has a significant improvement over ASMCG_PR and is superior to the particularly well-known limited memory conjugate gradient software packages CG_DESCENT (6.8) and CGOPT(2.0) for the CUTEr library.

math.OC↗

arXiv2022

Convergence Rate of Inertial Forward-Backward Algorithms Based on the Local Error Bound Condition

The "Inertial Forward-Backward algorithm" (IFB) is a powerful tool for convex nonsmooth minimization problems, it gives the well known "fast iterative shrinkage-thresholding algorithm " (FISTA), which enjoys $O\left( {\frac{1}{k^2}} \right)$ global convergence rate of function values, however, no convergence of iterates has been proved; by do a small modification, an accelerated IFB called "FISTA\_CD" improves the convergence rate of function values to $o\left( {\frac{1}{k^2}} \right)$ and shows the weak convergence of iterates. The local error bound condition is extremely useful in analyzing the convergence rates of a host of iterative methods for solving optimization problems, and in practical application, a large number of problems with special structure often satisfy the error bound condition. Naturally, using local error bound condition to derive or improve the convergence rate of IFB is a common means. In this paper, based on the local error bound condition, we exploit an new assumption condition for the important parameter $t_k$ in IFB, and establish the improved convergence rate of function values and strong convergence of the iterates generated by the IFB algorithms with six $t_k$ satisfying the above assumption condition in Hilbert space. It is remarkable that, under the local error bound condition, we establish the strong convergence of the iterates generated by the original FISTA, and prove that the convergence rates of function values for FISTA\_CD is actually related to the value of parameter $a,$ and show that the IFB algorithms with some $t_k$ mentioned above can achieve sublinear convergence rate $o\left( {\frac{1}{k^p}} \right)$ for any positive integer $p>1$. Some numerical experiments are conducted to illustrate our results.

math.OC↗

arXiv2022

Two efficient gradient methods with approximately optimal stepsizes based on regularization models for unconstrained optimization

It is widely accepted that the stepsize is of great significance to gradient method. Two efficient gradient methods with approximately optimal stepsizes mainly based on regularization models are proposed for unconstrained optimization. More exactly, if the objective function is not close to a quadratic function on the line segment between the current and latest iterates, regularization models are exploited carefully to generate approximately optimal stepsizes. Otherwise, quadratic approximation models are used. In particular, when the curvature is non-positive, special regularization models are developed. The convergence of the proposed methods is established under the weak conditions. Extensive numerical experiments indicated the proposed method is superior to the BBQ method (SIAM J. Optim. 2021,31(4), 3068-3096) and other efficient gradient methods, and is competitive to two famous and efficient conjugate gradient software packages CG$ \_ $DESCENT (5.0) (SIAM J. Optim. 16(1), 170-192, 2005) and CGOPT (1.0) (SIAM J. Optim. 23(1), 296-320, 2013) for the CUTEr library. Due to the surprising efficiency, we believe that gradient methods with approximately optimal stepsizes can become strong candidates for large-scale unconstrained optimization.

math.OC↗

arXiv2020

New subspace minimization conjugate gradient methods based on regularization model for unconstrained optimization

In this paper, two new subspace minimization conjugate gradient methods based on $p - $regularization models are proposed, where a special scaled norm in $p - $regularization model is analyzed. Different choices for special scaled norm lead to different solutions to the $p - $regularized subproblem. Based on the analyses of the solutions in a two-dimensional subspace, we derive new directions satisfying the sufficient descent condition. With a modified nonmonotone line search, we establish the global convergence of the proposed methods under mild assumptions. $R - $linear convergence of the proposed methods are also analyzed. Numerical results show that, for the CUTEr library, the proposed methods are superior to four conjugate gradient methods, which were proposed by Hager and Zhang (SIAM J Optim 16(1):170-192, 2005), Dai and Kou (SIAM J Optim 23(1):296-320, 2013), Liu and Liu (J Optim Theory Appl 180(3):879-906, 2019) and Li et al. (Comput Appl Math 38(1): 2019), respectively.

math.OC↗

arXiv2019

An Improved Gradient Method with Approximately Optimal Stepsize Based on Conic model for Unconstrained Optimization

A new type of stepsize, which was recently introduced by Liu and Liu (Optimization, 67(3), 427-440, 2018), is called approximately optimal stepsize and is quit efficient for gradient method. Interestingly, all gradient methods can be regarded as gradient methods with approximately optimal stepsizes. In this paper, based on the work (Numer. Algorithms 78(1), 21-39, 2018), we present an improved gradient method with approximately optimal stepsize based on conic model for unconstrained optimization. If the objective function $ f $ is not close to a quadratic on the line segment between the current and latest iterates, we construct a conic model to generate approximately optimal stepsize for gradient method if the conic model can be used; otherwise, we construct some quadratic models to generate approximately optimal stepsizes for gradient method. The convergence of the proposed method is analyzed under suitable conditions. Numerical comparisons with some well-known conjugate gradient software packages such as CG$ \_ $DESCENT (SIAM J. Optim. 16(1), 170-192, 2005) and CGOPT (SIAM J. Optim. 23(1), 296-320, 2013) indicate the proposed method is very promising.

math.OC↗