arXiv subjects

Xiaokai Chang

Publications and source records attributed to Xiaokai Chang.

11 recordsLinked to original sources

arXiv2026

Averaged proximal reflected gradient method for monotone variational inequalities

Projected reflected gradient (PRG) method proposed by Malitsky is efficient for solving monotone variational inequality (MVI), while the existing upper bound of step size is not tight due to the inequality scaling in the theoretical analysis. In this paper, we construct an averaged variant of PRG method for more general MVI and present a novel Lyapunov function to establish convergent theory. This averaged PRG method provides an improvement of the golden ratio algorithm [Y, Malitsky, Math. Program., 184, 383-410, 2020], and the involved step size is compatible with that for the classical methods, such as Popov's extragradient and forward-reflected-backward methods. Moreover, a fully adaptive strategy without linesearch is presented to adjust step sizes, which generates closed-form and potentially much larger step sizes. Numerical experiments on the Nash-Cournot equilibrium, HpHard, and image reconstruction problems demonstrate that the proposed algorithm significantly outperforms existing state-of-the-art methods.

math.OC↗

arXiv2026

A New Primal-Dual Algorithm with Convex Combination and Extrapolation for Convex-Concave Saddle Point Problems with Nonlinear Coupling Term

Convex-concave saddle point problems with nonlinear coupling term have wide applications in signal processing, machine learning, robust optimization, and generative models, among others. Primal-dual algorithms are widely used for convex-concave saddle point problems. However, when handling nonlinear coupling term in convex-concave saddle point problems, primal-dual algorithms usually encounter geometric mismatches between variable and mapping spaces, delayed gradient information, and strong dependence on linesearch, resulting in less stable performance and complex convergence analysis. To address these issues, we propose a new primal-dual algorithm named PDAce by combining convex combination and extrapolation strategies. More specifically, in the update of the primal variable, we construct a convex combination point to replace the current iterative point and compute the Jacobian matrix of the vector function in the nonlinear coupling term at the convex combination point. Besides, we use the latest information of convex combination points to extend extrapolation to the mapping space in the update of the dual variable. The core innovations lie in performing linearization of the vector function in the nonlinear coupling term at convex combination points and shifting extrapolation from the variable space to the nonlinear mapping space. This design completely eliminates nonlinear residual terms and allows for rigorous convergence analysis without linesearch. Under mild convex assumptions, we construct a new Lyapunov potential function to prove that PDAce is globally convergent with an ergodic convergence rate of $\mathcal{O}(1/N)$. Moreover, we develop an accelerated version of PDAce, termed aPDAce, which achieves $\mathcal{O}(1/N^2)$ rate under strong convexity of the primal function, and linear convergence when both the primal and dual functions are strongly convex.

math.OC↗

arXiv2025

A primal-dual splitting algorithm with convex combination and larger step sizes for composite monotone inclusion problems

The primal-dual splitting algorithm (PDSA) by Chambolle and Pock is efficient for solving structured convex optimization problems. It adopts an extrapolation step and achieves convergence under certain step size condition. Chang and Yang recently proposed a modified PDSA for bilinear saddle point problems, integrating a convex combination step to enable convergence with extended step sizes. In this paper, we focus on composite monotone inclusion problems (CMIPs), a generalization of convex optimization problems. While Vu extended PDSA to CMIPs, whether the modified PDSA can be directly adapted to CMIPs remains an open question. This paper introduces a new PDSA for CMIPs, featuring the inclusion of both an extrapolation step and a convex combination step. The proposed algorithm is reformulated as a fixed-point iteration by leveraging an extended firmly nonexpansive operator. Under a significantly relaxed step size condition, both its convergence and sublinear convergence rate results are rigorously established. For structured convex optimization problem, we establish its sublinear convergence rate results measured by function value gap and constraint violations. Moreover, we show through a concrete example that our condition on the involved parameters cannot be relaxed. Numerical experiments on image denoising, inpainting, matrix games, and LASSO problems are conducted to compare the proposed algorithm with state-of-the-art counterparts, demonstrating the efficiency of the proposed algorithm.

math.OC↗

arXiv2024

A convex combination based primal-dual algorithm with linesearch for general convex-concave saddle point problems

Using convex combination and linesearch techniques, we introduce a novel primal-dual algorithm for solving structured convex-concave saddle point problems with a generic smooth nonbilinear coupling term. Our adaptive linesearch strategy works under specific local smoothness conditions, allowing for potentially larger stepsizes. For an important class of structured convex optimization problems, the proposed algorithm reduces to a fully adaptive proximal gradient algorithm without linesearch, thereby representing an advancement over the golden ratio algorithm delineated in [Y. Malitsky, Math. Program. 2020]. We establish global pointwise and ergodic sublinear convergence rate of the algorithm measured by the primal-dual gap function in the general case. When the coupling term is linear in the dual variable, we measure the convergence rate by function value residual and constraint violation of an equivalent constrained optimization problem. Furthermore, an accelerated algorithm achieving the faster O(1/N^2) ergodic convergence rate is presented for the strongly convex case, where N denotes the iteration number. Our numerical experiments on quadratically constrained quadratic programming and sparse logistic regression problems indicate the new algorithm is significantly faster than the comparison algorithms.

math.OC↗

arXiv2021

Golden ratio primal-dual algorithm with linesearch

Golden ratio primal-dual algorithm (GRPDA) is a new variant of the classical Arrow-Hurwicz method for solving structured convex optimization problem, in which the objective function consists of the sum of two closed proper convex functions, one of which involves a composition with a linear transform. In this paper, we propose a linesearch strategy for GRPDA, which not only does not require the spectral norm of the linear transform but also allows adaptive and potentially much larger stepsizes. Within each linesearch step, only the dual variable needs to be updated, and it is thus quite cheap and does not require any extra matrix-vector multiplications for many special yet important applications, e.g., regularized least squares problem. Global convergence and ${\cal O}(1/N)$ ergodic convergence rate results measured by the primal-dual gap function are established, where $N$ denotes the iteration counter. When one of the component functions is strongly convex, faster ${\cal O}(1/N^2)$ ergodic convergence rate results are established by adaptively choosing some algorithmic parameters. Moreover, when both component functions are strongly convex, nonergodic linear converge results are established. Numerical experiments on matrix game and LASSO problems illustrate the effectiveness of the proposed linesearch strategy.

math.OC↗

arXiv2020

A golden ratio primal-dual algorithm for structured convex optimization

We design, analyze and test a golden ratio primal-dual algorithm (GRPDA) for solving structured convex optimization problem, where the objective function is the sum of two closed proper convex functions, one of which involves a composition with a linear transform. GRPDA preserves all the favorable features of the classical primal-dual algorithm (PDA), i.e., the primal and the dual variables are updated in a Gauss-Seidel manner, and the per iteration cost is dominated by the evaluation of the proximal point mappings of the two component functions and two matrix-vector multiplications. Compared with the classical PDA, which takes an extrapolation step, the novelty of GRPDA is that it is constructed based on a convex combination of essentially the whole iteration trajectory. We show that GRPDA converges within a broader range of parameters than the classical PDA, provided that the reciprocal of the convex combination parameter is bounded above by the golden ratio, which explains the name of the algorithm. An O(1/N) ergodic convergence rate result is also established based on the primal-dual gap function, where N denotes the number of iterations. When either the primal or the dual problem is strongly convex, an accelerated GRPDA is constructed to improve the ergodic convergence rate from O(1/N) to O(1/N2). Moreover, we show for regularized least-squares and linear equality constrained problems that the reciprocal of the convex combination parameter can be extended from the golden ratio to 2 and meanwhile a relaxation step can be taken. Our preliminary numerical results on LASSO, nonnegative least-squares and minimax matrix game problems, with comparisons to some state-of-the-art relative algorithms, demonstrate the efficiency of the proposed algorithms.

math.OC↗

arXiv2019

Proximal extrapolated gradient methods with prediction and correction for monotone variational inequalities

An efficient proximal-gradient-based method, called proximal extrapolated gradient method, is designed for solving monotone variational inequality in Hilbert space. The proposed method extends the acceptable range of parameters to obtain larger step sizes. The step size is predicted based a local information of the operator and corrected by linesearch procedures to satisfy a very weak condition, which is even weaker than the boundedness of sequence generated and always holds when the operator is the gradient of a convex function. We establish its convergence and ergodic convergence rate in theory under the larger range of parameters. Furthermore, we improve numerical efficiency by employing the proposed method with non-monotonic step size, and obtain the upper bound of the parameter relating to step size by an extremely simple example. Related numerical experiments illustrate the improvements in efficiency from the larger step size.

math.OC↗

arXiv2019

First-order primal-dual algorithm with correction

This paper is devoted to the design of efficient primal-dual algorithm (PDA) for solving convex optimization problems with known saddle-point structure. We present a new PDA with larger acceptable range of parameters and correction, which result in larger step sizes. The step sizes are predicted by using a local information of the linear operator and corrected by linesearch to satisfy a very weak condition, even weaker than the boundedness of sequence generated. The convergence and ergodic convergence rate are established for general cases, and in case when one of the prox-functions is strongly convex. The numerical experiments illustrate the improvements in efficiency from the larger step sizes and acceptable range of parameters.

math.OC↗

arXiv2019

A family of multi-parameterized proximal point algorithms

In this paper, a multi-parameterized proximal point algorithm combining with a relaxation step is developed for solving convex minimization problem subject to linear constraints. We show its global convergence and sublinear convergence rate from the prospective of variational inequality. Preliminary numerical experiments on testing a sparse minimization problem from signal processing indicate that the proposed algorithm performs better than some well-established methods

math.NA↗

arXiv2019

Convergence Revisit on Generalized Symmetric ADMM

In this note, we show a sublinear nonergodic convergence rate for the algorithm developed in [Bai, et al. Generalized symmetric ADMM for separable convex optimization. Comput. Optim. Appl. 70, 129-170 (2018)], as well as its linear convergence under assumptions that the sub-differential of each component objective function is piecewise linear and all the constraint sets are polyhedra. These remaining convergence results are established for the stepsize parameters of dual variables belonging to a special isosceles triangle region, which aims to strengthen our understanding for convergence of the generalized symmetric ADMM.

math.NA↗

arXiv2018

A three-operator splitting perspective of a three-block ADMM for convex quadratic semidefinite programming and extensions

In recent years, several convergent multi-block variants of the alternating direction method of multipliers (ADMM) have been proposed for solving the convex quadratic semidefinite programming via its dual, which is naturally a 3-block separable convex optimization problem with one coupled linear equality constraint. Among of these ADMM-type algorithms, the modified 3-block ADMM in [Chang et al., Neurocomput. 214: 575--586 (2016)] bears a peculiar feature that the augmented Lagrangian function is not necessarily to be minimized with respect to the block-variable corresponding to the quadratic term of the objective function. In this paper, we lay the theoretical foundation of this phenomena by interpreting this modified 3-block ADMM as a realization of a 3-operator splitting framework. Based on this perspective, we are able to extend this modified 3-block ADMM to a generalized 3-block ADMM, which not only applies to the more general convex composite quadratic programming setting but also admits the potential of achieving even a better numerical performance.

math.OC↗