Searcharxiv⌕ Search

arXiv subjects

Jingrong Wei

Publications and source records attributed to Jingrong Wei.

9 recordsLinked to original sources

Accelerated Transformed Primal Dual Methods for Smooth Convex Optimization Problems with Affine Equality Constraints

Accelerated transformed primal--dual (ATPD) methods are proposed for smooth convex optimization problems with affine equality constraints. The methods combine primal--dual dynamics with a transformation designed to accelerate convergence. In the strongly convex case, ATPD achieves an accelerated linear convergence rate and matches the known lower bounds on first-order oracle complexity in terms of gradient evaluations and matrix--vector products. The analysis distinguishes these two types of computational work when assessing the complexity of the method. For general convex objectives, a homotopy perturbation strategy is introduced, and an accelerated sublinear rate for the modified Lyapunov energy is proved under a uniform boundedness condition on the primal iterates at all homotopy stages. The acceleration mechanism is based on an exponentially stable ATPD flow and a modified discrete Lyapunov function. This continuous-time perspective guides the construction and analysis of the discrete methods. Numerical experiments on policy evaluation and constrained signal reconstruction problems support the theory and illustrate the efficiency of the proposed methods.

math.OC↗

Accelerated Gradient Methods Through Variable and Operator Splitting

This survey develops a variable and operator splitting (VOS) framework for accelerated first-order methods. Operator splitting separates the components of the optimality system, while variable splitting introduces auxiliary dynamics that generate acceleration. The strong Lyapunov property guides the stability analysis, and practical schemes follow from Accelerated Over-Relaxation (AOR) and Extrapolation by Predictor--Corrector (EPC) discretizations. For merely convex problems, a perturbed VOS flow combined with homotopy in the auxiliary scale yields accelerated rates. The framework gives a common derivation and analysis of Nesterov-type, FISTA-type, composite, mirror-descent-type, and bilinear saddle-point accelerated methods. Its main contribution is structural and expository: it organizes these methods within one Lyapunov and splitting template and explains how acceleration survives different splittings and discretizations.

math.OC↗

Several Accelerated and Stable Pseudo-Energy-Dissipative Lagrange Multiplier Methods based on Second-order Flow and Variable Splitting

Based on second-order inertial dynamics, we develop two accelerated Lagrange multiplier (LM)-based optimization methods: Linear LM-based second-order flow method and linear LM-based variable and operator splitting method for unconstrained non-convex optimization problems. Relaxed and adaptive variants of both methods are proposed to prevent degeneration of the Lagrange multiplier and to enable automatic adjustment of the time step size. Within the novel second-order LM-based optimization framework, the associated pseudo-energy functional is shown to dissipate monotonically along the iterative trajectory, and convergence of the generated iterates to stationary points is rigorously established. Extensive numerical experiments on function optimization and partial differential equation benchmark problems, including applications as optimizers for physics-informed neural networks (PINNs) and deep operator networks (DeepONets), demonstrate that the proposed methods effectively escape local minima, achieve high accuracy, and exhibit strong robustness with respect to the choice of the initial learning rate during neural network training.

math.OC↗

Accelerated Mirror Descent Method through Variable and Operator Splitting

Mirror descent uses the mirror function to encode geometry and constraints, improving convergence while preserving feasibility. Accelerated Mirror Descent Methods (Acc-MD) are derived from a discretization of an accelerated mirror ODE system using a variable--operator splitting framework. A geometric assumption, termed the Generalized Cauchy-Schwarz (GCS) condition, is introduced to quantify the compatibility between the objective and the mirror geometry, under which the first accelerated linear convergence for Acc-MD on a broad class of problems is established. Numerical experiments on smooth and composite optimization tasks demonstrate that Acc-MD consistently outperforms existing accelerated variants, both theoretically and empirically.

math.OC↗

A Novel Preconditioning Framework for Solving Nonlinear PDEs based on Fenchel-Rockafellar Duality and Transformed Primal-Dual Techniques

A DualTPD method is proposed for solving nonlinear partial differential equations. The method is characterized by three main features. First, decoupling via Fenchel--Rockafellar duality is achieved, so that nonlinear terms are discretized by discontinuous finite element spaces, yielding block-diagonal mass matrices and closed-form updates. Second, improved convergence is obtained by applying transformed primal--dual (TPD) dynamics to the nonlinear saddle-point system, which yields strongly monotone behavior. Third, efficient preconditioners are designed for the elliptic-type Schur complement arising from the separated differential operators, and multigrid solvers are applied effectively. Extensive numerical experiments on elliptic $p$-Laplacian and nonlinear $H(\curl)$ problems are presented, showing significant efficiency gains with global, mesh-independent convergence.

math.NA↗

Accelerated Over-Relaxation Heavy-Ball Method: Achieving Global Accelerated Convergence with Broad Generalization

The heavy-ball momentum method accelerates gradient descent with a momentum term but lacks accelerated convergence for general smooth strongly convex problems. This work introduces the Accelerated Over-Relaxation Heavy-Ball (AOR-HB) method, the first variant with provable global and accelerated convergence for such problems. AOR-HB closes a long-standing theoretical gap, extends to composite convex optimization and min-max problems, and achieves optimal complexity bounds. It offers three key advantages: (1) broad generalization ability, (2) potential to reshape acceleration techniques, and (3) conceptual clarity and elegance compared to existing methods.

math.OC↗

Accelerated Gradient and Skew-Symmetric Splitting Methods for a Class of Monotone Operator Equations

A class of monotone operator equations, which can be decomposed into sum of the gradient of a strongly convex function and a linear and skew-symmetric operator, is considered in this work. Based on discretization of the generalized gradient flow, gradient and skew-symmetric splitting (GSS) methods are proposed and proved to converge in linear rates. To further accelerate the convergence, an accelerated gradient flow is proposed and accelerated gradient and skew-symmetric splitting (AGSS) methods are developed, which extends the acceleration among the existing works on the convex minimization to a more general class of monotone operator equations. In particular, when applied to smooth saddle point systems with bilinear coupling, a linear convergent method with optimal lower iteration complexity is proposed. The robustness and efficiency of GSS and AGSS methods are verified via extensive numerical experiments.

math.OC↗

Transformed Primal-Dual Methods with Variable-Preconditioners

This paper introduces a novel Transformed Primal-Dual with variable-metric/preconditioner (TPDv) algorithm, designed to efficiently solve affine constrained optimization problems common in nonlinear partial differential equations (PDEs). Diverging from traditional methods, TPDv iteratively updates time-evolving preconditioning operators, enhancing adaptability. The algorithm is derived and analyzed, demonstrating global linear convergence rates under mild assumptions. Numerical experiments on challenging nonlinear PDEs, including the Darcy-Forchheimer model and a nonlinear electromagnetic problem, showcase the algorithm's superiority over existing methods in terms of iteration numbers and computational efficiency. The paper concludes with a comprehensive convergence analysis.

math.NA↗

Transformed Primal-Dual Methods For Nonlinear Saddle Point Systems

A transformed primal-dual (TPD) flow is developed for a class of nonlinear smooth saddle point system. The flow for the dual variable contains a Schur complement which is strongly convex. Exponential stability of the saddle point is obtained by showing the strong Lyapunov property. Several TPD iterations are derived by implicit Euler, explicit Euler, implicit-explicit and Gauss-Seidel methods with accelerated overrelaxation of the TPD flow. Generalized to the symmetric TPD iterations, linear convergence rate is preserved for convex-concave saddle point systems under assumptions that the regularized functions are strongly convex. The effectiveness of augmented Lagrangian methods can be explained as a regularization of the non-strongly convexity and a preconditioning for the Schur complement. The algorithm and convergence analysis depends crucially on appropriate inner products of the spaces for the primal variable and dual variable. A clear convergence analysis with nonlinear inexact inner solvers is also developed.

math.OC↗