arXiv subjects

Xinmin Yang

Publications and source records attributed to Xinmin Yang.

At least 19 recordsLinked to original sources

arXiv2026

Second-order Methods for Multiobjective Composite Optimization: Preconditioning Strategies, Subspace Variants and Inexact Solutions

Multiobjective composite optimization problems arise in sparse regularization, constrained multiobjective models, and multi-task learning, but their numerical solution remains challenging when the smooth components are ill-conditioned. Proximal gradient methods are inexpensive per iteration but may converge slowly, while proximal Newton and quasi-Newton methods exploit curvature information at the cost of evaluating expensive metric proximal mappings. To address these issues, we propose a preconditioned proximal Barzilai--Borwein method for multiobjective composite optimization. The method combines objective-wise Barzilai--Borwein scaling, which reduces imbalance among objectives, with a common preconditioner that captures shared curvature information. To avoid non-diagonal metric proximal mappings, we develop a subspace variant in which the search direction is computed in a two-dimensional subspace generated by a proximal-gradient-type direction and a projected historical direction. By constructing a conjugate basis with respect to the preconditioning metric, the subspace model decomposes into tractable one-dimensional subproblems. The framework is further extended to nonsmooth terms of the form $g_i(Ax)$ through a linear-operator-aware preconditioner, yielding explicit proximal evaluations via dual subproblems. We also analyze an inexact version based on relaxed descent conditions. We establish the global convergence of the inexact algorithm in the nonconvex setting and prove a linear convergence rate under an error-bound condition. Numerical experiments on ill-conditioned $\ell_1$-regularized, structured $\ell_1$-regularized, and linearly constrained problems demonstrate the effectiveness of the proposed method.

math.OC↗

arXiv2026

Preconditioned Proximal Gradient Methods with Conjugate Momentum: A Subspace Perspective

In this paper, we propose a descent method for composite optimization problems with linear operators. Specifically, we first design a structure-exploiting preconditioner tailored to the linear operator so that the resulting preconditioned proximal subproblem admits a closed-form solution through its dual formulation. However, such a structure-driven preconditioner may be poorly aligned with the local curvature of the smooth component, which can lead to slow practical convergence. To address this issue, we develop a subspace proximal Newton framework that incorporates curvature information within a low-dimensional subspace. At each iteration, the search direction is obtained by minimizing a proximal Newton model restricted to a two-dimensional subspace spanned by the current preconditioned proximal gradient direction and a momentum direction derived from the previous iterate. By orthogonalizing the subspace basis with respect to the local Hessian-induced metric, the solution of the original coupled two-dimensional nonsmooth subproblem can be well approximated by solving two decoupled one-dimensional subproblems, while keeping the per-iteration computational cost low. We establish global convergence of the proposed method and prove a $Q$-linear convergence rate under strong convexity. Comparative numerical experiments demonstrate the effectiveness of the proposed algorithm on ill-conditioned problems.

math.OC↗

arXiv2025

Barzilai-Borwein Diagonal Quasi-Newton Method for Nonconvex Multiobjective Optimization Problems

This paper addresses the challenge of developing efficient algorithms for large-scale nonconvex multiobjective optimization problems (MOPs). While quasi-Newton methods are effective, their traditional application to MOPs is computationally expensive as they require maintaining and inverting separate Hessian approximations for each objective function. To overcome this limitation, we propose a novel Barzilai-Borwein diagonal-type Quasi-Newton method (BB-DQN). Our key innovation is the use of a single, shared, and modified BB-type matrix, updated iteratively using function and gradient information, to approximate the Hessians of all objectives simultaneously. We theoretically demonstrate that this approximation matrix remains positive definite throughout the iterative process. Furthermore, we establish the global convergence of the BB-DQN method without convexity assumptions and prove its R-linear convergence under mild conditions. Numerical experiments on a diverse set of test problems confirm that BB-DQN outperforms existing methods like M-BFGSMO, achieving superior performance in terms of computational time, iteration count, and reliability, especially for large-scale instances.

math.OC↗

arXiv2025

An accelerated primal-dual flow for linearly constrained multiobjective optimization

In this paper, we propose a continuous-time primal-dual approach for linearly constrained multiobjective optimization problems. A novel dynamical model, called accelerated multiobjective primal-dual flow, is presented with a second-order equation for the primal variable and a first-order equation for the dual variable. It can be viewed as an extension of the accelerated primal-dual flow by Luo [arXiv:2109.12604, 2021] for the single objective case. To facilitate the convergence rate analysis, we introduce a new merit function, which motivates the use of the feasibility violation and the objective gap to measure the weakly Pareto optimality. By using a proper Lyapunov function, we establish the exponential decay rate in the continuous level. After that, we consider an implicit-explicit scheme, which yields an accelerated multiobjective primal-dual method with a quadratic subproblem, and prove the sublinear rates of the feasibility violation and the objective gap, under the convex case and the strongly convex case, respectively. Numerical results are provided to demonstrate the performance of the proposed method.

math.OC↗

arXiv2025

A Parallel Hybrid Action Space Reinforcement Learning Model for Real-world Adaptive Traffic Signal Control

Adaptive traffic signal control (ATSC) can effectively reduce vehicle travel times by dynamically adjusting signal timings but poses a critical challenge in real-world scenarios due to the complexity of real-time decision-making in dynamic and uncertain traffic conditions. The burgeoning field of intelligent transportation systems, bolstered by artificial intelligence techniques and extensive data availability, offers new prospects for the implementation of ATSC. In this study, we introduce a parallel hybrid action space reinforcement learning model (PH-DDPG) that optimizes traffic signal phase and duration of traffic signals simultaneously, eliminating the need for sequential decision-making seen in traditional two-stage models. Our model features a task-specific parallel hybrid action space tailored for adaptive traffic control, which directly outputs discrete phase selections and their associated continuous duration parameters concurrently, thereby inherently addressing dynamic traffic adaptation through unified parametric optimization. %Our model features a unique parallel hybrid action space that allows for the simultaneous output of each action and its optimal parameters, streamlining the decision-making process. Furthermore, to ascertain the robustness and effectiveness of this approach, we executed ablation studies focusing on the utilization of a random action parameter mask within the critic network, which decouples the parameter space for individual actions, facilitating the use of preferable parameters for each action. The results from these studies confirm the efficacy of this method, distinctly enhancing real-world applicability

cs.AI↗

arXiv2025

An accelerated gradient method with adaptive restart for convex multiobjective optimization problems

In this work, based on the continuous time approach, we propose an accelerated gradient method with adaptive residual restart for convex multiobjective optimization problems. For the first, we derive rigorously the continuous limit of the multiobjective accelerated proximal gradient method by Tanabe et al. [Comput. Optim. Appl., 2023]. It is a second-order ordinary differential equation (ODE) that involves a special projection operator and can be viewed as an extension of the ODE by Su et al. [J. Mach. Learn. Res., 2016] for Nesterov acceleration. Then, we introduce a novel accelerated multiobjective gradient (AMG) flow with tailored time scaling that adapts automatically to the convex case and the strongly convex case, and the exponential decay rate of a merit function along with the solution trajectory of AMG flow is established via the Lyapunov analysis. After that, we consider an implicit-explicit time discretization and obtain an accelerated multiobjective gradient method with a convex quadratic programming subproblem. The fast sublinear rate and linear rate are proved respectively for convex and strongly convex problems. In addition, we present an efficient residual based adaptive restart technique to overcome the oscillation issue and improve the convergence significantly. Numerical results are provided to validate the practical performance of the proposed method.

math.OC↗

arXiv2024

Scaled Proximal Gradient Methods for Multiobjective Optimization: Improved Linear Convergence and Nesterov's Acceleration

Over the past two decades, descent methods have received substantial attention within the multiobjective optimization field. Nonetheless, both theoretical analyses and empirical evidence reveal that existing first-order methods for multiobjective optimization converge slowly, even for well-conditioned problems, due to the objective imbalances. To address this limitation, we incorporate curvature information to scale each objective within the direction-finding subproblem, introducing a scaled proximal gradient method for multiobjective optimization (SPGMO). We demonstrate that the proposed method achieves improved linear convergence, exhibiting rapid convergence in well-conditioned scenarios. Furthermore, by applying small scaling to linear objectives, we prove that the SPGMO attains improved linear convergence for problems with multiple linear objectives. Additionally, integrating Nesterov's acceleration technique further enhances the linear convergence of SPGMO. To the best of our knowledge, this advancement in linear convergence is the first theoretical result that directly addresses objective imbalances in multiobjective first-order methods. Finally, we provide numerical experiments to validate the efficiency of the proposed methods and confirm the theoretical findings.

math.OC↗

arXiv2024

Generalized conditional gradient methods for multiobjective composite optimization problems with H{\"o}lder condition

In this paper, we deal with multiobjective composite optimization problems, where each objective function is a combination of smooth and possibly non-smooth functions. We first propose a parameter-dependent conditional gradient method to solve this problem. The step size in this method requires prior knowledge of the parameters related to the H{\"o}lder continuity of the gradient of the smooth function. The convergence properties of this method are then established. Given that these parameters may be unknown or, if known, may not be unique, the first method may encounter implementation challenges or slow convergence. To address this, we further propose a parameter-free conditional gradient method that determines the step size using a local quadratic upper approximation and an adaptive line search strategy, eliminating the need for any problem-specific parameters. The performance of the proposed methods is demonstrated on several test problems involving the indicator function and an uncertainty function.

math.OC↗

arXiv2024

Mirror descent method for stochastic multi-objective optimization

Stochastic multi-objective optimization (SMOO) has recently emerged as a powerful framework for addressing machine learning problems with multiple objectives. The bias introduced by the nonlinearity of the subproblem solution mapping complicates the convergence analysis of multi-gradient methods. In this paper, we propose a novel SMOO method called the Multi-gradient Stochastic Mirror Descent (MSMD) method, which incorporates stochastic mirror descent method to solve the SMOO subproblem, providing convergence guarantees. By selecting an appropriate Bregman function, our method enables analytical solutions of the weighting vector and requires only a single gradient sample at each iteration. We demonstrate the sublinear convergence rate of our MSMD method under four different inner and outer step setups. For SMOO with preferences, we propose a variant of MSMD method and demonstrate its convergence rate. Through extensive numerical experiments, we compare our method with both stochastic descent methods based on weighted sum and state-of-the-art SMOO methods. Our method consistently outperforms these methods in terms of generating superior Pareto fronts on benchmark test functions while also achieving competitive results in neural network training.

math.OC↗

arXiv2024

First-order Methods for Unconstrained Vector Optimization Problems: A Unified Majorization-Minimization Perspective

In this paper, we develop a unified majorization-minimization scheme and convergence analysis with first-order surrogate functions for unconstrained vector optimization problems (VOPs). By selecting different surrogate functions, the unified method can be reduced to various existing first-order methods. The unified convergence analysis reveals that the slow convergence of the steepest descent method is primarily attributed to the significant gap between the surrogate and objective functions. Consequently, narrowing this surrogate gap can enhance the performance of first-order methods for VOPs. To strike a better trade-off in terms of surrogate gap and per-iteration cost, we reformulate the direction-finding subproblem and elucidate that selecting a tighter surrogate function is equivalent to using an appropriate base of the dual cone in the direction-finding subproblem. Building on this insight, we employ the Barzilai-Borwein method to narrow the surrogate gap and propose a Barzilai-Borwein descent method for VOPs (BBDVO) with polyhedral cones. By reformulating the corresponding subproblem, we provide a novel perspective on the Barzilai-Borwein descent method, bridging the gap between this method and the steepest descent method. Finally, several numerical experiments are presented to validate the efficiency of the BBDVO.

math.OC↗

arXiv2024

Gradient-based algorithms for multi-objective bi-level optimization

Multi-Objective Bi-Level Optimization (MOBLO) addresses nested multi-objective optimization problems common in a range of applications. However, its multi-objective and hierarchical bilevel nature makes it notably complex. Gradient-based MOBLO algorithms have recently grown in popularity, as they effectively solve crucial machine learning problems like meta-learning, neural architecture search, and reinforcement learning. Unfortunately, these algorithms depend on solving a sequence of approximation subproblems with high accuracy, resulting in adverse time and memory complexity that lowers their numerical efficiency. To address this issue, we propose a gradient-based algorithm for MOBLO, called gMOBA, which has fewer hyperparameters to tune, making it both simple and efficient. Additionally, we demonstrate the theoretical validity by accomplishing the desirable Pareto stationarity. Numerical experiments confirm the practical efficiency of the proposed method and verify the theoretical results. To accelerate the convergence of gMOBA, we introduce a beneficial L2O neural network (called L2O-gMOBA) implemented as the initialization phase of our gMOBA algorithm. Comparative results of numerical experiments are presented to illustrate the performance of L2O-gMOBA.

math.OC↗

arXiv2024

Level proximal subdifferential, variational convexity, and pointwise quadratic approximation

Level proximal subdifferential was introduced by Rockafellar recently for studying proximal mappings of possibly nonconvex functions. In this paper a systematic study of level proximal subdifferential is given. We characterize variational convexity of a function by local firm nonexpansiveness of proximal mappings or local relative monotonicity of level proximal subdifferential, and use them to study local convergence of proximal gradient method and others for variationally convex functions. Variational sufficiency guarantees that proximal gradient method converges to local minimizers rather than just critical points. We also investigate the existence, single-valuedness and integration of level proximal subdifferential, and quantify pointwise quadratic approximation (or Lipschitz smoothness) of a function. As a powerful tool, level proximal subdifferential provides deep insights into variational analysis and optimization.

math.OC↗

arXiv2024

Last-iterate convergence of modified predictive method via high-resolution differential equation on bilinear game

This paper discusses the convergence of the modified predictive method (MPM) proposed by Liang and stokes corresponding to high-resolution differential equations (HRDE) in bilinear games. First, we present the high-resolution differential equations (MPM-HRDE) corresponding to the MPM. Then, we discuss the uniqueness of the solution for MPM-HRDE in bilinear games. Finally, we provide the convergence results of MPM-HRDE in bilinear games. The results obtained in this paper address the gap in the existing literature and extend the conclusions of related works.

math.OC↗

arXiv2024

A Branch and Bound Algorithm for Multiobjective Optimization Problems Using General Ordering Cones

Many existing branch and bound algorithms for multiobjective optimization problems require a significant computational cost to approximate the entire Pareto optimal solution set. In this paper, we propose a new branch and bound algorithm that approximates a part of the Pareto optimal solution set by introducing the additional preference information in the form of ordering cones. The basic idea is to replace the Pareto dominance induced by the nonnegative orthant with the cone dominance induced by a larger ordering cone in the discarding test. In particular, we consider both polyhedral and non-polyhedral cones, and propose the corresponding cone dominance-based discarding tests, respectively. In this way, the subboxes that do not contain efficient solutions with respect to the ordering cone will be removed, even though they may contain Pareto optimal solutions. We prove the global convergence of the proposed algorithm. Finally, the proposed algorithm is applied to a number of test instances as well as to 2- to 5-objective real-world constrained problems.

math.OC↗

arXiv2024

On the convergence of conditional gradient method for unbounded multiobjective optimization problems

This paper focuses on developing a conditional gradient algorithm for multiobjective optimization problems with an unbounded feasible region. We employ the concept of recession cone to establish the well-defined nature of the algorithm. The asymptotic convergence property and the iteration-complexity bound are established under mild assumptions. Numerical examples are provided to verify the algorithmic performance.

math.OC↗

arXiv2024

Obtaining properly Pareto optimal solutions of multiobjective optimization problems via the branch and bound method

In multiobjective optimization, most branch and bound algorithms provide the decision maker with the whole Pareto front, and then decision maker could select a single solution finally. However, if the number of objectives is large, the number of candidate solutions may be also large, and it may be difficult for the decision maker to select the most interesting solution. As we argue in this paper, the most interesting solutions are the ones whose trade-offs are bounded. These solutions are usually known as the properly Pareto optimal solutions. We propose a branch-and-bound-based algorithm to provide the decision maker with so-called $\epsilon$-properly Pareto optimal solutions. The discarding test of the algorithm adopts a dominance relation induced by a convex polyhedral cone instead of the common used Pareto dominance relation. In this way, the proposed algorithm excludes the subboxes which do not contain $\epsilon$-properly Pareto optimal solution from further exploration. We establish the global convergence results of the proposed algorithm. Finally, the algorithm is applied to benchmark problems as well as to two real-world optimization problems.

math.OC↗

arXiv2024

Improvements to steepest descent method for multi-objective optimization

In this paper, we propose a simple yet efficient strategy for improving the multi-objective steepest descent method proposed by Fliege and Svaiter (Math Methods Oper Res, 2000, 3: 479--494). The core idea behind this strategy involves incorporating a positive modification parameter into the iterative formulation of the multi-objective steepest descent algorithm in a multiplicative manner. This modification parameter captures certain second-order information associated with the objective functions. We provide two distinct methods for calculating this modification parameter, leading to the development of two improved multi-objective steepest descent algorithms tailored for solving multi-objective optimization problems. Under reasonable assumptions, we demonstrate the convergence of sequences generated by the first algorithm toward a critical point. Moreover, for strongly convex multi-objective optimization problems, we establish the linear convergence to Pareto optimality of the sequence of generated points. The performance of the new algorithms is empirically evaluated through a computational comparison on a set of multi-objective test instances. The numerical results underscore that the proposed algorithms consistently outperform the original multi-objective steepest descent algorithm.

math.OC↗

arXiv2023

The convergence rate of the accelerated proximal gradient algorithm for Multiobjective Optimization is faster than $O(1/k^2)$

In this paper, we propose a fast proximal gradient algorithm for multiobjective optimization, it is proved that the convergence rate of the accelerated algorithm for multiobjective optimization developed by Tanabe et al. can be improved from $O(1/k^2)$ to $o(1/k^2)$ by introducing different extrapolation term $\frac{k-1}{k+α-1}$ with $α>3$. Further, we establish the inexact version of the proposed algorithm when the error term is additive, which owns the same convergence rate. At last, the efficiency of the proposed algorithm is verified on some numerical experiments.

math.OC↗