arXiv subjects

Zai-Yun Peng

Publications and source records attributed to Zai-Yun Peng.

6 recordsLinked to original sources

arXiv2026

A Momentum-Based Variance-Reduced Algorithm for Federated Multiobjective Optimization

Federated learning has traditionally been formulated as a single-objective optimization problem, primarily focused on maximizing model utility. In real-world applications, however, machine learning models often need to optimize multiple and potentially conflicting objectives simultaneously. This motivates federated multiobjective optimization (FMOO), which provides a natural framework for jointly handling multiple task-specific objectives in federated learning. In this paper, we propose a momentum-based variance-reduced algorithm for federated multiobjective optimization. The method incorporates a momentum-driven gradient estimator into the local updates to reduce the variance of stochastic updates, leading to an improved convergence rate. We establish theoretical guarantees showing that the expected Pareto stationarity measure of a randomly selected output iterate decays at a rate of $\mathcal{O}(T^{-2/3})$, improving upon the $\mathcal{O}(T^{-1/2})$ rates established for existing methods such as FSMGDA and FedCMOO. Numerical experiments on federated multiobjective optimization benchmarks demonstrate the effectiveness and competitive performance of the proposed algorithm.

cs.LG↗

arXiv2026

Hager-Zhang Conjugate Gradient Method for Set Optimization with Set-Valued Objective Map of Finite Cardinality

This work introduces a nonlinear Hager-Zhang conjugate gradient method for solving set optimization problems. The objective function under consideration is defined by a finite collection of continuously differentiable functions. Notably, the proposed approach imposes restrictions neither on the existence of a finite generator of the ordering cone nor on any regularity condition at the optimal solution. As a result, the proposed method holds considerable significance for both set optimization and vector optimization problems, with the latter serving as a special case of the former. The study begins by discussing Wolfe line search conditions using Drummond-Svaiter scalarization function. Thereafter, we establish the existence of a step length satisfying the Wolfe line search conditions along a descent direction. The Hager-Zhang scalar conjugate parameter is introduced to derive the search direction for the proposed method. It is established that the direction generated by the proposed method is a descent direction. The well-definedness of the proposed method is given. Furthermore, we discuss some important results and a Zoutendijk-like condition to ensure global convergence. Subsequently, the global convergence of the proposed method is established in an asymptotic manner. Finally, numerical experiments on various test problems validate the practical performance and effectiveness of the proposed technique.

math.OC↗

arXiv2026

PRP, HS and LS Conjugate Gradient Methods for Interval-Valued Multiobjective Optimization Problems

In this article, we develop an efficient algorithm based on three special variants of the nonlinear conjugate gradient method, namely, the Polak--Ribiere--Polyak, Hestenes--Stiefel, and Liu--Story schemes for computing Pareto critical points in unconstrained interval-valued multiobjective optimization problems. The proposed algorithm incorporates a Wolfe line search strategy to determine a suitable range of step size that satisfies the standard Wolfe conditions. For each of the proposed variants of the nonlinear conjugate gradient method, we establish rigorous global convergence results under appropriate assumptions. To demonstrate the effectiveness of the proposed methods, we conduct numerical experiments on a set of benchmark test problems and present a comprehensive performance profile analysis.

math.OC↗

arXiv2025

Variable Smoothing Alternating Proximal Gradient Algorithm for Coupled Composite Optimization

In this paper, we consider a broad class of nonconvex and nonsmooth optimization problems, where one objective component is a nonsmooth weakly convex function composed with a linear operator. By integrating variable smoothing techniques with first-order methods, we propose a variable smoothing alternating proximal gradient algorithm that features flexible parameter choices for step sizes and smoothing levels. Under mild assumptions, we establish that the iteration complexity to reach an $\varepsilon$-approximate stationary point is $\mathcal{O}(\varepsilon^{-3})$. The proposed algorithm is evaluated on sparse signal recovery and image denoising problems. Numerical experiments demonstrate its effectiveness and superiority over existing algorithms.

math.OC↗

arXiv2025

Nonmonotone Trust-Region Methods for Optimization of Set-Valued Mapping of Finite Cardinality

Non-monotone trust-region methods are known to provide additional benefits for scalar and multi-objective optimization, such as enhancing the probability of convergence and improving the speed of convergence. For optimization of set-valued maps, non-monotone trust-region methods have not yet been explored and investigated to see if they show similar benefits. Thus, in this article, we propose two non-monotone trust-region schemes--max-type and average-type for set-valued optimization. Using these methods, the aim is to find \emph{K}-critical points for a non-convex unconstrained set optimization problem through vectorization and oriented-distance scalarization. The main modification in the existing trust region method for set optimization occurs in reduction ratios, where max-type uses the maximum over function values from the last few iterations, and avg-type uses an exponentially weighted moving average of successive previous function values till the current iteration. Under appropriate assumptions, we show the global convergence of the proposed methods. To verify their effectiveness, we numerically compare their performance with the existing trust region method, steepest descent method, and conjugate gradient method using performance profile in terms of three metrics: number of non-convergence, number of iterations, and computation time.

math.OC↗

arXiv2024

Nonlinear Conjugate Gradient Methods for Optimization of Set-Valued Mappings of Finite Cardinality

This article presents nonlinear conjugate gradient methods for finding local weakly minimal points of set-valued optimization problems under a lower set less ordering relation. The set-valued objective function of the optimization problem under consideration is defined by finitely many continuously differentiable vector-valued functions. For such optimization problems, at first, we propose a general scheme for nonlinear conjugate gradient methods and then introduce Dai-Yuan, Polak-Ribi{è}re-Polyak, and Hestenes-Stiefel conjugate gradient parameters for set-valued functions. Toward deriving the general scheme, we introduce a condition of sufficient decrease and Wolfe line searches for set-valued functions. For a given sequence of descent directions of a set-valued function, it is found that if the proposed standard Wolfe line search technique is employed, then the generated sequence of iterates for set optimization follows a Zoutendijk-like condition. With the help of the derived Zoutendijk-like condition, we report that all the proposed nonlinear conjugate gradient schemes are globally convergent under usual assumptions. It is important to note that the ordering cone used in the entire study is not restricted to be finitely generated, and no regularity assumption on the solution set of the problem is required for any of the reported convergence analyses. Finally, we demonstrate the performance of the proposed methods through numerical experiments. In the numerical experiments, we demonstrate the effectiveness of the proposed methods not only on the commonly used test instances for set optimization but also on a few newly introduced problems under general ordering cones that are neither nonnegative hyper-octant nor finitely generated.

math.OC↗