arXiv subjects

Yuyang Qiu

Publications and source records attributed to Yuyang Qiu.

4 recordsLinked to original sources

arXiv2026

Self-tuned Regularized Federated Methods with Guarantees for Optimal Solution Selection

We study a hierarchical federated learning (FL) problem, where clients cooperatively seek to select among multiple optimal solutions of a primary distributed learning problem, a solution that minimizes a secondary loss function. This problem arises from over-parameterized learning and ill-posed optimization problems. First, we consider the setting where the inner-level objective is convex and the outer-level objective is either convex or strongly convex. We propose a self-tuned regularized federated averaging (StR-FedAvg) method where the stepsize and regularization parameter are characterized by the number of communication rounds and problem parameters. We derive new complexity guarantees for addressing the optimal solution selection problem in FL. Second, when the outer-level objective is nonconvex, we propose a two-loop FL scheme in which the outer loop employs an inexact projected first-order method and the inner loop applies StR-FedAvg with an iteratively updated regularization parameter. We derive new communication complexity guarantees for computing a stationary point of the nonconvex solution-selection problem. To our knowledge, this is the first work to establish complexity guarantees for this class of problems in FL. Preliminary experiments validate our theoretical findings.

math.OC↗

arXiv2025

A Randomized Zeroth-Order Hierarchical Framework for Heterogeneous Federated Learning

Heterogeneity in federated learning (FL) is a critical and challenging aspect that significantly impacts model performance and convergence. In this paper, we propose a novel framework by formulating heterogeneous FL as a hierarchical optimization problem. This new framework captures both local and global training processes through a bilevel formulation and is capable of the following: (i) addressing client heterogeneity through a personalized learning framework; (ii) capturing the pre-training process on the server side; (iii) updating the global model through nonstandard aggregation; (iv) allowing for nonidentical local steps; and (v) capturing clients' local constraints. We design and analyze an implicit zeroth-order FL method (ZO-HFL), equipped with nonasymptotic convergence guarantees for both the server-agent and the individual client-agents, and asymptotic guarantees for both the server-agent and client-agents in an almost sure sense. Notably, our method does not rely on standard assumptions in heterogeneous FL, such as the bounded gradient dissimilarity condition. We implement our method on image classification tasks and compare with other methods under different heterogeneous settings.

math.OC↗

arXiv2025

Zeroth-Order Federated Methods for Stochastic MPECs and Nondifferentiable Nonconvex Hierarchical Optimization

Motivated by the emergence of federated learning (FL), we design and analyze federated methods for addressing: (i) Nondifferentiable nonconvex optimization; (ii) Bilevel optimization; (iii) Minimax problems; and (iv) Two-stage stochastic mathematical programs with equilibrium constraints (2s-SMPEC). Research on these problems has been limited and afflicted by reliance on strong assumptions, including the need for differentiability of the implicit function and the absence of constraints in the lower-level problem, among others. We make the following contributions. In (i), by leveraging convolution-based smoothing and Clarke's subdifferential calculus, we devise a randomized smoothing-enabled zeroth-order FL method and derive communication and iteration complexity guarantees for computing an approximate Clarke stationary point. To contend with (ii) and (iii), we devise a unifying randomized implicit zeroth-order FL framework, equipped with explicit communication and iteration complexities. Importantly, our method utilizes delays during local steps to skip calls to the inexact lower-level FL oracle. This results in significant reduction in communication overhead. In (iv), we devise an inexact implicit variant of the method in (i). Remarkably, this method achieves a total communication complexity matching that of single-level nonsmooth nonconvex optimization in FL. We empirically validate the theoretical findings on instances of federated nonsmooth and hierarchical problems.

math.OC↗

arXiv2024

Iteratively Regularized Gradient Tracking Methods for Optimal Equilibrium Seeking

In noncooperative Nash games, equilibria are often inefficient. This is exemplified by the Prisoner's Dilemma and was first provably shown in the 1980s. Since then, understanding the quality of Nash equilibrium (NE) received considerable attention, leading to the emergence of inefficiency measures characterized by the best or the worst equilibrium. Traditionally, computing an optimal NE in monotone regimes is done through two-loop schemes which lack scalability and provable performance guarantees. The goal in this work lies in the development of among the first single-timescale distributed gradient tracking optimization methods for optimal NE seeking over networks. Our main contributions are as follows. By employing a regularization-based relaxation approach within two existing distributed gradient tracking methods, namely Push-Pull and DSGT, we devise and analyze two single-timescale iteratively regularized gradient tracking algorithms. The first method addresses computing the optimal NE over directed networks, while the second method addresses a stochastic variant of this problem over undirected networks. For both methods, we establish the convergence to the optimal NE and derive new convergence rate statements for the consensus error of the generated iterates. We provide preliminary numerical results on a Nash-Cournot game.

math.OC↗