arXiv subjects

Runze You

Publications and source records attributed to Runze You.

5 recordsLinked to original sources

arXiv2025

FedSUM Family: Efficient Federated Learning Methods under Arbitrary Client Participation

Federated Learning (FL) methods are often designed for specific client participation patterns, limiting their applicability in practical deployments. We introduce the FedSUM family of algorithms, which supports arbitrary client participation without additional assumptions on data heterogeneity. Our framework models participation variability with two delay metrics, the maximum delay $\tau_{\max}$ and the average delay $\tau_{\text{avg}}$. The FedSUM family comprises three variants: FedSUM-B (basic version), FedSUM (standard version), and FedSUM-CR (communication-reduced version). We provide unified convergence guarantees demonstrating the effectiveness of our approach across diverse participation patterns, thereby broadening the applicability of FL in real-world scenarios.

cs.LG↗

arXiv2025

Stochastic Push-Pull for Decentralized Nonconvex Optimization

To understand the convergence behavior of the Push-Pull method for decentralized optimization with stochastic gradients (Stochastic Push-Pull), this paper presents a comprehensive analysis. Specifically, we first clarify the algorithm's underlying assumptions, particularly those regarding the network structure and weight matrices. Then, to establish the convergence rate under smooth nonconvex objectives, we introduce a general analytical framework that not only encompasses a broad class of decentralized optimization algorithms, but also recovers or enhances several state-of-the-art results for distributed stochastic gradient tracking methods. A key highlight is the derivation of a sufficient condition under which the Stochastic Push-Pull algorithm achieves linear speedup, matching the scalability of centralized stochastic gradient methods -- a result not previously reported. Extensive numerical experiments validate our theoretical findings, demonstrating the algorithm's effectiveness and robustness across various decentralized optimization scenarios.

math.OC↗

arXiv2025

Decentralized Min-Max Optimization with Gradient Tracking

This paper presents a novel distributed formulation of the min-max optimization problem. Such a formulation enables enhanced flexibility among agents when optimizing their maximization variables. To address the problem, we propose two distributed gradient methods over networks, termed Distributed Gradient Tracking Ascent (DGTA) and Distributed Stochastic Gradient Tracking Ascent (DSGTA). We demonstrate that DGTA achieves an iteration complexity of $\mathcal{O}(\kappa^2\varepsilon^{-2})$, and DSGTA attains a sample complexity of $\mathcal{O}(\kappa^3\varepsilon^{-4})$ for nonconvex strongly concave (NC-SC) objective functions. Both results match those of their centralized counterparts up to constant factors related to the communication network. Numerical experiments further demonstrate the superior empirical performance of the proposed algorithms compared to existing methods.

math.OC↗

arXiv2025

Distributed Learning over Arbitrary Topology: Linear Speed-Up with Polynomial Transient Time

We study a distributed learning problem in which $n$ agents, each with potentially heterogeneous local data, collaboratively minimize the sum of their local cost functions via peer-to-peer communication. We propose a novel algorithm, \emph{Spanning Tree Push-Pull} (STPP), which employs two spanning trees extracted from a general communication graph to distribute both model parameters and stochastic gradients. Unlike prior approaches that rely heavily on spectral gap properties, STPP leverages a more flexible topological characterization, enabling robust information flow and efficient updates. Theoretically, we prove that STPP achieves linear speedup and polynomial transient iteration complexity -- up to $\mathcal{O}(n^7)$ for smooth nonconvex objectives and $\tilde{\mathcal{O}}(n^3)$ for smooth strongly convex objectives -- under arbitrary network topologies. Moreover, compared with existing methods, STPP achieves faster convergence rates on sparse and non-regular topologies (e.g., directed rings) and reduces communication overhead on dense networks (e.g., static exponential graphs). Numerical experiments further demonstrate the strong performance of STPP across various graph architectures.

math.OC↗

arXiv2024

B-ary Tree Push-Pull Method is Provably Efficient for Distributed Learning on Heterogeneous Data

This paper considers the distributed learning problem where a group of agents cooperatively minimizes the summation of their local cost functions based on peer-to-peer communication. Particularly, we propose a highly efficient algorithm, termed ``B-ary Tree Push-Pull'' (BTPP), that employs two B-ary spanning trees for distributing the information related to the parameters and stochastic gradients across the network. The simple method is efficient in communication since each agent interacts with at most $(B+1)$ neighbors per iteration. More importantly, BTPP achieves linear speedup for smooth nonconvex and strongly convex objective functions with only $\tilde{O}(n)$ and $\tilde{O}(1)$ transient iterations, respectively, significantly outperforming the state-of-the-art results to the best of our knowledge. Our code is available at https://github.com/ryou98/BTPP.

math.OC↗