Searcharxiv⌕ Search
Home/Archive/arXiv

arXiv · 2610.08508

Budget-Constrained Multi-Consensus Decentralized Gradient Descent

Shuyi Ren·Nicol`o Michelusi·Erik G. Larsson
View original source ↗Read source document ↗Export metadata ↓

Abstract

We investigate decentralized gradient descent (DGD) with emphasis on efficient communication and computation resource utilization under budget constraints. As a first step toward the broader communication-computation allocation problem, we consider and analyze a \textit{multi-consensus decentralized gradient descent} (mcDGD) scheme, where the number of consensus rounds and the stepsize are allowed to vary across iterations. Building on a unified analytical framework for DGD, we derive finite-time convergence bounds that explicitly characterize the interaction between consensus quality and optimization dynamics. Our analysis requires only convexity of the local objective functions while assuming smoothness and strong convexity of the global objective. The resulting bounds enable a principled consensus-allocation strategy under resource constraints, for which we show that equal allocation of consensus rounds across iterations is optimal under our stepsize rule, up to integer rounding. Numerical experiments corroborate the theoretical findings and demonstrate favorable communication-computation tradeoffs compared with existing multi-consensus decentralized optimization baselines.

Explore related subjects

math.OCcs.SYeess.SY

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Shuyi Ren, Nicol`o Michelusi, Erik G. Larsson. 2026-10-06. Budget-Constrained Multi-Consensus Decentralized Gradient Descent. https://arxiv.org/abs/2610.08508

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Multilevel Regularized Newton Methods with Fast Convergence Rates

We introduce new multilevel methods for solving large-scale unconstrained optimization problems. Specifically, the philosophy of multilevel methods is applied to Newton-type methods that regularize the Newton sub-problem using second order information from a coarse (low dimensional) sub-problem. The new \emph{regularized multilevel methods} provably converge from any initialization point and enjoy faster convergence rates than Gradient Descent. In particular, for arbitrary functions with Lipschitz continuous Hessians, we show that their convergence rate interpolates between the rate of Gradient Descent and that of the cubic Newton method. If, additionally, the objective function is assumed to be convex, then the proposed method converges with the fast $\mathcal{O}(k^{-2})$ rate. Hence, since the updates are generated using a \emph{coarse} model in low dimensions, the theoretical results of this paper significantly speed-up the convergence of Newton-type or preconditioned gradient methods in practical applications. Preliminary numerical results suggest that the proposed multilevel algorithms are significantly faster than current state-of-the-art methods.

math.OC↗
arXiv2026

Adaptive Algorithms for Robust Phase Retrieval

This paper considers robust phase retrieval, which can be cast as a nonsmooth and nonconvex optimization problem. We propose two first-order algorithms with adaptive step sizes: the subgradient algorithm (AdaSubGrad) and the inexact proximal linear algorithm (AdaIPL). Our contribution lies in a novel design of adaptive step sizes based on quantiles of the absolute residuals. We analyze local linear convergence of both algorithms across different hyperparameter regimes under i.i.d. centered sub-Gaussian measurements, a stability condition linking the measurement distribution and the corruption level, and an additional uniform small-ball condition. Numerical experiments on synthetic datasets and image recovery also demonstrate that our methods are competitive with existing methods in the literature that utilize predetermined (possibly impractical) step sizes, such as subgradient methods and the inexact proximal linear method.

math.OC↗
arXiv2026

A Simple yet Highly Accurate Prediction-Correction Algorithm for Time-Varying Optimization

This paper proposes a simple yet highly accurate prediction-correction algorithm, SHARP, for unconstrained time-varying optimization problems. Its prediction is based on an extrapolation derived from the Lagrange interpolation of past solutions. Since this extrapolation can be computed without Hessian matrices or even gradients, the computational cost is low. To ensure the stability of the prediction, the algorithm includes an acceptance condition that rejects the prediction when the update is excessively large. The proposed method achieves a tracking error of $O(h^{p})$, where $h$ is the sampling period, assuming that the $p$-th derivative of the target trajectory is bounded and the convergence of the correction step is locally linear. We also prove that the method can track a trajectory of stationary points even if the objective function is non-convex. Numerical experiments demonstrate the high accuracy of the proposed algorithm.

math.OC↗