Accelerated Consensus and Decentralized Optimization over Slowly Varying Networks

Abstract

We study average consensus over slowly time-varying networks and its application to decentralized optimization. We introduce WAVE, a windowed Chebyshev method that limits the accumulated effect of network variation by restarting the recurrence after finite windows. If $χ$ bounds the network condition number and successive communication operators satisfy $\|L_{k+1}-L_k\|\leqβ$, WAVE reaches $ε$-consensus in $O((\sqrtχ+\min\lbraceβχ^2,χ\rbrace)\ln(e/ε))$ communication rounds. This rate matches the fixed-network $\sqrtχ$ dependence when $β=O(χ^{-3/2})$ and smoothly interpolates across the full range of network variation. For piecewise-constant networks where each change is detected when it occurs and consecutive changes are at least $τ$ rounds apart, WAVE reaches $ε$-consensus in $O((\sqrtχ+χ/τ)\ln(e/ε))$ communication rounds. Finally, we use WAVE as the consensus step in an accelerated decentralized optimization method for $α$-smooth convex local objectives with a $μ$-strongly convex average. For sufficiently slow network variation, every agent obtains an $ε$-solution to the global optimization problem after $\widetilde O(\sqrt{α/μ}\,\sqrtχ)$ communication rounds, matching the fixed-network dependence.