PLADOS: A Parameter-free Landing Algorithm for Decentralized Optimization on the Stiefel Manifold

Abstract

Decentralized optimization on the Stiefel manifold has broad applications in machine learning and signal processing. We propose PLADOS, a parameter-free landing algorithm for decentralized optimization on the Stiefel manifold. Each iteration combines local stochastic gradient updates with a single retraction-free communication step over the communication graph. A key convex-like property of the intersection of the nonconvex Stiefel manifold and consensus constraints is characterized through a restricted secant inequality for the penalty-only scheme, ensuring local contraction of the manifold consensus error without introducing additional parameters that require tuning. For $n$ nodes using independent local batches of size $b$, we prove that PLADOS achieves linear speedup with respect to the number of nodes with an asymptotic convergence rate of $O(σ/\sqrt{nbK}+(nξ^2)^{1/3}/K^{2/3})$ after $K$ iterations, where $σ^2>0$ is the sampling variance and $ξ^2$ quantifies gradient heterogeneity. Experiments demonstrate stability across tested stepsizes and networks, time efficiency, and linear speedup with respect to the number of nodes.