A Robust Compressed Push-Pull Method for Decentralized Nonconvex Optimization

Abstract

In the modern paradigm of multi-agent networks, communication has become one of the main bottlenecks for decentralized optimization, where a large number of agents are involved in minimizing the average of the local cost functions. In this paper, we propose a robust compressed push-pull algorithm (RCPP) that combines gradient tracking with communication compression. In particular, RCPP is robust under a much more general class of compression operators that allow both relative and absolute compression errors, in contrast to the existing works which can handle either one of them or assume convex problems. We show that RCPP enjoys sublinear convergence rate for smooth and possibly nonconvex objective functions over general directed networks. Moreover, under the additional Polyak-{\L}ojasiewicz condition, linear convergence rate can be achieved for RCPP. Numerical examples verify the theoretical findings and demonstrate the efficiency, flexibility, and robustness of the proposed algorithm.