Algorithm duality from resolvent Moreau identities

Abstract

Duality is classically defined as a relationship between objective functions, one that breaks down beyond convexity. We show that a more fundamental duality persists between the resolvent operators used to solve associated problems. We establish a framework from which we obtain complete characterizations of the duality-like correspondences between proximal fixed-point iterations and their alternating minimization scheme counterparts. Namely, we show the known primal-dual relationships between forward--backward splitting and Tseng's alternating minimization algorithm, and between the Douglas-Rachford method and alternating direction method of multipliers, to hold generically and irrespective of problem assumptions. The approach considered herein avoids the machinery of convex analysis entirely, providing a definitive formulation of algorithmic dualities in maximal generality while greatly simplifying their proofs. In particular, assumptions such as monotonicity, weak monotonicity, Lipschitz continuity, and injectivity of linear operators are all shown to be unnecessary. In addition, we utilize our framework to reveal and analyze a novel primal-dual relationship between backward-backward splitting and a special instance of the alternating minimization algorithm. This illustrates how our duality framework provides a principled basis for primal-dual algorithm design and analysis in nonmonotone settings. Finally, our experiments show that our backward--backward splitting--based approach reliably outperforms the FISTA and SR3 algorithms for the LASSO problem.