Linear convergence of dual coordinate descent on non-polyhedral convex problems

Abstract

This paper deals with constrained convex problems, where the objective function is smooth strongly convex and the feasible set is given as the intersection of a large number of closed convex (possibly non-polyhedral) sets. In order to deal efficiently with the complicated constraints we consider a dual formulation of this problem. We prove that the corresponding dual function satisfies a quadratic growth property on any sublevel set, provided that the objective function is smooth and strongly convex and the sets verify the Slater's condition. To the best of our knowledge, this work is the first deriving a quadratic growth condition for the dual under these general assumptions. Existing works derive similar quadratic growth conditions under more conservative assumptions, e.g., the sets need to be either polyhedral or compact. Then, for finding the minimum of the dual problem, due to its special composite structure, we propose random (accelerated) coordinate descent algorithms. However, with the existing theory one can prove that such methods converge only sublinearly. Based on our new quadratic growth property derived for the dual, we now show that such methods have faster convergence, that is the dual random (accelerated) coordinate descent algorithms converge linearly. Besides providing a general dual framework for the analysis of randomized coordinate descent schemes, our results resolve an open problem in the literature related to the convergence of Dykstra algorithm on the best feasibility problem for a collection of convex sets. That is, we establish linear convergence rate for the randomized Dykstra algorithm when the convex sets satisfy the Slater's condition and derive also a new accelerated variant for the Dykstra algorithm.