Equilibrate Parametrization: Optimal Metric Selection with Provable One-iteration Convergence for $ l_1 $-minimization

Abstract

Incorporating a non-Euclidean variable metric to first-order algorithms is known to bring enhancement. However, due to the lack of an optimal choice, such an enhancement appears significantly underestimated. In this work, we establish a metric selection principle via optimizing a convergence rate upper-bound. For general l1-minimization, we propose an optimal metric choice with closed-form expressions guaranteed. Equipping such a variable metric, we prove that the optimal solution to the l1 problem will be obtained via a one-time proximal operator evaluation. Our technique applies to a large class of fixed-point algorithms, particularly the ADMM, which is popular, general, and requires minimum assumptions. The key to our success is the employment of an unscaled/equilibrate upper-bound. We show that there exists an implicit scaling that poses a hidden obstacle to optimizing parameters. This turns out to be a fundamental issue induced by the classical parametrization. We note that the conventional way always associates the parameter to the range of a function/operator. This turns out not a natural way, causing certain symmetry losses, definition inconsistencies, and unnecessary complications, with the well-known Moreau identity being the best example. We propose equilibrate parametrization, which associates the parameter to the domain of a function, and to both the domain and range of a monotone operator. A series of powerful results are obtained owing to the new parametrization. Quite remarkably, the preconditioning technique can be shown as equivalent to the metric selection issue.