arXiv subjects

Yifan Ran

Publications and source records attributed to Yifan Ran.

4 recordsLinked to original sources

arXiv2024

Towards Scalable Semidefinite Programming: Optimal Metric ADMM with A Worst-case Performance Guarantee

Despite the numerous uses of semidefinite programming (SDP) and its universal solvability via interior point methods (IPMs), it is rarely applied to practical large-scale problems. This mainly owes to the computational cost of IPMs that increases in a bad exponential way with the data size. While first-order algorithms such as ADMM can alleviate this issue, but the scalability improvement appears far not enough. In this work, we aim to achieve extra acceleration for ADMM by appealing to a non-Euclidean metric space, while maintaining everything in closed-form expressions. The efficiency gain comes from the extra degrees of freedom of a variable metric compared to a scalar step-size, which allows us to capture some additional ill-conditioning structures. On the application side, we consider the quadratically constrained quadratic program (QCQP), which naturally appears in an SDP form after a dualization procedure. This technique, known as semidefinite relaxation, has important uses across different fields, particularly in wireless communications. Numerically, we observe that the scalability property is significantly improved. Depending on the data generation process, the extra acceleration can easily surpass the scalar-parameter efficiency limit, and the advantage is rapidly increasing as the data conditioning becomes worse.

math.OC↗

arXiv2024

General Optimal Step-size for ADMM-type Algorithms: Domain Parametrization and Optimal Rates

In this work, we solve a 49-year open problem, the general optimal step-size for ADMM-type algorithms. For a convex program: $\text{min.} \,\, f({x}) + g({z})$, $\text{s.t.}\, {A}{x} - {B}{z} = {c} $, given an arbitrary fixed-point initialization $ ζ^0 $, an optimal step-size choice is given by a root of the following polynomial: \begin{equation*} ρ^4\Vert {A}{x}^\star\Vert^2 - ρ^3\langle {A}{x}^\star, ζ^0\rangle + ρ\langle λ^\star,ζ^0\rangle - \Vertλ^\star\Vert^2 = 0, \end{equation*} with $ ρ\neq 0 $ a domain step-size, which relates to the classical positive one via $ γ= ρ^2$. We denote by $ \cdot^\star $ the optimal solution, by $ λ $ the Lagrange multiplier associated with the equality constraint (dual variable). The above polynomial always admits a closed-form solution. The optimality is in the sense that a worst-case fixed-point convergence rate is minimized, which is a balance of the normalized primal and dual iterates convergence speed (reciprocally related). In cases where either the primal or dual solution is trivial (a zero vector), improvement can be made by accelerating the non-trivial sequence only. For practical use, adaptively replace the above optimal solutions with the current iterates, which are known at every iteration. Numerically, it exhibits almost identical performance as the theoretical one (after a few iterations), similar to the underlying best fixed step-size (found by exhaustive grid search).

math.OC↗

arXiv2023

Equilibrate Parametrization: Optimal Metric Selection with Provable One-iteration Convergence for $ l_1 $-minimization

Incorporating a non-Euclidean variable metric to first-order algorithms is known to bring enhancement. However, due to the lack of an optimal choice, such an enhancement appears significantly underestimated. In this work, we establish a metric selection principle via optimizing a convergence rate upper-bound. For general l1-minimization, we propose an optimal metric choice with closed-form expressions guaranteed. Equipping such a variable metric, we prove that the optimal solution to the l1 problem will be obtained via a one-time proximal operator evaluation. Our technique applies to a large class of fixed-point algorithms, particularly the ADMM, which is popular, general, and requires minimum assumptions. The key to our success is the employment of an unscaled/equilibrate upper-bound. We show that there exists an implicit scaling that poses a hidden obstacle to optimizing parameters. This turns out to be a fundamental issue induced by the classical parametrization. We note that the conventional way always associates the parameter to the range of a function/operator. This turns out not a natural way, causing certain symmetry losses, definition inconsistencies, and unnecessary complications, with the well-known Moreau identity being the best example. We propose equilibrate parametrization, which associates the parameter to the domain of a function, and to both the domain and range of a monotone operator. A series of powerful results are obtained owing to the new parametrization. Quite remarkably, the preconditioning technique can be shown as equivalent to the metric selection issue.

math.OC↗

arXiv2023

A Generic Closed-form Optimal Step-size for ADMM

In this work, we present a generic step-size choice for the ADMM type proximal algorithms. It admits a closed-form expression and is theoretically optimal with respect to a worst-case convergence rate bound. It is simply given by the ratio of Euclidean norms of the dual and primal solutions, i.e., $ ||λ^\star|| / ||{x}^\star||$. Numerical tests show that its practical performance is near-optimal in general. The only challenge is that such a ratio is not known a priori and we provide two strategies to address it. The derivation of our step-size choice is based on studying the fixed-point structure of ADMM using the proximal operator. However, we demonstrate that the classical proximal operator definition contains an input scaling issue. This leads to a scaled step-size optimization problem which would yield a false solution. Such an issue is naturally avoided by our proposed new definition of the proximal operator. A series of its properties is established.

math.OC↗