A Decomposed Bilevel Search for Variable-Metric Proximal Gradient Methods

Abstract

Variable-metric proximal methods accelerate composite convex optimization, but the scaled proximal map induced by a quasi-Newton metric rarely has a closed form. We develop \emph{Decomposed Bilevel Search} (DBS), based on a diagonal-plus-rank-one factor \(X=D+uv^\top\) whose diagonal scaling satisfies a weak secant equation. The induced inverse metric \(B^{-1}=XX^\top\) recovers zero-memory DFP/BFGS-type Broyden members, while the factor form reduces each scaled proximal step to a two-dimensional monotone residual system. Each residual evaluation requires one diagonal-metric proximal map, and a certified bilevel solve reaches target accuracy \(\epsilon\) in \(\mathcal O((d+T_p)\log^2(1/\epsilon))\) work, where \(T_p\) is the cost of that proximal map. Under strong convexity, the outer method converges linearly with exact and inexact inner solves. In the scalar specialization \(D=\alpha I\), the oracle uses only ordinary proximal evaluations of the regularizer and no generalized Jacobian or active-set information. Experiments on ordered-weighted \(\ell_1\) (SLOPE/OWL) and group-lasso logistic regression evaluate both the inner oracle and the full outer method. The oracle solves high-dimensional SLOPE scaled proximal subproblems of condition number up to \(4\times10^6\) using a few hundred ordinary proximal evaluations. In SLOPE outer benchmarks, the warm-started oracle keeps the scaled-proximal overhead controlled and DBS reaches stringent targets with far fewer gradient evaluations than Lipschitz-normalized FISTA; on \texttt{real-sim} this becomes a clear target-time advantage. On group-lasso logistic regression, DBS is reliable on synthetic correlated instances and fastest on a real-data grouped-copy instance.