Global and Unconditional $R$-Linear Convergence of Rank-Compressed Weighted LMSD Sweeps for Strictly Convex Quadratics

Abstract

We study limited memory steepest descent (LMSD) with exact algebraic rank compression for strictly convex quadratic optimization. At each cycle, the method restricts the gradient history to its column space and applies the reciprocals of all Ritz values of the compressed projection in the next sweep. If the block-start gradient has at most $p$ active distinct eigenvalues, the delayed sweep terminates finitely. For nonterminating trajectories, a determinant and Cauchy--Binet formula for the complete-sweep polynomial yields global convergence without a full-rank history or a run-wise normalized-conditioning bound. Consecutive sweep endpoints define a continuous positively homogeneous map on a closed cone of compatible states. Compactness then gives an $R$-linear endpoint estimate, uniform over compatible initial states, with constants depending only on $H$ and $p$; the estimate extends to inner gradients, iterate errors, and objective gaps. Every fixed positive spectral weight $W=ω(H)$ is linearly conjugate to the standard method. The weighted methods therefore share its decay factor, while the Euclidean-norm prefactor is increased by at most $\sqrt{κ_2(W)}$. This includes harmonic-Ritz LMSD, fixed power weights, and the delayed BB1 and BB2 recurrences.