Consecutive Support Matching Induced Parameter Tuning Accelerates Momentum Iterative Hard Thresholding

Abstract

Momentum-based acceleration of iterative hard thresholding (IHT) can dramatically speed up sparse signal recovery from linear measurements, but its effectiveness hinges on careful parameter tuning -- a task complicated by the frequent support changes inherent to hard thresholding. We propose CosMIHT(Consecutive Support Matching Induced Momentum IHT), which resolves this difficulty through a simple adaptive rule: start with the conservative parameters and whenever two consecutive iterates share the same support, estimate the extreme eigenvalues of the support restricted Gram matrix via a lightweight power method and switch to the corresponding optimal heavy-ball parameters. This mechanism allows CosMIHT to automatically interpolate between cautious MIHT-like behavior during support discovery and near-optimal accelerated convergence after support identification. Under standard restricted isometry assumptions, we develop a two-phase convergence theory. In the \emph{wandering phase}, we establish a linear contraction of the recovery error up to a noise floor and derive an explicit upper bound on the number of iterations required to identify the correct support. In the \emph{lock-in} phase, we establish that, with a randomly initialized power method based eigenvalue estimates that depend on the number of power iterations, the algorithm enjoys, with high probability, a near-optimal accelerated convergence rate akin to the heavy ball method. We corroborate the theoretical findings with extensive numerical experiments on both noiseless and noisy measurements demonstrating that CosMIHT achieves faster convergence than state-of-the-art iterative sparse recovery techniques without compromising the recovery performance.