High-dimensional sparse recovery from function samples Decoders, guarantees and instance optimality

Abstract

We investigate the reconstruction of multivariate functions from samples using sparse recovery techniques. For Square Root Lasso, Orthogonal Matching Pursuit, and Compressive Sampling Matching Pursuit, we demonstrate both theoretically and empirically that they allow us to recover functions from a small number of random samples. In contrast to Basis Pursuit Denoising, the deployed decoders only require a search space $V_J$ spanned by dictionary elements indexed by $J$ and a sparsity parameter $n$ to guarantee an $L_2$-approximation error decaying no worse than a best $n$-term approximation error and the truncation error with respect to the search space $V_J$ and the uniform norm. We show that this happens simultaneously for all admissible functions if the number of samples scales as $n\log^2 n\log |J|$, coming from known bounds for the RIP for matrices built upon bounded orthonormal systems. As a consequence, we obtain bounds for sampling widths in function classes. In addition, we establish lower bounds on the required sample complexity, which show that the log-factor in $\vert J \vert$ is indeed necessary to obtain such {\em instance-optimal} error guarantees. Finally, we conduct several numerical experiments to show that our theoretical bounds are reasonable and compare the discussed decoders in practice.