Structured Recovery with Heavy-tailed Measurements: A Thresholding Procedure and Optimal Rates

Abstract

This paper introduces a general regularized thresholded least-square procedure estimating a structured signal $\theta_*\in\mathbb{R}^d$ from the following observations: $y_i = f(\langle\mathbf{x}_i, \theta_*\rangle, \xi_i),~i\in\{1,2,\cdots,N\}$, with i.i.d. heavy-tailed measurements $\{(\mathbf{x}_i,y_i)\}_{i=1}^N$. A general framework analyzing the thresholding procedure is proposed, which boils down to computing three critical radiuses of the bounding balls of the estimator. Then, we demonstrate these critical radiuses can be tightly bounded in the following two scenarios: (1) The link function $f(\cdot)$ is linear, i.e. $y = \langle\mathbf{x},\theta_*\rangle + \xi$, with $\theta_*$ being a sparse vector and $\{\mathbf{x}_i\}_{i=1}^N$ being general heavy-tailed random measurements with bounded $(20+\epsilon)$-moments. (2) The function $f(\cdot)$ is arbitrary unknown (possibly discontinuous) and $\{\mathbf{x}_i\}_{i=1}^N$ are heavy-tailed elliptical random vectors with bounded $(4+\epsilon)$-moments. In both scenarios, we show under these rather minimal bounded moment assumptions, such a procedure and corresponding analysis lead to optimal sample and error bounds with high probability in terms of the structural properties of $\theta_*$.