On optimal solutions of the constrained $\ell_0$ regularization and its penalty problem

Abstract

The constrained $\ell_0$ regularization plays an important role in sparse reconstruction. A widely used approach for solving this problem is the penalty method, of which the least square penalty problem is a special case. However, the connections between global minimizers of the constrained $\ell_0$ problem and its penalty problem have never been studied in a systematic way. This work provides a comprehensive investigation on optimal solutions of these two problems and their connections. We give detailed descriptions of optimal solutions of the two problems, including existence, stability with respect to the parameter, cardinality and strictness. In particular, we find that the optimal solution set of the penalty problem is piecewise constant with respect to the penalty parameter. Then we analyze in-depth the relationship between optimal solutions of the two problems. It is shown that, in the noisy case the least square penalty problem probably has no common optimal solutions with the constrained $\ell_0$ problem for any penalty parameter. Under a mild condition on the penalty function, we establish that the penalty problem has the same optimal solution set as the constrained $\ell_0$ problem when the penalty parameter is sufficiently large. Based on the conditions, we further propose exact penalty problems for the constrained $\ell_0$ problem. Finally, we present a numerical example to illustrate our main theoretical results.