Safe Screening Rules for Generalized Double Sparsity Learning

Abstract

In a high-dimensional setting, sparse model has shown its power in computational and statistical efficiency. We consider variables selection problem with a broad class of simultaneous sparsity regularization, enforcing both feature-wise and group-wise sparsity at the same time. The analysis leverages an introduction of $\epsilon q$-norm in vector space, which is proved to has close connection with the mixture regularization and naturally leads to a dual formulation. Properties of primal/dual optimal solution and optimal values are discussed, which motivates the design of screening rules. We several fast safe screening rules in the general framework, rules that discard inactive features/groups at an early stage that are guaranteed to be inactive in the exact solution, leading to a significant gain in computation speed.