L-PASSO: Adaptive Log-penalized Variable Selector, with Sparse Selection and Robustness to the Pilot Estimator

Abstract

We study L-PASSO, a penalized least-squares regression method based on the logarithmic penalty $λ\sum_j\log(1+|β_j|/τ)$, and its two-stage adaptive form, in which the scales $τ_j$ come from a pilot estimator. Although this concave penalty has been proposed previously, its variable-selection properties and sparsity relative to other popular penalties have not been fully characterized. We derive the exact orthogonal-design solution, and show that the single-stage estimator selects the true model under a weighted irrepresentable condition that holds for every design once $τ$ is small. We further establish that, under matched tuning, the two-stage estimator possesses the oracle property and has half the asymptotic bias of the adaptive LASSO. Our main theoretical contribution concerns the robustness of the two-stage procedure to the pilot estimator. We show that two-stage L-PASSO remains selection consistent even when the weights associated with relevant variables are as small as $\exp\{-o(1/ζ_n)\}$, whereas the adaptive LASSO fails to retain relevant variables whose pilot weights are sufficiently small relative to its tuning parameter $ζ_n$. We establish that both thresholds are sharp. In high-dimensional settings, we prove support recovery for a local minimizer without imposing any lower-bound condition on the weights of relevant variables, and provide conditions under which this local minimizer is also the unique global minimizer. We additionally establish the selection consistency of a BIC-based tuning-parameter selection procedure. Finally, simulations comparing L-PASSO with the LASSO, SCAD, MCP, SELO, uniLasso, and the adaptive LASSO show that L-PASSO most frequently recovers the true support. Two real-data examples further illustrate the method.