Oracle high-dimensional $M$-estimation using smooth reparameterization for sparsity

Abstract

This paper establishes a unified non-linear regularization framework for high-dimensional $M$-estimation, encompassing both linear models and Cox's proportional hazards models. Rather than relying on traditional additive non-convex penalties, the proposed paradigm embeds sparsity directly into the transformation for the physical parameter $θ=ϕ(β)$ using a smooth ($C^2$) component-wise "ReParametrization map for Sparsity (RePS)" $ϕ$, and the penalty term is $λ\Vert β\Vert_1$ rather than $λ\Vert θ\Vert_1$. This structural formulation dynamically adapts to local parameter scales, suppressing high-dimensional noise while simultaneously recovering unbiased oracle asymptotic normality under the large-sample limit. Through the Primal-Dual Witness method, we establish a unified oracle equivalence for both model classes under a liberated micro-penalty scaling regime $λ\ll n^{-1/2}$, accommodating model-specific structures such as the shift-invariance in survival analysis. Extensive Monte Carlo simulations demonstrate that the proposed framework consistently achieves superior false-positive control and high 95\% confidence interval coverage in high-dimensional linear models, while also delivering performance comparable or superior in all aspects to state-of-the-art methods like SCAD and MCP in proportional hazards models.