arXiv subjects

Ulrike Schneider

Publications and source records attributed to Ulrike Schneider.

11 recordsLinked to original sources

arXiv2026

Beyond the Oracle Property: Adaptive LASSO in Cointegrating Regressions with Local-to-Unity Regressors

This paper derives new asymptotic results for the adaptive LASSO estimator in cointegrating regressions, allowing for uncertainty about whether the regressors are exact unit root processes. We study model selection probabilities, estimator consistency, and limiting distributions under standard and moving-parameter asymptotics. We further derive uniform convergence rates and the fastest local-to-zero rates detectable by the estimator under conservative and consistent tuning. For consistent tuning, we construct confidence regions that are easy to implement, uniformly valid over the parameter space, and achieve sure asymptotic coverage without requiring knowledge or estimation of local-to-unity or long-run covariance parameters. Simulation results reveal that the finite-sample distribution of the adaptive LASSO estimator can deviate substantially from the oracle property, whereas moving-parameter asymptotics provide much more accurate approximations. Consequently, in addition to being infeasible in applications due to their dependence on non-estimable nuisance parameters, oracle-based confidence regions are often too small to achieve adequate coverage in empirically relevant scenarios with small but non-zero coefficients. In contrast, the proposed confidence regions are always feasible and deliver reliable coverage across the parameter space. An empirical application to predicting the U.S. unemployment rate illustrates their practical usefulness for quantifying uncertainty around adaptive LASSO estimates.

econ.EM↗

arXiv2025

A Unified Framework for Pattern Recovery in Penalized and Thresholded Estimation and its Geometry

We consider the framework of penalized estimation where the penalty term is given by a real-valued polyhedral gauge, which encompasses methods such as LASSO, generalized LASSO, SLOPE, OSCAR, PACS and others. Each of these estimators is defined through an optimization problem and can uncover a different structure or ``pattern'' of the unknown parameter vector. We define a novel and general notion of patterns based on subdifferentials and formalize an approach to measure pattern complexity. For pattern recovery, we provide a minimal condition for a particular pattern to be detected by the procedure with positive probability, the so-called accessibility condition. Using our approach, we also introduce the stronger noiseless recovery condition. For the LASSO, it is well known that the irrepresentability condition is necessary for pattern recovery with probability larger than $1/2$ and we show that the noiseless recovery plays exactly the same role in our general framework, thereby unifying and extending the irrepresentability condition to a broad class of penalized estimators. We also show that the noiseless recovery condition can be relaxed when turning to so-called thresholded penalized estimators: we prove that the necessary condition of accessibility is already sufficient for sure pattern recovery by thresholded penalized estimation provided that the noise is small enough. Throughout the article, we demonstrate how our findings can be interpreted through a geometrical lens.

math.ST↗

arXiv2023

Uniformly Valid Inference Based on the Lasso in Linear Mixed Models

Linear mixed models (LMMs) are suitable for clustered data and are common in biometrics, medicine, survey statistics and many other fields. In those applications, it is essential to carry out valid inference after selecting a subset of the available variables. We construct confidence sets for the fixed effects in Gaussian LMMs that are based on Lasso-type estimators. Aside from providing confidence regions, this also allows to quantify the joint uncertainty of both variable selection and parameter estimation in the procedure. To show that the resulting confidence sets for the fixed effects are uniformly valid over the parameter spaces of both the regression coefficients and the covariance parameters, we also prove the novel result on uniform Cramer consistency of the restricted maximum likelihood (REML) estimators of the covariance parameters. The superiority of the constructed confidence sets to naive post-selection procedures is validated in simulations and illustrated with a study of the acid neutralization capacity of lakes in the United States.

stat.ME↗

arXiv2022

The Geometry of Uniqueness, Sparsity and Clustering in Penalized Estimation

We provide a necessary and sufficient condition for the uniqueness of penalized least-squares estimators whose penalty term is given by a norm with a polytope unit ball, covering a wide range of methods including SLOPE, PACS, fused, clustered and classical LASSO as well as the related method of basis pursuit. We consider a strong type of uniqueness that is relevant for statistical problems. The uniqueness condition is geometric and involves how the row span of the design matrix intersects the faces of the dual norm unit ball, which for SLOPE is given by the signed permutahedron. Further considerations based this condition also allow to derive results on sparsity and clustering features. In particular, we define the notion of a SLOPE pattern to describe both sparsity and clustering properties of this method and also provide a geometric characterization of accessible SLOPE patterns.

math.ST↗

arXiv2021

Uniform Asymptotics and Confidence Regions Based on the Adaptive Lasso with Partially Consistent Tuning

We consider the adaptive Lasso estimator with componentwise tuning in the framework of a low-dimensional linear regression model. In our setting, at least one of the components is penalized at the rate of consistent model selection and certain components may not be penalized at all. We perform a detailed study of the consistency properties and the asymptotic distribution which includes the effects of componentwise tuning within a so-called moving-parameter framework. These results enable us to explicitly provide a set $\mathcal{M}$ such that every open superset acts as a confidence set with uniform asymptotic coverage equal to 1, whereas removing an arbitrarily small open set along the boundary yields a confidence set with uniform asymptotic coverage equal to 0. The shape of the set $\mathcal{M}$ depends on the regressor matrix as well as the deviations within the componentwise tuning parameters. Our findings can be viewed as a broad generalization of Pötscher & Schneider (2009, 2010) who considered distributional properties and confidence intervals based on components of the adaptive Lasso estimator for the case of orthogonal regressors.

math.ST↗

arXiv2020

On the Distribution, Model Selection Properties and Uniqueness of the Lasso Estimator in Low and High Dimensions

We derive expressions for the finite-sample distribution of the Lasso estimator in the context of a linear regression model in low as well as in high dimensions by exploiting the structure of the optimization problem defining the estimator. In low dimensions, we assume full rank of the regressor matrix and present expressions for the cumulative distribution function as well as the densities of the absolutely continuous parts of the estimator. Our results are presented for the case of normally distributed errors, but do not hinge on this assumption and can easily be generalized. Additionally, we establish an explicit formula for the correspondence between the Lasso and the least-squares estimator. We derive analogous results for the distribution in less explicit form in high dimensions where we make no assumptions on the regressor matrix at all. In this setting, we also investigate the model selection properties of the Lasso and show that possibly only a subset of models might be selected by the estimator, completely independently of the observed response vector. Finally, we present a condition for uniqueness of the estimator that is necessary as well as sufficient.

math.ST↗

arXiv2018

Uniformly Valid Confidence Sets Based on the Lasso

In a linear regression model of fixed dimension $p \leq n$, we construct confidence regions for the unknown parameter vector based on the Lasso estimator that uniformly and exactly hold the prescribed in finite samples as well as in an asymptotic setup. We thereby quantify estimation uncertainty as well as the "post-model selection error" of this estimator. More concretely, in finite samples with Gaussian errors and asymptotically in the case where the Lasso estimator is tuned to perform conservative model selection, we derive exact formulas for computing the minimal coverage probability over the entire parameter space for a large class of shapes for the confidence sets, thus enabling the construction of valid confidence regions based on the Lasso estimator in these settings. The choice of shape for the confidence sets and comparison with the confidence ellipse based on the least-squares estimator is also discussed. Moreover, in the case where the Lasso estimator is tuned to enable consistent model selection, we give a simple confidence region with minimal coverage probability converging to one. Finally, we also treat the case of unknown error variance and present some ideas for extensions.

math.ST↗

arXiv2014

Confidence Sets Based on Thresholding Estimators in High-Dimensional Gaussian Regression Models

We study confidence intervals based on hard-thresholding, soft-thresholding, and adaptive soft-thresholding in a linear regression model where the number of regressors $k$ may depend on and diverge with sample size $n$. In addition to the case of known error variance, we define and study versions of the estimators when the error variance is unknown. In the known variance case, we provide an exact analysis of the coverage properties of such intervals in finite samples. We show that these intervals are always larger than the standard interval based on the least-squares estimator. Asymptotically, the intervals based on the thresholding estimators are larger even by an order of magnitude when the estimators are tuned to perform consistent variable selection. For the unknown-variance case, we provide non-trivial lower bounds for the coverage probabilities in finite samples and conduct an asymptotic analysis where the results from the known-variance case can be shown to carry over asymptotically if the number of degrees of freedom $n-k$ tends to infinity fast enough in relation to the thresholding parameter.

math.ST↗

arXiv2011

Distributional Results for Thresholding Estimators in High-Dimensional Gaussian Regression Models

We study the distribution of hard-, soft-, and adaptive soft-thresholding estimators within a linear regression model where the number of parameters k can depend on sample size n and may diverge with n. In addition to the case of known error-variance, we define and study versions of the estimators when the error-variance is unknown. We derive the finite-sample distribution of each estimator and study its behavior in the large-sample limit, also investigating the effects of having to estimate the variance when the degrees of freedom n-k does not tend to infinity or tends to infinity very slowly. Our analysis encompasses both the case where the estimators are tuned to perform consistent model selection and the case where the estimators are tuned to perform conservative model selection. Furthermore, we discuss consistency, uniform consistency and derive the uniform convergence rate under either type of tuning.

math.ST↗

arXiv2010

Confidence Sets Based on Penalized Maximum Likelihood Estimators in Gaussian Regression

Confidence intervals based on penalized maximum likelihood estimators such as the LASSO, adaptive LASSO, and hard-thresholding are analyzed. In the known-variance case, the finite-sample coverage properties of such intervals are determined and it is shown that symmetric intervals are the shortest. The length of the shortest intervals based on the hard-thresholding estimator is larger than the length of the shortest interval based on the adaptive LASSO, which is larger than the length of the shortest interval based on the LASSO, which in turn is larger than the standard interval based on the maximum likelihood estimator. In the case where the penalized estimators are tuned to possess the `sparsity property', the intervals based on these estimators are larger than the standard interval by an order of magnitude. Furthermore, a simple asymptotic confidence interval construction in the `sparse' case, that also applies to the smoothly clipped absolute deviation estimator, is discussed. The results for the known-variance case are shown to carry over to the unknown-variance case in an appropriate asymptotic sense.

math.ST↗

arXiv2008

On the Distribution of the Adaptive LASSO Estimator

We study the distribution of the adaptive LASSO estimator (Zou (2006)) in finite samples as well as in the large-sample limit. The large-sample distributions are derived both for the case where the adaptive LASSO estimator is tuned to perform conservative model selection as well as for the case where the tuning results in consistent model selection. We show that the finite-sample as well as the large-sample distributions are typically highly non-normal, regardless of the choice of the tuning parameter. The uniform convergence rate is also obtained, and is shown to be slower than $n^{-1/2}$ in case the estimator is tuned to perform consistent model selection. In particular, these results question the statistical relevance of the `oracle' property of the adaptive LASSO estimator established in Zou (2006). Moreover, we also provide an impossibility result regarding the estimation of the distribution function of the adaptive LASSO estimator.The theoretical results, which are obtained for a regression model with orthogonal design, are complemented by a Monte Carlo study using non-orthogonal regressors.

math.ST↗