Searcharxiv⌕ Search

arXiv subjects

Tate Jacobson

Publications and source records attributed to Tate Jacobson.

6 recordsLinked to original sources

Global Inference for High Dimensional Quantile Regression with Heterogeneity

Quantile regression has received considerable attention in high dimensional methodology due to its robustness and ability to accommodate heteroskedasticity. Despite this attention, current inferential methods for high dimensional quantile regression have significant limitations---only testing significance, testing at a single quantile at a time, or assuming homoskedasticity. To generalize the inferences that can be drawn in high dimensional quantile regression, we introduce partial penalized Wald and rank score tests for testing linear hypotheses uniformly over a set of quantiles $\mathcal T$ under a heteroskedastic model. We propose globally adaptive local linear approximation (LLA) algorithms for folded concave penalized estimation and show that our global LLA algorithms recover ``testing oracles'' for the full and reduced models with high probability. We then show that our global LLA estimators achieve the $\sqrt{s/n}$ oracle estimation rate uniformly over $\mathcal T$, establish Bahadur representations, and derive asymptotic distributions for the test statistics under the null and local alternatives. We develop a dual formulation incorporating adaptive penalty weights for the rank scores and use the strong oracle property to establish their oracle equivalence. Using empirical process techniques, we derive reference distributions based on suprema of squared Bessel processes, extending partial penalized inference beyond pointwise chi-square calibration.

stat.ME↗

Large-scale linear hypothesis testing for high-dimensional online M-estimation

With the growing prevalence of streaming data, renewable estimation has become an important tool for updating statistical analyses without retaining historical raw observations. We study estimation and linear hypothesis testing in high-dimensional online M-estimation. Existing renewable procedures based on quadratic approximations can incur first-order approximation errors because the gradient of the unpenalized loss generally does not vanish at a preceding penalized estimator, posing a particular challenge for valid inference. To address this issue, we propose a corrected renewable loss that preserves historical first-order information and combine it with partial folded concave regularization and an iterative local linear approximation. We establish finite sample $\ell_1$- and $\ell_2$-error bounds, together with contraction bounds and strong oracle properties for both the constrained and unconstrained iterative estimators. Building on these results, we derive oracle Bahadur representations and establish asymptotically valid Wald and score tests for general linear hypotheses, allowing both the testing-oracle dimension and the number of restrictions to diverge. We further characterize their asymptotic power under local alternatives. The proposed procedure requires only recursively updated summaries while recovering the first-order inferential behavior of its pooled-data oracle counterpart. Simulation studies and real-data applications demonstrate the finite-sample performance of the proposed estimation and testing procedures.

stat.ME↗

Nonparametric Envelopes for Flexible Response Reduction

Envelope methods improve the estimation efficiency in multivariate linear regression by identifying and separating the material and immaterial parts of the responses or the predictors and estimating the regression coefficients using only the material part. Though envelopes have been extended to other models, such as GLMs, these extensions still largely fall under the restrictive parametric modeling framework. In this paper, we introduce a flexible, nonparametric extension of response envelopes for improving efficiency in nonlinear multivariate regressions. We propose the kernel envelope (KENV) estimator for simultaneously estimating the response envelope subspace and the enveloped nonparametric conditional mean function in a reproducing kernel Hilbert space, with a novel penalty that accounts for the envelope structure. We prove that the prediction risk for KENV converges to the optimal risk as the sample size diverges and show that KENV achieves a lower in-sample prediction risk than kernel ridge regression when the response has a non-trivial immaterial component. We compare the prediction performance of KENV with other envelope methods and kernel regression methods in simulations and a real data example, finding that KENV delivers more accurate predictions than both the envelope-based and kernel-based alternatives in both low and high dimensions.

stat.ME↗

Envelope-Guided Regularization for Improved Prediction in High-Dimensional Multivariate Regression

Envelope methods perform dimension reduction of predictors or responses in multivariate regression, exploiting the relationship between them to improve estimation efficiency. While most research on envelopes has focused on their estimation properties, certain envelope estimators have been shown to excel at prediction in both low and high dimensions. In this paper, we propose to further improve prediction through envelope-guided regularization (EgReg), a novel method which uses envelope-derived information to guide shrinkage along the principal components (PCs) of the predictor matrix. We situate EgReg among other PC-based regression methods and envelope methods to motivate its development. We show that EgReg delivers lower prediction risk than a closely related non-shrinkage envelope estimator when the number of predictors $p$ and observations $n$ are fixed and in any alignment. In an asymptotic regime where the true intrinsic dimension of the predictors and $n$ diverge proportionally, we find that the limiting prediction risk of the non-shrinkage envelope estimator exhibits a double descent phenomenon and is consistently larger than the limiting risk for EgReg. We compare the prediction performance of EgReg with envelope methods and other PC-based prediction methods in simulations and a real data example, observing improved prediction performance over these alternative approaches in general.

stat.ME↗

Strong Oracle Guarantees for Partial Penalized Tests of High Dimensional Generalized Linear Models

Partial penalized tests provide flexible approaches to testing linear hypotheses in high dimensional generalized linear models. However, because the estimators used in these tests are local minimizers of potentially non-convex folded-concave penalized objectives, the solutions one computes in practice may not coincide with the unknown local minima for which we have nice theoretical guarantees. To close this gap between theory and computation, we introduce local linear approximation (LLA) algorithms to compute the full and reduced model estimators for these tests and develop theory specifically for the LLA solutions. We prove that our LLA algorithms converge to oracle estimators for the full and reduced models in two steps with overwhelming probability. We then leverage this strong oracle result and the asymptotic properties of the oracle estimators to show that the partial penalized test statistics evaluated at the two-step LLA solutions are approximately chi-square in large samples, giving us guarantees for the tests using specific computed solutions and thereby closing the theoretical gap. We conduct simulation studies to assess the finite-sample performance of our testing procedures, finding that partial penalized tests using the LLA solutions agree with tests using the oracle estimators, and demonstrate our testing procedures in a real data application.

math.ST↗

High-dimensional Censored Regression via the Penalized Tobit Likelihood

High-dimensional regression and regression with a left-censored response are each well-studied topics. In spite of this, few methods have been proposed which deal with both of these complications simultaneously. The Tobit model -- long the standard method for censored regression in economics -- has not been adapted for high-dimensional regression at all. To fill this gap and bring up-to-date techniques from high-dimensional statistics to the field of high-dimensional left-censored regression, we propose several penalized Tobit models. We develop a fast algorithm which combines quadratic minimization with coordinate descent to compute the penalized Tobit solution path. Theoretically, we analyze the Tobit lasso and Tobit with a folded concave penalty, bounding the $\ell_2$ estimation loss for the former and proving that a local linear approximation estimator for the latter possesses the strong oracle property. Through an extensive simulation study, we find that our penalized Tobit models provide more accurate predictions and parameter estimates than other methods. We use a penalized Tobit model to analyze high-dimensional left-censored HIV viral load data from the AIDS Clinical Trials Group and identify potential drug resistance mutations in the HIV genome. Appendices contain intermediate theoretical results and technical proofs.

stat.ME↗