Searcharxiv⌕ Search

arXiv subjects

R. Dennis Cook

Publications and source records attributed to R. Dennis Cook.

18 recordsLinked to original sources
arXiv2020

Fundamentals of path analysis in the social sciences

Motivated by a recent series of diametrically opposed articles on the relative value of statistical methods for the analysis of path diagrams in the social sciences, we discuss from a primarily theoretical perspective selected fundamental aspects of path modeling and analysis based on a common re reflexive setting. Since there is a paucity of technical support evident in the debate, our aim is to connect it to mainline statistics literature and to address selected foundational issues that may help move the discourse. We do not intend to advocate for or against a particular method or analysis philosophy.

stat.ME↗
arXiv2020

Enveloped Huber Regression

Huber regression (HR) is a popular robust alternative to the least squares regression when the error follows a heavy-tailed distribution. We propose a new method called the enveloped Huber regression (EHR) by considering the envelope assumption that there exists some subspace of the predictors that has no association with the response, which is referred to as the immaterial part. More efficient estimation is achieved via the removal of the immaterial part. Different from the envelope least squares (ENV) model whose estimation is based on maximum normal likelihood, the estimation of the EHR model is through Generalized Method of Moments. The asymptotic normality of the EHR estimator is established, and it is shown that EHR is more efficient than HR. Moreover, EHR is more efficient than ENV when the error distribution is heavy-tailed, while maintaining a small efficiency loss when the error distribution is normal. Moreover, our theory also covers the heteroscedastic case in which the error may depend on the covariates. Extensive simulation studies confirm the messages from the asymptotic theory. EHR is further illustrated on a real dataset.

stat.ME↗
arXiv2018

A convex formulation for high-dimensional sparse sliced inverse regression

Sliced inverse regression is a popular tool for sufficient dimension reduction, which replaces covariates with a minimal set of their linear combinations without loss of information on the conditional distribution of the response given the covariates. The estimated linear combinations include all covariates, making results difficult to interpret and perhaps unnecessarily variable, particularly when the number of covariates is large. In this paper, we propose a convex formulation for fitting sparse sliced inverse regression in high dimensions. Our proposal estimates the subspace of the linear combinations of the covariates directly and performs variable selection simultaneously. We solve the resulting convex optimization problem via the linearized alternating direction methods of multiplier algorithm, and establish an upper bound on the subspace distance between the estimated and the true subspaces. Through numerical studies, we show that our proposal is able to identify the correct covariates in the high-dimensional setting.

stat.ML↗
arXiv2018

Multivariate Design of Experiments for Engineering Dimensional Analysis

We consider the design of dimensional analysis experiments when there is more than a single response. We first give a brief overview of dimensional analysis experiments and the dimensional analysis (DA) procedure. The validity of the DA method for univariate responses was established by the Buckingham $Π$-Theorem in the early 20th century. We extend the theorem to the multivariate case, develop basic criteria for multivariate design of DA and give guidelines for design construction. Finally, we illustrate the construction of designs for DA experiments for an example involving the design of a heat exchanger.

stat.AP↗
arXiv2018

Combining Envelope Methodology and Aster Models for Variance Reduction in Life History Analyses

Precise estimation of expected Darwinian fitness, the expected lifetime number of offspring of organism, is a central component of life history analysis. The aster model serves as a defensible statistical model for distributions of Darwinian fitness. The aster model is equipped to incorporate the major life stages an organism travels through which separately may effect Darwinian fitness. Envelope methodology reduces asymptotic variability by establishing a link between unknown parameters of interest and the asymptotic covariance matrices of their estimators. It is known both theoretically and in applications that incorporation of envelope methodology reduces asymptotic variability. We develop an envelope framework, including a new envelope estimator, that is appropriate for aster analyses. The level of precision provided from our methods allows researchers to draw stronger conclusions about the driving forces of Darwinian fitness from their life history analyses than they could with the aster model alone. Our methods are illustrated on a simulated dataset and a life history analysis of \emph{Mimulus guttatus} flowers is provided. Useful variance reduction is obtained in both analyses.

stat.AP↗
arXiv2018

Inverse regression for ridge recovery II: Numerics

We investigate the application of sufficient dimension reduction (SDR) to a noiseless data set derived from a deterministic function of several variables. In this context, SDR provides a framework for ridge recovery. In this second part, we explore the numerical subtleties associated with using two inverse regression methods---sliced inverse regression (SIR) and sliced average variance estimation (SAVE)---for ridge recovery. This includes a detailed numerical analysis of the eigenvalues of the resulting matrices and the subspaces spanned by their columns. After this analysis, we demonstrate the methods on several numerical test problems.

math.NA↗
arXiv2017

Matrix-Variate Regressions and Envelope Models

Modern technology often generates data with complex structures in which both response and explanatory variables are matrix-valued. Existing methods in the literature are able to tackle matrix-valued predictors but are rather limited for matrix-valued responses. In this article, we study matrix-variate regressions for such data, where the response Y on each experimental unit is a random matrix and the predictor X can be either a scalar, a vector, or a matrix, treated as non-stochastic in terms of the conditional distribution Y|X. We propose models for matrix-variate regressions and then develop envelope extensions of these models. Under the envelope framework, redundant variation can be eliminated in estimation and the number of parameters can be notably reduced when the matrix-variate dimension is large, possibly resulting in significant gains in efficiency. The proposed methods are applicable to high dimensional settings.

stat.ME↗
arXiv2017

Weighted envelope estimation to handle variability in model selection

Envelope methodology can provide substantial efficiency gains in multivariate statistical problems, but in some applications the estimation of the envelope dimension can induce selection volatility that may mitigate those gains. Current envelope methodology does not account for the added variance that can result from this selection. In this article, we circumvent dimension selection volatility through the development of a weighted envelope estimator. Theoretical justification is given for our estimator and validity of the residual bootstrap for estimating its asymptotic variance is established. A simulation study and an analysis on a real data set illustrate the utility of our weighted envelope estimator.

stat.ME↗
arXiv2017

Inverse regression for ridge recovery: A data-driven approach for parameter reduction in computer experiments

Parameter reduction can enable otherwise infeasible design and uncertainty studies with modern computational science models that contain several input parameters. In statistical regression, techniques for sufficient dimension reduction (SDR) use data to reduce the predictor dimension of a regression problem. A computational scientist hoping to use SDR for parameter reduction encounters a problem: a computer prediction is best represented by a deterministic function of the inputs, so data comprised of computer simulation queries fail to satisfy the SDR assumptions. To address this problem, we interpret SDR methods sliced inverse regression (SIR) and sliced average variance estimation (SAVE) as estimating the directions of a ridge function, which is a composition of a low-dimensional linear transformation with a nonlinear function. Within this interpretation, SIR and SAVE estimate matrices of integrals whose column spaces are contained in the ridge directions' span; we analyze and numerically verify convergence of these column spaces as the number of computer model queries increases. Moreover, we show example functions that are not ridge functions but whose inverse conditional moment matrices are low-rank. Consequently, the computational scientist should beware when using SIR and SAVE for parameter reduction, since SIR and SAVE may mistakenly suggest that truly important directions are unimportant.

math.NA↗
arXiv2014

Algorithms for envelope estimation

Envelopes were recently proposed as methods for reducing estimative variation in multivariate linear regression. Estimation of an envelope usually involves optimization over Grassmann manifolds. We propose a fast and widely applicable one-dimensional (1D) algorithm for estimating an envelope in general. We reveal an important structural property of envelopes that facilitates our algorithm, and we prove both Fisher consistency and root-n-consistency of the algorithm.

stat.ME↗
arXiv2012

Coordinate-independent sparse sufficient dimension reduction and variable selection

Sufficient dimension reduction (SDR) in regression, which reduces the dimension by replacing original predictors with a minimal set of their linear combinations without loss of information, is very helpful when the number of predictors is large. The standard SDR methods suffer because the estimated linear combinations usually consist of all original predictors, making it difficult to interpret. In this paper, we propose a unified method - coordinate-independent sparse estimation (CISE) - that can simultaneously achieve sparse sufficient dimension reduction and screen out irrelevant and redundant variables efficiently. CISE is subspace oriented in the sense that it incorporates a coordinate-independent penalty term with a broad series of model-based and model-free SDR approaches. This results in a Grassmann manifold optimization problem and a fast algorithm is suggested. Under mild conditions, based on manifold theories and techniques, it can be shown that CISE would perform asymptotically as well as if the true irrelevant predictors were known, which is referred to as the oracle property. Simulation studies and a real-data example demonstrate the effectiveness and efficiency of the proposed approach.

math.ST↗
arXiv2012

Estimating sufficient reductions of the predictors in abundant high-dimensional regressions

We study the asymptotic behavior of a class of methods for sufficient dimension reduction in high-dimension regressions, as the sample size and number of predictors grow in various alignments. It is demonstrated that these methods are consistent in a variety of settings, particularly in abundant regressions where most predictors contribute some information on the response, and oracle rates are possible. Simulation results are presented to support the theoretical conclusion.

math.ST↗
arXiv2009

Principal Fitted Components for Dimension Reduction in Regression

We provide a remedy for two concerns that have dogged the use of principal components in regression: (i) principal components are computed from the predictors alone and do not make apparent use of the response, and (ii) principal components are not invariant or equivariant under full rank linear transformation of the predictors. The development begins with principal fitted components [Cook, R. D. (2007). Fisher lecture: Dimension reduction in regression (with discussion). Statist. Sci. 22 1--26] and uses normal models for the inverse regression of the predictors on the response to gain reductive information for the forward regression of interest. This approach includes methodology for testing hypotheses about the number of components and about conditional independencies among the predictors.

stat.ME↗
arXiv2007

Elevated soil lead: Statistical modeling and apportionment of contributions from lead-based paint and leaded gasoline

While it is widely accepted that lead-based paint and leaded gasoline are primary sources of elevated concentrations of lead in residential soils, conclusions regarding their relative contributions are mixed and generally study specific. We develop a novel nonlinear regression for soil lead concentrations over time. It is argued that this methodology provides useful insights into the partitioning of the average soil lead concentration by source and time over large residential areas. The methodology is used to investigate soil lead concentrations from the 1987 Minnesota Lead Study and the 1990 National Lead Survey. Potential litigation issues are discussed briefly.

stat.AP↗
arXiv2007

Fisher Lecture: Dimension Reduction in Regression

Beginning with a discussion of R. A. Fisher's early written remarks that relate to dimension reduction, this article revisits principal components as a reductive method in regression, develops several model-based extensions and ends with descriptions of general approaches to model-based and model-free dimension reduction in regression. It is argued that the role for principal components and related methodology may be broader than previously seen and that the common practice of conditioning on observed values of the predictors may unnecessarily limit the choice of regression methodology.

stat.ME↗
arXiv2005

Determining the dimension of iterative Hessian transformation

The central mean subspace (CMS) and iterative Hessian transformation (IHT) have been introduced recently for dimension reduction when the conditional mean is of interest. Suppose that X is a vector-valued predictor and Y is a scalar response. The basic problem is to find a lower-dimensional predictor η^TX such that E(Y|X)=E(Y|η^TX). The CMS defines the inferential object for this problem and IHT provides an estimating procedure. Compared with other methods, IHT requires fewer assumptions and has been shown to perform well when the additional assumptions required by those methods fail. In this paper we give an asymptotic analysis of IHT and provide stepwise asymptotic hypothesis tests to determine the dimension of the CMS, as estimated by IHT. Here, the original IHT method has been modified to be invariant under location and scale transformations. To provide empirical support for our asymptotic results, we will present a series of simulation studies. These agree well with the theory. The method is applied to analyze an ozone data set.

math.ST↗
arXiv2004

Testing predictor contributions in sufficient dimension reduction

We develop tests of the hypothesis of no effect for selected predictors in regression, without assuming a model for the conditional distribution of the response given the predictors. Predictor effects need not be limited to the mean function and smoothing is not required. The general approach is based on sufficient dimension reduction, the idea being to replace the predictor vector with a lower-dimensional version without loss of information on the regression. Methodology using sliced inverse regression is developed in detail.

math.ST↗