Searcharxiv⌕ Search
Home/Archive/arXiv

arXiv · math/0504499

Analysis of variance--why it is more important than ever

Andrew Gelman
View original source ↗Read source document ↗Export metadata ↓

Also available from

Abstract

Analysis of variance (ANOVA) is an extremely important method in exploratory and confirmatory data analysis. Unfortunately, in complex problems (e.g., split-plot designs), it is not always easy to set up an appropriate ANOVA. We propose a hierarchical analysis that automatically gives the correct ANOVA comparisons even in complex scenarios. The inferences for all means and variances are performed under a model with a separate batch of effects for each row of the ANOVA table. We connect to classical ANOVA by working with finite-sample variance components: fixed and random effects models are characterized by inferences about existing levels of a factor and new levels, respectively. We also introduce a new graphical display showing inferences about the standard deviations of each batch of effects. We illustrate with two examples from our applied data analysis, first illustrating the usefulness of our hierarchical computations and displays, and second showing how the ideas of ANOVA are helpful in understanding a previously fit hierarchical model.

Explore related subjects

math.STstat.TH

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Andrew Gelman. 2005-08-26. Analysis of variance--why it is more important than ever. https://doi.org/10.1214/009053604000001048

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Regression Functions as Fields on Covariate Space: Evolution Equations, the Hessian, and Identification

For each time $t$, the regression function $m_t(x)=E[Y_t\mid X_t=x]$ of an outcome on contemporaneous covariates can be regarded as a field on covariate space. When the covariates of units move as a diffusion and each unit's outcome has its own dynamics, we show that $m_t$ satisfies a second-order evolution equation whose diffusion term is $\frac12\mathrm{tr}(Σ\nabla^2 m_t)$, the Hessian of the regression function weighted by the covariance $Σ$ of covariate increments, whose transport term is the drift of the time-reversed covariate process, and whose source term is the conditional mean rate of change of the outcome; this yields a Feynman-Kac representation along reversed covariate paths. We relate this Hessian to quadratic regression (under Gaussian design the best quadratic approximation has Hessian $E[\nabla^2 m(X)]$) and to kernel smoothing bias. For bistable replicator dynamics on covariate space, travelling fronts are exactly logistic in $(x,t)$, so one logistic regression identifies the diffusion-reaction ratio and the invading strategy. We then ask whether spread in covariate space reflects fluctuating unit characteristics or imitation among similar units. Imitation acts through exactly the same operator as covariate fluctuation, so the two are observationally equivalent in the regression field; panel data separate them because they identify $Σ$ directly. The two cannot be separated on a travelling front, and we propose a split-sample instrumental-variables test that works on seeded, non-front profiles, with size and power evaluated in agent-based experiments.

math.ST↗
arXiv2026

Scalable Bayesian Quadrature via Operator-Adapted Features

Kernel methods face two recurrent bottlenecks: the cubic cost of inverting an $n\times n$ kernel matrix, as in kernel ridge regression, and the evaluation of kernel integrals, as required in kernel quadrature. Existing low-rank kernel and Gaussian process approximations primarily address the former bottleneck, but do not control the downstream quadrature error when both the kernel matrix and kernel mean are approximated, particularly when the integrand lies outside the kernel reproducing kernel Hilbert space. We address both challenges through an operator-adapted finite-feature approximation. Specifically, we construct a finite-dimensional working kernel from the eigenvalues and eigenfunctions of a second-order differential operator. We establish convergence guarantees for the resulting finite-feature estimators in kernel quadrature, with extension to other related kernel methods. Our results show that, with $m \lesssim n$ basis functions, the truncated estimator attains the corresponding benchmark statistical rates. The theory covers fixed/random designs, noisy/noiseless observations, oversmoothing/undersmoothing regimes, and Matérn/Gaussian-type weights. We provide explicit error bounds that quantify the roles of the spectral cutoff, regularization parameter, lengthscale, sampling geometry, and operator boundary condition. We verify the convergence results in numerical experiments.

math.ST↗
arXiv2026

New explanations and inference for least angle regression

Efron et al. (2004) introduced least angle regression (LAR) as an algorithm for linear predictions, intended as an alternative to forward selection with connections to penalized regression. However, LAR has remained somewhat of a "black box," where some basic behavioral properties of LAR output are not well understood, including an appropriate termination point for the algorithm. We provide a novel framework for inference with LAR, which also allows LAR to be understood from new perspectives with several newly developed mathematical properties. The LAR algorithm at a data level can viewed as estimating a population counterpart "path" that organizes a response mean along regressor variables which are ordered according to a decreasing series of population "correlation" parameters; such parameters are shown to have meaningful interpretations for explaining variable contributions whereby zero correlations denote unimportant variables. In the output of LAR, estimates of all non-zero population correlations turn out to have independent normal distributions for use in inference, while estimates of zero-valued population correlations have a certain non-normal joint distribution. These properties help to provide a formal rule for stopping the LAR algorithm. While the standard bootstrap for regression can fail for LAR, a modified bootstrap provides a practical and formally justified tool for interpreting the entrance of variables and quantifying uncertainty in estimation. The LAR inference method is studied through simulation and illustrated with data examples.

math.ST↗