arXiv subjects

A. B. Owen

Publications and source records attributed to A. B. Owen.

4 recordsLinked to original sources

arXiv2017

Estimation and Inference for Very Large Linear Mixed Effects Models

Linear mixed models with large imbalanced crossed random effects structures pose severe computational problems for maximum likelihood estimation and for Bayesian analysis. The costs can grow as fast as $N^{3/2}$ when there are N observations. Such problems arise in any setting where the underlying factors satisfy a many to many relationship (instead of a nested one) and in electronic commerce applications, the N can be quite large. Methods that do not account for the correlation structure can greatly underestimate uncertainty. We propose a method of moments approach that takes account of the correlation structure and that can be computed at O(N) cost. The method of moments is very amenable to parallel computation and it does not require parametric distributional assumptions, tuning parameters or convergence diagnostics. For the regression coefficients, we give conditions for consistency and asymptotic normality as well as a consistent variance estimate. For the variance components, we give conditions for consistency and we use consistent estimates of a mildly conservative variance estimate. All of these computations can be done in O(N) work. We illustrate the algorithm with some data from Stitch Fix where the crossed random effects correspond to clients and items.

stat.ME↗

arXiv2015

Bi-cross-validation for factor analysis

Factor analysis is over a century old, but it is still problematic to choose the number of factors for a given data set. The scree test is popular but subjective. The best performing objective methods are recommended on the basis of simulations. We introduce a method based on bi-cross-validation, using randomly held-out submatrices of the data to choose the number of factors. We find it performs better than the leading methods of parallel analysis (PA) and Kaiser's rule. Our performance criterion is based on recovery of the underlying factor-loading (signal) matrix rather than identifying the true number of factors. Like previous comparisons, our work is simulation based. Recent advances in random matrix theory provide principled choices for the number of factors when the noise is homoscedastic, but not for the heteroscedastic case. The simulations we choose are designed using guidance from random matrix theory. In particular, we include factors too small to detect, factors large enough to detect but not large enough to improve the estimate, and two classes of factors large enough to be useful. Much of the advantage of bi-cross-validation comes from cases with factors large enough to detect but too small to be well estimated. We also find that a form of early stopping regularization improves the recovery of the signal matrix.

stat.ME↗

arXiv2015

Scrambled geometric net integration over general product spaces

Quasi-Monte Carlo (QMC) sampling has been developed for integration over $[0,1]^s$ where it has superior accuracy to Monte Carlo (MC) for integrands of bounded variation. Scrambled net quadrature gives allows replication based error estimation for QMC with at least the same accuracy and for smooth enough integrands even better accuracy than plain QMC. Integration over triangles, spheres, disks and Cartesian products of such spaces is more difficult for QMC because the induced integrand on a unit cube may fail to have the desired regularity. In this paper, we present a construction of point sets for numerical integration over Cartesian products of $s$ spaces of dimension $d$, with triangles ($d=2$) being of special interest. The point sets are transformations of randomized $(t,m,s)$-nets using recursive geometric partitions. The resulting integral estimates are unbiased and their variance is $o(1/n)$ for any integrand in $L^2$ of the product space. Under smoothness assumptions on the integrand, our randomized QMC algorithm has variance $O(n^{-1 - 2/d} (\log n)^{s-1})$, for integration over $s$-fold Cartesian products of $d$-dimensional domains, compared to $O(n^{-1})$ for ordinary Monte Carlo.

math.NA↗

arXiv2011

Consistency of Markov chain quasi-Monte Carlo on continuous state spaces

The random numbers driving Markov chain Monte Carlo (MCMC) simulation are usually modeled as independent U(0,1) random variables. Tribble [Markov chain Monte Carlo algorithms using completely uniformly distributed driving sequences (2007) Stanford Univ.] reports substantial improvements when those random numbers are replaced by carefully balanced inputs from completely uniformly distributed sequences. The previous theoretical justification for using anything other than i.i.d. U(0,1) points shows consistency for estimated means, but only applies for discrete stationary distributions. We extend those results to some MCMC algorithms for continuous stationary distributions. The main motivation is the search for quasi-Monte Carlo versions of MCMC. As a side benefit, the results also establish consistency for the usual method of using pseudo-random numbers in place of random ones.

math.ST↗