arXiv subjects

Miles E. Lopes

Publications and source records attributed to Miles E. Lopes.

At least 19 recordsLinked to original sources

arXiv2026

A Goodness-of-Fit Test for Independent Component Models in High Dimensions

Independent component (IC) models are a standard tool for representing multivariate data in statistics, signal processing, and machine learning. Despite the extensive use of IC models, much less attention has been given to goodness-of-fit tests for assessing their compatibility with data. We develop the first goodness-of-fit test for IC models that is supported by a theoretical validity guarantee when the data dimension and sample size diverge proportionally. This is made possible by the fact that the test does not rely on a pre-whitening step, which often limits the applicability of other goodness-of-fit tests in high dimensions. Our theoretical analysis is complemented with numerical experiments that demonstrate the test's size control and power under a range of conditions. In addition, we provide examples involving gene-expression data to illustrate that the test has potential for effective diagnostic use in practice.

math.ST↗

arXiv2025

Robust Max Statistics for High-Dimensional Inference

Although much progress has been made in the theory and application of bootstrap approximations for max statistics in high dimensions, the literature has largely been restricted to cases involving light-tailed data. To address this issue, we propose an approach to inference based on robust max statistics, and we show that their distributions can be accurately approximated via bootstrapping when the data are both high-dimensional and heavy-tailed. In particular, the data are assumed to satisfy an extended version of the well-established $L^{4}$-$L^2$ moment equivalence condition, as well as a weak variance decay condition. In this setting, we show that near-parametric rates of bootstrap approximation can be achieved in the Kolmogorov metric, independently of the data dimension. Moreover, this theoretical result is complemented by encouraging empirical results involving both Euclidean and functional data.

stat.ME↗

arXiv2025

Tracy-Widom, Gaussian, and Bootstrap: Approximations for Leading Eigenvalues in High-Dimensional PCA

Under certain conditions, the largest eigenvalue of a sample covariance matrix undergoes a well-known phase transition when the sample size $n$ and data dimension $p$ diverge proportionally. In the subcritical regime, this eigenvalue has fluctuations of order $n^{-2/3}$ that can be approximated by a Tracy-Widom distribution, while in the supercritical regime, it has fluctuations of order $n^{-1/2}$ that can be approximated with a Gaussian distribution. However, the statistical problem of determining which regime underlies a given dataset is far from resolved. We develop a new testing framework and procedure to address this problem. In particular, we demonstrate that the procedure has an asymptotically controlled level, and that it is power consistent for certain alternatives. Also, this testing procedure enables the design a new bootstrap method for approximating the distributions of functionals of the leading sample eigenvalues within the subcritical regime -- which is the first such method that is supported by theoretical guarantees.

math.ST↗

arXiv2025

Empirical Error Estimates for Graph Sparsification

Graph sparsification is a well-established technique for accelerating graph-based learning algorithms, which uses edge sampling to approximate dense graphs with sparse ones. Because the sparsification error is random and unknown, users must contend with uncertainty about the reliability of downstream computations. Although it is possible for users to obtain conceptual guidance from theoretical error bounds in the literature, such results are typically impractical at a numerical level. Taking an alternative approach, we propose to address these issues from a data-driven perspective by computing empirical error estimates. The proposed error estimates are highly versatile, and we demonstrate this in four use cases: Laplacian matrix approximation, graph cut queries, graph-structured regression, and spectral clustering. Moreover, we provide two theoretical guarantees for the error estimates, and explain why the cost of computing them is manageable in comparison to the overall cost of a typical graph sparsification workflow.

cs.LG↗

arXiv2025

Testing Elliptical Models in High Dimensions

Due to the broad applications of elliptical models, there is a long line of research on goodness-of-fit tests for empirically validating them. However, the existing literature on this topic is generally confined to low-dimensional settings, and to the best of our knowledge, there are no established goodness-of-fit tests for elliptical models that are supported by theoretical guarantees in high dimensions. In this paper, we propose a new goodness-of-fit test for this problem, and our main result shows that the test is asymptotically valid when the dimension and sample size diverge proportionally. Remarkably, it also turns out that the asymptotic validity of the test requires no assumptions on the population covariance matrix. With regard to numerical performance, we confirm that the empirical level of the test is close to the nominal level across a range of conditions, and that the test is able to reliably detect non-elliptical distributions. Moreover, when the proposed test is specialized to the problem of testing normality in high dimensions, we show that it compares favorably with a state-of-the-art method, and hence, this way of using the proposed test is of independent interest.

math.ST↗

arXiv2024

Improved Rates of Bootstrap Approximation for the Operator Norm: A Coordinate-Free Approach

Let $\hatΣ=\frac{1}{n}\sum_{i=1}^n X_i\otimes X_i$ denote the sample covariance operator of centered i.i.d.~observations $X_1,\dots,X_n$ in a real separable Hilbert space, and let $Σ=\mathbb{E}(X_1\otimes X_1)$. The focus of this paper is to understand how well the bootstrap can approximate the distribution of the operator norm error $\sqrt n\|\hatΣ-Σ\|_{\text{op}}$, in settings where the eigenvalues of $Σ$ decay as $λ_j(Σ)\asymp j^{-2β}$ for some fixed parameter $β>1/2$. Our main result shows that the bootstrap can approximate the distribution of $\sqrt n\|\hatΣ-Σ\|_{\text{op}}$ at a rate of order $n^{-\frac{β-1/2}{2β+4+ε}}$ with respect to the Kolmogorov metric, for any fixed $ε>0$. In particular, this shows that the bootstrap can achieve near $n^{-1/2}$ rates in the regime of large $β$ -- which substantially improves on previous near $n^{-1/6}$ rates in the same regime. In addition to obtaining faster rates, our analysis leverages a fundamentally different perspective based on coordinate-free techniques. Moreover, our result holds in greater generality, and we propose a model that is compatible with both elliptical and Marčenko-Pastur models in high-dimensional Euclidean spaces, which may be of independent interest.

math.ST↗

arXiv2023

A Bootstrap Method for Spectral Statistics in High-Dimensional Elliptical Models

Although there is an extensive literature on the eigenvalues of high-dimensional sample covariance matrices, much of it is specialized to independent components (IC) models -- in which observations are represented as linear transformations of random vectors with independent entries. By contrast, less is known in the context of elliptical models, which violate the independence structure of IC models and exhibit quite different statistical phenomena. In particular, very little is known about the scope of bootstrap methods for doing inference with spectral statistics in high-dimensional elliptical models. To fill this gap, we show how a bootstrap approach developed previously for IC models can be extended to handle the different properties of elliptical models. Within this setting, our main theoretical result guarantees that the proposed method consistently approximates the distributions of linear spectral statistics, which play a fundamental role in multivariate analysis. We also provide empirical results showing that the proposed method performs well for a variety of nonlinear spectral statistics.

math.ST↗

arXiv2023

Randomized Numerical Linear Algebra : A Perspective on the Field With an Eye to Software

Randomized numerical linear algebra - RandNLA, for short - concerns the use of randomization as a resource to develop improved algorithms for large-scale linear algebra computations. The origins of contemporary RandNLA lay in theoretical computer science, where it blossomed from a simple idea: randomization provides an avenue for computing approximate solutions to linear algebra problems more efficiently than deterministic algorithms. This idea proved fruitful in the development of scalable algorithms for machine learning and statistical data analysis applications. However, RandNLA's true potential only came into focus upon integration with the fields of numerical analysis and "classical" numerical linear algebra. Through the efforts of many individuals, randomized algorithms have been developed that provide full control over the accuracy of their solutions and that can be every bit as reliable as algorithms that might be found in libraries such as LAPACK. Recent years have even seen the incorporation of certain RandNLA methods into MATLAB, the NAG Library, NVIDIA's cuSOLVER, and SciKit-Learn. For all its success, we believe that RandNLA has yet to realize its full potential. In particular, we believe the scientific community stands to benefit significantly from suitably defined "RandBLAS" and "RandLAPACK" libraries, to serve as standards conceptually analogous to BLAS and LAPACK. This 200-page monograph represents a step toward defining such standards. In it, we cover topics spanning basic sketching, least squares and optimization, low-rank approximation, full matrix decompositions, leverage score sampling, and sketching data with tensor product structures (among others). Much of the provided pseudo-code has been tested via publicly available MATLAB and Python implementations.

math.NA↗

arXiv2023

Error Estimation for Random Fourier Features

Random Fourier Features (RFF) is among the most popular and broadly applicable approaches for scaling up kernel methods. In essence, RFF allows the user to avoid costly computations on a large kernel matrix via a fast randomized approximation. However, a pervasive difficulty in applying RFF is that the user does not know the actual error of the approximation, or how this error will propagate into downstream learning tasks. Up to now, the RFF literature has primarily dealt with these uncertainties using theoretical error bounds, but from a user's standpoint, such results are typically impractical -- either because they are highly conservative or involve unknown quantities. To tackle these general issues in a data-driven way, this paper develops a bootstrap approach to numerically estimate the errors of RFF approximations. Three key advantages of this approach are: (1) The error estimates are specific to the problem at hand, avoiding the pessimism of worst-case bounds. (2) The approach is flexible with respect to different uses of RFF, and can even estimate errors in downstream learning tasks. (3) The approach enables adaptive computation, so that the user can quickly inspect the error of a rough initial kernel approximation and then predict how much extra work is needed. Lastly, in exchange for all of these benefits, the error estimates can be obtained at a modest computational cost.

stat.ML↗

arXiv2022

Randomized Algorithms for Scientific Computing (RASC)

Randomized algorithms have propelled advances in artificial intelligence and represent a foundational research area in advancing AI for Science. Future advancements in DOE Office of Science priority areas such as climate science, astrophysics, fusion, advanced materials, combustion, and quantum computing all require randomized algorithms for surmounting challenges of complexity, robustness, and scalability. This report summarizes the outcomes of that workshop, "Randomized Algorithms for Scientific Computing (RASC)," held virtually across four days in December 2020 and January 2021.

cs.AI↗

arXiv2022

Rates of Bootstrap Approximation for Eigenvalues in High-Dimensional PCA

In the context of principal components analysis (PCA), the bootstrap is commonly applied to solve a variety of inference problems, such as constructing confidence intervals for the eigenvalues of the population covariance matrix $Σ$. However, when the data are high-dimensional, there are relatively few theoretical guarantees that quantify the performance of the bootstrap. Our aim in this paper is to analyze how well the bootstrap can approximate the joint distribution of the leading eigenvalues of the sample covariance matrix $\hatΣ$, and we establish non-asymptotic rates of approximation with respect to the multivariate Kolmogorov metric. Under certain assumptions, we show that the bootstrap can achieve the dimension-free rate of ${\tt{r}}(Σ)/\sqrt n$ up to logarithmic factors, where ${\tt{r}}(Σ)$ is the effective rank of $Σ$, and $n$ is the sample size. From a methodological standpoint, our work also illustrates that applying a transformation to the eigenvalues of $\hatΣ$ before bootstrapping is an important consideration in high-dimensional settings.

math.ST↗

arXiv2022

Central Limit Theorem and Bootstrap Approximation in High Dimensions: Near $1/\sqrt{n}$ Rates via Implicit Smoothing

Non-asymptotic bounds for Gaussian and bootstrap approximation have recently attracted significant interest in high-dimensional statistics. This paper studies Berry-Esseen bounds for such approximations with respect to the multivariate Kolmogorov distance, in the context of a sum of $n$ random vectors that are $p$-dimensional and i.i.d. Up to now, a growing line of work has established bounds with mild logarithmic dependence on $p$. However, the problem of developing corresponding bounds with near $n^{-1/2}$ dependence on $n$ has remained largely unresolved. Within the setting of random vectors that have sub-Gaussian or sub-exponential entries, this paper establishes bounds with near $n^{-1/2}$ dependence, for both Gaussian and bootstrap approximation. In addition, the proofs are considerably distinct from other recent approaches and make use of an "implicit smoothing" operation in the Lindeberg interpolation.

math.ST↗

arXiv2021

A Sharp Lower-tail Bound for Gaussian Maxima with Application to Bootstrap Methods in High Dimensions

Although there is an extensive literature on the maxima of Gaussian processes, there are relatively few non-asymptotic bounds on their lower-tail probabilities. The aim of this paper is to develop such a bound, while also allowing for many types of dependence. Let $(ξ_1,\dots,ξ_N)$ be a centered Gaussian vector with standardized entries, whose correlation matrix $R$ satisfies $\max_{i\neq j} R_{ij}\leq ρ_0$ for some constant $ρ_0\in (0,1)$. Then, for any $ε_0\in(0,\sqrt{1-ρ_0})$, we establish an upper bound on the probability $\mathbb{P}(\max_{1\leq j\leq N} ξ_j\leq ε_0\sqrt{2\log(N)})$ in terms of $(ρ_0,ε_0,N)$. The bound is also sharp, in the sense that it is attained up to a constant, independent of $N$. Next, we apply this result in the context of high-dimensional statistics, where we simplify and weaken conditions that have recently been used to establish near-parametric rates of bootstrap approximation. Lastly, an interesting aspect of this application is that it makes use of recent refinements of Bourgain and Tzafriri's "restricted invertibility principle".

math.PR↗

arXiv2021

High-dimensional MANOVA via Bootstrapping and its Application to Functional and Sparse Count Data

We propose a new approach to the problem of high-dimensional multivariate ANOVA via bootstrapping max statistics that involve the differences of sample mean vectors. The proposed method proceeds via the construction of simultaneous confidence regions for the differences of population mean vectors. It is suited to simultaneously test the equality of several pairs of mean vectors of potentially more than two populations. By exploiting the variance decay property that is a natural feature in relevant applications, we are able to provide dimension-free and nearly-parametric convergence rates for Gaussian approximation, bootstrap approximation, and the size of the test. We demonstrate the proposed approach with ANOVA problems for functional data and sparse count data. The proposed methodology is shown to work well in simulations and several real data applications.

stat.ME↗

arXiv2020

Error Estimation for Sketched SVD via the Bootstrap

In order to compute fast approximations to the singular value decompositions (SVD) of very large matrices, randomized sketching algorithms have become a leading approach. However, a key practical difficulty of sketching an SVD is that the user does not know how far the sketched singular vectors/values are from the exact ones. Indeed, the user may be forced to rely on analytical worst-case error bounds, which do not account for the unique structure of a given problem. As a result, the lack of tools for error estimation often leads to much more computation than is really necessary. To overcome these challenges, this paper develops a fully data-driven bootstrap method that numerically estimates the actual error of sketched singular vectors/values. In particular, this allows the user to inspect the quality of a rough initial sketched SVD, and then adaptively predict how much extra work is needed to reach a given error tolerance. Furthermore, the method is computationally inexpensive, because it operates only on sketched objects, and it requires no passes over the full matrix being factored. Lastly, the method is supported by theoretical guarantees and a very encouraging set of experimental results.

stat.ML↗

arXiv2019

Bootstrapping the Operator Norm in High Dimensions: Error Estimation for Covariance Matrices and Sketching

Although the operator (spectral) norm is one of the most widely used metrics for covariance estimation, comparatively little is known about the fluctuations of error in this norm. To be specific, let $\hatΣ$ denote the sample covariance matrix of $n$ observations in $\mathbb{R}^p$ that arise from a population matrix $Σ$, and let $T_n=\sqrt{n}\|\hatΣ-Σ\|_{\text{op}}$. In the setting where the eigenvalues of $Σ$ have a decay profile of the form $λ_j(Σ)\asymp j^{-2β}$, we analyze how well the bootstrap can approximate the distribution of $T_n$. Our main result shows that up to factors of $\log(n)$, the bootstrap can approximate the distribution of $T_n$ at the dimension-free rate of $n^{-\frac{β-1/2}{6β+4}}$, with respect to the Kolmogorov metric. Perhaps surprisingly, a result of this type appears to be new even in settings where $p< n$. More generally, we discuss the consequences of this result beyond covariance matrices and show how the bootstrap can be used to estimate the errors of sketching algorithms in randomized numerical linear algebra (RandNLA). An illustration of these ideas is also provided with a climate data example.

math.ST↗

arXiv2019

Measuring the Algorithmic Convergence of Randomized Ensembles: The Regression Setting

When randomized ensemble methods such as bagging and random forests are implemented, a basic question arises: Is the ensemble large enough? In particular, the practitioner desires a rigorous guarantee that a given ensemble will perform nearly as well as an ideal infinite ensemble (trained on the same data). The purpose of the current paper is to develop a bootstrap method for solving this problem in the context of regression --- which complements our companion paper in the context of classification (Lopes 2019). In contrast to the classification setting, the current paper shows that theoretical guarantees for the proposed bootstrap can be established under much weaker assumptions. In addition, we illustrate the flexibility of the method by showing how it can be adapted to measure algorithmic convergence for variable selection. Lastly, we provide numerical results demonstrating that the method works well in a range of situations.

stat.ML↗

arXiv2019

Bootstrapping Max Statistics in High Dimensions: Near-Parametric Rates Under Weak Variance Decay and Application to Functional and Multinomial Data

In recent years, bootstrap methods have drawn attention for their ability to approximate the laws of "max statistics" in high-dimensional problems. A leading example of such a statistic is the coordinate-wise maximum of a sample average of $n$ random vectors in $\mathbb{R}^p$. Existing results for this statistic show that the bootstrap can work when $n\ll p$, and rates of approximation (in Kolmogorov distance) have been obtained with only logarithmic dependence in $p$. Nevertheless, one of the challenging aspects of this setting is that established rates tend to scale like $n^{-1/6}$ as a function of $n$. The main purpose of this paper is to demonstrate that improvement in rate is possible when extra model structure is available. Specifically, we show that if the coordinate-wise variances of the observations exhibit decay, then a nearly $n^{-1/2}$ rate can be achieved, independent of $p$. Furthermore, a surprising aspect of this dimension-free rate is that it holds even when the decay is very weak. Lastly, we provide examples showing how these ideas can be applied to inference problems dealing with functional and multinomial data.

math.ST↗