Searcharxiv⌕ Search

arXiv subjects

Jianfeng Yao

Publications and source records attributed to Jianfeng Yao.

At least 37 records · Page 2Linked to original sources

Limiting distributions for eigenvalues of sample correlation matrices from heavy-tailed populations

Consider a $p$-dimensional population ${\mathbf x} \in\mathbb{R}^p$ with iid coordinates in the domain of attraction of a stable distribution with index $α\in (0,2)$. Since the variance of ${\mathbf x}$ is infinite, the sample covariance matrix ${\mathbf S}_n=n^{-1}\sum_{i=1}^n {{\mathbf x}_i}{\mathbf x}'_i$ based on a sample ${\mathbf x}_1,\ldots,{\mathbf x}_n$ from the population is not well behaved and it is of interest to use instead the sample correlation matrix ${\mathbf R}_n= \{\operatorname{diag}({\mathbf S}_n)\}^{-1/2}\, {\mathbf S}_n \{\operatorname{diag}({\mathbf S}_n)\}^{-1/2}$. This paper finds the limiting distributions of the eigenvalues of ${\mathbf R}_n$ when both the dimension $p$ and the sample size $n$ grow to infinity such that $p/n\to γ\in (0,\infty)$. The family of limiting distributions $\{H_{α,γ}\}$ is new and depends on the two parameters $α$ and $γ$. The moments of $H_{α,γ}$ are fully identified as sum of two contributions: the first from the classical Marčenko-Pastur law and a second due to heavy tails. Moreover, the family $\{H_{α,γ}\}$ has continuous extensions at the boundaries $α=2$ and $α=0$ leading to the Marčenko-Pastur law and a modified Poisson distribution, respectively. Our proofs use the method of moments, the path-shortening algorithm developed in [18] and some novel graph counting combinatorics. As a consequence, the moments of $H_{α,γ}$ are expressed in terms of combinatorial objects such as Stirling numbers of the second kind. A simulation study on these limiting distributions $H_{α,γ}$ is also provided for comparison with the Marčenko-Pastur law.

math.PR↗

An Eigenvalue Ratio Approach to Inferring Population Structure from Whole Genome Sequencing Data

Inference of population structure from genetic data plays an important role in population and medical genetics studies. With the advancement and decreasing cost of sequencing technology, the increasingly available whole genome sequencing data provide much richer information about the underlying population structure. The traditional method (Patterson, Price, and Reich, 2006) originally developed for array-based genotype data for computing and selecting top principal components that capture population structure may not perform well on sequencing data for two reasons. First, the number of genetic variants p is much larger than the sample size n in sequencing data such that the sample-to-marker ratio n/p is nearly zero, violating the assumption of the Tracy-Widom test used in their method. Second, their method might not be able to handle the linkage disequilibrium well in sequencing data. To resolve those two practical issues, we propose a new method called ERStruct to determine the number of top informative principal components based on sequencing data. More specifically, we propose to use the ratio of consecutive eigenvalues as a more robust test statistic, and then we approximate its null distribution using modern random matrix theory. Both simulation studies and applications to two public data sets from the HapMap 3 and the 1000 Genomes Projects demonstrate the empirical performance of our ERStruct method.

stat.AP↗

Impact of classification difficulty on the weight matrices spectra in Deep Learning and application to early-stopping

Much research effort has been devoted to explaining the success of deep learning. Random Matrix Theory (RMT) provides an emerging way to this end: spectral analysis of large random matrices involved in a trained deep neural network (DNN) such as weight matrices or Hessian matrices with respect to the stochastic gradient descent algorithm. To have more comprehensive understanding of weight matrices spectra, we conduct extensive experiments on weight matrices in different modules, e.g., layers, networks and data sets. Following the previous work of \cite{martin2018implicit}, we classify the spectra in the terminal stage into three main types: Light Tail (LT), Bulk Transition period (BT) and Heavy Tail(HT). These different types, especially HT, implicitly indicate some regularization in the DNNs. A main contribution from the paper is that we identify the difficulty of the classification problem as a driving factor for the appearance of heavy tail in weight matrices spectra. Higher the classification difficulty, higher the chance for HT to appear. Moreover, the classification difficulty can be affected by the signal-to-noise ratio of the dataset, or by the complexity of the classification problem (complex features, large number of classes) as well. Leveraging on this finding, we further propose a spectral criterion to detect the appearance of heavy tails and use it to early stop the training process without testing data. Such early stopped DNNs have the merit of avoiding overfitting and unnecessary extra training while preserving a much comparable generalization ability. These findings from the paper are validated in several NNs, using Gaussian synthetic data and real data sets (MNIST and CIFAR10).

cs.LG↗

On spectral distribution of sample covariance matrices from large dimensional and large $k$-fold tensor products

We study the eigenvalue distributions for sums of independent rank-one $k$-fold tensor products of large $n$-dimensional vectors. Previous results in the literature assume that $k=o(n)$ and show that the eigenvalue distributions converge to the celebrated Marčenko-Pastur law under appropriate moment conditions on the base vectors. In this paper, motivated by quantum information theory, we study the regime where $k$ grows faster, namely $k=O(n)$. We show that the moment sequences of the eigenvalue distributions have a limit, which is different from the Marčenko-Pastur law. As a byproduct, we show that the Marčenko-Pastur law limit holds if and only if $k=o(n)$ for this tensor model. The approach is based on the method of moments.

math.PR↗

Central limit theorem for linear spectral statistics of block-Wigner-type matrices

Motivated by the stochastic block model, we investigate a class of Wigner-type matrices with certain block structures, and establish a CLT for the corresponding linear spectral statistics via the large-deviation bounds from local law and the cumulant expansion formula. We apply the results to the stochastic block model. Specifically, a class of renormalized adjacency matrices will be block-Wigner-type matrices. Further, we show that for certain estimator of such renormalized adjacency matrices, which will be no longer Wigner-type but share long-range non-decaying weak correlations among the entries, the linear spectral statistics of such estimators will still share the same limiting behavior as those of the block-Wigner-type matrices, thus enabling hypothesis testing about stochastic block model.

math.PR↗

Recent advances on eigenvalues of matrix-valued stochastic processes

Since the introduction of Dyson's Brownian motion in early 1960's, there have been a lot of developments in the investigation of stochastic processes on the space of Hermitian matrices. Their properties, especially, the properties of their eigenvalues have been studied in great details. In particular, the limiting behaviors of the eigenvalues are found when the dimension of the matrix space tends to infinity, which connects with random matrix theory. This survey reviews a selection of results on the eigenvalues of stochastic processes from the literature of the past three decades. For most recent variations of such processes, such as matrix-valued processes driven by fractional Brownian motion or Brownian sheet, the eigenvalues of them are also discussed in this survey. In the end, some open problems in the area are also proposed.

math.PR↗

Asymptotic normality for eigenvalue statistics of a general sample covariance matrix when $p/n \to \infty$ and applications

The asymptotic normality for a large family of eigenvalue statistics of a general sample covariance matrix is derived under the ultra-high dimensional setting, that is, when the dimension to sample size ratio $p/n \to \infty$. Based on this CLT result, we first adapt the covariance matrix test problem to the new ultra-high dimensional context. Then as a second application, we develop a new test for the separable covariance structure of a matrix-valued white noise. Simulation experiments are conducted for the investigation of finite-sample properties of the general asymptotic normality of eigenvalue statistics, as well as the second test for separable covariance structure of matrix-valued white noise.

stat.ME↗

On singular values of data matrices with general independent columns

In this paper, we analyse singular values of a large $p\times n$ data matrix $\mathbf{X}_n= (\mathbf{x}_{n1},\ldots,\mathbf{x}_{nn})$ where the column $\mathbf{x}_{nj}$'s are independent $p$-dimensional vectors, possibly with different distributions. Such data matrices are common in high-dimensional statistics. Under a key assumption that the covariance matrices $\mathbfΣ_{nj}=\text{Cov}(\mathbf{x}_{nj})$ can be asymptotically simultaneously diagonalizable, and appropriate convergence of their spectra, we establish a limiting distribution for the singular values of $\mathbf{X}_n$ when both dimension $p$ and $n$ grow to infinity in a comparable magnitude. The matrix model goes beyond and includes many existing works on different types of sample covariance matrices, including the weighted sample covariance matrix, the Gram matrix model and the sample covariance matrix of linear times series models. Furthermore, we develop two applications of our general approach. First, we obtain the existence and uniqueness of a new limiting spectral distribution of realized covariance matrices for a multi-dimensional diffusion process with anisotropic time-varying co-volatility processes. Secondly, we derive the limiting spectral distribution for singular values of the data matrix for a recent matrix-valued auto-regressive model. Finally, for a generalized finite mixture model, the limiting spectral distribution for singular values of the data matrix is obtained.

math.ST↗

Linear regression under model uncertainty

We reexamine the classical linear regression model when the model is subject to two types of uncertainty: (i) some of covariates are either missing or completely inaccessible, and (ii) the variance of the measurement error is undetermined and changing according to a mechanism unknown to the statistician. By following the recent theory of sublinear expectation, we propose to characterize such mean and variance uncertainty in the response variable by two specific nonlinear random variables, which encompass an infinite family of probability distributions for the response variable in the sense of (linear) classical probability theory. The approach enables a family of estimators under various loss functions for the regression parameter and the parameters related to model uncertainty. The consistency of the estimators is established under mild conditions on the data generation process. Three applications are introduced to assess the quality of the approach including a forecasting model for the S\&P Index.

math.ST↗

Eigenvalue distribution of a high-dimensional distance covariance matrix with application

We introduce a new random matrix model called distance covariance matrix in this paper, whose normalized trace is equivalent to the distance covariance. We first derive a deterministic limit for the eigenvalue distribution of the distance covariance matrix when the dimensions of the vectors and the sample size tend to infinity simultaneously. This limit is valid when the vectors are independent or weakly dependent through a finite-rank perturbation. It is also universal and independent of the details of the distributions of the vectors. Furthermore, the top eigenvalues of this distance covariance matrix are shown to obey an exact phase transition when the dependence of the vectors is of finite rank. This finding enables the construction of a new detector for such weak dependence where classical methods based on large sample covariance matrices or sample canonical correlations may fail in the considered high-dimensional framework.

math.ST↗

On eigenvalue distributions of large auto-covariance matrices

In this article, we establish a limiting distribution for eigenvalues of a class of auto-covariance matrices. The same distribution has been found in the literature for a regularized version of these auto-covariance matrices. The original non-regularized auto-covariance matrices are non invertible which introduce supplementary diffculties for the study of their eigenvalues through Girko's Hermitization scheme. The key result in this paper is a new polynomial lower bound for the least singular value of the resolvent matrices associated to a rank-defective quadratic function of a random matrix with independent and identically distributed entries. Another improvement in the paper is that the lag of the auto-covariance matrices can grow to infinity with the matrix dimension.

math.PR↗

Extension of the Lagrange multiplier test for error cross-section independence to large panels with non normal errors

This paper reexamines the seminal Lagrange multiplier test for cross-section independence in a large panel model where both the number of cross-sectional units n and the number of time series observations T can be large. The first contribution of the paper is an enlargement of the test with two extensions: firstly the new asymptotic normality is derived in a simultaneous limiting scheme where the two dimensions (n, T) tend to infinity with comparable magnitudes; second, the result is valid for general error distribution (not necessarily normal). The second contribution of the paper is a new test statistic based on the sum of the fourth powers of cross-section correlations from OLS residuals, instead of their squares used in the Lagrange multiplier statistic. This new test is generally more powerful, and the improvement is particularly visible against alternatives with weak or sparse cross-section dependence. Both simulation study and real data analysis are proposed to demonstrate the advantages of the enlarged Lagrange multiplier test and the power enhanced test in comparison with the existing procedures.

econ.EM↗

On eigenvalues of a high-dimensional spatial-sign covariance matrix

This paper investigates limiting properties of eigenvalues of multivariate sample spatial-sign covariance matrices when both the number of variables and the sample size grow to infinity. The underlying p-variate populations are general enough to include the popular independent components model and the family of elliptical distributions. A first result of the paper establishes that the distribution of the eigenvalues converges to a deterministic limit that belongs to the family of generalized Marcenko-Pastur distributions. Furthermore, a new central limit theorem is established for a class of linear spectral statistics. We develop two applications of these results to robust statistics for a high-dimensional shape matrix. First, two statistics are proposed for testing the sphericity. Next, a spectrum-corrected estimator using the sample spatial-sign covariance matrix is proposed. Simulation experiments show that in high dimension, the sample spatial-sign covariance matrix provides a valid and robust tool for mitigating influence of outliers.

math.ST↗

Eigenvalue distributions of high-dimensional matrix processes driven by fractional Brownian motion

In this article, we study high-dimensional behavior of empirical spectral distributions $\{L_N(t), t\in[0,T]\}$ for a class of $N\times N$ symmetric/Hermitian random matrices, whose entries are generated from the solution of stochastic differential equation driven by fractional Brownian motion with Hurst parameter $H \in(1/2,1)$. For Wigner-type matrices, we obtain almost sure relative compactness of $\{L_N(t), t\in[0,T]\}_{N\in\mathbb N}$ in $C([0,T], \mathbf P(\mathbb R))$ following the approach in \cite{Anderson2010}; for Wishart-type matrices, we obtain tightness of $\{L_N(t), t\in[0,T]\}_{N\in\mathbb N}$ on $C([0,T], \mathbf P(\mathbb R))$ by tightness criterions provided in Appendix \ref{subset:tightness argument}. The limit of $\{L_N(t), t\in[0,T]\}$ as $N\to \infty$ is also characterised.

math.PR↗

Improving Value-at-Risk prediction under model uncertainty

Several well-established benchmark predictors exist for Value-at-Risk (VaR), a major instrument for financial risk management. Hybrid methods combining AR-GARCH filtering with skewed-$t$ residuals and the extreme value theory-based approach are particularly recommended. This study introduces yet another VaR predictor, G-VaR, which follows a novel methodology. Inspired by the recent mathematical theory of sublinear expectation, G-VaR is built upon the concept of model uncertainty, which in the present case signifies that the inherent volatility of financial returns cannot be characterized by a single distribution but rather by infinitely many statistical distributions. By considering the worst scenario among these potential distributions, the G-VaR predictor is precisely identified. Extensive experiments on both the NASDAQ Composite Index and S\&P500 Index demonstrate the excellent performance of the G-VaR predictor, which is superior to most existing benchmark VaR predictors.

q-fin.RM↗

On Laplacian spectrum of dendrite trees

For dendrite graphs from biological experiments on mouse's retinal ganglion cells, a paper by Nakatsukasa, Saito and Woei reveals a mysterious phase transition phenomenon in the spectra of the corresponding graph Laplacian matrices. While the bulk of the spectrum can be well understood by structures resembling starlike trees, mysteries about the spikes, that is, isolated eigenvalues outside the bulk spectrum, remain unexplained. In this paper, we bring new insights on these mysteries by considering a class of uniform trees. Exact relationships between the number of such spikes and the number of T-junctions are analyzed in function of the number of vertices separating the T-junctions. Using these theoretic results, predictions are proposed for the number of spikes observed in real-life dendrite graphs. Interestingly enough, these predictions match well the observed numbers of spikes, thus confirm the practical meaningness of our theoretical results.

math.ST↗

High-dimensional limits of eigenvalue distributions for general Wishart process

In this article, we obtain an equation for the high-dimensional limit measure of eigenvalues of generalized Wishart processes, and the results is extended to random particle systems that generalize SDEs of eigenvalues. We also introduce a new set of conditions on the coefficient matrices for the existence and uniqueness of a strong solution for the SDEs of eigenvalues. The equation of the limit measure is further discussed assuming self-similarity on the eigenvalues.

math.PR↗

High-dimensional central limit theorems for eigenvalue distributions of generalized Wishart processes

We consider eigenvalues of generalized Wishart processes as well as particle systems, of which the empirical measures converge to deterministic measures as the dimension goes to infinity. In this paper, we obtain central limit theorems to characterize the fluctuations of the empirical measures around the limit measures by using stochastic calculus. As applications, central limit theorems for the Dyson's Brownian motion and the eigenvalues of the Wishart process are recovered under slightly more general initial conditions, and a central limit theorem for the eigenvalues of a symmetric Ornstein-Uhlenbeck matrix process is obtained.

math.PR↗