arXiv subjects

Xiucai Ding

Publications and source records attributed to Xiucai Ding.

At least 37 records · Page 2Linked to original sources

arXiv2022

Impact of signal-to-noise ratio and bandwidth on graph Laplacian spectrum from high-dimensional noisy point cloud

We systematically study the spectrum of kernel-based graph Laplacian (GL) constructed from high-dimensional and noisy random point cloud in the nonnull setup. The problem is motived by studying the model when the clean signal is sampled from a manifold that is embedded in a low-dimensional Euclidean subspace, and corrupted by high-dimensional noise. We quantify how the signal and noise interact over different regions of signal-to-noise ratio (SNR), and report the resulting peculiar spectral behavior of GL. In addition, we explore the impact of chosen kernel bandwidth on the spectrum of GL over different regions of SNR, which lead to an adaptive choice of kernel bandwidth that coincides with the common practice in real data. This result paves the way to a theoretical understanding of how practitioners apply GL when the dataset is noisy.

math.ST↗

arXiv2022

A Riemann--Hilbert approach to the perturbation theory for orthogonal polynomials: Applications to numerical linear algebra and random matrix theory

We establish a new perturbation theory for orthogonal polynomials using a Riemann--Hilbert approach and consider applications in numerical linear algebra and random matrix theory. This new approach shows that the orthogonal polynomials with respect to two measures can be effectively compared using the difference of their Stieltjes transforms on a suitably chosen contour. Moreover, when two measures are close and satisfy some regularity conditions, we use the theta functions of a hyperelliptic Riemann surface to derive explicit and accurate expansion formulae for the perturbed orthogonal polynomials. In contrast to other approaches, a key strength of the methodology is that estimates can remain valid as the degree of the polynomial grows. The results are applied to analyze several numerical algorithms from linear algebra, including the Lanczos tridiagonalization procedure, the Cholesky factorization and the conjugate gradient algorithm. As a case study, we investigate these algorithms applied to a general spiked sample covariance matrix model by considering the eigenvector empirical spectral distribution and its limits. For the first time, we give precise estimates on the output of the algorithms, applied to this wide class of random matrices, as the number of iterations diverges. In this setting, beyond the first order expansion, we also derive a new mesoscopic central limit theorem for the associated orthogonal polynomials and other quantities relevant to numerical algorithms.

math.PR↗

arXiv2022

Local laws for multiplication of random matrices

Consider the random matrix model $A^{1/2} UBU^* A^{1/2},$ where $A$ and $B$ are two $N \times N$ deterministic matrices and $U$ is either an $N \times N$ Haar unitary or orthogonal random matrix. It is well-known that on the macroscopic scale, the limiting empirical spectral distribution (ESD) of the above model is given by the free multiplicative convolution of the limiting ESDs of $A$ and $B,$ denoted as $μ_α\boxtimes μ_β,$ where $μ_α$ and $μ_β$ are the limiting ESDs of $A$ and $B,$ respectively. In this paper, we study the asymptotic microscopic behavior of the edge eigenvalues and eigenvectors statistics. We prove that both the density of $μ_A \boxtimes μ_B,$ where $μ_A$ and $μ_B$ are the ESDs of $A$ and $B,$ respectively and the associated subordination functions have a regular behavior near the edges. Moreover, we establish the local laws near the edges on the optimal scale. In particular, we prove that the entries of the resolvent are close to some functionals depending only on the eigenvalues of $A, B$ and the subordination functions with optimal convergence rates. Our proofs and calculations are based on the techniques developed for the additive model $A+UBU^*$ in [3,5,6,8], and our results can be regarded as the counterparts of [8] for the multiplicative model.

math.PR↗

arXiv2022

Edge statistics of large dimensional deformed rectangular matrices

We consider the edge statistics of large dimensional deformed rectangular matrices of the form $Y_t=Y+\sqrt{t}X,$ where $Y$ is a $p \times n$ deterministic signal matrix whose rank is comparable to $n$, $X$ is a $p\times n$ random noise matrix with centered i.i.d. entries with variance $n^{-1}$, and $t>0$ gives the noise level. This model is referred to as the interference-plus-noise matrix in the study of massive multiple-input multiple-output (MIMO) system, which belongs to the category of the so-called signal-plus-noise model. For the case $t=1$, the spectral statistics of this model have been studied to a certain extent in the literature. In this paper, we study the singular value and singular vector statistics of $Y_t$ around the right-most edge of the singular value spectrum in the harder regime $n^{-2/3}\ll t \ll 1$. This regime is harder than the $t=1$ case, because on one hand, the edge behavior of the empirical spectral distribution (ESD) of $YY^\top$ has a strong effect on the edge statistics of $Y_tY_t^\top$ since $t\ll 1$ is "small", while on the other hand, the edge statistics of $Y_t$ is also not merely a perturbation of those of $Y$ since $t\gg n^{-2/3}$ is "large". Under certain regularity assumptions on $Y,$ we prove the edge universality, eigenvalues rigidity and eigenvector delocalization for the matrices $Y_tY_t^\top$ and $Y_t^\top Y_t$. These results can be used to estimate and infer the massive MIMO system. To prove the main results, we analyze the edge behavior of the asymptotic ESD for $Y_tY_t^\top$, and establish some sharp local laws on the resolvent of $Y_tY_t^\top$. These results can be of independent interest, and used as useful inputs for many other problems regarding the spectral statistics of $Y_t$.

math.PR↗

arXiv2022

Tracy-Widom distribution for heterogeneous Gram matrices with applications in signal detection

Detection of the number of signals corrupted by high-dimensional noise is a fundamental problem in signal processing and statistics. This paper focuses on a general setting where the high-dimensional noise has an unknown complicated heterogeneous variance structure. We propose a sequential test which utilizes the edge singular values (i.e., the largest few singular values) of the data matrix. It also naturally leads to a consistent sequential testing estimate of the number of signals. We describe the asymptotic distribution of the test statistic in terms of the Tracy-Widom distribution. The test is shown to be accurate and have full power against the alternative, both theoretically and numerically. The theoretical analysis relies on establishing the Tracy-Widom law for a large class of Gram type random matrices with non-zero means and completely arbitrary variance profiles, which can be of independent interest.

math.ST↗

arXiv2022

Modified Multidimensional Scaling and High Dimensional Clustering

Multidimensional scaling is an important dimension reduction tool in statistics and machine learning. Yet few theoretical results characterizing its statistical performance exist, not to mention any in high dimensions. By considering a unified framework that includes low, moderate and high dimensions, we study multidimensional scaling in the setting of clustering noisy data. Our results suggest that, the classical multidimensional scaling can be modified to further improve the quality of embedded samples, especially when the noise level increases. To this end, we propose {\it modified multidimensional scaling} which applies a nonlinear transformation to the sample eigenvalues. The nonlinear transformation depends on the dimensionality, sample size and moment of noise. We show that modified multidimensional scaling followed by various clustering algorithms can achieve exact recovery, i.e., all the cluster labels can be recovered correctly with probability tending to one. Numerical simulations and two real data applications lend strong support to our proposed methodology.

stat.ME↗

arXiv2021

Simultaneous Sieve Inference for Time-Inhomogeneous Nonlinear Time Series Regression

In this paper, we consider the time-inhomogeneous nonlinear time series regression for a general class of locally stationary time series. On one hand, we propose sieve nonparametric estimators for the time-varying regression functions which can achieve the min-max optimal rate. On the other hand, we develop a unified simultaneous inferential theory which can be used to conduct both structural and exact form testings on the functions. Our proposed statistics are powerful even under locally weak alternatives. We also propose a multiplier bootstrapping procedure for practical implementation. Our methodology and theory do not require any structural assumptions on the regression functions and we also allow the functions to be supported in an unbounded domain. We also establish sieve approximation theory for 2-D functions in unbounded domain and a Gaussian approximation result for affine and quadratic forms for high dimensional locally stationary time series, which can be of independent interest. Numerical simulations and a real financial data analysis are provided to support our results.

math.ST↗

arXiv2021

How do kernel-based sensor fusion algorithms behave under high dimensional noise?

We study the behavior of two kernel based sensor fusion algorithms, nonparametric canonical correlation analysis (NCCA) and alternating diffusion (AD), under the nonnull setting that the clean datasets collected from two sensors are modeled by a common low dimensional manifold embedded in a high dimensional Euclidean space and the datasets are corrupted by high dimensional noise. We establish the asymptotic limits and convergence rates for the eigenvalues of the associated kernel matrices assuming that the sample dimension and sample size are comparably large, where NCCA and AD are conducted using the Gaussian kernel. It turns out that both the asymptotic limits and convergence rates depend on the signal-to-noise ratio (SNR) of each sensor and selected bandwidths. On one hand, we show that if NCCA and AD are directly applied to the noisy point clouds without any sanity check, it may generate artificial information that misleads scientists' interpretation. On the other hand, we prove that if the bandwidths are selected adequately, both NCCA and AD can be made robust to high dimensional noise when the SNRs are relatively large.

stat.ML↗

arXiv2021

The conjugate gradient algorithm on a general class of spiked covariance matrices

We consider the conjugate gradient algorithm applied to a general class of spiked sample covariance matrices. The main result of the paper is that the norms of the error and residual vectors at any finite step concentrate on deterministic values determined by orthogonal polynomials with respect to a deformed Marchenko--Pastur law. The first-order limits and fluctuations are shown to be universal. Additionally, for the case where the bulk eigenvalues lie in a single interval we show a stronger universality result in that the asymptotic rate of convergence of the conjugate gradient algorithm only depends on the support of the bulk, provided the spikes are well-separated from the bulk. In particular, this shows that the classical condition number bound for the conjugate gradient algorithm is pessimistic for spiked matrices.

math.NA↗

arXiv2020

Multivariate functional responses low rank regression with an application to brain imaging data

We propose a multivariate functional responses low rank regression model with possible high dimensional functional responses and scalar covariates. By expanding the slope functions on a set of sieve basis, we reconstruct the basis coefficients as a matrix. To estimate these coefficients, we propose an efficient procedure using nuclear norm regularization. We also derive error bounds for our estimates and evaluate our method using simulations. We further apply our method to the Human Connectome Project neuroimaging data to predict cortical surface motor task-evoked functional magnetic resonance imaging signals using various clinical covariates to illustrate the usefulness of our results.

stat.ME↗

arXiv2020

On the spectral property of kernel-based sensor fusion algorithms of high dimensional data

We apply local laws of random matrices and free probability theory to study the spectral properties of two kernel-based sensor fusion algorithms, nonparametric canonical correlation analysis (NCCA) and alternating diffusion (AD), for two simultaneously recorded high dimensional datasets under the null hypothesis. The matrix of interest is the product of the kernel matrices associated with the databsets, which may not be diagonalizable in general. We prove that in the regime where dimensions of both random vectors are comparable to the sample size, if NCCA and AD are conducted using a smooth kernel function, then the first few nontrivial eigenvalues will converge to real deterministic values provided the datasets are independent Gaussian random vectors. Toward the claimed result, we also provide a convergence rate of eigenvalues of a kernel affinity matrix.

math.ST↗

arXiv2020

Statistical inference for principal components of spiked covariance matrices

In this paper, we study the asymptotic behavior of the extreme eigenvalues and eigenvectors of the high dimensional spiked sample covariance matrices, in the supercritical case when a reliable detection of spikes is possible. Especially, we derive the joint distribution of the extreme eigenvalues and the generalized components of the associated eigenvectors, i.e., the projections of the eigenvectors onto arbitrary given direction, assuming that the dimension and sample size are comparably large. In general, the joint distribution is given in terms of linear combinations of finitely many Gaussian and Chi-square variables, with parameters depending on the projection direction and the spikes. Our assumption on the spikes is fully general. First, the strengths of spikes are only required to be slightly above the critical threshold and no upper bound on the strengths is needed. Second, multiple spikes, i.e., spikes with the same strength, are allowed. Third, no structural assumption is imposed on the spikes. Thanks to the general setting, we can then apply the results to various high dimensional statistical hypothesis testing problems involving both the eigenvalues and eigenvectors. Specifically, we propose accurate and powerful statistics to conduct hypothesis testing on the principal components. These statistics are data-dependent and adaptive to the underlying true spikes. Numerical simulations also confirm the accuracy and powerfulness of our proposed statistics and illustrate significantly better performance compared to the existing methods in the literature. Especially, our methods are accurate and powerful even when either the spikes are small or the dimension is large.

math.ST↗

arXiv2020

Principal components of spiked covariance matrices in the supercritical regime

In this paper, we study the asymptotic behavior of the extreme eigenvalues and eigenvectors of the spiked covariance matrices, in the supercritical regime. Specifically, we derive the joint distribution of the extreme eigenvalues and the generalized components of their associated eigenvectors in this regime.

math.ST↗

arXiv2020

Singular vector and singular subspace distribution for the matrix denoising model

In this paper, we study the matrix denosing model $Y=S+X$, where $S$ is a low-rank deterministic signal matrix and $X$ is a random noise matrix, and both are $M\times n$. In the scenario that $M$ and $n$ are comparably large and the signals are supercritical, we study the fluctuation of the outlier singular vectors of $Y$. More specifically, we derive the limiting distribution of angles between the principal singular vectors of $Y$ and their deterministic counterparts, the singular vectors of $S$. Further, we also derive the distribution of the distance between the subspace spanned by the principal singular vectors of $Y$ and that spanned by the singular vectors of $S$. It turns out that the limiting distributions depend on the structure of the singular vectors of $S$ and the distribution of $X$, and thus they are non-universal.

math.ST↗

arXiv2020

Spiked separable covariance matrices and principal components

We introduce a class of separable sample covariance matrices of the form $\widetilde{\mathcal{Q}}_1:=\widetilde A^{1/2} X \widetilde B X^* \widetilde A^{1/2}.$ Here $\widetilde{A}$ and $\widetilde{B}$ are positive definite matrices whose spectrums consist of bulk spectrums plus several spikes, i.e. larger eigenvalues that are separated from the bulks. Conceptually, we call $\widetilde{\mathcal{Q}}_1$ a \emph{spiked separable covariance matrix model}. On the one hand, this model includes the spiked covariance matrix as a special case with $\widetilde{B}=I$. On the other hand, it allows for more general correlations of datasets. In particular, for spatio-temporal dataset, $\widetilde{A}$ and $\widetilde{B}$ represent the spatial and temporal correlations, respectively. In this paper, we study the outlier eigenvalues and eigenvectors, i.e. the principal components, of the spiked separable covariance model $\widetilde{\mathcal{Q}}_1$. We prove the convergence of the outlier eigenvalues $\widetilde λ_i$ and the generalized components (i.e. $\langle \mathbf v, \widetilde{\mathbfξ}_i \rangle$ for any deterministic vector $\mathbf v$) of the outlier eigenvectors $\widetilde{\mathbfξ}_i$ with optimal convergence rates. Moreover, we also prove the delocalization of the non-outlier eigenvectors. We state our results in full generality, in the sense that they also hold near the so-called BBP transition and for degenerate outliers. Our results highlight both the similarity and difference between the spiked separable covariance matrix model and the spiked covariance model. In particular, we show that the spikes of both $\widetilde{A}$ and $\widetilde{B}$ will cause outliers of the eigenvalue spectrum, and the eigenvectors can help us to select the outliers that correspond to the spikes of $\widetilde{A}$ (or $\widetilde{B}$).

math.ST↗

arXiv2019

Spiked sample covariance matrices with possibly multiple bulk components

In this paper, we study the convergent limits and rates of the eigenvalues and eigenvectors for spiked sample covariance matrices whose spectrum can have multiple bulk components. Our model is an extension of Johnstone's spiked covariance matrix model. Based on our results, we can extend many statistical applications based on Johnstone's spiked covariance matrix model.

math.PR↗

arXiv2019

Globally Optimal And Adaptive Short-Term Forecast of Locally Stationary Time Series And A Test for Its Stability

Forecasting the evolution of complex systems is one of the grand challenges of modern data science. The fundamental difficulty lies in understanding the structure of the observed stochastic process. In this paper, we show that every uniformly-positive-definite-in-covariance and sufficiently short-range dependent non-stationary and nonlinear time series can be well approximated globally by an auto-regressive process of slowly diverging order. When linear prediction with ${\cal L}^2$ loss is concerned, the latter result facilitates a unified globally-optimal short-term forecasting theory for a wide class of locally stationary time series asymptotically. A nonparametric sieve method is proposed to globally and adaptively estimate the optimal forecasting coefficient functions and the associated mean squared error of forecast. An adaptive stability test is proposed to check whether the optimal forecasting coefficients are time-varying, a frequently-encountered question for practitioners and researchers of time series. Furthermore, partial auto-correlation functions (PACF) of general non-stationary time series are studied and used as a visual tool to explore the linear dependence structure of such series. We use extensive numerical simulations and two real data examples to illustrate the usefulness of our results.

math.ST↗

arXiv2019

High dimensional deformed rectangular matrices with applications in matrix denoising

We consider the recovery of a low rank $M \times N$ matrix $S$ from its noisy observation $\tilde{S}$ in two different regimes. Under the assumption that $M$ is comparable to $N$, we propose two consistent estimators for $S$. Our analysis relies on the local behavior of the large dimensional rectangular matrices with finite rank perturbation. We also derive the convergent limits and rates for the singular values and vectors of such matrices.

math.ST↗