arXiv subjects

Weidong Liu

Publications and source records attributed to Weidong Liu.

At least 55 records · Page 3Linked to original sources

arXiv2018

Spectral Analysis of High-dimensional Time Series

A useful approach for analysing multiple time series is via characterising their spectral density matrix as the frequency domain analog of the covariance matrix. When the dimension of the time series is large compared to their length, regularisation based methods can overcome the curse of dimensionality, but the existing ones lack theoretical justification. This paper develops the first non-asymptotic result for characterising the difference between the sample and population versions of the spectral density matrix, allowing one to justify a range of high-dimensional models for analysing time series. As a concrete example, we apply this result to establish the convergence of the smoothed periodogram estimators and sparse estimators of the inverse of spectral density matrices, namely precision matrices. These results, novel in the frequency domain time series analysis, are corroborated by simulations and an analysis of the Google Flu Trends data.

math.ST↗

arXiv2018

Quantile Regression Under Memory Constraint

This paper studies the inference problem in quantile regression (QR) for a large sample size $n$ but under a limited memory constraint, where the memory can only store a small batch of data of size $m$. A natural method is the naïve divide-and-conquer approach, which splits data into batches of size $m$, computes the local QR estimator for each batch, and then aggregates the estimators via averaging. However, this method only works when $n=o(m^2)$ and is computationally expensive. This paper proposes a computationally efficient method, which only requires an initial QR estimator on a small batch of data and then successively refines the estimator via multiple rounds of aggregations. Theoretically, as long as $n$ grows polynomially in $m$, we establish the asymptotic normality for the obtained estimator and show that our estimator with only a few rounds of aggregations achieves the same efficiency as the QR estimator computed on all the data. Moreover, our result allows the case that the dimensionality $p$ goes to infinity. The proposed method can also be applied to address the QR problem under distributed computing environment (e.g., in a large-scale sensor network) or for real-time streaming data.

stat.ME↗

arXiv2017

Graph Estimation for Matrix-variate Gaussian Data

Matrix-variate Gaussian graphical models (GGM) have been widely used for modeling matrix-variate data. Since the support of sparse precision matrix represents the conditional independence graph among matrix entries, conducting support recovery yields valuable information. A commonly used approach is the penalized log-likelihood method. However, due to the complicated structure of precision matrices in the form of Kronecker product, the log-likelihood is non-convex, which presents challenges for both computation and theoretical analysis. In this paper, we propose an alternative approach by formulating the support recovery problem into a multiple testing problem. A new test statistic is developed and based on that, we use the popular Benjamini and Hochberg's procedure to control false discovery rate (FDR) asymptotically. Our method involves only convex optimization, making it computationally attractive. Theoretically, our method allows very weak conditions, i.e., even when the sample size is finite and the dimensions go to infinity, the asymptotic normality of the test statistics and FDR control can still be guaranteed. We further provide the power analysis result. The finite sample performance of the proposed method is illustrated by both simulated and real data analysis.

stat.ME↗

arXiv2017

Testing independence with high-dimensional correlated samples

Testing independence among a number of (ultra) high-dimensional random samples is a fundamental and challenging problem. By arranging $n$ identically distributed $p$-dimensional random vectors into a $p \times n$ data matrix, we investigate the problem of testing independence among columns under the matrix-variate normal modeling of data. We propose a computationally simple and tuning-free test statistic, characterize its limiting null distribution, analyze the statistical power and prove its minimax optimality. As an important by-product of the test statistic, a ratio-consistent estimator for the quadratic functional of a covariance matrix from correlated samples is developed. We further study the effect of correlation among samples to an important high-dimensional inference problem --- large-scale multiple testing of Pearson's correlation coefficients. Indeed, blindly using classical inference results based on the assumed independence of samples will lead to many false discoveries, which suggests the need for conducting independence testing before applying existing methods. To address the challenge arising from correlation among samples, we propose a "sandwich estimator" of Pearson's correlation coefficient by de-correlating the samples. Based on this approach, the resulting multiple testing procedure asymptotically controls the overall false discovery rate at the nominal level while maintaining good statistical power. Both simulated and real data experiments are carried out to demonstrate the advantages of the proposed methods.

math.ST↗

arXiv2016

Fast and Adaptive Sparse Precision Matrix Estimation in High Dimensions

This paper proposes a new method for estimating sparse precision matrices in the high dimensional setting. It has been popular to study fast computation and adaptive procedures for this problem. We propose a novel approach, called Sparse Column-wise Inverse Operator, to address these two issues. We analyze an adaptive procedure based on cross validation, and establish its convergence rate under the Frobenius norm. The convergence rates under other matrix norms are also established. This method also enjoys the advantage of fast computation for large-scale problems, via a coordinate descent algorithm. Numerical merits are illustrated using both simulated and real datasets. In particular, it performs favorably on an HIV brain tissue dataset and an ADHD resting-state fMRI dataset.

stat.ME↗

arXiv2014

Incorporation of Sparsity Information in Large-scale Multiple Two-sample $t$ Tests

Large-scale multiple two-sample {\em Student}'s $t$ testing problems often arise from the statistical analysis of scientific data. To detect components with different values between two mean vectors, a well-known procedure is to apply the Benjamini and Hochberg (B-H) method and two-sample {\em Student}'s $t$ statistics to control the false discovery rate (FDR). In many applications, mean vectors are expected to be sparse or asymptotically sparse. When dealing with such type of data, {\em can we gain more power than the standard procedure such as the B-H method with Student's $t$ statistics while keeping the FDR under control?} The answer is positive. By exploiting the possible sparsity information in mean vectors, we present an uncorrelated screening-based (US) FDR control procedure, which is shown to be more powerful than the B-H method. The US testing procedure depends on a novel construction of screening statistics, which are asymptotically uncorrelated with two-sample {\em Student}'s $t$ statistics. The US testing procedure is different from some existing {\em testing following screening} methods (Reiner, et al., 2007; Yekutieli, 2008) in which independence between screening and testing is crucial to control the FDR, while the independence often requires additional data or splitting of samples. An inappropriate splitting of samples may result in a loss rather than an improvement of statistical power. Instead, the uncorrelated screening US is based on the original data and does not need to split the samples. Theoretical results show that the US testing procedure controls the desired FDR asymptotically. Numerical studies are conducted and indicate that the proposed procedure works quite well.

stat.ME↗

arXiv2014

Komlós-Major-Tusnády approximation under dependence

The celebrated results of Komlós, Major and Tusnády [Z. Wahrsch. Verw. Gebiete 32 (1975) 111-131; Z. Wahrsch. Verw. Gebiete 34 (1976) 33-58] give optimal Wiener approximation for the partial sums of i.i.d. random variables and provide a powerful tool in probability and statistics. In this paper we extend KMT approximation for a large class of dependent stationary processes, solving a long standing open problem in probability theory. Under the framework of stationary causal processes and functional dependence measures of Wu [Proc. Natl. Acad. Sci. USA 102 (2005) 14150-14154], we show that, under natural moment conditions, the partial sum processes can be approximated by Wiener process with an optimal rate. Our dependence conditions are mild and easily verifiable. The results are applied to ergodic sums, as well as to nonlinear time series and Volterra processes, an important class of nonlinear processes.

math.PR↗

arXiv2013

Phase Transition and Regularized Bootstrap in Large Scale $t$-tests with False Discovery Rate Control

Applying Benjamini and Hochberg (B-H) method to multiple Student's $t$ tests is a popular technique in gene selection in microarray data analysis. Because of the non-normality of the population, the true p-values of the hypothesis tests are typically unknown. Hence, it is common to use the standard normal distribution N(0,1), Student's $t$ distribution $t_{n-1}$ or the bootstrap method to estimate the p-values. In this paper, we first study N(0,1) and $t_{n-1}$ calibrations. We prove that, when the population has the finite 4-th moment and the dimension $m$ and the sample size $n$ satisfy $\log m=o(n^{1/3})$, B-H method controls the false discovery rate (FDR) at a given level $α$ asymptotically with p-values estimated from N(0,1) or $t_{n-1}$ distribution. However, a phase transition phenomenon occurs when $\log m\geq c_{0}n^{1/3}$. In this case, the FDR of B-H method may be larger than $α$ or even tends to one. In contrast, the bootstrap calibration is accurate for $\log m=o(n^{1/2})$ as long as the underlying distribution has the sub-Gaussian tails. However, such light tailed condition can not be weakened in general. The simulation study shows that for the heavy tailed distributions, the bootstrap calibration is very conservative. In order to solve this problem, a regularized bootstrap correction is proposed and is shown to be robust to the tails of the distributions. The simulation study shows that the regularized bootstrap method performs better than the usual bootstrap method.

stat.ME↗

arXiv2013

Self-normalized Cramér type moderate deviations for the maximum of sums

Let $X_1,X_2,...$ be independent random variables with zero means and finite variances, and let $S_n=\sum_{i=1}^nX_i$ and $V^2_n=\sum_{i=1}^nX^2_i$. A Cramér type moderate deviation for the maximum of the self-normalized sums $\max_{1\leq k\leq n}S_k/V_n$ is obtained. In particular, for identically distributed $X_1,X_2,...,$ it is proved that $P(\max_{1\leq k\leq n}S_k\geq xV_n)/(1-Φ(x))\rightarrow2$ uniformly for $0<x\leq\mathrm{o}(n^{1/6})$ under the optimal finite third moment of $X_1$.

math.ST↗

arXiv2013

Gaussian Graphical Model Estimation with False Discovery Rate Control

This paper studies the estimation of high dimensional Gaussian graphical model (GGM). Typically, the existing methods depend on regularization techniques. As a result, it is necessary to choose the regularized parameter. However, the precise relationship between the regularized parameter and the number of false edges in GGM estimation is unclear. Hence, it is impossible to evaluate their performance rigorously. In this paper, we propose an alternative method by a multiple testing procedure. Based on our new test statistics for conditional dependence, we propose a simultaneous testing procedure for conditional dependence in GGM. Our method can control the false discovery rate (FDR) asymptotically. The numerical performance of the proposed method shows that our method works quite well.

stat.ME↗

arXiv2013

A Cramér moderate deviation theorem for Hotelling's $T^2$-statistic with applications to global tests

A Cramer moderate deviation theorem for Hotelling's $T^2$-statistic is proved under a finite $(3+δ)$th moment. The result is applied to large scale tests on the equality of mean vectors and is shown that the number of tests can be as large as $e^{o(n^{1/3})}$ before the chi-squared distribution calibration becomes inaccurate. As an application of the moderate deviation results, a global test on the equality of m mean vectors based on the maximum of Hotelling's $T^2$-statistics is developed and its asymptotic null distribution is shown to be an extreme value type I distribution. A novel intermediate approximation to the null distribution is proposed to improve the slow convergence rate of the extreme distribution approximation. Numerical studies show that the new test procedure works well even for a small sample size and performs favorably in analyzing a breast cancer dataset.

math.ST↗

arXiv2012

Estimating Sparse Precision Matrix: Optimal Rates of Convergence and Adaptive Estimation

Precision matrix is of significant importance in a wide range of applications in multivariate analysis. This paper considers adaptive minimax estimation of sparse precision matrices in the high dimensional setting. Optimal rates of convergence are established for a range of matrix norm losses. A fully data driven estimator based on adaptive constrained $\ell_1$ minimization is proposed and its rate of convergence is obtained over a collection of parameter spaces. The estimator, called ACLIME, is easy to implement and performs well numerically. A major step in establishing the minimax rate of convergence is the derivation of a rate-sharp lower bound. A "two-directional" lower bound technique is applied to obtain the minimax lower bound. The upper and lower bounds together yield the optimal rates ofconvergence for sparse precision matrix estimation and show that the ACLIME estimator is adaptively minimax rate optimal for a collection of parameter spaces and a range of matrix norm losses simultaneously.

math.ST↗

arXiv2011

A Direct Estimation Approach to Sparse Linear Discriminant Analysis

This paper considers sparse linear discriminant analysis of high-dimensional data. In contrast to the existing methods which are based on separate estimation of the precision matrix $Ø$ and the difference $\de$ of the mean vectors, we introduce a simple and effective classifier by estimating the product $Ø\de$ directly through constrained $\ell_1$ minimization. The estimator can be implemented efficiently using linear programming and the resulting classifier is called the linear programming discriminant (LPD) rule. The LPD rule is shown to have desirable theoretical and numerical properties. It exploits the approximate sparsity of $Ø\de$ and as a consequence allows cases where it can still perform well even when $Ø$ and/or $\de$ cannot be estimated consistently. Asymptotic properties of the LPD rule are investigated and consistency and rate of convergence results are given. The LPD classifier has superior finite sample performance and significant computational advantages over the existing methods that require separate estimation of $Ø$ and $\de$. The LPD rule is also applied to analyze real datasets from lung cancer and leukemia studies. The classifier performs favorably in comparison to existing methods.

stat.ME↗

arXiv2011

On non-stationary threshold autoregressive models

In this paper we study the limiting distributions of the least-squares estimators for the non-stationary first-order threshold autoregressive (TAR(1)) model. It is proved that the limiting behaviors of the TAR(1) process are very different from those of the classical unit root model and the explosive AR(1).

math.ST↗

arXiv2011

A Constrained L1 Minimization Approach to Sparse Precision Matrix Estimation

A constrained L1 minimization method is proposed for estimating a sparse inverse covariance matrix based on a sample of $n$ iid $p$-variate random variables. The resulting estimator is shown to enjoy a number of desirable properties. In particular, it is shown that the rate of convergence between the estimator and the true $s$-sparse precision matrix under the spectral norm is $s\sqrt{\log p/n}$ when the population distribution has either exponential-type tails or polynomial-type tails. Convergence rates under the elementwise $L_{\infty}$ norm and Frobenius norm are also presented. In addition, graphical model selection is considered. The procedure is easily implementable by linear programming. Numerical performance of the estimator is investigated using both simulated and real data. In particular, the procedure is applied to analyze a breast cancer dataset. The procedure performs favorably in comparison to existing methods.

stat.ME↗

arXiv2011

Adaptive Thresholding for Sparse Covariance Matrix Estimation

In this paper we consider estimation of sparse covariance matrices and propose a thresholding procedure which is adaptive to the variability of individual entries. The estimators are fully data driven and enjoy excellent performance both theoretically and numerically. It is shown that the estimators adaptively achieve the optimal rate of convergence over a large class of sparse covariance matrices under the spectral norm. In contrast, the commonly used universal thresholding estimators are shown to be sub-optimal over the same parameter spaces. Support recovery is also discussed. The adaptive thresholding estimators are easy to implement. Numerical performance of the estimators is studied using both simulated and real data. Simulation results show that the adaptive thresholding estimators uniformly outperform the universal thresholding estimators. The method is also illustrated in an analysis on a dataset from a small round blue-cell tumors microarray experiment. A supplement to this paper which contains additional technical proofs is available online.

stat.ME↗

arXiv2010

Simultaneous nonparametric inference of time series

We consider kernel estimation of marginal densities and regression functions of stationary processes. It is shown that for a wide class of time series, with proper centering and scaling, the maximum deviations of kernel density and regression estimates are asymptotically Gumbel. Our results substantially generalize earlier ones which were obtained under independence or beta mixing assumptions. The asymptotic results can be applied to assess patterns of marginal densities or regression functions via the construction of simultaneous confidence bands for which one can perform goodness-of-fit tests. As an application, we construct simultaneous confidence bands for drift and volatility functions in a dynamic short-term rate model for the U.S. Treasury yield curve rates data.

math.ST↗

arXiv2009

On maxima of periodograms of stationary processes

We consider the limit distribution of maxima of periodograms for stationary processes. Our method is based on $m$-dependent approximation for stationary processes and a moderate deviation result.

math.ST↗