Searcharxiv⌕ Search

arXiv subjects

Cun-Hui Zhang

Publications and source records attributed to Cun-Hui Zhang.

72 records · Page 4Linked to original sources
arXiv2012

A General Theory of Concave Regularization for High Dimensional Sparse Estimation Problems

Concave regularization methods provide natural procedures for sparse recovery. However, they are difficult to analyze in the high dimensional setting. Only recently a few sparse recovery results have been established for some specific local solutions obtained via specialized numerical procedures. Still, the fundamental relationship between these solutions such as whether they are identical or their relationship to the global minimizer of the underlying nonconvex formulation is unknown. The current paper fills this conceptual gap by presenting a general theoretical framework showing that under appropriate conditions, the global solution of nonconvex regularization leads to desirable recovery performance; moreover, under suitable conditions, the global solution corresponds to the unique sparse local solution, which can be obtained via different numerical procedures. Under this unified framework, we present an overview of existing results and discuss their connections. The unified view of this work leads to a more satisfactory treatment of concave high dimensional sparse estimation procedures, and serves as guideline for developing further numerical procedures for concave regularization.

stat.ML↗
arXiv2011

Estimation And Selection Via Absolute Penalized Convex Minimization And Its Multistage Adaptive Applications

The $\ell_1$-penalized method, or the Lasso, has emerged as an important tool for the analysis of large data sets. Many important results have been obtained for the Lasso in linear regression which have led to a deeper understanding of high-dimensional statistical problems. In this article, we consider a class of weighted $\ell_1$-penalized estimators for convex loss functions of a general form, including the generalized linear models. We study the estimation, prediction, selection and sparsity properties of the weighted $\ell_1$-penalized estimator in sparse, high-dimensional settings where the number of predictors $p$ can be much larger than the sample size $n$. Adaptive Lasso is considered as a special case. A multistage method is developed to apply an adaptive Lasso recursively. We provide $\ell_q$ oracle inequalities, a general selection consistency theorem, and an upper bound on the dimension of the Lasso estimator. Important models including the linear regression, logistic regression and log-linear models are used throughout to illustrate the applications of the general results.

stat.ML↗
arXiv2011

The sparse Laplacian shrinkage estimator for high-dimensional regression

We propose a new penalized method for variable selection and estimation that explicitly incorporates the correlation patterns among predictors. This method is based on a combination of the minimax concave penalty and Laplacian quadratic associated with a graph as the penalty function. We call it the sparse Laplacian shrinkage (SLS) method. The SLS uses the minimax concave penalty for encouraging sparsity and Laplacian quadratic penalty for promoting smoothness among coefficients associated with the correlated predictors. The SLS has a generalized grouping property with respect to the graph represented by the Laplacian quadratic. We show that the SLS possesses an oracle property in the sense that it is selection consistent and equal to the oracle Laplacian shrinkage estimator with high probability. This result holds in sparse, high-dimensional settings with p >> n under reasonable conditions. We derive a coordinate descent algorithm for computing the SLS estimates. Simulation studies are conducted to evaluate the performance of the SLS method and a real data example is used to illustrate its application.

math.ST↗
arXiv2010

Optimal rates of convergence for covariance matrix estimation

Covariance matrix plays a central role in multivariate statistical analysis. Significant advances have been made recently on developing both theory and methodology for estimating large covariance matrices. However, a minimax theory has yet been developed. In this paper we establish the optimal rates of convergence for estimating the covariance matrix under both the operator norm and Frobenius norm. It is shown that optimal procedures under the two norms are different and consequently matrix estimation under the operator norm is fundamentally different from vector estimation. The minimax upper bound is obtained by constructing a special class of tapering estimators and by studying their risk properties. A key step in obtaining the optimal rate of convergence is the derivation of the minimax lower bound. The technical analysis requires new ideas that are quite different from those used in the more conventional function/sequence estimation problems.

math.ST↗
arXiv2010

Nearly unbiased variable selection under minimax concave penalty

We propose MC+, a fast, continuous, nearly unbiased and accurate method of penalized variable selection in high-dimensional linear regression. The LASSO is fast and continuous, but biased. The bias of the LASSO may prevent consistent variable selection. Subset selection is unbiased but computationally costly. The MC+ has two elements: a minimax concave penalty (MCP) and a penalized linear unbiased selection (PLUS) algorithm. The MCP provides the convexity of the penalized loss in sparse regions to the greatest extent given certain thresholds for variable selection and unbiasedness. The PLUS computes multiple exact local minimizers of a possibly nonconvex penalized loss function in a certain main branch of the graph of critical points of the penalized loss. Its output is a continuous piecewise linear path encompassing from the origin for infinite penalty to a least squares solution for zero penalty. We prove that at a universal penalty level, the MC+ has high probability of matching the signs of the unknowns, and thus correct selection, without assuming the strong irrepresentable condition required by the LASSO. This selection consistency applies to the case of $p\gg n$, and is proved to hold for exactly the MC+ solution among possibly many local minimizers. We prove that the MC+ attains certain minimax convergence rates in probability for the estimation of regression coefficients in $\ell_r$ balls. We use the SURE method to derive degrees of freedom and $C_p$-type risk estimates for general penalized LSE, including the LASSO and MC+ estimators, and prove their unbiasedness. Based on the estimated degrees of freedom, we propose an estimator of the noise level for proper choice of the penalty level.

math.ST↗
arXiv2009

Asymptotic normality of a nonparametric estimator of sample coverage

This paper establishes a necessary and sufficient condition for the asymptotic normality of the nonparametric estimator of sample coverage proposed by Good [Biometrica 40 (1953) 237--264]. This new necessary and sufficient condition extends the validity of the asymptotic normality beyond the previously proven cases.

math.ST↗
arXiv2009

General maximum likelihood empirical Bayes estimation of normal means

We propose a general maximum likelihood empirical Bayes (GMLEB) method for the estimation of a mean vector based on observations with i.i.d. normal errors. We prove that under mild moment conditions on the unknown means, the average mean squared error (MSE) of the GMLEB is within an infinitesimal fraction of the minimum average MSE among all separable estimators which use a single deterministic estimating function on individual observations, provided that the risk is of greater order than $(\log n)^5/n$. We also prove that the GMLEB is uniformly approximately minimax in regular and weak $\ell_p$ balls when the order of the length-normalized norm of the unknown means is between $(\log n)^{κ_1}/n^{1/(p\wedge2)}$ and $n/(\log n)^{κ_2}$. Simulation experiments demonstrate that the GMLEB outperforms the James--Stein and several state-of-the-art threshold estimators in a wide range of settings without much down side.

math.ST↗
arXiv2008

The sparsity and bias of the Lasso selection in high-dimensional linear regression

Meinshausen and Buhlmann [Ann. Statist. 34 (2006) 1436--1462] showed that, for neighborhood selection in Gaussian graphical models, under a neighborhood stability condition, the LASSO is consistent, even when the number of variables is of greater order than the sample size. Zhao and Yu [(2006) J. Machine Learning Research 7 2541--2567] formalized the neighborhood stability condition in the context of linear regression as a strong irrepresentable condition. That paper showed that under this condition, the LASSO selects exactly the set of nonzero regression coefficients, provided that these coefficients are bounded away from zero at a certain rate. In this paper, the regression coefficients outside an ideal model are assumed to be small, but not necessarily zero. Under a sparse Riesz condition on the correlation of design variables, we prove that the LASSO selects a model of the correct order of dimensionality, controls the bias of the selected model at a level determined by the contributions of small regression coefficients and threshold bias, and selects all coefficients of greater order than the bias of the selected model. Moreover, as a consequence of this rate consistency of the LASSO in model selection, it is proved that the sum of error squares for the mean response and the $\ell_α$-loss for the regression coefficients converge at the best possible rates under the given conditions. An interesting aspect of our results is that the logarithm of the number of variables can be of the same order as the sample size for certain random dependent designs.

math.ST↗
arXiv2007

Complex Datasets and Inverse Problems. Tomography, Networks and Beyond

This book is a collection of papers dedicated to the memory of Yehuda Vardi. Yehuda was the chair of the Department of Statistics of Rutgers University when he passed away unexpectedly on January 13, 2005. On October 21--22, 2005, some 150 leading scholars from many different fields, including statistics, telecommunications, biomedical engineering, bioinformatics, biostatistics and epidemiology, gathered at Rutgers in a conference in his honor. This conference was on ``Complex Datasets and Inverse Problems: Tomography, Networks, and Beyond,'' and was organized by the editors. The present collection includes research work presented at the conference, as well as contributions from Yehuda's colleagues. The theme of the conference was networks and other important and emerging areas of research involving incomplete data and statistical inverse problems. Networks are abundant around us: communication, computer, traffic, social and energy are just a few examples. As enormous amounts of network data are collected in this information age, the field has attracted a great amount of attention from researchers in statistics and computer engineering as well as telecommunication providers and various government agencies. However, few statistical tools have been developed for analyzing network data as they are typically governed by time-varying and mutually dependent communication protocols sitting on complicated graph-structured network topologies. Many prototypical applications in these and other important technologies can be viewed as statistical inverse problems with complex, massive, high-dimensional and possibly biased/incomplete data. This unifying theme of inverse problems is particularly appropriate for a conference and volume dedicated to the memory of Yehuda. Indeed he made influential contributions to these fields, especially in medical tomography, biased data, statistical inverse problems, and network tomography.

math.ST↗
arXiv2007

An iterative tomogravity algorithm for the estimation of network traffic

This paper introduces an iterative tomogravity algorithm for the estimation of a network traffic matrix based on one snapshot observation of the link loads in the network. The proposed method does not require complete observation of the total load on individual edge links or proper tuning of a penalty parameter as existing methods do. Numerical results are presented to demonstrate that the iterative tomogravity method controls the estimation error well when the link data is fully observed and produces robust results with moderate amount of missing link data.

stat.AP↗
arXiv2007

Empirical Bayes methods for controlling the false discovery rate with dependent data

False discovery rate (FDR) has been widely used as an error measure in large scale multiple testing problems, but most research in the area has been focused on procedures for controlling the FDR based on independent test statistics or the properties of such procedures for test statistics with certain types of stochastic dependence. Based on an approach proposed in Tang and Zhang (2005), we further develop in this paper empirical Bayes methods for controlling the FDR with dependent data. We implement our methodology in a time series model and report the results of a simulation study to demonstrate the advantages of the empirical Bayes approach.

stat.ME↗
arXiv2006

Nonlinear renewal theorems for random walks with perturbations of intermediate order

We develop nonlinear renewal theorems for a perturbed random walk without assuming stochastic boundedness of centered perturbation terms. A second order expansion of the expected stopping time is obtained via the uniform integrability of the difference between certain linear and nonlinear stopping rules. An intermediate renewal theorem is obtained which provides expansions between the nonlinear versions of the elementary and regular renewal theorems. The expected sample size of a two-sample rank sequential probability ratio test is considered as the motivating example.

math.ST↗
arXiv2006

A Conversation with Yuan Shih Chow

Yuan Shih Chow was born in Hubei province in China, on September 1, 1924. The eldest child of a local militia and political leader, he grew up in war and turmoil. His hometown was on the front line during most of the Japanese invasion and occupation of China. When he was 16, Y. S. Chow journeyed, mostly on foot, to Chongqing (Chung-King), the wartime Chinese capital, to finish his high school education. When the Communist party gained power in China, Y. S. Chow had already followed his university job to Taiwan. In Taiwan, he taught mathematics as an assistant at National Taiwan University until he came to the United States in 1954. At the University of Illinois, he studied under J. L. Doob and received his Ph.D. in 1958. He served as a staff mathematician and adjunct faculty at the IBM Watson Research Laboratory and Columbia University from 1959 to 1962. He was a member of the Statistics Department at Purdue University from 1962 to 1968. From 1968 until his retirement in 1993, Y. S. Chow served as Professor of Mathematical Statistics at Columbia University. At different times, he was a visiting professor at the University of California at Berkeley, University of Heidelberg (Germany) and the National Central University, Taiwan. He served as Director of the Institute of Mathematics of Academia Sinica, Taiwan, and Director of the Center of Applied Statistics at Nankai University, Tianjin, China. He was instrumental in establishing the Institute of Statistics of Academia Sinica in Taiwan. He is currently Professor Emeritus at Columbia University. Y. S. Chow is a fellow of the Institute of Mathematical Statistics, a member of the International Statistical Institute and a member of Taiwan's Academia Sinica. He has numerous publications, including Great Expectations: The Theory of Optimal Stopping (1971), in collaboration with Herbert Robbins and David Siegmund, and Probability Theory (1978), in collaboration with Henry Teicher. Y. S. Chow has a strong interest in mathematics education. He taught high school mathematics for one year in 1947 and wrote a book on high school algebra in collaboration with J. H. Teng and M. L. Chu. In 1992, Y. S. Chow, together with I. S. Chang and W. C. Ho, established the Chinese Institute of Probability and Statistics in Taiwan. This conversation took place in the fall of 2003 in Dobbs Ferry, New York.

math.ST↗
arXiv2006

Estimation of sums of random variables: Examples and information bounds

This paper concerns the estimation of sums of functions of observable and unobservable variables. Lower bounds for the asymptotic variance and a convolution theorem are derived in general finite- and infinite-dimensional models. An explicit relationship is established between efficient influence functions for the estimation of sums of variables and the estimation of their means. Certain ``plug-in'' estimators are proved to be asymptotically efficient in finite-dimensional models, while ``$u,v$'' estimators of Robbins are proved to be efficient in infinite-dimensional mixture models. Examples include certain species, network and data confidentiality problems.

math.ST↗
arXiv2005

Network Inference from TraceRoute Measurements: Internet Topology `Species'

Internet mapping projects generally consist in sampling the network from a limited set of sources by using traceroute probes. This methodology, akin to the merging of spanning trees from the different sources to a set of destinations, leads necessarily to a partial, incomplete map of the Internet. Accordingly, determination of Internet topology characteristics from such sampled maps is in part a problem of statistical inference. Our contribution begins with the observation that the inference of many of the most basic topological quantities -- including network size and degree characteristics -- from traceroute measurements is in fact a version of the so-called `species problem' in statistics. This observation has important implications, since species problems are often quite challenging. We focus here on the most fundamental example of a traceroute internet species: the number of nodes in a network. Specifically, we characterize the difficulty of estimating this quantity through a set of analytical arguments, we use statistical subsampling principles to derive two proposed estimators, and we illustrate the performance of these estimators on networks with various topological characteristics.

cs.NI↗
arXiv2005

General empirical Bayes wavelet methods and exactly adaptive minimax estimation

In many statistical problems, stochastic signals can be represented as a sequence of noisy wavelet coefficients. In this paper, we develop general empirical Bayes methods for the estimation of true signal. Our estimators approximate certain oracle separable rules and achieve adaptation to ideal risks and exact minimax risks in broad collections of classes of signals. In particular, our estimators are uniformly adaptive to the minimum risk of separable estimators and the exact minimax risks simultaneously in Besov balls of all smoothness and shape indices, and they are uniformly superefficient in convergence rates in all compact sets in Besov spaces with a finite secondary shape parameter. Furthermore, in classes nested between Besov balls of the same smoothness index, our estimators dominate threshold and James-Stein estimators within an infinitesimal fraction of the minimax risks. More general block empirical Bayes estimators are developed. Both white noise with drift and nonparametric regression are considered.

math.ST↗
arXiv2005

Equivalence theory for density estimation, Poisson processes and Gaussian white noise with drift

This paper establishes the global asymptotic equivalence between a Poisson process with variable intensity and white noise with drift under sharp smoothness conditions on the unknown function. This equivalence is also extended to density estimation models by Poissonization. The asymptotic equivalences are established by constructing explicit equivalence mappings. The impact of such asymptotic equivalence results is that an investigation in one of these nonparametric models automatically yields asymptotically analogous results in the other models.

math.ST↗