arXiv subjects

Heng Peng

Publications and source records attributed to Heng Peng.

16 recordsLinked to original sources

arXiv2026

Model-free Feature Screening via Revised Chatterjee's Rank Correlation for Ultra-high Dimensional Censored Data

In large-scale biomedical research, it's common to gather ultra-high dimensional data that includes right-censored survival times. Feature screening has emerged as a crucial statistical technique for handling such data. In this paper, we introduce a straightforward and robust feature screening approach, leveraging the modified Chatterjee's rank correlation, suitable for a broad range of survival models. With reasonably mild regularity assumptions, we establish the properties of sure screening and ranking consistency. The computation involved in our proposed method is quite direct and simple. Through simulation studies and real gene expression data analysis, we demonstrate the superior efficacy of our proposed approach.

stat.ME↗

arXiv2024

A Two-Step Projection-Based Goodness-of-Fit Test for Ultra-High Dimensional Sparse Regressions

This paper proposes a novel two-step strategy for testing the goodness-of-fit of parametric regression models in ultra-high dimensional sparse settings, where the predictor dimension far exceeds the sample size. This regime usually renders existing goodness-of-fit tests for regressions infeasible, primarily due to the curse of dimensionality or their reliance on the asymptotic linearity and normality of parameter estimators -- properties that may no longer hold under ultra-high dimensional settings. To address these limitations, our strategy first constructs multiple test statistics based on projected predictors from distinct projections and establishes their asymptotic properties under both the null and alternative hypotheses. This projection-based approach significantly mitigates the dimensionality problem, enabling our tests to detect local alternatives converging to the null at the rate as if the predictor were univariate. An important finding is that the resulting test statistics based on linearly independent projections are asymptotically independent under the null hypothesis. Based on this, our second step employs powerful $p$-value combination procedures, such as the minimum $p$-value and the Fisher combination of $p$-value, to form our final tests and enhance power. Theoretically, our tests only require the standard convergence rate of parameter estimators to derive their limiting distributions, thereby circumventing the need for asymptotic linearity or normality of parameter estimators. Simulations and real-data applications confirm that our approach provides robust and powerful goodness-of-fit testing in ultra-high dimensional settings.

stat.ME↗

arXiv2024

Model Free Prediction with Uncertainty Assessment

Deep nonparametric regression, characterized by the utilization of deep neural networks to learn target functions, has emerged as a focus of research attention in recent years. Despite considerable progress in understanding convergence rates, the absence of asymptotic properties hinders rigorous statistical inference. To address this gap, we propose a novel framework that transforms the deep estimation paradigm into a platform conducive to conditional mean estimation, leveraging the conditional diffusion model. Theoretically, we develop an end-to-end convergence rate for the conditional diffusion model and establish the asymptotic normality of the generated samples. Consequently, we are equipped to construct confidence regions, facilitating robust statistical inference. Furthermore, through numerical experiments, we empirically validate the efficacy of our proposed methodology.

stat.ML↗

arXiv2019

BOLT-SSI: A Statistical Approach to Screening Interaction Effects for Ultra-High Dimensional Data

Detecting interaction effects among predictors on the response variable is a crucial step in various applications. In this paper, we first propose a simple method for sure screening interactions (SSI). Although its computation complexity is $O(p^2n)$, SSI works well for problems of moderate dimensionality (e.g., $p=10^3\sim10^4$), without the heredity assumption. To ultra-high dimensional problems (e.g., $p = 10^6$), motivated by discretization associated Boolean representation and operations and the contingency table for discrete variables, we propose a fast algorithm, named "BOLT-SSI". The statistical theory has been established for SSI and BOLT-SSI, guaranteeing their sure screening property. The performance of SSI and BOLT-SSI are evaluated by comprehensive simulation and real case studies. Numerical results demonstrate that SSI and BOLT-SSI can often outperform their competitors in terms of computational efficiency and statistical accuracy. The proposed method can be applied for fully detecting interactions with more than 300,000 predictors. Based on this study, we believe that there is a great need to rethink the relationship between statistical accuracy and computational efficiency. We have shown that the computational performance of a statistical method can often be greatly improved by exploring the advantages of computational architecture with a tolerable loss of statistical accuracy.

stat.ME↗

arXiv2018

Fast Inference Procedures for Semivarying Coefficient Models via Local Averaging

The semivarying coefficient models are widely used in the application of finance, economics, medical science and many other areas. The functional coefficients are commonly estimated by local smoothing methods, e.g. local linear estimator. This implies that one should implement the estimation procedure for hundreds of times to obtain an estimate of one function. So the computation cost is very severe. In this paper, we give an insight to the trade-off between statistical efficiency and computation simplicity, and proposes a fast inference procedure for semivarying coefficient model. In our method, the coefficient functions are approximated by piecewise constants, which is a simple and rough approximation. This makes our estimators easy to implement and avoid repeat estimation. In this work, we shall show that though these estimators are not asymptotically optimal, they are efficient enough for building further inference procedure. Furthermore, three tests are brought out to check whether certain coefficient is constant. Our results clearly show that when the room for improving the asymptotic efficiency is limited, a proper trade-off between statistical efficiency and computation simplicity can be taken into consideration to improve the performance of the inference procedure.

stat.ME↗

arXiv2018

BIVAS: A scalable Bayesian method for bi-level variable selection with applications

In this paper, we consider a Bayesian bi-level variable selection problem in high-dimensional regressions. In many practical situations, it is natural to assign group membership to each predictor. Examples include that genetic variants can be grouped at the gene level and a covariate from different tasks naturally forms a group. Thus, it is of interest to select important groups as well as important members from those groups. The existing Markov Chain Monte Carlo (MCMC) methods are often computationally intensive and not scalable to large data sets. To address this problem, we consider variational inference for bi-level variable selection (BIVAS). In contrast to the commonly used mean-field approximation, we propose a hierarchical factorization to approximate the posterior distribution, by utilizing the structure of bi-level variable selection. Moreover, we develop a computationally efficient and fully parallelizable algorithm based on this variational approximation. We further extend the developed method to model data sets from multi-task learning. The comprehensive numerical results from both simulation studies and real data analysis demonstrate the advantages of BIVAS for variable selection, parameter estimation and computational efficiency over existing methods. The method is implemented in R package `bivas' available at https://github.com/mxcai/bivas.

stat.AP↗

arXiv2018

Varying Coefficient Panel Data Model with Interactive Fixed Effects

In this paper, we propose a varying coefficient panel data model with unobservable multiple interactive fixed effects that are correlated with the regressors. We approximate each coefficient function by B-spline, and propose a robust nonlinear iteration scheme based on the least squares method to estimate the coefficient functions of interest. We also establish the asymptotic theory of the resulting estimators under certain regularity assumptions, including the consistency, the convergence rate and the asymptotic distribution. Furthermore, we develop a least squares dummy variable method to study an important special case of the proposed model: the varying coefficient panel data model with additive fixed effects. To construct the pointwise confidence intervals for the coefficient functions, a residual-based block bootstrap method is proposed to reduce the computational burden as well as to avoid the accumulative errors. Simulation studies and a real data analysis are also carried out to assess the performance of our proposed methods.

stat.ME↗

arXiv2018

Unsupervised Learning of Mixture Regression Models for Longitudinal Data

This paper is concerned with learning of mixture regression models for individuals that are measured repeatedly. The adjective "unsupervised" implies that the number of mixing components is unknown and has to be determined, ideally by data driven tools. For this purpose, a novel penalized method is proposed to simultaneously select the number of mixing components and to estimate the mixing proportions and unknown parameters in the models. The proposed method is capable of handling both continuous and discrete responses by only requiring the first two moment conditions of the model distribution. It is shown to be consistent in both selecting the number of components and estimating the mixing proportions and unknown regression parameters. Further, a modified EM algorithm is developed to seamlessly integrate model selection and estimation. Simulation studies are conducted to evaluate the finite sample performance of the proposed procedure. And it is further illustrated via an analysis of a primary biliary cirrhosis data set.

stat.ME↗

arXiv2014

Nonparametric independence screening and structure identification for ultra-high dimensional longitudinal data

Ultra-high dimensional longitudinal data are increasingly common and the analysis is challenging both theoretically and methodologically. We offer a new automatic procedure for finding a sparse semivarying coefficient model, which is widely accepted for longitudinal data analysis. Our proposed method first reduces the number of covariates to a moderate order by employing a screening procedure, and then identifies both the varying and constant coefficients using a group SCAD estimator, which is subsequently refined by accounting for the within-subject correlation. The screening procedure is based on working independence and B-spline marginal models. Under weaker conditions than those in the literature, we show that with high probability only irrelevant variables will be screened out, and the number of selected variables can be bounded by a moderate order. This allows the desirable sparsity and oracle properties of the subsequent structure identification step. Note that existing methods require some kind of iterative screening in order to achieve this, thus they demand heavy computational effort and consistency is not guaranteed. The refined semivarying coefficient model employs profile least squares, local linear smoothing and nonparametric covariance estimation, and is semiparametric efficient. We also suggest ways to implement the proposed methods, and to select the tuning parameters. An extensive simulation study is summarized to demonstrate its finite sample performance and the yeast cell cycle data is analyzed.

stat.ME↗

arXiv2014

Estimation of Partially Linear Regression Model under Partial Consistency Property

In this paper, utilizing recent theoretical results in high dimensional statistical modeling, we propose a model-free yet computationally simple approach to estimate the partially linear model $Y=Xβ+g(Z)+\varepsilon$. Motivated by the partial consistency phenomena, we propose to model $g(Z)$ via incidental parameters. Based on partitioning the support of $Z$, a simple local average is used to estimate the response surface. The proposed method seeks to strike a balance between computation burden and efficiency of the estimators while minimizing model bias. Computationally this approach only involves least squares. We show that given the inconsistent estimator of $g(Z)$, a root $n$ consistent estimator of parametric component $β$ of the partially linear model can be obtained with little cost in efficiency. Moreover, conditional on the $β$ estimates, an optimal estimator of $g(Z)$ can then be obtained using classic nonparametric methods. The statistical inference problem regarding $β$ and a two-population nonparametric testing problem regarding $g(Z)$ are considered. Our results show that the behavior of test statistics are satisfactory. To assess the performance of our method in comparison with other methods, three simulation studies are conducted and a real dataset about risk factors of birth weights is analyzed.

stat.ME↗

arXiv2013

Bridging Information Criteria and Parameter Shrinkage for Model Selection

Model selection based on classical information criteria, such as BIC, is generally computationally demanding, but its properties are well studied. On the other hand, model selection based on parameter shrinkage by $\ell_1$-type penalties is computationally efficient. In this paper we make an attempt to combine their strengths, and propose a simple approach that penalizes the likelihood with data-dependent $\ell_1$ penalties as in adaptive Lasso and exploits a fixed penalization parameter. Even for finite samples, its model selection results approximately coincide with those based on information criteria; in particular, we show that in some special cases, this approach and the corresponding information criterion produce exactly the same model. One can also consider this approach as a way to directly determine the penalization parameter in adaptive Lasso to achieve information criteria-like model selection. As extensions, we apply this idea to complex models including Gaussian mixture model and mixture of factor analyzers, whose model selection is traditionally difficult to do; by adopting suitable penalties, we provide continuous approximators to the corresponding information criteria, which are easy to optimize and enable efficient model selection.

stat.ML↗

arXiv2013

Model Selection for Gaussian Mixture Models

This paper is concerned with an important issue in finite mixture modelling, the selection of the number of mixing components. We propose a new penalized likelihood method for model selection of finite multivariate Gaussian mixture models. The proposed method is shown to be statistically consistent in determining of the number of components. A modified EM algorithm is developed to simultaneously select the number of components and to estimate the mixing weights, i.e. the mixing probabilities, and unknown parameters of Gaussian distributions. Simulations and a real data analysis are presented to illustrate the performance of the proposed method.

stat.ME↗

arXiv2012

Robust rank correlation based screening

Independence screening is a variable selection method that uses a ranking criterion to select significant variables, particularly for statistical models with nonpolynomial dimensionality or "large p, small n" paradigms when p can be as large as an exponential of the sample size n. In this paper we propose a robust rank correlation screening (RRCS) method to deal with ultra-high dimensional data. The new procedure is based on the Kendall τcorrelation coefficient between response and predictor variables rather than the Pearson correlation of existing methods. The new method has four desirable features compared with existing independence screening methods. First, the sure independence screening property can hold only under the existence of a second order moment of predictor variables, rather than exponential tails or alikeness, even when the number of predictor variables grows as fast as exponentially of the sample size. Second, it can be used to deal with semiparametric models such as transformation regression models and single-index models under monotonic constraint to the link function without involving nonparametric estimation even when there are nonparametric functions in the models. Third, the procedure can be largely used against outliers and influence points in the observations. Last, the use of indicator functions in rank correlation screening greatly simplifies the theoretical derivation due to the boundedness of the resulting statistics, compared with previous studies on variable screening. Simulations are carried out for comparisons with existing methods and a real data example is analyzed.

stat.ME↗

arXiv2012

Nonconcave Penalized Spline

Regression spline is a useful tool in nonparametric regression. However, finding the optimal knot locations is a known difficult problem. In this article, we introduce the Non-concave Penalized Regression Spline. This proposal method not only produces smoothing spline with optimal convergence rate, but also can adaptively select optimal knots simultaneously. It is insensitive to the number of origin knots. The method's performance in a simulation has been studied to compare the other methods. The problem of how to choose smoothing parameters, i.e. penalty parameters in the non-concave regression spline is addressed.

stat.ME↗

arXiv2010

Component Selection in the Additive Regression Model

Similar to variable selection in the linear regression model, selecting significant components in the popular additive regression model is of great interest. However, such components are unknown smooth functions of independent variables, which are unobservable. As such, some approximation is needed. In this paper, we suggest a combination of penalized regression spline approximation and group variable selection, called the lasso-type spline method (LSM), to handle this component selection problem with a diverging number of strongly correlated variables in each group. It is shown that the proposed method can select significant components and estimate nonparametric additive function components simultaneously with an optimal convergence rate simultaneously. To make the LSM stable in computation and able to adapt its estimators to the level of smoothness of the component functions, weighted power spline bases and projected weighted power spline bases are proposed. Their performance is examined by simulation studies across two set-ups with independent predictors and correlated predictors, respectively, and appears superior to the performance of competing methods. The proposed method is extended to a partial linear regression model analysis with real data, and gives reliable results.

stat.ME↗

arXiv2004

Nonconcave penalized likelihood with a diverging number of parameters

A class of variable selection procedures for parametric models via nonconcave penalized likelihood was proposed by Fan and Li to simultaneously estimate parameters and select important variables. They demonstrated that this class of procedures has an oracle property when the number of parameters is finite. However, in most model selection problems the number of parameters should be large and grow with the sample size. In this paper some asymptotic properties of the nonconcave penalized likelihood are established for situations in which the number of parameters tends to \infty as the sample size increases. Under regularity conditions we have established an oracle property and the asymptotic normality of the penalized likelihood estimators. Furthermore, the consistency of the sandwich formula of the covariance matrix is demonstrated. Nonconcave penalized likelihood ratio statistics are discussed, and their asymptotic distributions under the null hypothesis are obtained by imposing some mild conditions on the penalty functions.

math.ST↗