arXiv subjects

Xinwei Deng

Publications and source records attributed to Xinwei Deng.

51 records · Page 3Linked to original sources

arXiv2021

Sequential Design of Computer Experiments with Quantitative and Qualitative Factors in Applications to HPC Performance Optimization

Computer experiments with both qualitative and quantitative factors are widely used in many applications. Motivated by the emerging need of optimal configuration in the high-performance computing (HPC) system, this work proposes a sequential design, denoted as adaptive composite exploitation and exploration (CEE), for optimization of computer experiments with qualitative and quantitative factors. The proposed adaptive CEE method combines the predictive mean and standard deviation based on the additive Gaussian process to achieve a meaningful balance between exploitation and exploration for optimization. Moreover, the adaptiveness of the proposed sequential procedure allows the selection of next design point from the adaptive design region. Theoretical justification of the adaptive design region is provided. The performance of the proposed method is evaluated by several numerical examples in simulations. The case study of HPC performance optimization further elaborates the merits of the proposed method.

cs.DC↗

arXiv2020

Investigating the Robustness of Artificial Intelligent Algorithms with Mixture Experiments

Artificial intelligent (AI) algorithms, such as deep learning and XGboost, are used in numerous applications including computer vision, autonomous driving, and medical diagnostics. The robustness of these AI algorithms is of great interest as inaccurate prediction could result in safety concerns and limit the adoption of AI systems. In this paper, we propose a framework based on design of experiments to systematically investigate the robustness of AI classification algorithms. A robust classification algorithm is expected to have high accuracy and low variability under different application scenarios. The robustness can be affected by a wide range of factors such as the imbalance of class labels in the training dataset, the chosen prediction algorithm, the chosen dataset of the application, and a change of distribution in the training and test datasets. To investigate the robustness of AI classification algorithms, we conduct a comprehensive set of mixture experiments to collect prediction performance results. Then statistical analyses are conducted to understand how various factors affect the robustness of AI classification algorithms. We summarize our findings and provide suggestions to practitioners in AI applications.

stat.ML↗

arXiv2020

Bayesian Auxiliary Variable Model for Birth Records Data with Qualitative and Quantitative Responses

Many applications involve data with qualitative and quantitative responses. When there is an association between the two responses, a joint model will provide improved results than modeling them separately. In this paper, we propose a Bayesian method to jointly model such data. The joint model links the qualitative and quantitative responses and can assess their dependency strength via a latent variable. The posterior distributions of parameters are obtained through an efficient MCMC sampling algorithm. The simulation shows that the proposed method can improve the prediction capacity for both responses. We apply the proposed joint model to the birth records data acquired by the Virginia Department of Health and study the mutual dependence between preterm birth of infants and their birth weights.

stat.ME↗

arXiv2020

A Maximin $Φ_{p}$-Efficient Design for Multivariate GLM

Experimental designs for a generalized linear model (GLM) often depend on the specification of the model, including the link function, the predictors, and unknown parameters, such as the regression coefficients. To deal with uncertainties of these model specifications, it is important to construct optimal designs with high efficiency under such uncertainties. Existing methods such as Bayesian experimental designs often use prior distributions of model specifications to incorporate model uncertainties into the design criterion. Alternatively, one can obtain the design by optimizing the worst-case design efficiency with respect to uncertainties of model specifications. In this work, we propose a new Maximin $Φ_p$-Efficient (or Mm-$Φ_p$ for short) design which aims at maximizing the minimum $Φ_p$-efficiency under model uncertainties. Based on the theoretical properties of the proposed criterion, we develop an efficient algorithm with sound convergence properties to construct the Mm-$Φ_p$ design. The performance of the proposed Mm-$Φ_p$ design is assessed through several numerical examples.

stat.ME↗

arXiv2020

Scalable Algorithms for the Sparse Ridge Regression

Sparse regression and variable selection for large-scale data have been rapidly developed in the past decades. This work focuses on sparse ridge regression, which enforces the sparsity by use of the L0 norm. We first prove that the continuous relaxation of the mixed integer second order conic (MISOC) reformulation using perspective formulation is equivalent to that of the convex integer formulation proposed in recent work. We also show that the convex hull of the constraint system of MISOC formulation is equal to its continuous relaxation. Based upon these two formulations (i.e., the MISOC formulation and convex integer formulation), we analyze two scalable algorithms, the greedy and randomized algorithms, for sparse ridge regression with desirable theoretical properties. The proposed algorithms are proved to yield near-optimal solutions under mild conditions. We further propose to integrate the greedy algorithm with the randomized algorithm, which can greedily search the features from the nonzero subset identified by the continuous relaxation of the MISOC formulation. The merits of the proposed methods are illustrated through numerical examples in comparison with several existing ones.

stat.CO↗

arXiv2020

An Efficient Algorithm for Elastic I-optimal Design of Generalized Linear Models

The generalized linear models (GLMs) are widely used in statistical analysis and the related design issues are undoubtedly challenging. The state-of-the-art works mostly apply to design criteria on the estimates of regression coefficients. The prediction accuracy is usually critical in modern decision making and artificial intelligence applications. It is of importance to study optimal designs from the prediction aspects for generalized linear models. In this work, we consider the Elastic I-optimality as a prediction-oriented design criterion for generalized linear models, and develop efficient algorithms for such $\text{EI}$-optimal designs. By investigating theoretical properties for the optimal weights of any set of design points and extending the general equivalence theorem to the $\text{EI}$-optimality for GLMs, the proposed efficient algorithm adequately combines the Fedorov-Wynn algorithm and multiplicative algorithm. It achieves great computational efficiency with guaranteed convergence. Numerical examples are conducted to evaluate the feasibility and computational efficiency of the proposed algorithm.

stat.ME↗

arXiv2020

Multivariate Regression of Mixed Responses for Evaluation of Visualization Designs

Information visualization significantly enhances human perception by graphically representing complex data sets. The variety of visualization designs makes it challenging to efficiently evaluate all possible designs catering to users' preferences and characteristics. Most of existing evaluation methods perform user studies to obtain multivariate qualitative responses from users via questionnaires and interviews. However, these methods cannot support online evaluation of designs as they are often time-consuming. A statistical model is desired to predict users' preferences on visualization designs based on non-interference measurements (i.e., wearable sensor signals). In this work, we propose a multivariate regression of mixed responses (MRMR) to facilitate quantitative evaluation of visualization designs. The proposed MRMR method is able to provide accurate model prediction with meaningful variable selection. A simulation study and a user study of evaluating visualization designs with 14 effective participants are conducted to illustrate the merits of the proposed model.

stat.ME↗

arXiv2020

An Improved Modified Cholesky Decomposition Method for Precision Matrix Estimation

The modified Cholesky decomposition is commonly used for precision matrix estimation given a specified order of random variables. However, the order of variables is often not available or cannot be pre-determined. In this work, we propose to address the variable order issue in the modified Cholesky decomposition for sparse precision matrix estimation. The key idea is to effectively combine a set of estimates obtained from multiple permutations of variable orders, and to efficiently encourage the sparse structure for the resultant estimate by the thresholding technique on the ensemble Cholesky factor matrix. The consistent property of the proposed estimate is established under some weak regularity conditions. Simulation studies are conducted to evaluate the performance of the proposed method in comparison with several existing approaches. The proposed method is also applied into linear discriminant analysis of real data for classification.

stat.ML↗

arXiv2020

On Variable Ordination of Modified Cholesky Decomposition for Sparse Covariance Matrix Estimation

Estimation of large sparse covariance matrices is of great importance for statistical analysis, especially in the high-dimensional settings. The traditional approach such as the sample covariance matrix performs poorly due to the high dimensionality. The modified Cholesky decomposition (MCD) is a commonly used method for sparse covariance matrix estimation. However, the MCD method relies on the order of variables, which is often not available or cannot be pre-determined in practice. In this work, we solve this order issue by obtaining a set of covariance matrix estimates under different orders of variables used in the MCD. Then we consider an ensemble estimator as the "center" of such a set of covariance matrix estimates with respect to the Frobenius norm. The proposed method not only ensures the estimator to be positive definite, but also can capture the underlying sparse structure of the covariance matrix. Under some weak regularity conditions, we establish both algorithmic convergence and asymptotical convergence of the proposed method. The merits of the proposed method are illustrated through simulation studies and one real data example.

math.ST↗

arXiv2018

Spatial Variable Selection and An Application to Virginia Lyme Disease Emergence

Lyme disease is an infectious disease that is caused by a bacterium called Borrelia burgdorferi sensu stricto. In the United States, Lyme disease is one of the most common infectious diseases. The major endemic areas of the disease are New England, Mid-Atlantic, East-North Central, South Atlantic, and West North-Central. Virginia is on the front-line of the disease's diffusion from the northeast to the south. One of the research objectives for the infectious disease community is to identify environmental and economic variables that are associated with the emergence of Lyme disease. In this paper, we use a spatial Poisson regression model to link the spatial disease counts and environmental and economic variables, and develop a spatial variable selection procedure to effectively identify important factors by using an adaptive elastic net penalty. The proposed methods can automatically select important covariates, while adjusting for possible spatial correlations of disease counts. The performance of the proposed method is studied and compared with existing methods via a comprehensive simulation study. We apply the developed variable selection methods to the Virginia Lyme disease data and identify important variables that are new to the literature. Supplementary materials for this paper are available online.

stat.AP↗

arXiv2016

Sparse Estimation of Multivariate Poisson Log-Normal Models from Count Data

Modeling data with multivariate count responses is a challenging problem due to the discrete nature of the responses. Existing methods for univariate count responses cannot be easily extended to the multivariate case since the dependency among multiple responses needs to be properly accommodated. In this paper, we propose a multivariate Poisson log-normal regression model for multivariate data with count responses. By simultaneously estimating the regression coefficients and inverse covariance matrix over the latent variables with an efficient Monte Carlo EM algorithm, the proposed regression model takes advantages of association among multiple count responses to improve the model prediction performance. Simulation studies and applications to real world data are conducted to systematically evaluate the performance of the proposed method in comparison with conventional methods.

stat.ME↗

arXiv2016

A Parallel Implementation of the Ensemble Kalman Filter Based on Modified Cholesky Decomposition

This paper discusses an efficient parallel implementation of the ensemble Kalman filter based on the modified Cholesky decomposition. The proposed implementation starts with decomposing the domain into sub-domains. In each sub-domain a sparse estimation of the inverse background error covariance matrix is computed via a modified Cholesky decomposition; the estimates are computed concurrently on separate processors. The sparsity of this estimator is dictated by the conditional independence of model components for some radius of influence. Then, the assimilation step is carried out in parallel without the need of inter-processor communication. Once the local analysis states are computed, the analysis sub-domains are mapped back onto the global domain to obtain the analysis ensemble. Computational experiments are performed using the Atmospheric General Circulation Model (SPEEDY) with the T-63 resolution on the Blueridge cluster at Virginia Tech. The number of processors used in the experiments ranges from 96 to 2,048. The proposed implementation outperforms in terms of accuracy the well-known local ensemble transform Kalman filter (LETKF) for all the model variables. The computational time of the proposed implementation is similar to that of the parallel LETKF method (where no covariance estimation is performed). Finally, for the largest number of processors, the proposed parallel implementation is 400 times faster than the serial version of the proposed method.

math.NA↗

arXiv2016

An Ensemble Kalman Filter Implementation Based on Modified Cholesky Decomposition for Inverse Covariance Matrix Estimation

This paper develops an efficient implementation of the ensemble Kalman filter based on a modified Cholesky decomposition for inverse covariance matrix estimation. This implementation is named EnKF-MC. Background errors corresponding to distant model components with respect to some radius of influence are assumed to be conditionally independent. This allows to obtain sparse estimators of the inverse background error covariance matrix. The computational effort of the proposed method is discussed and different formulations based on various matrix identities are provided. Furthermore, an asymptotic proof of convergence with regard to the ensemble size is presented. In order to assess the performance and the accuracy of the proposed method, experiments are performed making use of the Atmospheric General Circulation Model SPEEDY. The results are compared against those obtained using the local ensemble transform Kalman filter (LETKF). Tests are performed for dense observations ($100\%$ and $50\%$ of the model components are observed) as well as for sparse observations (only $12\%$, $6\%$, and $4\%$ of model components are observed). The results reveal that the use of modified Cholesky for inverse covariance matrix estimation can reduce the impact of spurious correlations during the assimilation cycle, i.e., the results of the proposed method are of better quality than those obtained via the LETKF in terms of root mean square error.

math.ST↗

arXiv2012

Estimation in high-dimensional linear models with deterministic design matrices

Because of the advance in technologies, modern statistical studies often encounter linear models with the number of explanatory variables much larger than the sample size. Estimation and variable selection in these high-dimensional problems with deterministic design points is very different from those in the case of random covariates, due to the identifiability of the high-dimensional regression parameter vector. We show that a reasonable approach is to focus on the projection of the regression parameter vector onto the linear space generated by the design matrix. In this work, we consider the ridge regression estimator of the projection vector and propose to threshold the ridge regression estimator when the projection vector is sparse in the sense that many of its components are small. The proposed estimator has an explicit form and is easy to use in application. Asymptotic properties such as the consistency of variable selection and estimation and the convergence rate of the prediction mean squared error are established under some sparsity conditions on the projection vector. A simulation study is also conducted to examine the performance of the proposed estimator.

math.ST↗

arXiv2011

Sparse linear discriminant analysis by thresholding for high dimensional data

In many social, economical, biological and medical studies, one objective is to classify a subject into one of several classes based on a set of variables observed from the subject. Because the probability distribution of the variables is usually unknown, the rule of classification is constructed using a training sample. The well-known linear discriminant analysis (LDA) works well for the situation where the number of variables used for classification is much smaller than the training sample size. Because of the advance in technologies, modern statistical studies often face classification problems with the number of variables much larger than the sample size, and the LDA may perform poorly. We explore when and why the LDA has poor performance and propose a sparse LDA that is asymptotically optimal under some sparsity conditions on the unknown parameters. For illustration of application, we discuss an example of classifying human cancer into two classes of leukemia based on a set of 7,129 genes and a training sample of size 72. A simulation is also conducted to check the performance of the proposed method.

math.ST↗