arXiv subjects

Xiaoning Kang

Publications and source records attributed to Xiaoning Kang.

9 recordsLinked to original sources

arXiv2026

Robust and Sparse Generalized Linear Models for High-Dimensional Data via Maximum Mean Discrepancy

High-dimensional datasets are frequently subject to contamination by outliers and heavy-tailed noise, which can severely bias standard regularized estimators like the Lasso. While Maximum Mean Discrepancy (MMD) has recently been introduced as a ``universal'' framework for robust regression, its application to high-dimensional Generalized Linear Models (GLMs) remains largely unexplored, particularly regarding variable selection. In this paper, we propose a penalized MMD framework for robust estimation and feature selection in GLMs. We introduce an $\ell_1$-penalized MMD objective and develop two versions of the estimator: a full $O(n^2)$ version and a computationally efficient $O(n)$ approximation. To solve the resulting non-convex optimization problem, we employ an algorithm based on the Alternating Direction Method of Multipliers (ADMM) combined with AdaGrad. Through extensive simulation studies involving Gaussian linear regression and binary logistic regression, we demonstrate that our proposed methods are highly competitive with classical penalized GLMs and existing robust benchmarks. Our approach shows particular resilience in maintaining a balance between estimation accuracy and variable selection across diverse contamination scenarios, especially in handling high-leverage points and heavy-tailed error distributions where traditional methods may fluctuate in performance.

stat.ME↗

arXiv2025

Weighted Average Ensemble for Cholesky-based Covariance Matrix Estimation

The modified Cholesky decomposition (MCD) is an efficient technique for estimating a covariance matrix. However, it is known that the MCD technique often requires a pre-specified variable ordering in the estimation procedure. In this work, we propose a weighted average ensemble covariance estimation for high-dimensional data based on the MCD technique. It can flexibly accommodate the high-dimensional case and ensure the positive definiteness property of the resultant estimate. Our key idea is to obtain different weights for different candidate estimates by minimizing an appropriate risk function with respect to the Frobenius norm. Different from the existing ensemble estimation based on the MCD, the proposed method provides a sparse weighting scheme such that one can distinguish which variable orderings employed in the MCD are useful for the ensemble matrix estimate. The asymptotically theoretical convergence rate of the proposed ensemble estimate is established under regularity conditions. The merits of the proposed method are examined by the simulation studies and a portfolio allocation example of real stock data.

stat.ME↗

arXiv2023

On Block Cholesky Decomposition for Sparse Inverse Covariance Estimation

The modified Cholesky decomposition is popular for inverse covariance estimation, but often needs pre-specification on the full information of variable ordering. In this work, we propose a block Cholesky decomposition (BCD) for estimating inverse covariance matrix under the partial information of variable ordering, in the sense that the variables can be divided into several groups with available ordering among groups, but variables within each group have no orderings. The proposed BCD model provides a unified framework for several existing methods including the modified Cholesky decomposition and the Graphical lasso. By utilizing the partial information on variable ordering, the proposed BCD model guarantees the positive definiteness of the estimated matrix with statistically meaningful interpretation. Theoretical results are established under regularity conditions. Simulation and case studies are conducted to evaluate the proposed BCD model.

stat.ME↗

arXiv2022

Bayesian Sparse Regression for Mixed Multi-Responses with Application to Runtime Metrics Prediction in Fog Manufacturing

Fog manufacturing can greatly enhance traditional manufacturing systems through distributed Fog computation units, which are governed by predictive computational workload offloading methods under different Industrial Internet architectures. It is known that the predictive offloading methods highly depend on accurate prediction and uncertainty quantification of runtime performance metrics, containing multivariate mixed-type responses (i.e., continuous, counting, binary). In this work, we propose a Bayesian sparse regression for multivariate mixed responses to enhance the prediction of runtime performance metrics and to enable the statistical inferences. The proposed method considers both group and individual variable selection to jointly model the mixed types of runtime performance metrics. The conditional dependency among multiple responses is described by a graphical model using the precision matrix, where a spike-and-slab prior is used to enable the sparse estimation of the graph. The proposed method not only achieves accurate prediction, but also makes the predictive model more interpretable with statistical inferences on model parameters and prediction in the Fog manufacturing. A simulation study and a real case example in a Fog manufacturing are conducted to demonstrate the merits of the proposed model.

stat.ME↗

arXiv2021

A Generative Approach to Joint Modeling of Quantitative and Qualitative Responses

In many scientific areas, data with quantitative and qualitative (QQ) responses are commonly encountered with a large number of predictors. By exploring the association between QQ responses, existing approaches often consider a joint model of QQ responses given the predictor variables. However, the dependency among predictive variables also provides useful information for modeling QQ responses. In this work, we propose a generative approach to model the joint distribution of the QQ responses and predictors. The proposed generative model provides efficient parameter estimation under a penalized likelihood framework. It achieves accurate classification for qualitative response and accurate prediction for quantitative response with efficient computation. Because of the generative approach framework, the asymptotic optimality of classification and prediction of the proposed method can be established under some regularity conditions. The performance of the proposed method is examined through simulations and real case studies in material science and genetics.

stat.ME↗

arXiv2020

Bayesian Auxiliary Variable Model for Birth Records Data with Qualitative and Quantitative Responses

Many applications involve data with qualitative and quantitative responses. When there is an association between the two responses, a joint model will provide improved results than modeling them separately. In this paper, we propose a Bayesian method to jointly model such data. The joint model links the qualitative and quantitative responses and can assess their dependency strength via a latent variable. The posterior distributions of parameters are obtained through an efficient MCMC sampling algorithm. The simulation shows that the proposed method can improve the prediction capacity for both responses. We apply the proposed joint model to the birth records data acquired by the Virginia Department of Health and study the mutual dependence between preterm birth of infants and their birth weights.

stat.ME↗

arXiv2020

Multivariate Regression of Mixed Responses for Evaluation of Visualization Designs

Information visualization significantly enhances human perception by graphically representing complex data sets. The variety of visualization designs makes it challenging to efficiently evaluate all possible designs catering to users' preferences and characteristics. Most of existing evaluation methods perform user studies to obtain multivariate qualitative responses from users via questionnaires and interviews. However, these methods cannot support online evaluation of designs as they are often time-consuming. A statistical model is desired to predict users' preferences on visualization designs based on non-interference measurements (i.e., wearable sensor signals). In this work, we propose a multivariate regression of mixed responses (MRMR) to facilitate quantitative evaluation of visualization designs. The proposed MRMR method is able to provide accurate model prediction with meaningful variable selection. A simulation study and a user study of evaluating visualization designs with 14 effective participants are conducted to illustrate the merits of the proposed model.

stat.ME↗

arXiv2020

An Improved Modified Cholesky Decomposition Method for Precision Matrix Estimation

The modified Cholesky decomposition is commonly used for precision matrix estimation given a specified order of random variables. However, the order of variables is often not available or cannot be pre-determined. In this work, we propose to address the variable order issue in the modified Cholesky decomposition for sparse precision matrix estimation. The key idea is to effectively combine a set of estimates obtained from multiple permutations of variable orders, and to efficiently encourage the sparse structure for the resultant estimate by the thresholding technique on the ensemble Cholesky factor matrix. The consistent property of the proposed estimate is established under some weak regularity conditions. Simulation studies are conducted to evaluate the performance of the proposed method in comparison with several existing approaches. The proposed method is also applied into linear discriminant analysis of real data for classification.

stat.ML↗

arXiv2020

On Variable Ordination of Modified Cholesky Decomposition for Sparse Covariance Matrix Estimation

Estimation of large sparse covariance matrices is of great importance for statistical analysis, especially in the high-dimensional settings. The traditional approach such as the sample covariance matrix performs poorly due to the high dimensionality. The modified Cholesky decomposition (MCD) is a commonly used method for sparse covariance matrix estimation. However, the MCD method relies on the order of variables, which is often not available or cannot be pre-determined in practice. In this work, we solve this order issue by obtaining a set of covariance matrix estimates under different orders of variables used in the MCD. Then we consider an ensemble estimator as the "center" of such a set of covariance matrix estimates with respect to the Frobenius norm. The proposed method not only ensures the estimator to be positive definite, but also can capture the underlying sparse structure of the covariance matrix. Under some weak regularity conditions, we establish both algorithmic convergence and asymptotical convergence of the proposed method. The merits of the proposed method are illustrated through simulation studies and one real data example.

math.ST↗