arXiv subjects

Weichang Yu

Publications and source records attributed to Weichang Yu.

7 recordsLinked to original sources

arXiv2026

Variational approximate penalized credible regions for Bayesian grouped regression

We develop a fast and accurate grouped penalized credible region approach for variable selection and prediction in Bayesian high-dimensional regression. Most existing Bayesian methods either are subject to high computational costs due to long Markov Chain Monte Carlo runs or yield ambiguous variable selection results due to non-sparse solution output. The penalized credible region framework yields sparse post-processed estimates that facilitates unambiguous grouped variable selection. High estimation accuracy is achieved by shrinking noise from unimportant groups using a grouped global-local shrinkage prior. To ensure computational scalability, we approximate posterior summaries using coordinate ascent variational inference and recast the penalized credible region framework as a convex optimization problem that admits efficient computations. We prove that the resultant post-processed estimators are both parameter-consistent and variable selection consistent in high-dimensional settings. Theory is developed to justify running the coordinate ascent algorithm for at least two cycles. Through extensive simulations, we demonstrate that our proposed method outperforms state-of-the-art methods in grouped variable selection, prediction, and computation time for several common models including ANOVA and nonparametric varying coefficient models.

stat.ME↗

arXiv2025

Expectation-Propagation for Bayesian Empirical Likelihood Inference

Bayesian inference typically relies on specifying a parametric model that approximates the data-generating process. However, misspecified models can yield poor convergence rates and unreliable posterior calibration. Bayesian empirical likelihood offers a semi-parametric alternative by replacing the parametric likelihood with a profile empirical likelihood defined through moment constraints, thereby avoiding explicit distributional assumptions. Despite these advantages, Bayesian empirical likelihood faces substantial computational challenges, including the need to solve a constrained optimization problem for each likelihood evaluation and difficulties with non-convex posterior support, particularly in small-sample settings. This paper introduces a variational approach based on expectation-propagation to approximate the Bayesian empirical-likelihood posterior, balancing computational cost and accuracy without altering the target posterior via adjustments such as pseudo-observations. Empirically, we show that our approach can achieve a superior cost-accuracy trade-off relative to existing methods, including Hamiltonian Monte Carlo and variational Bayes. Theoretically, we show that the approximation and the Bayesian empirical-likelihood posterior are asymptotically equivalent.

stat.ME↗

arXiv2023

Cutting Feedback in Misspecified Copula Models

In copula models the marginal distributions and copula function are specified separately. We treat these as two modules in a modular Bayesian inference framework, and propose conducting modified Bayesian inference by "cutting feedback". Cutting feedback limits the influence of potentially misspecified modules in posterior inference. We consider two types of cuts. The first limits the influence of a misspecified copula on inference for the marginals, which is a Bayesian analogue of the popular Inference for Margins (IFM) estimator. The second limits the influence of misspecified marginals on inference for the copula parameters by using a pseudo likelihood of the ranks to define the cut model. We establish that if only one of the modules is misspecified, then the appropriate cut posterior gives accurate uncertainty quantification asymptotically for the parameters in the other module. Computation of the cut posteriors is difficult, and new variational inference methods to do so are proposed. The efficacy of the new methodology is demonstrated using both simulated data and a substantive multivariate time series application from macroeconomic forecasting. In the latter, cutting feedback from misspecified marginals to a 1096 dimension copula improves posterior inference and predictive accuracy greatly, compared to conventional Bayesian inference.

stat.ME↗

arXiv2022

Moment Propagation

Mean-field variational Bayes is a fast and scalable approach to approximate Bayesian inference, but its independence assumptions often lead to underestimated posterior uncertainty. We introduce moment propagation (MP), a framework for improving marginal posterior approximations by propagating conditional posterior moment information between parameter blocks and matching these moments within convenient approximating families. In conjugate settings, MP identifies variance terms omitted by mean-field approximations and uses them to construct corrected marginal updates. We develop both mean-field and Gaussian variants of MP, the latter using conditional Gaussian structure to obtain low-dimensional local updates for non-conjugate models. We establish consistency and asymptotic variance correctness for two-component models, and show that MP recovers exact marginal posteriors for linear regression and multivariate normal models. We also derive algorithms for multivariate normal models with missing data, probit regression, generalized linear models, and Bayesian Lasso regression. Numerical experiments show that MP, particularly Gaussian MP, can substantially improve marginal posterior accuracy over standard variational approximations while remaining much faster than MCMC.

stat.CO↗

arXiv2018

Variational Discriminant Analysis with Variable Selection

A fast Bayesian method that seamlessly fuses classification and hypothesis testing via discriminant analysis is developed. Building upon the original discriminant analysis classifier, modelling components are added to identify discriminative variables. A combination of cake priors and a novel form of variational Bayes we call reverse collapsed variational Bayes gives rise to variable selection that can be directly posed as a multiple hypothesis testing approach using likelihood ratio statistics. Some theoretical arguments are presented showing that Chernoff-consistency (asymptotically zero type I and type II error) is maintained across all hypotheses. We apply our method on some publicly available genomics datasets and show that our method performs well in practice for its computational cost. An R package VaDA has also been made available on Github.

stat.ME↗

arXiv2018

Variational Nonparametric Discriminant Analysis

Variable selection and classification are common objectives in the analysis of high-dimensional data. Most such methods make distributional assumptions that may not be compatible with the diverse families of distributions data can take. A novel Bayesian nonparametric discriminant analysis model that performs both variable selection and classification within a seamless framework is proposed. P{\'o}lya tree priors are assigned to the unknown group-conditional distributions to account for their uncertainty, and allow prior beliefs about the distributions to be incorporated simply as hyperparameters. The adoption of collapsed variational Bayes inference in combination with a chain of functional approximations led to an algorithm with low computational cost. The resultant decision rules carry heuristic interpretations and are related to an existing two-sample Bayesian nonparametric hypothesis test. By an application to some simulated and publicly available real datasets, the proposed method exhibits good performance when compared to current state-of-the-art approaches.

stat.ME↗

arXiv2017

Bayesian hypothesis tests with diffuse priors: Can we have our cake and eat it too?

We introduce a new class of priors for Bayesian hypothesis testing, which we name "cake priors". These priors circumvent Bartlett's paradox (also called the Jeffreys-Lindley paradox); the problem associated with the use of diffuse priors leading to nonsensical statistical inferences. Cake priors allow the use of diffuse priors (having one's cake) while achieving theoretically justified inferences (eating it too). We demonstrate this methodology for Bayesian hypotheses tests for scenarios under which the one and two sample t-tests, and linear models are typically derived. The resulting Bayesian test statistic takes the form of a penalized likelihood ratio test statistic. By considering the sampling distribution under the null and alternative hypotheses we show for independent identically distributed regular parametric models that Bayesian hypothesis tests using cake priors are Chernoff-consistent, i.e., achieve zero type I and II errors asymptotically. Lindley's paradox is also discussed. We argue that a true Lindley's paradox will only occur with small probability for large sample sizes.

math.ST↗