arXiv subjects

Aoxiang Chen

Publications and source records attributed to Aoxiang Chen.

3 recordsLinked to original sources

arXiv2026

Generalized reparametrized variational Bayes with skew-symmetric normalization

Bayesian hierarchical models with high-dimensional latent structure require scalable posterior approximations that preserve key dependencies while remaining computationally tractable. Mean-field variational inference (MFVI) is efficient, but can be unreliable when local variables are strongly correlated or tightly coupled to global variables. We propose KNorm-RVB, a generalized reparametrized variational Bayes framework for latent Gaussian and latent non-Gaussian models with sparse local precision matrices. KNorm-RVB maps the conditional posterior of local variables toward a standard Gaussian via normalization followed by skewness reduction, enabled by a novel K-component skew-symmetric density representation. This reparametrization centers the transformed conditional local posterior at an optimized reflection point and decorrelates local and global variables, making MFVI much more effective. Under symmetry conditions, we show that MFVI recovers the local posterior mean and correlation matrix exactly, motivating KNorm-RVB's normalization and symmetrization of the conditional local posterior before applying MFVI. We combine a Gaussian variational family for reparametrized local variables with a flexible closed skew normal family for the remaining variables. Across generalized linear mixed models, mixed multinomial logit models, spatial autoregressive models, and stochastic volatility models, KNorm-RVB improves posterior approximation accuracy over existing methods.

stat.ME↗

arXiv2025

Weighted Fisher divergence for high-dimensional Gaussian variational inference

Bayesian inference has many advantages for complex models, but standard Monte Carlo methods for summarizing the posterior can be computationally demanding, and it is attractive to consider optimization-based variational methods. Our work considers Gaussian approximations with sparse precision matrices which are tractable to optimize in high-dimensions. The optimal Gaussian approximation is usually defined as being closest to the posterior in Kullback-Leibler divergence, but it is useful to consider other divergences when the Gaussian assumption is crude, to capture important posterior features for given applications. Our work studies the weighted Fisher divergence, which focuses on gradient differences between the target posterior and its approximation, with the Fisher and score-based divergences as special cases. We make three main contributions. First, we compare approximations for weighted Fisher divergences under mean-field assumptions for Gaussian and non-Gaussian targets with Kullback-Leibler approximations. Second, we go beyond mean-field and consider approximations with sparse precision matrices reflecting posterior conditional independence structure for hierarchical models. Using stochastic gradient descent to enforce sparsity, we develop two approaches to minimize the Fisher and score-based divergences, based on the reparametrization trick and a batch approximation of the objective. Finally, we study the performances of our methods using logistic regression, generalized linear mixed models and stochastic volatility models.

stat.CO↗

arXiv2024

Variational inference based on a subclass of closed skew normals

Gaussian distributions are widely used in Bayesian variational inference to approximate intractable posterior densities, but the ability to accommodate skewness can improve approximation accuracy significantly, when data or prior information is scarce. We study the properties of a subclass of closed skew normals constructed using affine transformation of independent standardized univariate skew normals as the variational density, and illustrate how it provides increased flexibility and accuracy in approximating the joint posterior in various applications, by overcoming limitations in existing skew normal variational approximations. The evidence lower bound is optimized using stochastic gradient ascent, where analytic natural gradient updates are derived. We also demonstrate how problems in maximum likelihood estimation of skew normal parameters occur similarly in stochastic variational inference, and can be resolved using the centered parametrization. Supplemental materials are available online.

stat.ME↗