arXiv subjects

Don van den Bergh

Publications and source records attributed to Don van den Bergh.

6 recordsLinked to original sources

arXiv2026

Accelerating Bayesian Variable Selection using Piecewise Deterministic Markov Processes

Bayesian variable selection becomes computationally challenging when models contain many dependent parameters. We study Piecewise Deterministic Markov Process (PDMP) samplers as a continuous-time alternative to conventional Markov chain Monte Carlo for spike-and-slab variable selection. In sticky PDMP samplers, active parameters evolve continuously until they reach zero, where they may remain for a random duration before re-entering the model. While one parameter enters or leaves the model, the remaining parameters continue to evolve along the deterministic flow, offering a potentially advantageous mechanism for exploring posteriors with strongly dependent parameters. We make two methodological contributions. First, we extend existing sticky PDMP methods beyond independent spike-and-slab priors to dependent model priors and dependent slab distributions. Second, we investigate the use of unbiased stochastic gradients to reduce the computational cost of variable selection when the likelihood decomposes into many factors while retaining the same target distribution. We study these extensions to two psychometric models: a Gaussian random intercept cross-lagged panel model and an ordinal Markov random field. For the former, marginalization yields a fixed-dimensional sufficient-statistic representation that permits efficient model evaluation. For the latter, the model factorizes, which enables subsampling over person-node contributions. In simulation studies, we compare ZigZag, Bouncy Particle, and Boomerang dynamics with reversible-jump MCMC, and examine the effects of prior dependence and stochastic-gradient subsampling on sampling efficiency. We illustrate the methodology using data from an empirical study on mental well-being. Finally, we discuss the advantages and challenges when using PDMP samplers for Bayesian variable selection.

stat.ME↗

arXiv2026

Comparing Variable Selection and Model Averaging Methods for Logistic Regression

Model uncertainty is a central challenge in statistical models for binary outcomes such as logistic regression, arising when it is unclear which predictors should be included in the model. Many methods have been proposed to address this issue for logistic regression, but their relative performance under realistic conditions remains poorly understood. We therefore conducted a preregistered, simulation-based comparison of 28 established methods for variable selection and inference under model uncertainty, using 11 empirical datasets spanning a range of sample sizes and number of predictors, in cases both with and without separation. We found that Bayesian model averaging (BMA) methods based on g-priors, particularly g = max(n, p^2), show the strongest overall performance when separation is absent. When separation occurs, penalized likelihood approaches, especially the LASSO, provide the most stable results, while BMA with the local empirical Bayes (EB-local) prior is competitive in both situations. These findings offer practical guidance for applied researchers on how to effectively address model uncertainty in logistic regression in modern empirical and machine learning research.

stat.ME↗

arXiv2026

Reversible Jump MCMC With No Regrets: Bayesian Variable Selection Using Mixtures of Mutually Singular Distributions

Bayesian variable selection requires sampling from a posterior distribution that combines discrete model indicators with continuously varying parameters, a challenge often addressed through reversible jump Markov chain Monte Carlo (RJMCMC). Despite its generality, RJMCMC is widely regarded as difficult to design and implement correctly. We present mixtures of mutually singular (MoMS) distributions as a transparent alternative in which competing models are represented within a single fixed-dimensional parameter space partitioned into mutually singular subspaces. We show that this formulation reproduces the exact spike-and-slab interpretation of Bayesian variable selection and that, under appropriate constructions, MoMS and RJMCMC share the same Metropolis--Hastings acceptance probability. On a benchmark dataset with ten predictors, both methods recover posterior inclusion probabilities that match full enumeration, while MoMS achieves comparable or superior effective sample size per second relative to a carefully engineered RJMCMC scheme. We further illustrate the approach in a mixed-effects logistic regression for a sleep-and-memory experiment and in factor-loading selection for a multidimensional generalized partial credit model. Together, these results show that Bayesian variable selection can be carried out within standard fixed-dimensional Markov chain Monte Carlo methodology -- without regret.

stat.ME↗

arXiv2026

Efficient Bayes Factor Sensitivity Analysis via Posterior Density Ratios

Bayes factor sensitivity analysis examines how the evidence for one hypothesis over another depends on the prior distribution. In complex models, the standard approach refits the model at each hyper-parameter value, and the total computational cost scales linearly in the grid size. We propose a method that recovers the entire sensitivity curve from a single additional model fit. The key identity decomposes the Bayes factor at any hyper-parameter value $γ_x$ into an ``anchor'' Bayes factor at a fixed reference $γ_0$ and a Savage--Dickey density ratio in an extended model that places a hyper-prior on $γ$. Once this extended model is fit, the Bayes factor at any $γ_x$ follows from the anchor value and a ratio of two posterior density ordinates. To approximate this ratio, we employ the importance-weighted marginal density estimator (IWMDE). Because the sensitivity parameter enters the model only through the prior distribution on the model parameters, the data likelihood cancels in the IWMDE, reducing it to a simple ratio of prior density evaluations on the MCMC draws, without any additional likelihood computation. The resulting estimator is fast, remains accurate even with small MCMC samples, and substantially outperforms kernel density estimation across the full sensitivity range. The method extends naturally to simultaneous sensitivity over multiple hyper-parameters and to Bayesian model averaging. We illustrate it on a univariate Bayesian $t$-test with exact Bayes factors for validation, a bivariate informed $t$-test, and a Bayesian model-averaged meta-analysis, obtaining accurate sensitivity curves at a fraction of the brute-force cost.

stat.ME↗

arXiv2025

Flexible Bayesian Multiple Comparison Adjustment Using Dirichlet Process and Beta-Binomial Model Priors

Researchers frequently wish to assess the equality or inequality of groups, but this poses the challenge of adequately adjusting for multiple comparisons. Statistically, all possible configurations of equality and inequality constraints can be uniquely represented as partitions of groups, where any number of groups are equal if they are in the same subset of the partition. In a Bayesian framework, one can adjust for multiple comparisons by constructing a suitable prior distribution over all possible partitions. Inspired by work on variable selection in regression, we propose a class of flexible beta-binomial priors for multiple comparison adjustment. We compare this prior setup to the Dirichlet process prior suggested by Gopalan and Berry (1998) and multiple comparison adjustment methods that do not specify a prior over partitions directly. Our approach not only allows researchers to assess pairwise equality constraints but simultaneously all possible equalities among all groups. Since the space of possible partitions grows rapidly -- for ten groups, there are already 115,975 possible partitions -- we use a stochastic search algorithm to efficiently explore the space. Our method is implemented in the Julia package EqualitySampler, and we illustrate it on examples related to the comparison of means, standard deviations, and proportions.

stat.ME↗

arXiv2022

Default Bayes Factors for Testing the (In)equality of Several Population Variances

Testing the (in)equality of variances is an important problem in many statistical applications. We develop default Bayes factor tests to assess the (in)equality of two or more population variances, as well as a test for whether the population variances equal a specific value. The resulting test can be used to check assumptions for commonly used procedures such as the $t$-test or ANOVA, or test substantive hypotheses concerning variances directly. We show that our Bayes factor fulfills a number of desiderata. Researchers may have directed hypotheses such as $σ_{1}^{2} > σ_{2}^{2}$, they may want to extend $\mathcal{H}_{0}$ to have a null-region, or wish to combine hypotheses about equality with hypotheses about inequality, for example $σ_{1}^{2} = σ_{2}^{2} > (σ_{3}^{2}, σ_{4}^{2})$. We extend our Bayes factor test to allow for these deviations from our proposed default and illustrate it on a number of practical examples. Our procedure is implemented in the R package $bfvartest$.

stat.ME↗