arXiv subjects

Mark Steel

Publications and source records attributed to Mark Steel.

9 recordsLinked to original sources

arXiv2026

Inference on counterfactual distributions using martingale posteriors

Causal inference is often focused on average effects, which can hide important aspects of the effect distributions. Here we consider the entire posterior effects distribution by estimating full counterfactual outcome distributions. We propose a methodology for inference on counterfactual distributions which builds upon the martingale posterior framework of Fong et al. (2023). This provides a highly flexible approach to estimating densities, distribution functions, and derived quantities such as quantiles, which coherently quantifies the epistemic uncertainty on any target estimand of interest. As the predictive recursions are based on an underlying nonparametric model (a Dirichlet process mixture model), our method naturally inherits robustness with respect to restrictive parametric assumptions. In addition, implementation of our method is typically very fast. This approach can be applied to marginal or conditional counterfactual distributions and is easily extended to an instrumental variables setup. Using the concept of almost conditionally identically distributed random variables, we prove convergence of the martingale posterior inference on the counterfactual outcome distributions for the causal models considered in the paper. We illustrate our approach on both simulated and real data. Using the latter, we investigate the effect of zinc lozenges on common cold duration, the impact of vitamin A supplementation on children's survival rates with one-sided non-compliance (analysed in Imbens and Rubin, 1997a) and the effect of job training (LaLonde, 1986).

stat.ME↗

arXiv2025

Modelling Skewed and Heavy-Tailed Errors in Bayesian Mediation Analysis

Traditional mediation models in both the frequentist and Bayesian frameworks typically assume normality of the error terms. Violations of this assumption can impair the estimation and hypothesis testing of the mediation effect in conventional approaches. This study addresses the non-normality issue by explicitly modelling skewed and heavy-tailed error terms within the Bayesian mediation framework. Building on the work of Fern\'andez and Steel (1998), this study introduces a novel family of distributions, termed the Centred Two-Piece Student $t$ Distribution (CTPT). The new distribution incorporates a skewness parameter into the Student t distribution and centres it to have a mean of zero, enabling flexible modelling of error terms in Bayesian regression and mediation analysis. A class of standard improper priors is employed, and conditions for the existence of the posterior distribution and posterior moments are established, while enabling inference on both skewness and tail parameters. Simulation studies are conducted to examine parameter recovery accuracy and statistical power in testing mediation effects. Compared to traditional Bayesian and frequentist methods, particularly bootstrap-based approaches, our method gives greater statistical power when correctly specified, while maintaining robustness against model misspecification. The application of the proposed approach is illustrated through real data analysis. Additionally, we have developed an R package FlexBayesMed to implement our methods in linear regression and mediation analysis, available at https://github.com/Zongyu-Li/FlexBayesMed.

stat.ME↗

arXiv2025

Bayesian Model Averaging in Causal Instrumental Variable Models

Instrumental variables are a popular tool to infer causal effects under unobserved confounding, but choosing suitable instruments is challenging in practice. We propose gIVBMA, a Bayesian model averaging procedure that addresses this challenge by averaging across different sets of instrumental variables and covariates in a structural equation model. This allows for data-driven selection of valid and relevant instruments and provides additional robustness against invalid instruments. Our approach extends previous work through a scale-invariant prior structure and accommodates non-Gaussian outcomes and treatments, offering greater flexibility than existing methods. The computational strategy uses conditional Bayes factors to update models separately for the outcome and treatments. We prove that this model selection procedure is consistent. In simulation experiments, gIVBMA outperforms current state-of-the-art methods. We demonstrate its usefulness in two empirical applications: the effects of malaria and institutions on income per capita and the returns to schooling. A software implementation of gIVBMA is available in Julia.

stat.ME↗

arXiv2024

Your copula is a classifier in disguise: classification-based copula density estimation

We propose reinterpreting copula density estimation as a discriminative task. Under this novel estimation scheme, we train a classifier to distinguish samples from the joint density from those of the product of independent marginals, recovering the copula density in the process. We derive equivalences between well-known copula classes and classification problems naturally arising in our interpretation. Furthermore, we show our estimator achieves theoretical guarantees akin to maximum likelihood estimation. By identifying a connection with density ratio estimation, we benefit from the rich literature and models available for such problems. Empirically, we demonstrate the applicability of our approach by estimating copulas of real and high-dimensional datasets, outperforming competing copula estimators in density evaluation as well as sampling.

stat.ME↗

arXiv2024

Quasi-Bayes meets Vines

Recently proposed quasi-Bayesian (QB) methods initiated a new era in Bayesian computation by directly constructing the Bayesian predictive distribution through recursion, removing the need for expensive computations involved in sampling the Bayesian posterior distribution. This has proved to be data-efficient for univariate predictions, but extensions to multiple dimensions rely on a conditional decomposition resulting from predefined assumptions on the kernel of the Dirichlet Process Mixture Model, which is the implicit nonparametric model used. Here, we propose a different way to extend Quasi-Bayesian prediction to high dimensions through the use of Sklar's theorem by decomposing the predictive distribution into one-dimensional predictive marginals and a high-dimensional copula. Thus, we use the efficient recursive QB construction for the one-dimensional marginals and model the dependence using highly expressive vine copulas. Further, we tune hyperparameters using robust divergences (eg. energy score) and show that our proposed Quasi-Bayesian Vine (QB-Vine) is a fully non-parametric density estimator with \emph{an analytical form} and convergence rate independent of the dimension of data in some situations. Our experiments illustrate that the QB-Vine is appropriate for high dimensional distributions ($\sim$64), needs very few samples to train ($\sim$200) and outperforms state-of-the-art methods with analytical forms for density estimation and supervised tasks by a considerable margin.

stat.ML↗

arXiv2017

In Search of Lost (Mixing) Time: Adaptive Markov chain Monte Carlo schemes for Bayesian variable selection with very large p

The availability of data sets with large numbers of variables is rapidly increasing. The effective application of Bayesian variable selection methods for regression with these data sets has proved difficult since available Markov chain Monte Carlo methods do not perform well in typical problem sizes of interest. The current paper proposes new adaptive Markov chain Monte Carlo algorithms to address this shortcoming. The adaptive design of these algorithms exploits the observation that in large $p$ small $n$ settings, the majority of the $p$ variables will be approximately uncorrelated a posteriori. The algorithms adaptively build suitable non-local proposals that result in moves with squared jumping distance significantly larger than standard methods. Their performance is studied empirically in high-dimensional problems (with both simulated and actual data) and speedups of up to 4 orders of magnitude are observed. The proposed algorithms are easily implementable on multi-core architectures and are well suited for parallel tempering or sequential Monte Carlo implementations.

stat.CO↗

arXiv2016

Methods and Tools for Bayesian Variable Selection and Model Averaging in Univariate Linear Regression

In this paper we briefly review the main methodological aspects concerned with the application of the Bayesian approach to model choice and model averaging in the context of variable selection in regression models. This includes prior elicitation, summaries of the posterior distribution and computational strategies. We then examine and compare various publicly available {\tt R}-packages for its practical implementation summarizing and explaining the differences between packages and giving recommendations for applied users. We find that all packages reviewed lead to very similar results, but there are potentially important differences in flexibility and efficiency of the packages.

stat.CO↗

arXiv2016

On choosing mixture components via non-local priors

Choosing the number of mixture components remains an elusive challenge. Model selection criteria can be either overly liberal or conservative and return poorly-separated components of limited practical use. We formalize non-local priors (NLPs) for mixtures and show how they lead to well-separated components with non-negligible weight, interpretable as distinct subpopulations. We also propose an estimator for posterior model probabilities under local and non-local priors, showing that Bayes factors are ratios of posterior to prior empty-cluster probabilities. The estimator is widely applicable and helps set thresholds to drop unoccupied components in overfitted mixtures. We suggest default prior parameters based on multi-modality for Normal/T mixtures and minimal informativeness for categorical outcomes. We characterise theoretically the NLP-induced sparsity, derive tractable expressions and algorithms. We fully develop Normal, Binomial and product Binomial mixtures but the theory, computation and principles hold more generally. We observed a serious lack of sensitivity of the Bayesian information criterion (BIC), insufficient parsimony of the AIC and a local prior, and a mixed behavior of the singular BIC. We also considered overfitted mixtures, their performance was competitive but depended on tuning parameters. Under our default prior elicitation NLPs offered a good compromise between sparsity and power to detect meaningfully-separated components.

stat.ME↗

arXiv2014

Individual adaptation: an adaptive MCMC scheme for variable selection problems

The increasing size of data sets has lead to variable selection in regression becoming increasingly important. Bayesian approaches are attractive since they allow uncertainty about the choice of variables to be formally included in the analysis. The application of fully Bayesian variable selection methods to large data sets is computationally challenging. We describe an adaptive Markov chain Monte Carlo approach called Individual Adaptation which adjusts a general proposal to the data. We show that the algorithm is ergodic and discuss its use within parallel tempering and sequential Monte Carlo approaches. We illustrate the use of the method on two data sets including a gene expression analysis with 22 577 variables.

stat.CO↗