Dangers of Bayesian analyses and how to address them

Abstract

In light of the US FDA announcement supporting the use of Bayesian methods in clinical trials, we present a nontechnical review of problems with Bayesian analyses and methods to address them. Our focus is on the well-known sensitivities of Bayesian results to biases embedded in their prior distributions. This vulnerability calls for special safety features, including detailed justification of informative priors based on empirical-research literature rather than on singular expert opinions. Crucially, the information contributed by priors should be realistic in light of actual background data rather than merely judged so by available experts. We illustrate our recommendations with using a textbook clinical-trial example which covered these points. We then use the example describe basic diagnostic procedures and safety recommendations for Bayesian analyses and their presentation. These include reference analyses which exclude informative priors, and justification for informative priors based on empirical-research literature rather than on philosophical arguments. We also critique some common priors that violate our realism requirements. One strategy to aid judgments about priors is to translate them into data to be added to the study data. The source of these prior data should be examined with the same critical attitude as the source of study data. We illustrate some simple summary methods for comparing prior and data information, including prior predictions of data, variance ratios, and their translation to effective sample sizes (ESS). Finally, we argue that Bayesian analyses should be treated as extensions or supplements to conventional analyses, rather than replace them, for that treatment puts prior construction and justification at the fore of the transition from frequentist to Bayesian analyses in teaching and practice.