arXiv subjects

Maximilian Scholz

Publications and source records attributed to Maximilian Scholz.

5 recordsLinked to original sources

arXiv2025

Primed Priors for Simulation-Based Validation of Bayesian Models

Simulation-based calibration (SBC) is a method for validating inference algorithms and model implementations through repeated inference on data simulated from a generative model. For a model to be generative, one must specify proper priors. However, in all but the simplest of cases, choosing priors for every model parameter is a nontrivial task. In particular, priors that are too broad can produce numerical issues due to extreme parameter values while overly narrow ones can exclude precisely those regions of the parameter space where legitimate problems in the implementation would have manifested. When the data to be analyzed is already available, the issue can be sidestepped by checking calibration on the corresponding posterior, but that is not always a viable option. In this paper, we adapt the framework of catalytic priors, which have been recently proposed for construction of data-based prior distributions, and propose primed priors, which do not require real data and can therefore facilitate prior specification in SBC. We discuss relevant connections of primed priors to the theory of catalytic priors and show their use for SBC in three simulation studies.

stat.ME↗

arXiv2024

Prediction can be safely used as a proxy for explanation in causally consistent Bayesian generalized linear models

Bayesian modeling provides a principled approach to quantifying uncertainty in model parameters and model structure and has seen a surge of applications in recent years. Within the context of a Bayesian workflow, we are concerned with model selection for the purpose of finding models that best explain the data, that is, help us understand the underlying data generating process. Since we rarely have access to the true process, all we are left with during real-world analyses is incomplete causal knowledge from sources outside of the current data and model predictions of said data. This leads to the important question of when the use of prediction as a proxy for explanation for the purpose of model selection is valid. We approach this question by means of large-scale simulations of Bayesian generalized linear models where we investigate various causal and statistical misspecifications. Our results indicate that the use of prediction as proxy for explanation is valid and safe only when the models under consideration are sufficiently consistent with the underlying causal structure of the true data generating process.

stat.ME↗

arXiv2023

Posterior accuracy and calibration under misspecification in Bayesian generalized linear models

Generalized linear models (GLMs) are popular for data-analysis in almost all quantitative sciences, but the choice of likelihood family and link function is often difficult. This motivates the search for likelihoods and links that minimize the impact of potential misspecification. We perform a large-scale simulation study on double-bounded and lower-bounded response data where we systematically vary both true and assumed likelihoods and links. In contrast to previous studies, we also study posterior calibration and uncertainty metrics in addition to point-estimate accuracy. Our results indicate that certain likelihoods and links can be remarkably robust to misspecification, performing almost on par with their respective true counterparts. Additionally, normal likelihood models with identity link (i.e., linear regression) often achieve calibration comparable to the more structurally faithful alternatives, at least in the studied scenarios. On the basis of our findings, we provide practical suggestions for robust likelihood and link choices in GLMs.

stat.ME↗

arXiv2023

Some models are useful, but how do we know which ones? Towards a unified Bayesian model taxonomy

Probabilistic (Bayesian) modeling has experienced a surge of applications in almost all quantitative sciences and industrial areas. This development is driven by a combination of several factors, including better probabilistic estimation algorithms, flexible software, increased computing power, and a growing awareness of the benefits of probabilistic learning. However, a principled Bayesian model building workflow is far from complete and many challenges remain. To aid future research and applications of a principled Bayesian workflow, we ask and provide answers for what we perceive as two fundamental questions of Bayesian modeling, namely (a) "What actually is a Bayesian model?" and (b) "What makes a good Bayesian model?". As an answer to the first question, we propose the PAD model taxonomy that defines four basic kinds of Bayesian models, each representing some combination of the assumed joint distribution of all (known or unknown) variables (P), a posterior approximator (A), and training data (D). As an answer to the second question, we propose ten utility dimensions according to which we can evaluate Bayesian models holistically, namely, (1) causal consistency, (2) parameter recoverability, (3) predictive performance, (4) fairness, (5) structural faithfulness, (6) parsimony, (7) interpretability, (8) convergence, (9) estimation speed, and (10) robustness. Further, we propose two example utility decision trees that describe hierarchies and trade-offs between utilities depending on the inferential goals that drive model building and testing.

stat.ME↗

arXiv2021

An empirical study of Linespots: A novel past-fault algorithm

This paper proposes the novel past-faults fault prediction algorithm Linespots, based on the Bugspots algorithm. We analyze the predictive performance and runtime of Linespots compared to Bugspots with an empirical study using the most significant self-built dataset as of now, including high-quality samples for validation. As a novelty in fault prediction, we use Bayesian data analysis and Directed Acyclic Graphs to model the effects. We found consistent improvements in the predictive performance of Linespots over Bugspots for all seven evaluation metrics. We conclude that Linespots should be used over Bugspots in all cases where no real-time performance is necessary.

cs.SE↗