Adeline P. Guthrie | Searcharxiv

arXiv2025

BRcal: An R Package to Boldness-Recalibrate Probability Predictions

When probability predictions are too cautious for decision making, boldness-recalibration enables responsible emboldening while maintaining the probability of calibration required by the user. We formulate boldness-recalibration as a nonlinear optimization of boldness with a nonlinear inequality constraint on calibration. We further show that recalibration based on the maximized linear log odds likelihood also maximizes the posterior probability of calibration. We introduce BRcal, an R package implementing boldness-recalibration and supporting methodology as recently proposed. The BRcal package provides direct control of the calibration-boldness tradeoff and visualizes how different calibration levels change individual predictions. We present a new real world case study involving housing foreclosure predictions. The BRcal package is available on the Comprehensive R Archive Network (CRAN) (https://cran.r-project.org/web/packages/BRcal/index.html) and on Github (https://github.com/apguthrie/BRcal).

stat.ME↗

arXiv2024

Boldness-Recalibration for Binary Event Predictions

Probability predictions are essential to inform decision making across many fields. Ideally, probability predictions are (i) well calibrated, (ii) accurate, and (iii) bold, i.e., spread out enough to be informative for decision making. However, there is a fundamental tension between calibration and boldness, since calibration metrics can be high when predictions are overly cautious, i.e., non-bold. The purpose of this work is to develop a Bayesian model selection-based approach to assess calibration, and a strategy for boldness-recalibration that enables practitioners to responsibly embolden predictions subject to their required level of calibration. Specifically, we allow the user to pre-specify their desired posterior probability of calibration, then maximally embolden predictions subject to this constraint. We demonstrate the method with a case study on hockey home team win probabilities and then verify the performance of our procedures via simulation. We find that very slight relaxation of calibration probability (e.g., from 0.99 to 0.95) can often substantially embolden predictions when they are well calibrated and accurate (e.g., widening hockey predictions range from .26-.78 to .10-.91).

stat.ME↗