The Actuary's Final Word on Algorithmic Decision Making

Abstract

Paul Meehl's foundational work "Clinical versus Statistical Prediction," provided early theoretical justification and empirical evidence of the superiority of statistical methods over clinical judgment. Despite a century of empirical evidence supporting Meehl's central thesis, from early parole prediction studies in the 1920s to modern meta-analyses, confusion persists regarding when and why his troubling finding applies. This paper provides a contemporary theoretical justification for Meehl's result. Importantly, Meehl's prediction problems require a small set of possible outcomes and machine-readable data. Second, individual predictions and decisions are evaluated only on average. This formulation leads to a natural analysis from statistical decision theory, which shows that statistical rules are more accurate than clinical intuition almost by definition. Meehl's prediction paradox is an example of metrical determinism, where the rules of evaluation implicitly determine the best procedure. The decision-theoretic analysis of Meehl's problem elucidates the utility of algorithmic systems as decision-support tools, but also reveals their natural shortcomings, inducing expertise erosion, decision fatigue, and the usurpation of discretionary judgment.