Interpreting relative utility for probabilistic predictions

Linard Hoessly

Home/Archive/arXiv

arXiv · 2609.29133

Interpreting relative utility for probabilistic predictions

Linard Hoessly

View original source ↗Read source document ↗Export metadata ↓

Abstract

At a fixed threshold, relative utility (RU) measures the net-benefit gain of a prediction model over the better of treat-all and treat-none relative to the corresponding gain under perfect outcome classification. We illustrate that RU equal to 1 therefore represents perfect outcome classification at that fixed threshold, not perfect probabilistic prediction. However, even when every predicted probability equals the true probability, observed RU can equal 0. In a simple constant-risk setting, this occurs with probability approaching 1 as the sample size increases. Consequently, the distance from observed RU to 1 should not in general be interpreted as improvement achievable by a better prediction for binary probabilities.

Explore related subjects

stat.AP

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Linard Hoessly. 2026-09-24. Interpreting relative utility for probabilistic predictions. https://arxiv.org/abs/2609.29133

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Auditing Bayesian Graph Alignment: Diagnostic Comparisons and Reference Failure

Bayesian graph alignment estimates correspondence probabilities, but convergence of an alignment-score trace need not imply accurate correspondence marginals. We audit this gap on 240 new exact graph pairs from four source families, 240 larger pairs with 20-100 vertices, and a separate 60-case exact implementation check. Under an explicit edge-flip likelihood, we compare three samplers and score, marginal, indicator, categorical, and classifier-based diagnostics. Marginal disagreement improves error discrimination over score R-hat for the exact informed sampler, but its improvement for vanilla local sampling is uncertain. Assignment-based R* and short indicator panels are competitive; no diagnostic dominates across samplers and endpoints. At larger sizes, diagnostics predict subsequent marginal changes, not posterior error, and classification performance depends on the drift threshold. Disjoint-window and held-out-chain checks attenuate but preserve positive associations. Only 22 of 240 original reference sets pass an agreement screen. On forty failure-selected cases, eightfold SMC particle escalation does not resolve disagreement, whereas additional rejuvenation helps. Longer informed runs remain unstable. An elementary feasible-alignment bound demonstrates severely unrepresentative SMC and informed-chain scores in concentrated 100-vertex cases, independently of approximate reference consensus. We also exhibit common-start chains with near-zero disagreement despite exact marginal error near .967. These results support assignment-sensitive auditing while identifying limits of finite budgets, diagnostic rankings, and reference agreement as evidence of accuracy.

stat.AP↗

arXiv2026

GeoDose-CP: Graph-Local Conformal Inference for Continuous-Treatment Earth Observation

Reliable intervention-oriented uncertainty quantification from Earth observation (EO) remains challenging when continuous treatment shifts, spatial dependence, limited support, and satellite-outcome uncertainty must be addressed simultaneously. Existing causal, conformal, and spatial approaches address parts of this problem, but their direct combination does not generally recover the appropriate interventional reference law because candidate reassignment jointly alters treatment likelihood, standardized residuals, and graph-dependent residual likelihood. This study presents GeoDose-CP, a support-aware conformal framework for localized stochastic potential outcomes under continuous or mixed continuous-atomic treatment. Its central methodological contribution is a graph-local target-orbit law that jointly represents intervention-induced treatment shift, the inverse outcome-scale Jacobian, and spatial residual dependence. The framework further provides exact weighted candidate inversion, a scalable sparse approximation with explicit discrepancy accounting, and refusal under inadequate support. Evaluation used controlled known-truth experiments, MineDoseBench, treatment-density sensitivity analysis, external conformal comparators, and a multi-mine New South Wales (NSW) study. In MineDoseBench, GeoDose-CP achieved mean selective coverage of 0.9692 across 27 configurations and a minimum local q0.05 of 0.8951; exact-sparse auditing produced nine inclusion disagreements over 2,700 targets. In the NSW study, the absence of an auditable longitudinal rehabilitation treatment rendered treatment-dependent inference nonoperational rather than forcing inference through a proxy exposure.

stat.AP↗

arXiv2026

Hybrid Models for Short-Term Sea-Level Forecasting

Accurate tide forecasts are essential for coastal management, navigation, flood-risk reduction, and infrastructure protection. Observed sea level can be decomposed into astronomical and non-astronomical components, the latter mainly driven by meteorological effects. This study investigates a hybrid framework for hourly sea-level forecasting that combines harmonic analysis (HA) for the astronomical component with data-driven models for the non-astronomical contribution. The approach is evaluated at six tide-gauge stations with different tidal regimes: Venice, Trieste, Saint-Malo, Vardø, Nikiski, and Nagasaki. Four data-driven model classes are considered: (i) linear parametric models, represented by autoregressive models with exogenous variables; (ii) functional parametric models, based on functional autoregressive models with exogenous variables; (iii) semiparametric and nonlinear models, including generalized additive models and autoregressive neural networks; and (iv) a semi-functional non-standard k-nearest-neighbours approach combining similarity in recent non-astronomical trajectories and meteorological conditions. Results reveal that hybrid models reduce forecast errors by 52.9-54.9% on average relative to HA. The generalized additive model is the most competitive across locations, while k-nearest neighbours performs best at Saint-Malo and the autoregressive model with exogenous variables is favoured in Nagasaki. For Venice, an economic decision-making case study assesses the operational use of sea-level forecasts in managing the MoSE flood-barrier system.

stat.AP↗