Alessandro Rovetta | Searcharxiv

arXiv2026

Statistical Compatibility, Refutational Information, and Acceptability

This paper develops an interpretive framework for divergence P-values and S-values within a descriptive frequentist perspective. Statistical analysis is framed as operating within idealized worlds defined by a set of assumptions and a target hypothesis, where probabilities describe the behavior of data under the model but do not assign truth values to hypotheses. Within this view, P-values are interpreted as graded indices of compatibility between the observed result and the predictions generated by the assumed model; accordingly, small P-values should not be read as indicating logical impossibility or strict inconsistency of the model itself. Building on this distinction, the paper argues that practical inference requires moving beyond the internal logic of the model toward judgments of overall acceptability, which depend not only on data-model compatibility but also on multiple contextual considerations such as subject-matter knowledge, plausibility of assumptions, data quality, usefulness, and loss - all interpreted through the competence, intentions, perceptions, and moral values of the specific analyst. S-values are therefore interpreted not as evidence against the epistemic status of the model, but as a specific form of refutational information that contributes to the broader body of information used by the analyst to judge whether a model remains acceptable for an intended practical purpose. The paper also examines the linguistic and conceptual risks associated with the language of incompatibility, distinguishes probability from rarity, and clarifies different notions of surprise - including a possible definition of Shannon-type surprise, to be distinguished from Bayesian belief revision. Overall, the article proposes a more cautious and explicit interpretation of frequentist measures, centered on model-based description, analyst responsibility, and decision acceptability.

stat.OT↗

arXiv2025

On the Ambiguities of Incompatibility in Frequentist Inference

The interpretation of the P-value and its monotone transform s=-log2(p), or S-value, remains debated despite decades of dedicated literature. Within the neo-Fisherian framework, these values are often described as indices of (in)compatibility between the observed data and a set of ideal assumptions (i.e., the statistical model). In this regard, this paper proposes the distinction between two domains: the model domain, where assumptions are taken as perfectly true and every admissible outcome is, by construction, fully compatible with the model; and the real domain, where assumptions may fail and face empirical scrutiny. I argue that, although interpreted through an objective numerical index, any level of incompatibility can arise only in the latter domain, where the epistemic status of the model under examination is uncertain and a genuine conflict between data and hypotheses can therefore occur. The extent to which P- and S-values are taken as indicating incompatibility is a matter of contextual judgment. Within this framework, descriptive approaches serve to quantify the numerical values of P and S; these can be interpreted as indicative of a certain degree (or amount) of incompatibility between data and hypotheses once causal knowledge of the data-generating process and information about the costs and benefits of related decisions become clearer. Although the distinction between the model domain and the real domain may appear merely theoretical or even philosophical, I argue that this perspective is useful for developing a clear mental representation of how statistical estimates should be evaluated in practical settings and applications.

stat.OT↗