arXiv subjects

Sebastian Arnold

Publications and source records attributed to Sebastian Arnold.

12 recordsLinked to original sources

arXiv2026

Betting on Bets: Anytime-Valid Tests for Stochastic Dominance

How can we monitor, in real time, whether one uncertain prospect has any upside over another? To answer this question, we develop a novel family of sequential, anytime-valid tests for stochastic dominance (SD), a classical and popular notion for comparing entire distribution functions. The problem is distinct from that of testing mean dominance, and it is particularly useful when comparing distributions with similar means or with ordinal outcomes. We first derive powerful, nonparametric e-processes that quantify evidence against the null hypothesis that one prospect is stochastically dominated by another. For first-order SD, these e-processes are based on mixtures of growth-rate optimal e-variables, yielding a test of power one that retains validity under continuous monitoring. We then generalize the approach to sequential testing for higher-order SD and other integral stochastic orders. Empirically, we find that the tests are competitive in power with classical, non-anytime-valid SD tests. Our real-world application examines a controversial phenomenon in baseball analytics, known as the "third-time-through-the-order (3TTO) penalty," viewed as a monitoring problem. We close by sketching the complementary problem of testing whether a prospect has a definite upside, formalizing conditions under which we can derive a powerful anytime-valid test.

stat.ME↗

arXiv2026

Optimal e-values for testing the mean of a bounded random variable against a composite alternative

We derive explicitly the e-values with optimal (relative) growth rate in the worst case for testing the mean of a bounded random variable, thereby providing the first application of the (RE)GROW quality criteria beyond the assumption of mutually absolutely continuous hypotheses for e-values originally proposed by Gr\"unwald et al. (2024). For both criteria, we explicitly characterise the alternatives that are most difficult to test against and show that they admit a meaningful interpretation. We give two important examples in which REGROW provides a powerful quality criterion to choose optimal e-variables whereas GROW leads to trivial solutions.

math.ST↗

arXiv2025

An exploration of sequential Bayesian variable selection -- A comment on Garc\'{i}a-Donato et al. (2025). "Model uncertainty and missing data: An objective Bayesian perspective"

Our comment on Garc\'ia-Donato et al. (2025). "Model uncertainty and missing data: An objective Bayesian perspective" explores a further extension of the proposed methodology. Specifically, we consider the sequential setting where (potentially missing) data accumulate over time, with the goal of continuously monitoring statistical evidence, as opposed to assessing it only once data collection terminates. We explore a new variable selection method based on sequential model confidence sets, as proposed by Arnold et al. (2024), and show that it can help stabilise the inference of Garc\'ia-Donato et al. (2025). To be published as "Invited discussion" in Bayesian Analysis.

stat.ME↗

arXiv2024

Sequential model confidence sets

In most prediction and estimation situations, scientists consider various statistical models for the same problem, and naturally want to select amongst the best. Hansen et al. (2011) provide a powerful solution to this problem by the so-called model confidence set, a subset of the original set of available models that contains the best models with a given level of confidence. Importantly, model confidence sets respect the underlying selection uncertainty by being flexible in size. However, they presuppose a fixed sample size which stands in contrast to the fact that model selection and forecast evaluation are inherently sequential tasks where we successively collect new data and where the decision to continue or conclude a study may depend on the previous outcomes. In this article, we extend model confidence sets sequentially over time by relying on sequential testing methods. Recently, e-processes and confidence sequences have been introduced as new, safe methods for assessing statistical evidence. Sequential model confidence sets allow to continuously monitor the models' performances and come with time-uniform, nonasymptotic coverage guarantees.

stat.ME↗

arXiv2023

Decompositions of the mean continuous ranked probability score

The continuous ranked probability score (crps) is the most commonly used scoring rule in the evaluation of probabilistic forecasts for real-valued outcomes. To assess and rank forecasting methods, researchers compute the mean crps over given sets of forecast situations, based on the respective predictive distributions and outcomes. We propose a new, isotonicity-based decomposition of the mean crps into interpretable components that quantify miscalibration (MSC), discrimination ability (DSC), and uncertainty (UNC), respectively. In a detailed theoretical analysis, we compare the new approach to empirical decompositions proposed earlier, generalize to population versions, analyse their properties and relationships, and relate to a hierarchy of notions of calibration. The isotonicity-based decomposition guarantees the nonnegativity of the components and quantifies calibration in a sense that is stronger than for other types of decompositions, subject to the nondegeneracy of empirical decompositions. We illustrate the usage of the isotonicity-based decomposition in case studies from weather prediction and machine learning.

stat.ME↗

arXiv2023

Isotonic conditional laws

We introduce isotonic conditional laws (ICL) which extend the classical notion of conditional laws by the additional requirement that there exists an isotonic relationship between the random variable of interest and the conditioning random object. We show existence and uniqueness of ICL building on conditional expectations given $\sigma$-lattices. ICL corresponds to a classical conditional law if and only if the latter is already isotonic. ICL is motivated from a statistical point of view by showing that ICL emerges equivalently as the minimizer of an expected score where the scoring rule may be taken from a large class comprising the continuous ranked probability score (CRPS). Furthermore, ICL is calibrated in the sense that it is invariant to certain conditioning operations, and the corresponding event probabilities and quantiles are simultaneously optimal with respect to all relevant scoring functions. We develop a new notion of general conditional functionals given $\sigma$-lattices which is of independent interest.

math.ST↗

arXiv2021

Sequentially valid tests for forecast calibration

Forecasting and forecast evaluation are inherently sequential tasks. Predictions are often issued on a regular basis, such as every hour, day, or month, and their quality is monitored continuously. However, the classical statistical tools for forecast evaluation are static, in the sense that statistical tests for forecast calibration are only valid if the evaluation period is fixed in advance. Recently, e-values have been introduced as a new, dynamic method for assessing statistical significance. An e-value is a non-negative random variable with expected value at most one under a null hypothesis. Large e-values give evidence against the null hypothesis, and the multiplicative inverse of an e-value is a conservative p-value. E-values are particularly suitable for sequential forecast evaluation, since they naturally lead to statistical tests which are valid under optional stopping. This article proposes e-values for testing probabilistic calibration of forecasts, which is one of the most important notions of calibration. The proposed methods are also more generally applicable for sequential goodness-of-fit testing. We demonstrate that the e-values are competitive in terms of power when compared to extant methods, which do not allow sequential testing. Furthermore, they provide important and useful insights in the evaluation of probabilistic weather forecasts.

stat.ME↗

arXiv2021

Self-supervised Answer Retrieval on Clinical Notes

Retrieving answer passages from long documents is a complex task requiring semantic understanding of both discourse and document context. We approach this challenge specifically in a clinical scenario, where doctors retrieve cohorts of patients based on diagnoses and other latent medical aspects. We introduce CAPR, a rule-based self-supervision objective for training Transformer language models for domain-specific passage matching. In addition, we contribute a novel retrieval dataset based on clinical notes to simulate this scenario on a large corpus of clinical notes. We apply our objective in four Transformer-based architectures: Contextual Document Vectors, Bi-, Poly- and Cross-encoders. From our extensive evaluation on MIMIC-III and three other healthcare datasets, we report that CAPR outperforms strong baselines in the retrieval of domain-specific passages and effectively generalizes across rule-based and human-labeled passages. This makes the model powerful especially in zero-shot scenarios where only limited training data is available.

cs.IR↗

arXiv2020

Learning Contextualized Document Representations for Healthcare Answer Retrieval

We present Contextual Discourse Vectors (CDV), a distributed document representation for efficient answer retrieval from long healthcare documents. Our approach is based on structured query tuples of entities and aspects from free text and medical taxonomies. Our model leverages a dual encoder architecture with hierarchical LSTM layers and multi-task training to encode the position of clinical entities and aspects alongside the document discourse. We use our continuous representations to resolve queries with short latency using approximate nearest neighbor search on sentence level. We apply the CDV model for retrieving coherent answer passages from nine English public health resources from the Web, addressing both patients and medical professionals. Because there is no end-to-end training data available for all application scenarios, we train our model with self-supervised data from Wikipedia. We show that our generalized model significantly outperforms several state-of-the-art baselines for healthcare passage ranking and is able to adapt to heterogeneous domains without additional fine-tuning.

cs.CL↗

arXiv2019

Bivariate distributions with ordered marginals

This paper provides a characterization of all possible dependency structures between two stochastically ordered random variables. The answer is given in terms of copulas that are compatible with the stochastic order and the marginal distributions. The extremal values for Kendall's $τ$ and Spearman's $ρ$ for all these copulas are given in closed form. We also find an explicit form for the joint distribution with the maximal entropy. A multivariate extension and a generalization to random elements in partially ordered spaces are also provided.

math.PR↗

arXiv2019

SECTOR: A Neural Model for Coherent Topic Segmentation and Classification

When searching for information, a human reader first glances over a document, spots relevant sections and then focuses on a few sentences for resolving her intention. However, the high variance of document structure complicates to identify the salient topic of a given section at a glance. To tackle this challenge, we present SECTOR, a model to support machine reading systems by segmenting documents into coherent sections and assigning topic labels to each section. Our deep neural network architecture learns a latent topic embedding over the course of a document. This can be leveraged to classify local topics from plain text and segment a document at topic shifts. In addition, we contribute WikiSection, a publicly available dataset with 242k labeled sections in English and German from two distinct domains: diseases and cities. From our extensive evaluation of 20 architectures, we report a highest score of 71.6% F1 for the segmentation and classification of 30 topics from the English city domain, scored by our SECTOR LSTM model with bloom filter embeddings and bidirectional segmentation. This is a significant improvement of 29.5 points F1 compared to state-of-the-art CNN classifiers with baseline segmentation.

cs.CL↗

arXiv2016

Robust Named Entity Recognition in Idiosyncratic Domains

Named entity recognition often fails in idiosyncratic domains. That causes a problem for depending tasks, such as entity linking and relation extraction. We propose a generic and robust approach for high-recall named entity recognition. Our approach is easy to train and offers strong generalization over diverse domain-specific language, such as news documents (e.g. Reuters) or biomedical text (e.g. Medline). Our approach is based on deep contextual sequence learning and utilizes stacked bidirectional LSTM networks. Our model is trained with only few hundred labeled sentences and does not rely on further external knowledge. We report from our results F1 scores in the range of 84-94% on standard datasets.

cs.CL↗