arXiv subjects

Ivo V. Stoepker

Publications and source records attributed to Ivo V. Stoepker.

6 recordsLinked to original sources

arXiv2026

Anytime-Valid Tests for Sparse Anomalies

We consider the problem of testing sequentially for the presence of sparse anomalies among a large number of data streams. To this end, we design and analyze Anytime-Valid (AV) tests, which retain type-I error control at arbitrary stopping times. Existing results address exclusively the nonsequential case, which exhibits a subtle phase transition between two regimes where tests are either powerless or powerful. In our sequential setting, we argue, two challenges arise: (1) the standard analysis of AV tests cannot be executed in the relevant sample-size regime; and (2) standard constructions of parameter-adaptive AV tests are either analytically intractable or computationally unfeasible. This work addresses these challenges. Borrowing insights from the nonsequential literature, we propose a framework to analyze AV tests and their shortest possible sample sizes. Under this framework, we show that, in the Gaussian location setting, the oracle AV test has a delicate threshold behavior that is related to -- but not implied by -- the phase transition observed in optimal nonsequential tests. Our main results include a computationally efficient, parameter-adaptive AV test; we show that it achieves the same threshold behavior as the oracle AV test. Numerical simulations illustrate these theoretical findings.

math.ST↗

arXiv2025

Efficient Sampling in Disease Surveillance through Subpopulations: Sampling Canaries in the Coal Mine

We consider outbreak detection settings of endemic diseases where the population under study consists of various subpopulations available for stratified surveillance. These subpopulations can for example be based on age cohorts, but may also correspond to other subgroups of the population under study such as international travellers. Rather than sampling uniformly across the population, one may elevate the effectiveness of the detection methodology by optimally choosing a sampling subpopulation. We show (under some assumptions) the relative sampling efficiency between two subpopulations is inversely proportional to the ratio of their respective baseline disease risks. This implies one can increase sampling efficiency by sampling from the subpopulation with higher baseline disease risk. Our results require careful treatment of the power curves of exact binomial tests as a function of their sample size, which are non-monotonic due to the underlying discreteness. A case study of COVID-19 cases in the Netherlands illustrates our theoretical findings.

stat.ME↗

arXiv2025

Sparse Anomaly Detection Across Referentials: A Rank-Based Higher Criticism Approach

Detecting anomalies in large sets of observations is crucial in various applications, such as epidemiological studies, gene expression studies, and systems monitoring. We consider settings where the units of interest result in multiple independent observations from potentially distinct referentials. Scan statistics and related methods are commonly used in such settings, but rely on stringent modeling assumptions for proper calibration. We instead propose a rank-based variant of the higher criticism statistic that only requires independent observations originating from ordered spaces. We show under what conditions the resulting methodology is able to detect the presence of anomalies. These conditions are stated in a general, non-parametric manner, and depend solely on the probabilities of anomalous observations exceeding nominal observations. The analysis requires a refined understanding of the distribution of the ranks under the presence of anomalies, and in particular of the rank-induced dependencies. The methodology is robust against heavy-tailed distributions through the use of ranks. Within the exponential family and a family of convolutional models, we analytically quantify the asymptotic performance of our methodology and the performance of the oracle, and show the difference is small for many common models. Simulations confirm these results. We show the applicability of the methodology through an analysis of quality control data of a pharmaceutical manufacturing process.

stat.ME↗

arXiv2024

Inference with Sequential Monte-Carlo Computation of $p$-values: Fast and Valid Approaches

Hypothesis tests calibrated by (re)sampling methods (such as permutation, rank and bootstrap tests) are useful tools for statistical analysis, at the computational cost of requiring Monte-Carlo sampling for calibration. It is common and almost universal practice to execute such tests with predetermined and large number of Monte-Carlo samples, and disregard any randomness from this sampling at the time of drawing and reporting inference. At best, this approach leads to computational inefficiency, and at worst to invalid inference. That being said, a number of approaches in the literature have been proposed to adaptively guide analysts in choosing the number of Monte-Carlo samples, by sequentially deciding when to stop collecting samples and draw inference. These works introduce varying competing notions of what constitutes "valid" inference, complicating the landscape for analysts seeking suitable methodology. Furthermore, the majority of these approaches solely guarantee a meaningful estimate of the testing outcome, not the $p$-value itself $\unicode{x2014}$ which is insufficient for many practical applications. In this paper, we survey the relevant literature, and build bridges between the scattered validity notions, highlighting some of their complementary roles. We also introduce a new practical methodology that provides an estimate of the $p$-value of the Monte-Carlo test, endowed with practically relevant validity guarantees. Moreover, our methodology is sequential, updating the $p$-value estimate after each new Monte-Carlo sample has been drawn, while retaining important validity guarantees regardless of the selected stopping time. We conclude this paper with a set of recommendations for the practitioner, both in terms of selection of methodology and manner of reporting results.

stat.ME↗

arXiv2024

A Holistic Approach for Bitcoin Confirmation Times & Optimal Fee Selection

Bitcoin is currently subject to a significant pay-for-speed trade-off. This is caused by lengthy and highly variable transaction confirmation times, especially during times of congestion. Users can reduce their transaction confirmation times by increasing their transaction fee. In this paper, based on the inner workings of Bitcoin, we propose a model-based approach (based on the Cramér-Lundberg model) that can be used to determine the optimal fee, via, for example, the mean or quantiles, and models accurately the confirmation time distribution for a given fee. The proposed model is highly suitable as it arises as the limiting model for the mempool process (that tracks the unconfirmed transactions), which we rigorously show via a fluid limit and we extend this to the diffusion limit (an approximation of the Cramér-Lundberg model for fast computations in highly congested instances). We also propose methods (incorporating the real-time data) to estimate the model parameters, thereby combining model and data-driven approaches. The model-based approach is validated on real-world data and the resulting transaction fees outperform, in most instances, the data-driven ones.

math.PR↗

arXiv2022

Anomaly Detection for a Large Number of Streams: A Permutation-Based Higher Criticism Approach

Anomaly detection when observing a large number of data streams is essential in a variety of applications, ranging from epidemiological studies to monitoring of complex systems. High-dimensional scenarios are usually tackled with scan-statistics and related methods, requiring stringent modeling assumptions for proper calibration. In this work we take a non-parametric stance, and propose a permutation-based variant of the higher criticism statistic not requiring knowledge of the null distribution. This results in an exact test in finite samples which is asymptotically optimal in the wide class of exponential models. We demonstrate the power loss in finite samples is minimal with respect to the oracle test. Furthermore, since the proposed statistic does not rely on asymptotic approximations it typically performs better than popular variants of higher criticism that rely on such approximations. We include recommendations such that the test can be readily applied in practice, and demonstrate its applicability in monitoring the content uniformity of an active ingredient for a batch-produced drug product.

stat.ME↗