Tim Ramsay | Searcharxiv

arXiv2026

A Multi-Stage Drop-the-Loser Design with Superiority Boundaries

Multi-arm multi-stage (MAMS) trials have gained popularity, due to their improved efficiency in evaluating multiple treatments. A traditional MAMS trial often decreases the expected sample size of the trial compared to just running a multi-arm approach, but with the drawback of an increase in maximum sample size. For academic led trials this poses a particular challenge, as funding is typically based on the maximum required sample size. To address this, drop-the-loser designs were introduced, where a fixed number of treatments are dropped at each interim stage, thereby reducing the maximum sample size. In this work, we propose an enhanced multi-stage drop-the-loser design that also allows for early stopping of the entire trial for superiority. This approach aims to retain the benefits of a reduced maximum sample size while also lowering the expected sample size. The proposed design is motivated by a trial in atrial fibrillation. We derive analytical expressions for the type I error rate, power, and expected sample size, and compare the proposed design's performance to alternative methods. We outline the key requirements for implementing the proposed design and discuss the contexts in which it should be considered. For the motivating example the results show that the proposed design substantially reduces the expected sample size compared to a standard drop-the-loser design, while lowering the maximum sample size relative to running a traditional MAMS trial or multiple separate trials.

stat.ME↗

arXiv2021

Deep ROC Analysis and AUC as Balanced Average Accuracy to Improve Model Selection, Understanding and Interpretation

Optimal performance is critical for decision-making tasks from medicine to autonomous driving, however common performance measures may be too general or too specific. For binary classifiers, diagnostic tests or prognosis at a timepoint, measures such as the area under the receiver operating characteristic curve, or the area under the precision recall curve, are too general because they include unrealistic decision thresholds. On the other hand, measures such as accuracy, sensitivity or the F1 score are measures at a single threshold that reflect an individual single probability or predicted risk, rather than a range of individuals or risk. We propose a method in between, deep ROC analysis, that examines groups of probabilities or predicted risks for more insightful analysis. We translate esoteric measures into familiar terms: AUC and the normalized concordant partial AUC are balanced average accuracy (a new finding); the normalized partial AUC is average sensitivity; and the normalized horizontal partial AUC is average specificity. Along with post-test measures, we provide a method that can improve model selection in some cases and provide interpretation and assurance for patients in each risk group. We demonstrate deep ROC analysis in two case studies and provide a toolkit in Python.

stat.ME↗