Bernhard Haller | Searcharxiv

arXiv2026

Investigations of Heterogeneity in Diagnostic Test Accuracy Meta-Analysis: A Methodological Review

Background: Subgroup analyses and meta-regression are commonly used to investigate heterogeneity in diagnostic test accuracy (DTA) meta-analyses (MA), but adherence to methodological guidance is unclear. This methodological review summarizes investigations of heterogeneity (IoH) in DTA-MAs, examining their frequency, characteristics, and alignment with recommendations. Methods: We included DTA-MAs published in 2024 reporting at least one pair of summary sensitivity and specificity. Non-DTA reviews, narrative syntheses, studies reporting only alternative measures, and overviews of systematic reviews were excluded. MEDLINE (via Ovid) was searched for English-language publications, with the final search in January 2025. Results: From 403 records, the most recent 100 DTA-MAs were included, each contributing one index test. IoH were reported in 61 analyses. The number of primary studies was positively associated with conducting an investigation (OR 1.66; p = 0.008). Subgroup analyses were used in 35/61 (57%), while 26/61 (43%) applied meta-regression alone or with subgroup analyses. Subgroup analyses examined fewer variables than meta-regression (p < 0.001). Among 44/61 (72%) analyses with sufficient detail to identify a statistical model, the bivariate model was used in 28/44 (64%), univariate random-effects models in 14/44 (32%), and the HSROC model in 5/44 (11%). Formal tests for subgroup differences were reported in 37/61 (61%). Protocols were available for 43/61 (70%) analyses, of which 19/43 (44%) fully prespecified IoH. Discussion: IoH were common and more likely when more primary studies were available, although individual subgroups were often supported by limited data. Reporting of statistical models and model choice was frequently unclear. Greater prespecification of IoH in protocols may reduce spurious findings and improve transparency in diagnostic research.

stat.ME↗

arXiv2022

Sequential Permutation Testing of Random Forest Variable Importance Measures

Hypothesis testing of random forest (RF) variable importance measures (VIMP) remains the subject of ongoing research. Among recent developments, heuristic approaches to parametric testing have been proposed whose distributional assumptions are based on empirical evidence. Other formal tests under regularity conditions were derived analytically. However, these approaches can be computationally expensive or even practically infeasible. This problem also occurs with non-parametric permutation tests, which are, however, distribution-free and can generically be applied to any type of RF and VIMP. Embracing this advantage, it is proposed here to use sequential permutation tests and sequential p-value estimation to reduce the high computational costs associated with conventional permutation tests. The popular and widely used permutation VIMP serves as a practical and relevant application example. The results of simulation studies confirm that the theoretical properties of the sequential tests apply, that is, the type-I error probability is controlled at a nominal level and a high power is maintained with considerably fewer permutations needed in comparison to conventional permutation testing. The numerical stability of the methods is investigated in two additional application studies. In summary, theoretically sound sequential permutation testing of VIMP is possible at greatly reduced computational costs. Recommendations for application are given. A respective implementation is provided through the accompanying R package $rfvimptest$. The approach can also be easily applied to any kind of prediction model.

stat.ME↗