arXiv subjects

Alexander Hapfelmeier

Publications and source records attributed to Alexander Hapfelmeier.

7 recordsLinked to original sources

arXiv2026

Tree models for covariate-dependent method agreement with repeated measurements in clinical research

Background: In clinical research, the Bland-Altman analysis is commonly used to assess agreement of metric measurements made by two or more techniques, devices or methods. The approach can also deal with repeated measurements per subject or observational unit. However, a strong and implicit assumption is that agreement of methods is homogeneous across subjects. Objective: To extend the previously introduced multivariable modeling of conditional method agreement with single measurements per subject to the frequent case of repeated measurements. Methods: Appropriate regression trees, called conditional method agreement trees (COAT), are generalized to capture the dependence of the parameters of the Bland-Altman analysis on covariates. These parameters, the expectation and variance of the differences between the methods, are decomposed into subject-specific components to account for repeated measurements. Whilst the theoretical, asymptotic properties of tree models are known, a simulation study was carried out to assess the performance of COAT in finite samples. A comparison of devices measuring cardiac output serves as an application example. Results: COAT is applicable to the two relevant cases of paired and unpaired repeated measurements. In the simulation study, it controlled the type-I error at the nominal level and could detect covariate-dependent method agreement with increasing sample size. The Adjusted Rand Index, a measure of concordance between the estimated and true subgroups, reached very high values close to the maximum of 1. The analysis of cardiac output showed that patients' characteristics may influence the agreement between measuring devices, with implications for use in patient care. Conclusion: COAT can explicitly define subgroups of heterogeneous method agreement in dependence of covariates with appropriate statistical testing in case of repeated measurements.

stat.ME↗

arXiv2026

Investigations of Heterogeneity in Diagnostic Test Accuracy Meta-Analysis: A Methodological Review

Background: Subgroup analyses and meta-regression are commonly used to investigate heterogeneity in diagnostic test accuracy (DTA) meta-analyses (MA), but adherence to methodological guidance is unclear. This methodological review summarizes investigations of heterogeneity (IoH) in DTA-MAs, examining their frequency, characteristics, and alignment with recommendations. Methods: We included DTA-MAs published in 2024 reporting at least one pair of summary sensitivity and specificity. Non-DTA reviews, narrative syntheses, studies reporting only alternative measures, and overviews of systematic reviews were excluded. MEDLINE (via Ovid) was searched for English-language publications, with the final search in January 2025. Results: From 403 records, the most recent 100 DTA-MAs were included, each contributing one index test. IoH were reported in 61 analyses. The number of primary studies was positively associated with conducting an investigation (OR 1.66; p = 0.008). Subgroup analyses were used in 35/61 (57%), while 26/61 (43%) applied meta-regression alone or with subgroup analyses. Subgroup analyses examined fewer variables than meta-regression (p < 0.001). Among 44/61 (72%) analyses with sufficient detail to identify a statistical model, the bivariate model was used in 28/44 (64%), univariate random-effects models in 14/44 (32%), and the HSROC model in 5/44 (11%). Formal tests for subgroup differences were reported in 37/61 (61%). Protocols were available for 43/61 (70%) analyses, of which 19/43 (44%) fully prespecified IoH. Discussion: IoH were common and more likely when more primary studies were available, although individual subgroups were often supported by limited data. Reporting of statistical models and model choice was frequently unclear. Greater prespecification of IoH in protocols may reduce spurious findings and improve transparency in diagnostic research.

stat.ME↗

arXiv2026

Unity Forests: Improving Interaction Modelling and Interpretability in Random Forests

Random forests (RFs) are widely used for prediction and variable importance analysis and are often believed to capture any types of interactions via recursive splitting. However, since the splits are chosen locally, interactions are only reliably captured when at least one involved covariate has a marginal effect. We introduce unity forests (UFOs), an RF variant designed to better exploit interactions involving covariates without marginal effects. In UFOs, the first few splits of each tree are optimized jointly across a random covariate subset to form a "tree root" capturing such interactions; the remainder is grown conventionally. We further propose the unity variable importance measure (VIM), which is based on out-of-bag split criterion values from the tree roots. Here, only a small fraction of tree root splits with the highest in-bag criterion values are considered per covariate, reflecting that covariates with purely interaction-based effects are discriminative only if a split in an interacting covariate occurred earlier in the tree. Finally, we introduce covariate-representative tree roots (CRTRs), which select representative tree roots per covariate and provide interpretable insight into the conditions - marginal or interactive - under which each covariate has its strongest effects. In a simulation study, the unity VIM reliably identified interacting covariates without marginal effects, unlike conventional RF-based VIMs. In a large-scale real-data comparison, UFOs achieved higher discrimination and predictive accuracy than standard RFs, with comparable calibration. The CRTRs reproduced the covariates' true effect types reliably in simulated data and provided interesting insights in a real data analysis.

stat.ME↗

arXiv2024

Multi forests: Variable importance for multi-class outcomes

In prediction tasks with multi-class outcomes, identifying covariates specifically associated with one or more outcome classes can be important. Conventional variable importance measures (VIMs) from random forests (RFs), like permutation and Gini importance, focus on overall predictive performance or node purity, without differentiating between the classes. Therefore, they can be expected to fail to distinguish class-associated covariates from covariates that only distinguish between groups of classes. We introduce a VIM called multi-class VIM, tailored for identifying exclusively class-associated covariates, via a novel RF variant called multi forests (MuFs). The trees in MuFs use both multi-way and binary splitting. The multi-way splits generate child nodes for each class, using a split criterion that evaluates how well these nodes represent their respective classes. This setup forms the basis of the multi-class VIM, which measures the discriminatory ability of the splits performed in the respective covariates with regard to this split criterion. Alongside the multi-class VIM, we introduce a second VIM, the discriminatory VIM. This measure, based on the binary splits, assesses the strength of the general influence of the covariates, irrespective of their class-associatedness. Simulation studies demonstrate that the multi-class VIM specifically ranks class-associated covariates highly, unlike conventional VIMs which also rank other types of covariates highly. Analyses of 121 datasets reveal that MuFs often have slightly lower predictive performance compared to conventional RFs. This is, however, not a limiting factor given the algorithm's primary purpose of calculating the multi-class VIM.

stat.ML↗

arXiv2023

Tree models for assessing covariate-dependent method agreement

Method comparison studies explore the agreement of measurements made by two or more methods. Commonly, agreement is evaluated by the well-established Bland-Altman analysis. However, the underlying assumption is that differences between measurements are identically distributed for all observational units and in all application settings. We introduce the concept of conditional method agreement and propose a respective modeling approach to alleviate this constraint. Therefore, the Bland-Altman analysis is embedded in the framework of recursive partitioning to explicitly define subgroups with heterogeneous agreement in dependence of covariates in an exploratory analysis. Three different modeling approaches, conditional inference trees with an appropriate transformation of the modeled differences (CTreeTrafo), distributional regression trees (DistTree), and model-based trees (MOB) are considered. The performance of these models is evaluated in terms of type-I error probability and power in several simulation studies. Further, the adjusted rand index (ARI) is used to quantify the models' ability to uncover given subgroups. An application example to real data of accelerometer device measurements is used to demonstrate the applicability. Additionally, a two-sample Bland-Altman test is proposed for exploratory or confirmatory hypothesis testing of differences in agreement between subgroups. Results indicate that all models were able to detect given subgroups with high accuracy as the sample size increased. Relevant covariates that may affect agreement could be detected in the application to accelerometer data. We conclude that conditional method agreement trees (COAT) enable the exploratory analysis of method agreement in dependence of covariates and the respective exploratory or confirmatory hypothesis testing of group differences. It is made publicly available through the R package coat.

stat.ME↗

arXiv2022

Sequential Permutation Testing of Random Forest Variable Importance Measures

Hypothesis testing of random forest (RF) variable importance measures (VIMP) remains the subject of ongoing research. Among recent developments, heuristic approaches to parametric testing have been proposed whose distributional assumptions are based on empirical evidence. Other formal tests under regularity conditions were derived analytically. However, these approaches can be computationally expensive or even practically infeasible. This problem also occurs with non-parametric permutation tests, which are, however, distribution-free and can generically be applied to any type of RF and VIMP. Embracing this advantage, it is proposed here to use sequential permutation tests and sequential p-value estimation to reduce the high computational costs associated with conventional permutation tests. The popular and widely used permutation VIMP serves as a practical and relevant application example. The results of simulation studies confirm that the theoretical properties of the sequential tests apply, that is, the type-I error probability is controlled at a nominal level and a high power is maintained with considerably fewer permutations needed in comparison to conventional permutation testing. The numerical stability of the methods is investigated in two additional application studies. In summary, theoretically sound sequential permutation testing of VIMP is possible at greatly reduced computational costs. Recommendations for application are given. A respective implementation is provided through the accompanying R package $rfvimptest$. The approach can also be easily applied to any kind of prediction model.

stat.ME↗

arXiv2019

Essential guidelines for computational method benchmarking

In computational biology and other sciences, researchers are frequently faced with a choice between several computational methods for performing data analyses. Benchmarking studies aim to rigorously compare the performance of different methods using well-characterized benchmark datasets, to determine the strengths of each method or to provide recommendations regarding suitable choices of methods for an analysis. However, benchmarking studies must be carefully designed and implemented to provide accurate, unbiased, and informative results. Here, we summarize key practical guidelines and recommendations for performing high-quality benchmarking analyses, based on our experiences in computational biology.

q-bio.QM↗