Ensemble YOLO Framework for Multi-Domain Mitotic Figure Detection in Histopathology Images

Abstract

The reliable identification of mitotic figures in whole-slide histopathological images remains difficult, owing to their low prevalence, substantial morphological heterogeneity, and the inconsistencies introduced by tissue processing and staining procedures. The MIDOG competition series provides standardized benchmarks for evaluating detection approaches across diverse domains, thus motivating the development of generalizable deep learning models. In this work, we investigate the performance of two modern one-stage detectors, YOLOv5 and YOLOv8, trained on MIDOG++, CMC, and CCMCT datasets. To enhance robustness, training incorporated stain-invariant color perturbations and texture-preserving augmentations. Ininternal validation, YOLOv5 achieved higher precision (84.3%), while YOLOv8 offered improved recall (82.6%), reflecting architectural trade-offs between anchor-based and anchor-free detections. To capitalize on their complementary strengths, weemployed an ensemble of the two models, which improved sensitivity (85.3%) while maintaining competitive precision, yielding the best F1 score of 83.1%. On the preliminary MIDOG 2025 test leaderboard, our ensemble ranked 5th with an F1 score of 79.2%, precision of 73.6%, and recall of 85.8%, confirming that the proposed strategy generalizes effectively across unseen test data. These findings highlight the effectiveness of combining anchor-based and anchor-free object detectors to advance automated mitosis detection in digital pathology.