arXiv subjects

Jennifer Eckhoff

Publications and source records attributed to Jennifer Eckhoff.

3 recordsLinked to original sources

arXiv2025

Current validation practice undermines surgical AI development

Surgical data science (SDS) is rapidly advancing, yet clinical adoption of artificial intelligence (AI) in surgery remains limited, with inadequate validation as an important contributing factor. Existing validation practices often neglect the temporal and hierarchical structure of intraoperative videos, yielding misleading or clinically irrelevant results. We introduce a comprehensive catalogue of validation pitfalls in AI-based surgical video analysis, derived from a multi-stage Delphi process with 92 international experts. Pitfalls span three categories: (1) data, (2) metric selection/configuration, and (3) aggregation and reporting. A systematic review of surgical AI papers reveals that these pitfalls are widespread. Experiments on surgical video datasets show that ignoring temporal and hierarchical data structures can understate uncertainty, obscure critical failure modes, and alter algorithm rankings. To address these shortcomings, we provide consensus-based best practices compiled. Together, this work provides an evidence-based framework for rigorous validation of surgical video analysis algorithms, guiding benchmarking, reporting, regulatory review, and clinical translation.

q-bio.OT↗

arXiv2025

The SAGES Critical View of Safety Challenge: A Global Benchmark for AI-Assisted Surgical Quality Assessment

Advances in artificial intelligence (AI) for surgical quality assessment promise to democratize access to expertise, with applications in training, guidance, and accreditation. This study presents the SAGES Critical View of Safety (CVS) Challenge, the first AI competition organized by a surgical society, using the CVS in laparoscopic cholecystectomy, a universally recommended yet inconsistently performed safety step, as an exemplar of surgical quality assessment. A global collaboration across 54 institutions in 24 countries engaged hundreds of clinicians and engineers to curate 1,000 videos annotated by 20 surgical experts according to a consensus-validated protocol. The challenge addressed key barriers to real-world deployment in surgery, including achieving high performance, capturing uncertainty in subjective assessment, and ensuring robustness to clinical variability. To enable this scale of effort, we developed EndoGlacier, a framework for managing large, heterogeneous surgical video and multi-annotator workflows. Thirteen international teams participated, achieving up to a 17% relative gain in assessment performance, over 80% reduction in calibration error, and a 17% relative improvement in robustness over the state-of-the-art. Analysis of results highlighted methodological trends linked to model performance, providing guidance for future research toward robust, clinically deployable AI for surgical quality assessment.

cs.CV↗

arXiv2024

Hypergraph-Transformer (HGT) for Interactive Event Prediction in Laparoscopic and Robotic Surgery

Understanding and anticipating intraoperative events and actions is critical for intraoperative assistance and decision-making during minimally invasive surgery. Automated prediction of events, actions, and the following consequences is addressed through various computational approaches with the objective of augmenting surgeons' perception and decision-making capabilities. We propose a predictive neural network that is capable of understanding and predicting critical interactive aspects of surgical workflow from intra-abdominal video, while flexibly leveraging surgical knowledge graphs. The approach incorporates a hypergraph-transformer (HGT) structure that encodes expert knowledge into the network design and predicts the hidden embedding of the graph. We verify our approach on established surgical datasets and applications, including the detection and prediction of action triplets, and the achievement of the Critical View of Safety (CVS). Moreover, we address specific, safety-related tasks, such as predicting the clipping of cystic duct or artery without prior achievement of the CVS. Our results demonstrate the superiority of our approach compared to unstructured alternatives.

cs.CV↗