A Prospective Randomized Clinical Trial for Measuring Radiology Study Reporting Time on Artificial Intelligence-Based Detection of Intracranial Hemorrhage in Emergent Care Head CT

Axel Wismüller; Larry Stockmaster

Home/Archive/arXiv

arXiv · 2002.12515

A Prospective Randomized Clinical Trial for Measuring Radiology Study Reporting Time on Artificial Intelligence-Based Detection of Intracranial Hemorrhage in Emergent Care Head CT

Axel Wismüller·Larry Stockmaster

View original source ↗Read source document ↗Export metadata ↓

Abstract

We propose Artificial Intelligence Prospective Randomized Observer Blinding Evaluation (AI-PROBE) for quantitative clinical performance evaluation of radiology AI systems within prospective randomized clinical trials. AI-PROBE encompasses a study design and a matching radiology IT infrastructure that randomly blinds radiologists for results provided by AI-based image analysis. To demonstrate the applicability of our evaluation framework, we present a first prospective randomized clinical trial on the effect of Intra-Cranial Hemorrhage (ICH) detection in emergent care head CT on radiology study Turn-Around Time (TAT). Here, we acquired 620 non-contrast head CT scans from inpatient and emergency room patients at a large academic hospital. Following acquisition, scans were automatically analyzed for the presence of ICH using commercially available software (Aidoc, Tel Aviv, Israel). Cases identified positive for ICH by AI (ICH-AI+) were flagged in radiologists' reading worklists, where flagging was randomly switched off with probability 50%. TAT was measured as time difference between study completion and first clinically communicated reporting, with time stamps automatically retrieved from various IT systems. TATs for flagged cases (73+/-143 min) were significantly lower than TATs for non-flagged (132+/-193 min) cases (p<0.05, one-sided t-test), where 105 of 122 ICH-AI+ cases were true positive. Total sensitivity, specificity, and accuracy over all analyzed cases were 95.0%, 96.7%, and 96.4%, respectively. We conclude that automatic identification of ICH reduces TAT for ICH in emergent care head CT, which carries the potential for improving timely clinical management of ICH. Our results suggest that AI-PROBE can contribute to systematic quantitative evaluation of AI systems in clinical practice using clinically meaningful quantities, such as TAT or diagnostic accuracy.

Explore related subjects

physics.med-ph eess.IV

Keep this discovery

BibTeX RIS

Axel Wismüller, Larry Stockmaster. 2020-02-28. A Prospective Randomized Clinical Trial for Measuring Radiology Study Reporting Time on Artificial Intelligence-Based Detection of Intracranial Hemorrhage in Emergent Care Head CT. https://arxiv.org/abs/2002.12515

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Phase-contrast micro-CT for intra-operative breast tumour margin assessment using a microfocus x-ray source and photon-counting detector

Objective: Intra-operative tumour margin assessment during breast-conserving surgery requires rapid, high-resolution imaging of excised tissue, allowing the surgical team to take appropriate action within a single operation. This study evaluates a custom propagation-based phase-contrast micro-computed tomography (micro-CT) system designed to meet these clinical constraints without specialised optical elements. Methods: The experimental setup pairs a microfocus x-ray source with a photon-counting detector in a cone-beam geometry. We explore how the spatial coherence of the source can provide propagation-based phase contrast -- with no additional specialised optical elements -- and balance this against maximising the x-ray flux of the cone-beam geometry. System performance was evaluated across two anode target materials and filtration configurations at various tube power settings. Imaging capabilities were validated using anthropomorphic breast tissue phantoms and a formalin-fixed paraffin-embedded (FFPE) breast tissue specimen, with reconstructions compared against gold-standard histology. Results: An unfiltered tungsten target operated at 40 kVp yielded optimal image quality. The optimised system achieved high-resolution CT reconstructions of a 5 cm diameter sample with an isotropic voxel size of 40.7 $\upmu\text{m}$ in a scan time of 12 minutes. Reconstructed volumes demonstrated strong visual correlation with corresponding histology slides. Conclusion: Combining a microfocus source with a photon-counting detector enables high-resolution, phase-contrast micro-CT within a clinically viable timeframe, demonstrating strong potential for intra-operative margin assessment.

physics.med-ph↗

arXiv2026

Understanding Search and Decision Errors in Liver Metastasis Detection and the Effects of Lower Radiation Dose

The detection performance of liver metastases decreases with the reduction of radiation dose, but misses are heterogeneous. Previous eye tracking work has characterized missed metastases into two categories: search errors i.e., the eyes never land on the lesion, and decision errors i.e., the lesion is seen but not recognized as malignant. We integrated three prior reader studies to answer this question. In all studies, radiologists interpreted the same set of 40 contrast enhanced abdominal CT exams containing 91 liver metastases whose locations had been previously marked. In two studies, the workstation recorded their gaze and eye movements. Using eye dwell times, metastases were classified as search-error-dominant (majority of misses had <2 sec gaze time) or decision-error-dominant (>2 sec gaze time). In the third study, exams were interpreted both at 120 and 200 quality reference mAs (QRM) by ten radiologists. The third study did not include eye tracking. Out of 91 liver metastases, we excluded 16 that were never missed in the eye tracking studies and used 75 liver metastases for the present study.

physics.med-ph↗

arXiv2026

Develop and Optimize 5DCT Imaging Simulation and Reconstruction Methods

Purpose: To develop and optimize a 5DCT (3D + cardiac phase + respiratory phase) imaging simulation and reconstruction pipeline, and to compare two sinogram-space interpolation methods for reconstructing images at arbitrary combinations of cardiac and respiratory phase. Methods: Helical CT projections were simulated from the 4D XCAT phantom across a range of cardiac and respiratory motion states, with Poisson and electronic noise added. Ground-truth-matched volumes were generated at 5 cardiac phases and 10 respiratory amplitudes (50 total phase combinations). Because acquired projections are sparsely and unevenly distributed across this joint phase space, each target slice was reconstructed by interpolating rebinned sinogram rows to the target cardiac phase and respiratory amplitude, using either 2D scattered barycentric interpolation or 2D scattered local linear interpolation with a circular kernel for cardiac phase. Reconstructed volumes were compared to phantom ground truth using mean absolute error (MAE), and to conventional respiratory-gated 4DCT (r4DCT) reconstructed from the same simulated data. Results: Both interpolation methods eliminated the severe axial misalignment artifacts present when helical projections were reconstructed without phase-space interpolation. Local linear interpolation achieved lower MAE than barycentric interpolation across most tested conditions, with the largest improvement at low pitch. The 5DCT pipeline also produced respiratory-only volumes with fewer residual cardiac-motion artifacts than conventional r4DCT reconstructed from the same projection data, including at standard clinical pitch (0.1). Conclusions: 5DCT reconstruction using sinogram-space interpolation is feasible and can jointly resolve cardiac and respiratory motion with better accuracy than conventional 4DCT reconstruction.

physics.med-ph↗