Searcharxiv⌕ Search
Home/Archive/arXiv

arXiv · 2610.18422

Augmenting Real-time Visualization in Minimally Invasive Surgery with Particle Filtering and Depth-Aware Occlusion Handling

Anuska Sarkar·Partha Acharya·Vishnu Nandury·Kaustav Basu·Subhamoy Mandal
View original source ↗Read source document ↗Export metadata ↓

Abstract

We present a real-time, hybrid framework for occlusion handling and depth-aware tracking in minimally invasive surgeries (MIS). Occlusions caused by surgical instruments often obscure the region of interest (ROI), affecting surgical precision. To address this, our approach integrates enhanced You Only Look Once (YOLO) based detection, Particle Filtering (PF), and monocular depth estimation using Multiple Depth Estimation Accuracy with Single Network (MiDaS). Occlusion is detected using a distance-based threshold between instruments and ROI. Experimental results demonstrate robust instrument detection with a confidence score of 0.85, effective occlusion event handling, and accurate depth prediction with an MAE of 0.34 cm and RMSE of 0.53 cm. Independent phantom-based validation against stereo depth measurements yielded an MAE of 0.65 cm and RMSE of 0.66 cm. The framework further achieved an occlusion detection accuracy of 81.95\% while maintaining real-time performance at 38.7 FPS. Preliminary surgeon feedback indicates that the proposed visualization framework is clinically relevant and holds promise for future intraoperative image-guided navigation and decision support in MIS.

Explore related subjects

eess.IVcs.RO

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Anuska Sarkar, Partha Acharya, Vishnu Nandury, Kaustav Basu, Subhamoy Mandal. 2026-10-13. Augmenting Real-time Visualization in Minimally Invasive Surgery with Particle Filtering and Depth-Aware Occlusion Handling. https://arxiv.org/abs/2610.18422

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

UEPS: Robust and Efficient MRI Reconstruction

Deep unrolled models (DUM) have become the state-of-the-art for accelerated MRI reconstruction, yet their robustness under domain shift remains a critical barrier to clinical adoption. In this work, we identify coil sensitivity map (CSM) estimation as the primary bottleneck limiting generalization. To address this, we propose UEPS, a novel DUM architecture featuring three key designs: (i) an Unrolled Expanded (UE) design that eliminates CSM dependency by reconstructing each coil independently; (ii) progressive resolution, which leverages k-space-to-image mapping for efficient coarse-to-fine refinement; and (iii) sparse attention tailored to MRI's 1D undersampling nature. These physics-grounded designs enable gains in robustness and computational efficiency. We construct a large-scale zero-shot transfer benchmark comprising 10 out-of-distribution (OOD) test sets spanning diverse clinical shifts, anatomy, view, contrast, vendor, field strength, and coil configurations. Extensive experiments demonstrate that UEPS consistently and substantially outperforms existing DUM, end-to-end, diffusion, and untrained methods across all OOD tests, achieving state-of-the-art robustness with low-latency inference suitable for real-time deployment.

eess.IV↗
arXiv2026

Forecasting Brain Tracer Distribution from Pre-Contrast MRI: A Comparison of Direct Regression and Flow Matching

Intrathecal contrast-enhanced MRI shows how a tracer injected into the cerebrospinal fluid spreads through the brain, but requires repeated scans over 24--48\,h. We therefore ask how much of the later tracer distribution can be predicted from the pre-contrast scan and elapsed time alone. Using 104 training patients (878 MRI volumes), we compared a direct image-to-image regression model with a more complex generative model based on conditional flow matching, and tested both on 23 held-out patients and on 51 patients from a separate cohort with idiopathic normal pressure hydrocephalus (INPH). The direct model reduced the error of simply copying the pre-contrast scan by about 60\% in held-out patients and 25\% in the INPH cohort, and its predicted change maps agreed with the observed tracer enhancement rather than only reproducing anatomy. The generative model was less accurate in both cohorts. A substantial part of the later tracer distribution is therefore predictable from the patient's brain anatomy as seen on pre-contrast MRI, and the added model complexity gave no benefit at this data scale. With prospective validation, such forecasts could be studied for patient stratification and treatment planning in intrathecal therapies.

eess.IV↗
arXiv2026

Background2Background: a self-supervised background removal framework for high dimensional microscopy

High-dimensional microscopy captures features across space, time, and spectrum, but complex backgrounds obscure signals. We present Background2Background (B2B), a self-supervised background removal framework leveraging the backgrounds larger correlation relative to the signal. Local pixel shuffling disrupts signal correlations while preserving background structure, transforming signals into effective noise for Noise2Noise-based background removal. Across fluorescence, stimulated Raman scattering and mid-infrared photothermal imaging, B2B achieves over 40-fold background suppression and reveals hidden signatures.

eess.IV↗