arXiv subjects

Mingxu Liu

Publications and source records attributed to Mingxu Liu.

8 recordsLinked to original sources

arXiv2026

Visual Cue Interactions in AR-Guided Needle Insertion: A Prostate Biopsy-Inspired Phantom Study

Despite the apparent simplicity of the motor action involved during percutaneous needle procedures, manipulating the tool's direction becomes challenging when clinicians cannot directly visualize internal anatomy and must rely on ultrasound images, which increase cognitive demand. Augmented reality (AR) offers the promise to assist with these tasks by providing pertinent visual information in the clinician's field of view. However, simply providing visual information that ignores meaningful visual cues can complicate depth perception and spatial understanding. In this work, we introduce and evaluate three visualization techniques for needle alignment developed during design sessions with medical experts: a localized focus-and-context window, a color-based proximity encoding, and an explicit trajectory overlay. These techniques were evaluated in a user study (n=26) including clinical experts (n=7) using a prostate-biopsy-inspired phantom. Results from this study suggest that cue effects depended on the surrounding cue configuration and user expertise. For novices, explicit trajectory overlay improved targeting accuracy and reduced retreat behavior, but its effect on completion time varied across cue configurations, with slower performance when the overlay was presented alone. For experts, the focus-and-context window reduced completion time and retreat events, while color-based proximity overlay improved completion time. Subjectively, color cues were often perceived as helpful even when their effects on accuracy were not consistent. These results suggest that AR guidance strategies for percutaneous interventions should consider user expertise and visual context.

cs.GR↗

arXiv2026

The Role of Mixed and Augmented Reality in Medical Visualization: Literature Review and A Context-Aware Taxonomy

The discovery and evolution of medical imaging technologies have enabled non-invasive visualization of internal anatomy that has be- come essential for supporting diagnosis, monitoring, and treatment. However, because medical imaging relies on complex physical processes and contrast mechanisms for image formation, imaging alone is not sufficient to enable humans to leverage the resulting in- formation fully. In addition, traditional methods to visualize the resulting information use two-dimensional displays to present three-dimensional anatomical structures. The introduction of Augmented and Mixed Reality (AR/MR) technologies offers an opportunity to provide valuable paradigms for medical imaging visualization, allowing users to observe, explore, and interact with anatomical infor- mation in more spatially intuitive ways. However, naive implementation without careful design considerations can lead to perceptual inconsistencies, potentially compromising utility and effectiveness. In this paper, we present a structured taxonomy of medical AR/MR visualization strategies aimed at providing clearer insight into how visualization design varies across clinical use cases. The taxonomy organizes techniques based on four core design components: image modality, data dimensionality, display technology, and clinical application. In addition, we introduce two critical dimensions that are often overlooked in the literature: visualization anchoring (the spatial relationship between virtual content and the physical world), and perceptual awareness (the use of visual cues to support spatial interpretation). Together, these components form a comprehensive taxonomy, offering a detailed framework for selecting appropriate visualization techniques in medical applications.

cs.GR↗

arXiv2026

X-GuideAR: An Augmented Reality Framework to Mitigate Radiation Exposure during Fluoroscopic Guidance

Achieving optimal screw placement for orthopedic surgeries requires frequent alignment checks and multiple anatomical views under X-ray -- a process known as "fluoro-hunting" that increases radiation exposure to patients and surgical teams. This work introduces X-GuideAR, an augmented reality (AR) framework for identifying optimal X-ray views, aimed at reducing radiation exposure while ensuring accurate screw placement. To exemplify the benefits of X-GuideAR, we focus on S2 alar-iliac (S2AI) screw placement. Our system provides radiation-free guidance for view acquisition and drilling by generating synthetic X-ray previews that accelerate fluoro-hunting. Once the required anatomical views are identified using these previews, a real X-ray is acquired, and the preview of the drilling trajectory is augmented onto it, facilitating precise screw placement with minimal additional radiation. A preliminary study involving eight S2AI trajectories performed by an expert spine surgeon demonstrated a 62.3% reduction in the number of X-rays. Post-procedure evaluations showed that trajectories done with X-GuideAR supported an average safe screw diameter of 12.95 mm compared to 5.9 mm under the conventional workflow, suggesting improved bony containment and potential biomechanical benefit. X-GuideAR shows great potential to reduce radiation exposure and streamline S2AI screw placement, offering a promising direction toward safer and more efficient surgeries.

cs.CV↗

arXiv2026

Extend Your Horizon: A Device-Agnostic Surgical Tool Tracking Framework with Multi-View Optimization for Augmented Reality

Surgical navigation provides real-time guidance by estimating the pose of patient anatomy and surgical instruments to visualize relevant intraoperative information. In conventional systems, instruments are typically tracked using fiducial markers and stationary optical tracking systems (OTS). Augmented reality (AR) has further enabled intuitive visualization and motivated tracking using sensors embedded in head-mounted displays (HMDs). However, most existing approaches rely on a clear line of sight, which is difficult to maintain in dynamic operating room environments due to frequent occlusions caused by equipment, surgical tools, and personnel. This work introduces a framework for tracking surgical instruments under occlusion by fusing multiple sensing modalities within a dynamic scene graph representation. The proposed approach integrates tracking systems with different accuracy levels and motion characteristics while estimating tracking reliability in real time. Experimental results demonstrate improved robustness and enhanced consistency of AR visualization in the presence of occlusions.

cs.HC↗

arXiv2025

Benchmark of Segmentation Techniques for Pelvic Fracture in CT and X-ray: Summary of the PENGWIN 2024 Challenge

The segmentation of pelvic fracture fragments in CT and X-ray images is crucial for trauma diagnosis, surgical planning, and intraoperative guidance. However, accurately and efficiently delineating the bone fragments remains a significant challenge due to complex anatomy and imaging limitations. The PENGWIN challenge, organized as a MICCAI 2024 satellite event, aimed to advance automated fracture segmentation by benchmarking state-of-the-art algorithms on these complex tasks. A diverse dataset of 150 CT scans was collected from multiple clinical centers, and a large set of simulated X-ray images was generated using the DeepDRR method. Final submissions from 16 teams worldwide were evaluated under a rigorous multi-metric testing scheme. The top-performing CT algorithm achieved an average fragment-wise intersection over union (IoU) of 0.930, demonstrating satisfactory accuracy. However, in the X-ray task, the best algorithm achieved an IoU of 0.774, which is promising but not yet sufficient for intra-operative decision-making, reflecting the inherent challenges of fragment overlap in projection imaging. Beyond the quantitative evaluation, the challenge revealed methodological diversity in algorithm design. Variations in instance representation, such as primary-secondary classification versus boundary-core separation, led to differing segmentation strategies. Despite promising results, the challenge also exposed inherent uncertainties in fragment definition, particularly in cases of incomplete fractures. These findings suggest that interactive segmentation approaches, integrating human decision-making with task-relevant information, may be essential for improving model reliability and clinical applicability.

eess.IV↗

arXiv2025

DualVision ArthroNav: Investigating Opportunities to Enhance Localization and Reconstruction in Image-based Arthroscopy Navigation via External Cameras

Arthroscopic procedures can greatly benefit from navigation systems that enhance spatial awareness, depth perception, and field of view. However, existing optical tracking solutions impose strict workspace constraints and disrupt surgical workflow. Vision-based alternatives, though less invasive, often rely solely on the monocular arthroscope camera, making them prone to drift, scale ambiguity, and sensitivity to rapid motion or occlusion. We propose DualVision ArthroNav, a multi-camera arthroscopy navigation system that integrates an external camera rigidly mounted on the arthroscope. The external camera provides stable visual odometry and absolute localization, while the monocular arthroscope video enables dense scene reconstruction. By combining these complementary views, our system resolves the scale ambiguity and long-term drift inherent in monocular SLAM and ensures robust relocalization. Experiments demonstrate that our system effectively compensates for calibration errors, achieving an average absolute trajectory error of 1.09 mm. The reconstructed scenes reach an average target registration error of 2.16 mm, with high visual fidelity (SSIM = 0.69, PSNR = 22.19). These results indicate that our system provides a practical and cost-efficient solution for arthroscopic navigation, bridging the gap between optical tracking and purely vision-based systems, and paving the way toward clinically deployable, fully vision-based arthroscopic guidance.

eess.IV↗

arXiv2024

StraightTrack: Towards Mixed Reality Navigation System for Percutaneous K-wire Insertion

In percutaneous pelvic trauma surgery, accurate placement of Kirschner wires (K-wires) is crucial to ensure effective fracture fixation and avoid complications due to breaching the cortical bone along an unsuitable trajectory. Surgical navigation via mixed reality (MR) can help achieve precise wire placement in a low-profile form factor. Current approaches in this domain are as yet unsuitable for real-world deployment because they fall short of guaranteeing accurate visual feedback due to uncontrolled bending of the wire. To ensure accurate feedback, we introduce StraightTrack, an MR navigation system designed for percutaneous wire placement in complex anatomy. StraightTrack features a marker body equipped with a rigid access cannula that mitigates wire bending due to interactions with soft tissue and a covered bony surface. Integrated with an Optical See-Through Head-Mounted Display (OST HMD) capable of tracking the cannula body, StraightTrack offers real-time 3D visualization and guidance without external trackers, which are prone to losing line-of-sight. In phantom experiments with two experienced orthopedic surgeons, StraightTrack improves wire placement accuracy, achieving the ideal trajectory within $5.26 \pm 2.29$ mm and $2.88 \pm 1.49$ degree, compared to over 12.08 mm and 4.07 degree for comparable methods. As MR navigation systems continue to mature, StraightTrack realizes their potential for internal fracture fixation and other percutaneous orthopedic procedures.

cs.RO↗

arXiv2024

Seamless Augmented Reality Integration in Arthroscopy: A Pipeline for Articular Reconstruction and Guidance

Arthroscopy is a minimally invasive surgical procedure used to diagnose and treat joint problems. The clinical workflow of arthroscopy typically involves inserting an arthroscope into the joint through a small incision, during which surgeons navigate and operate largely by relying on their visual assessment through the arthroscope. However, the arthroscope's restricted field of view and lack of depth perception pose challenges in navigating complex articular structures and achieving surgical precision during procedures. Aiming at enhancing intraoperative awareness, we present a robust pipeline that incorporates simultaneous localization and mapping, depth estimation, and 3D Gaussian splatting to realistically reconstruct intra-articular structures solely based on monocular arthroscope video. Extending 3D reconstruction to Augmented Reality (AR) applications, our solution offers AR assistance for articular notch measurement and annotation anchoring in a human-in-the-loop manner. Compared to traditional Structure-from-Motion and Neural Radiance Field-based methods, our pipeline achieves dense 3D reconstruction and competitive rendering fidelity with explicit 3D representation in 7 minutes on average. When evaluated on four phantom datasets, our method achieves RMSE = 2.21mm reconstruction error, PSNR = 32.86 and SSIM = 0.89 on average. Because our pipeline enables AR reconstruction and guidance directly from monocular arthroscopy without any additional data and/or hardware, our solution may hold the potential for enhancing intraoperative awareness and facilitating surgical precision in arthroscopy. Our AR measurement tool achieves accuracy within 1.59 +/- 1.81mm and the AR annotation tool achieves a mIoU of 0.721.

cs.CV↗