arXiv subjects

Andre Kyme

Publications and source records attributed to Andre Kyme.

3 recordsLinked to original sources

arXiv2026

Patient4D: Temporally Consistent Patient Body Mesh Recovery from Monocular Operating Room Video

Recovering a dense 3D body mesh from monocular video remains challenging under occlusion from draping and continuously moving camera viewpoints. This configuration arises in surgical augmented reality (AR), where an anesthetized patient lies under surgical draping while a surgeon's head-mounted camera continuously changes viewpoint. Existing human mesh recovery (HMR) methods are typically trained on upright, moving subjects captured from relatively stable cameras, leading to performance degradation under such conditions. To address this, we present Patient4D, a stationarity-constrained reconstruction pipeline that explicitly exploits the stationarity prior. The pipeline combines image-level foundation models for perception with lightweight geometric mechanisms that enforce temporal consistency across frames. Two key components enable robust reconstruction: Pose Locking, which anchors pose parameters using stable keyframes, and Rigid Fallback, which recovers meshes under severe occlusion through silhouette-guided rigid alignment. Together, these mechanisms stabilize predictions while remaining compatible with off-the-shelf HMR models. We evaluate Patient4D on 4,680 synthetic surgical sequences and three public HMR video benchmarks. Under surgical drape occlusion, Patient4D achieves a 0.75 mean IoU, reducing failure frames from 30.5% to 1.3% compared to the best baseline. Our findings demonstrate that exploiting stationarity priors can substantially improve monocular reconstruction in clinical AR scenarios.

cs.CV↗

arXiv2025

Multi-modal 3D Pose and Shape Estimation with Computed Tomography

In perioperative care, precise in-bed 3D patient pose and shape estimation (PSE) can be vital in optimizing patient positioning in preoperative planning, enabling accurate overlay of medical images for augmented reality-based surgical navigation, and mitigating risks of prolonged immobility during recovery. Conventional PSE methods relying on modalities such as RGB-D, infrared, or pressure maps often struggle with occlusions caused by bedding and complex patient positioning, leading to inaccurate estimation that can affect clinical outcomes. To address these challenges, we present the first multi-modal in-bed patient 3D PSE network that fuses detailed geometric features extracted from routinely acquired computed tomography (CT) scans with depth maps (mPSE-CT). mPSE-CT incorporates a shape estimation module that utilizes probabilistic correspondence alignment, a pose estimation module with a refined neural network, and a final parameters mixing module. This multi-modal network robustly reconstructs occluded body regions and enhances the accuracy of the estimated 3D human mesh model. We validated mPSE-CT using proprietary whole-body rigid phantom and volunteer datasets in clinical scenarios. mPSE-CT outperformed the best-performing prior method by 23% and 49.16% in pose and shape estimation respectively, demonstrating its potential for improving clinical outcomes in challenging perioperative environments.

cs.CV↗

arXiv2024

Characterisation of the Intel RealSense D415 Stereo Depth Camera for Motion-Corrected CT Perfusion Imaging

Even for short protocols (<1 min), head movement can compromise accurate haemodynamic modelling of cerebral CT perfusion (CTP) imaging in acute stroke. Frame-to-frame registration is the most common form of retrospective correction but neglects the fact that motion is continuous, not discrete. By contrast, external tracking devices provide continuous motion monitoring and thereby the opportunity to fully correct the acquired data for motion. The aim of this study was to characterise the Intel D415 stereo depth camera, a compact low-cost markerless tracking device, in terms of its suitability for retrospective CTP motion correction. The results showed that jitter was stable, and thermally-induced pose drift was {\le} 1.5 mm and {\le} 0.5{\deg} during the first 10-20 min, after which it also became stable. For static poses, the mean difference between the Intel D415 motion estimates and ground-truth poses for a head phantom was {\le} 1.24 \pm 0.01 mm and {\le} 0.68 \pm 0.01{\deg} for position and orientation, respectively. For dynamic poses measured while a head phantom travelled smooth continuous trajectories with median speed 0.031 ms^(-1) (speed range 0-0.500 ms^(-1)), the root-mean-square-error (RMSE) was {\le} 1.40 \pm 0.12 mm and {\le} 0.24 \pm 0.02{\deg}. When tracking a simulated patient head trajectory derived from a clinical CTP scan, the average RMSE was {\le} 0.86 \pm 0.03 mm and {\le} 0.16 \pm 0.03{\deg}. Tracking the head motion of a human volunteer inside a clinical CT scanner, the average RMSE was {\le} 2.72 \pm 0.24 mm and {\le} 0.55 \pm 0.07{\deg}. Overall, our results suggest that a single D415 tracking system can achieve promising pose estimation accuracy, though still worse than typical brain CT resolution, including CTP. The error is likely to be reduced to a practical level by combining multiple devices and this should be investigated in future work.

physics.med-ph↗