Towards Automated Initial Probe Placement in Transthoracic Teleultrasound Using Human Mesh and Skeleton Recovery
Cardiac and lung ultrasound are technically demanding because operators must identify patient-specific intercostal acoustic windows and then navigate between standard views by adjusting probe position, rotation, and force across different imaging planes. These challenges are amplified in teleultrasound, where the examination proceeds without in-person expert assistance: once the probe is approximately positioned, the expert can navigate in ultrasound image space, but guiding the initial placement remotely remains difficult given the limited 3D perception of the patient. We present a framework for automating patient registration and anatomy-informed initial probe placement guidance (PIPG) using RGB images obtained from a calibrated camera and a point cloud accumulated from depth images. The novice first captures the patient using the camera on a mixed reality (MR) head-mounted display (HMD), and an edge server then infers a patient-specific body-surface and skeleton model. By leveraging the patient's spatial and temporal consistency across multiview and point cloud data, we achieved robust, training-free human registration, verified in a healthcare setting. Using bony landmarks from the predicted skeleton, we estimate the intercostal region and project the guidance back onto the reconstructed body surface. To validate the framework, we rendered the reconstructed body mesh and the virtual probe pose guidance in the MR headset across multiple transthoracic echocardiography scan planes in situ and measured the quantitative placement error. Pilot experiments with five healthy volunteers suggest that the proposed probe placement prediction and MR guidance yield consistent initial placement, with a mean surface error of 15 mm, positional error to palpated anatomical landmarks of 41.0 mm, and torso orientation errors within 9 deg, acceptable for the teleultrasound setup.