Searcharxiv⌕ Search
Home/Archive/arXiv

arXiv · 2610.32525

From Freehand Demonstrations to Autonomous Robotic Ultrasound: An Anatomy-Anchored Learning Framework

Yuan Bi·Fangxu Han·Ye Su·Yiping Zhou·Zhongliang Jiang·Nassir Navab
View original source ↗Read source document ↗Export metadata ↓

Abstract

Ultrasound imaging is highly operator-dependent, motivating robotic systems that reproduce expert scanning skills consistently. These skills involve coordinated interpretation of anatomical landmarks, ultrasound images, and probe-tissue interactions that are difficult to encode in hand-designed control strategies. Imitation learning offers an alternative, but its observation and action representations should reflect the anatomically structured nature of scanning. We propose an anatomy-anchored imitation-learning framework that combines anatomy-guided probe placement with learned ultrasound navigation. The placement module identifies a task-specific anatomical starting point from external visual observations and establishes the initial probe pose as the reference frame for scanning. A flow-based policy then generates scanning motions from anchor-relative probe poses, ultrasound images, probe-mounted camera views, and contact forces, while a termination head predicts task completion. This avoids continuous reliance on third-person vision and reduces dependence on the robot-patient configuration. A handheld acquisition platform enables multimodal demonstrations to be collected during natural freehand scanning without a robot. On 16 unseen participants, task success reaches 96%, 94%, 88%, and 94% for kidney, gallbladder, and liver standard-plane acquisition and radial artery sweeping, respectively. Compared with a fine-tuned pi_0 model, success on the abdominal tasks improves by 10-42 percentage points, with lower time-to-target, peak contact force, and trajectory jerk. Ablations support the complementary roles of anatomical anchoring, egocentric vision, and force feedback. End-to-end trials with automatic placement achieve 90-100% success, and the policies transfer without retraining to a second robot with 83-100% success.

Explore related subjects

cs.RO

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Yuan Bi, Fangxu Han, Ye Su, Yiping Zhou, Zhongliang Jiang, Nassir Navab. 2026-10-23. From Freehand Demonstrations to Autonomous Robotic Ultrasound: An Anatomy-Anchored Learning Framework. https://arxiv.org/abs/2610.32525

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

CARE: Consistent Action-Response Execution for Human-Guided Robot Transport

Human-guided collaborative transport becomes interaction-critical when the operator can continue along a route segment that is not executable by the robot--payload system. In this regime, operator motion is not an exogenous reference: it simultaneously communicates a persistent route tendency and adapts to the robot's realized progress. A common modular formulation predicts one operator future from observation history, scores multiple robot proposals against that shared forecast, and then modifies the selected command for local feasibility. This can compare candidates under responses they would not induce and can condition prediction on an action that is never executed. We formulate local control over executable-action-indexed joint human--robot transitions and instantiate it in Consistent Action--Response Execution (CARE). Each nominal proposal or bounded alternative is first mapped to an executable action; that same action conditions the route-aligned operator-response prediction, the matched robot--operator rollout, and a fixed interaction utility for candidate selection. Bounded reciprocal refinement handles transient route--feasibility mismatch, whereas persistent progress loss replaces the nominal proposal with a progress-oriented recovery proposal while retaining the same downstream executable-action-conditioned evaluation. Across training and held-out maps, CARE preserves near-saturated performance when the indicated route remains locally usable and achieves the highest observed success and service availability when the operator can continue but the robot must detour. It also yields lower jerk-proxy values and measured controller latency than dynamic-window and Timed Elastic Band baselines. Factorial ablations distinguish reciprocal refinement from progress recovery, delayed-response tests probe temporal mismatch, and an indoor deployment demonstrates physical executability.

cs.RO↗
arXiv2026

VORL-EXPLORE: A Hybrid Learning Planning Approach to Multi-Robot Exploration in Dynamic Environments

Hierarchical multi-robot exploration decouples frontier allocation from local navigation, making systems brittle in dynamic environments. Without awareness of execution difficulty, allocators may send robots toward frontiers whose routes are congested or unsafe, causing bottlenecks, oscillatory replanning, and redundant coverage. We propose VORL-EXPLORE, a hybrid learning and planning framework that makes exploration execution-aware through execution fidelity, an estimate of local navigability and execution reliability. This fidelity score is used to penalize risky or crowded frontiers in a fidelity-coupled Voronoi allocation objective with inter-robot repulsion, and to arbitrate between global $A^*$ guidance and a reactive reinforcement learning policy during navigation. As a result, robots follow efficient global plans in reliable regions and rely more on safer local reactions in crowded or changing areas. The fidelity model is recalibrated using self-supervised pseudo-labels derived from recent progress and safety outcomes. Quantitative experiments on randomized dynamic grids and an ablation under severe traffic show higher success rates, shorter exploration lengths, and lower inter-robot coverage overlap. A Gazebo factory scenario is included as a deployment illustration. The source code is available at https://github.com/21ning/VORL-EXPLORE.

cs.RO↗
arXiv2026

Factor Graph-Based Continuum Robot State Estimation with Magnus Expansion Priors

Reconstructing the shape of continuum manipulators from sparse, noisy sensor data is a challenging task, owing to the infinite-dimensional nature of such systems. Existing approaches broadly trade off between parametric methods that yield compact state representations but lack probabilistic structure, and Cosserat rod inference on factor graphs, which provides principled uncertainty quantification at the cost of a state dimension that grows with the spatial discretization. This letter combines the strengths of both these approaches by estimating the coefficients of a compact Geometric Variable Strain (GVS) parameterization within a factor graph framework. A novel kinematic factor, derived from the Magnus expansion of the strain field, encodes the closed-form rod geometry as a prior constraint linking the GVS strain coefficients to the backbone pose variables. The resulting formulation yields a compact state vector, is directly amenable to model-based control, and it retains the modularity, probabilistic treatment, and computational efficiency of factor graph inference. The proposed method is evaluated on both simulated and experimental data and compared against a Gaussian Process (GP) regression baseline, demonstrating increased accuracy in partial and sparse measurement scenarios while preserving computational efficiency.

cs.RO↗