Searcharxiv⌕ Search

arXiv subjects

Eric Jing Du

Publications and source records attributed to Eric Jing Du.

6 recordsLinked to original sources

Human-Machine Interfaces for Subsea Telerobotics: From Soda-straw to Natural Language Interactions

This review explores the evolution of human-machine interfaces (HMIs) in subsea telerobotics, charting the progression from traditional first-person "soda-straw" consoles-- characterized by narrow field-of-view camera feeds-- to contemporary interfaces leveraging gesture recognition, virtual reality, and natural language processing. The review makes three key contributions. First, we introduce a structured framework that analyzes subsea HMIs through three interrelated aspects: operator experience, robotic autonomy, and bidirectional human-machine communication. Second, using these aspects, we compare visual, haptic, extended-reality, gesture, and natural-language interfaces; examine the progression from direct teleoperation to shared autonomy; and assess the distinct roles of simulators and digital twins in training, prototyping, and mission support. Third, we synthesize the reported evidence to distinguish established industrial tools, selectively deployed technologies, and research-stage capabilities, while identifying limitations in interoperability, communication reliability, immersive feedback, and field validation. From this synthesis, we outline standardization needs and research directions for hydrodynamic feedback, latency-aware shared autonomy, grounded language interaction, and real-to-simulation transfer. The report concludes by identifying open research questions and proposing a roadmap toward more intelligent and intuitive subsea HMI systems.

cs.RO↗

From Wearable Interfaces to Dexterous Policies: Contact Shifts and Tactile Representations

Unlike conventional teleoperation, wearable interfaces allow operators to collect dexterous demonstrations through their own hand motions while directly interacting with task objects. This direct interaction reduces dependence on the target robot during collection, but it also makes the collection hardware part of the physical process that generates each demonstration. Interface geometry can influence both how a task is performed and what tactile observations are recorded for learning. We study two versions of a DexUMI-family exoskeleton that share the same robot command definition, mapping procedure, and tactile module type but differ in hand-side geometry. The revised interface reduces reported physical demand, improves selected device ratings, enables tactile access in a precision grasp that is mechanically blocked by the baseline, and produces task-dependent changes in recorded contact. Lid twisting primarily exhibits a change in contact location, whereas egg carton opening primarily exhibits a change in contact occurrence. We then train matched policies using either a binary aggregate input or a spatial-plus-force input. On lid twisting and egg carton opening, policies trained on revised-interface demonstrations achieve higher success than those trained on baseline demonstrations, with a significant pooled interface effect. Across these and two additional tasks, USB insertion and soldering tool pick and place, the spatial-plus-force input likewise outperforms binary aggregation. These results show that wearable collection hardware is part of the data-generation process for robot learning. Such interfaces should therefore be evaluated not only through operator experience, but also through the tactile interactions they make recordable and the policy performance their demonstrations support.

cs.RO↗

ExoBridge: Learning a Bare Hand to Hand-Worn Exoskeleton Mapping through Human Limb Coupling

Human video offers a scalable source of experience for dexterous robot learning, but obtaining motion and tactile supervision while preserving bare hand interaction remains challenging. We present ExoBridge, a framework that leverages human limb coupling to learn a bridging function from bare hand video to the motion and tactile state of a sensorized exoskeleton. Our central idea is to use coordinated bimanual behavior to connect an uninstrumented visual source with a measured manipulation interface. During collection, one hand remains bare and provides visual observations, while the opposite hand wears the exoskeleton and supplies synchronized motion and tactile measurements. These paired demonstrations train a temporal visual model to predict fingertip contact, continuous tactile intensity, and relative encoder motion from bare hand video alone. The exoskeleton defines an intermediate state space whose motion coordinates are linked to a dexterous robot hand through existing calibration. Evaluation on 1,215 demonstrations across four manipulation tasks uses held out collection sessions and yields a pooled any contact AUROC of 0.916 and a Pearson correlation of 0.790 for tactile intensity. The learned bridge also predicts relative changes in exoskeleton configuration from bare hand video. These results demonstrate that human limb coupling can turn exoskeleton measurements into supervision for bare hand video, establishing a learned bridge between human visual demonstrations and a robot oriented manipulation interface.

cs.RO↗

DreamLedger: Where to Refuse World-Model Imagination Using Execution-Settled Credit

World-model predictions inform robot actions, yet instantaneous reliability signals do not retain the outcomes of comparable past predictions. DreamLedger registers consumed predictions as claims, settles them against execution outcomes, and uses persistent execution history from comparable operating conditions, regions, and prediction horizons to estimate credit before future reliance. Replayable records connect each decision to its supporting evidence and eventual outcome. In ten-seed navigation comparisons at matched refusal volume, removing history features or resetting history increases burn rate, measured as failures per consumed prediction. An independent ten-seed manipulation replication at matched refusal volume finds that, relative to random refusal, DreamLedger lowers burn rate by 4.8 percentage points (95% CI: 0.8-8.7) and uses fewer probes. Randomized audits directly measure higher failure rates among denied candidates, and post-warmup shifts isolate the contribution of newly accumulated settlements. Franka experiments establish online deployment through replay of all 1,062 prediction uses and demonstrate a prospective gate transition: new failures lower previously high credit below a frozen threshold, triggering refusal before the next action. Task completion and verification cost characterize the trade-offs of these interventions.

cs.RO↗

When to Personalize Household Object Search: A Rigidity-Gated Hybrid Policy

Service robots searching for household objects rely on spatial priors to reduce search cost, yet object locations can vary with resident traits. Collecting longitudinal, trait-specific in-home trajectories is invasive and hard to scale. We study when personalization helps and propose PerSim, a rigidity-gated hybrid policy that combines a trait-conditioned prior with a population-frequency baseline, personalizing only when placement behavior is variable. To scale resident-conditioned dynamics, we employ a human-calibrated simulation pipeline to generate and validate object-placement transitions in diverse home layouts, and train a predictor that injects continuous Big Five vectors to output room-level priors and within-room co-occurrence cues. In a unified human study (N=200), dual-layer validation shows that (i) synthetic transitions are judged behaviorally plausible (mean 3.85/5, p < 1e-6), and (ii) in a blinded A/B comparison, personalization is favored primarily for low-rigidity objects (p=0.005), while the population-frequency baseline remains strong for universally placed items, yielding a decision rule for when to personalize. In an offline objective test, we observe a small but significant improvement on unseen continuous trait vectors over nearest discrete configuration matching (p=0.035), supporting interpolation in five-dimensional trait space. Finally, in a home digital twin we show that PerSim reduces expected search cost by combining room visitation effort with within-room cue checking, demonstrating end-to-end gains beyond isolated prediction metrics.

cs.RO↗

TEGA: A Tactile-Enhanced Grasping Assistant for Assistive Robotics via Sensor Fusion and Closed-Loop Haptic Feedback

Recent advances in teleoperation have enabled sophisticated manipulation of dexterous robotic hands, with most systems concentrating on guiding finger positions to achieve desired grasp configurations. However, while accurate finger positioning is essential, it often overlooks the equally critical task of grasp force modulation, vital for handling objects of diverse hardness, texture, and shape. This limitation poses a significant challenge for users, especially individuals with upper limb disabilities who lack natural tactile feedback and rely on indirect cues to infer appropriate force levels. To address this gap, We present the tactile enhanced grasping assistant (TEGA), a closed loop assistive teleoperation framework that fuses EMG based intent2force inference with visuotactile sensing mapped into real time vibrotactile feedback via a wearable haptic vest, enabling intuitive, proportional force adjustment during manipulation. A wearable haptic vest delivers real time tactile feedback, allowing users to dynamically refine grasp force during manipulation. User studies confirm that the system substantially improves grasp stability and task success, underscoring its potential for assistive robotic applications.

cs.RO↗