arXiv subjects

Jeremy D. Brown

Publications and source records attributed to Jeremy D. Brown.

16 recordsLinked to original sources

arXiv2026

Toward Autonomous Soft Robotic Endovascular Navigation via Imitation Learning

In endovascular surgery, endovascular interventionists push a thin tube called a catheter, guided by a thin wire to a treatment site inside the patient's blood vessels to treat various conditions such as blood clots, aneurysms, and malformations. Robotic guidewires can enhance maneuverability but are difficult to model and control. Autonomous soft robotic guidewire navigation has the potential to overcome these challenges, increasing the precision and safety of endovascular navigation. As a first step, we establish a large-scale, 2D-projected environment for autonomous navigation. In other surgical domains, end-to-end imitation learning has shown promising results. Thus, we develop a transformer-based imitation learning framework with goal conditioning, relative action outputs, and automatic contrast dye injections to enable generalizable soft robot navigation in an aneurysm targeting task. We train the policy on 36 different modular bifurcated geometries, generating 647 total demonstrations under simulated fluoroscopy, and evaluate it on three previously unseen vascular geometries. The policy reaches the aneurysm with a success rate of 83% on the unseen geometries, outperforming several baselines. In addition, ablation and baseline studies evaluate the effectiveness of each design and data collection choice. Lastly, we extend the policy to achieve 75% success on an unseen patient-derived geometry. Project website: https://softrobotnavigation.github.io/

cs.RO↗

arXiv2026

Modeling and Control of a Pneumatic Soft Robotic Catheter Using Neural Koopman Operators

Catheter-based interventions are widely used for the diagnosis and treatment of cardiac diseases. Recently, robotic catheters have attracted attention for their ability to improve precision and stability over conventional manual approaches. However, accurate modeling and control of soft robotic catheters remain challenging due to their complex, nonlinear behavior. The Koopman operator enables lifting the original system data into a linear "lifted space", offering a data-driven framework for predictive control; however, manually chosen basis functions in the lifted space often oversimplify system behaviors and degrade control performance. To address this, we propose a neural network-enhanced Koopman operator framework that jointly learns the lifted space representation and Koopman operator in an end-to-end manner. Moreover, motivated by the need to minimize radiation exposure during X-ray fluoroscopy in cardiac ablation, we investigate open-loop control strategies using neural Koopman operators to reliably reach target poses without continuous imaging feedback. The proposed method is validated in two experimental scenarios: interactive position control and a simulated cardiac ablation task using an atrium-like cavity. Our approach achieves average errors of 2.1 +- 0.4 mm in position and 4.9 +- 0.6 degrees in orientation, outperforming not only model-based baselines but also other Koopman variants in targeting accuracy and efficiency. These results highlight the potential of the proposed framework for advancing soft robotic catheter systems and improving catheter-based interventions.

cs.RO↗

arXiv2025

Towards Autonomous Robotic Electrosurgery via Thermal Imaging

Electrosurgery is a surgical technique that can improve tissue cutting by reducing cutting force and bleeding. However, electrosurgery adds a risk of thermal injury to surrounding tissue. Expert surgeons estimate desirable cutting velocities based on experience but have no quantifiable reference to indicate if a particular velocity is optimal. Furthermore, prior demonstrations of autonomous electrosurgery have primarily used constant tool velocity, which is not robust to changes in electrosurgical tissue characteristics, power settings, or tool type. Thermal imaging feedback provides information that can be used to reduce thermal injury while balancing cutting force by controlling tool velocity. We introduce Thermography for Electrosurgical Rate Modulation via Optimization (ThERMO) to autonomously reduce thermal injury while balancing cutting force by intelligently controlling tool velocity. We demonstrate ThERMO in tissue phantoms and compare its performance to the constant velocity approach. Overall, ThERMO improves cut success rate by a factor of three and can reduce peak cutting force by a factor of two. ThERMO responds to varying environmental disturbances, reduces damage to tissue, and completes cutting tasks that would otherwise result in catastrophic failure for the constant velocity approach.

cs.RO↗

arXiv2024

HapKnob -- A Motorized Shape-changing Haptic Knob Interface

The absence of physical interfaces creates challenges when interacting with touchscreen technology. This study aims to investigate an innovative haptic solution for interacting with graphical user interfaces. A motorized shape-changing rotary knob interface, HapKnob, has been developed, achieving seven distinctive shape configurations and various force feedback renderings. HapKnob presents a compact design and can provide additional configurations and combinations based on user needs. The anticipated results hold the potential to advance the development of future user interfaces, especially in situations where visual interaction is unavailable.

cs.HC↗

arXiv2024

Tool Compensation and User Strategy during Human-Robot Teleoperation are Impacted by System Dynamics and Kinesthetic Feedback

Manipulating an environment remotely with a robotic teleoperator introduces novel electromechanical (EM) dynamics between the user and environment. While considerable effort has focused on minimizing these dynamics, there is limited research into understanding their impact on a user's internal model and resulting motor control strategy. Here we investigate to what degree the dynamics and kinesthetic feedback of the teleoperator influence task behavior and tool compensation. Our teleoperator testbed features a leader port controlled by user input via wrist rotation, a follower port connected to a virtual environment rendered by rotary motor, and three distinct transmissions (Rigid, Unilateral EM, Bilateral EM) in-between that can be engaged independently. 30 adult participants rotated a disk in a visco-elastic virtual environment through counterbalanced presentation of each transmission. Users tracked targets oscillating at 7 pre-defined random frequencies between 0.55 and 2.35 Hz. After session completion, trajectories of the target, leader, and follower were decomposed into components of gain and phase error for all frequencies. We found that while tracking performance at the follower port was similar across transmissions, users' adjustment at the leader port differed between Rigid and EM transmissions. Users applied different pinch forces between Rigid and Unilateral transmissions, suggesting that tracking strategy does change between dynamics and feedback. However, the users' ability to compensate dynamics diminished significantly as task speed got faster and more difficult. Therefore, there are limits to pursuit tracking at the human wrist when compensating teleoperator dynamics.

cs.HC↗

arXiv2024

Identifying patterns of proprioception and target matching acuity in healthy humans

Traditional approaches to measurement in upper-limb therapy have gaps that electronic sensing and recording can help fill. We highlight shortcomings in current kinematic recording devices, and we introduce a wrist sensing device that performs multimodal sensing during single-axis rotation. Our goal is to characterize normative kinesthetic perception and real-world performance as a multimodal sensory "fingerprint" that can serve as a reference point for identifying deficit in persons affected by stroke, and then as a jumping point for later neuroscientific interrogation. We present an experiment involving psychophysical measurements of passive stimuli discrimination, matching adjustment acuity, and ADL performance in 11 neurologically-intact persons. We found that passive velocity sense and active position sense of healthy controls, measured by velocity discrimination and position matching respectively, correlated in rank with each other, but other score comparisons of acuity or task performance had no statistically significant correlations. We also found that participants differed in acuity between passive and active velocity sense, which supports current understanding about muscle spindle activation being modulated by conscious motor command. The potential for our null correlation results to reveal dissociable aspects of deficit is discussed, as well as implications for future neuroscientific study with more kinematic measures and larger datasets.

cs.RO↗

arXiv2022

Dual-Modality Haptic Feedback Improves Dexterous Task Execution with Virtual EMG-Controlled Gripper

Upper-extremity amputees who use myoelectric prostheses currently lack the haptic sensory information needed to perform dexterous activities of daily living. While considerable research has focused on restoring this haptic information, these approaches often rely on single-modality feedback schemes which are necessary but insufficient for the feedforward and feedback control strategies employed by the central nervous system. Multi-modality feedback approaches have been gaining attention in several application domains, however, the utility for myoelectric prosthesis use remains unclear. In this study, we investigated the utility of dual-modality haptic feedback in a virtual EMG-controlled grasp-and-hold task with a brittle object and variable load force. We recruited N=20 non-amputee participants to perform the task in four conditions: no feedback, vibration feedback of incipient slip, squeezing feedback of grip force, and dual (vibration + squeezing) feedback of incipient slip and grip force. Results suggest that receiving any feedback is better than receiving none, however, dual-modality feedback is far superior to either single-modality feedback approach in terms of preventing the object from breaking or dropping, even after it started slipping. Control with dual-modality feedback was also seen as more intuitive than with either of the single-modality feedback approaches.

cs.RO↗

arXiv2022

Haptic Shared Control Improves Neural Efficiency During Myoelectric Prosthesis Use

Clinical myoelectric prostheses lack the sensory feedback and sufficient dexterity required to complete activities of daily living efficiently and accurately. Providing haptic feedback of relevant environmental cues to the user or imbuing the prosthesis with autonomous control authority have been separately shown to improve prosthesis utility. Few studies, however, have investigated the effect of combining these two approaches in a shared control paradigm, and none have evaluated such an approach from the perspective of neural efficiency (the relationship between task performance and mental effort measured directly from the brain). In this work, we analyzed the neural efficiency of 30 non-amputee participants in a grasp-and-lift task of a brittle object. Here, a myoelectric prosthesis featuring vibrotactile feedback of grip force and autonomous control of grasping was compared with a standard myoelectric prosthesis with and without vibrotactile feedback. As a measure of mental effort, we captured the prefrontal cortex activity changes using functional near infrared spectroscopy during the experiment. Results showed that only the haptic shared control system enabled users to achieve high neural efficiency, and that vibrotactile feedback was important for grasping with the appropriate grip force. These results indicate that the haptic shared control system synergistically combines the benefits of haptic feedback and autonomous controllers, and is well-poised to inform such hybrid advancements in myoelectric prosthesis technology.

cs.RO↗

arXiv2022

The Utility of Synthetic Reflexes and Haptic Feedback for Upper-Limb Prostheses in a Dexterous Task Without Direct Vision

Individuals who use myoelectric upper-limb prostheses often rely heavily on vision to complete their daily activities. They thus struggle in situations where vision is overloaded, such as multitasking, or unavailable, such as poor lighting conditions. Non-amputees can easily accomplish such tasks due to tactile reflexes and haptic sensation guiding their upper-limb motor coordination. Based on these principles, we developed and tested two novel prosthesis systems that incorporate autonomous controllers and provide the user with touch-location feedback through either vibration or distributed pressure. These capabilities were made possible by installing a custom contact-location sensor on thefingers of a commercial prosthetic hand, along with a custom pressure sensor on the thumb. We compared the performance of the two systems against a standard myoelectric prosthesis and a myoelectric prosthesis with only autonomous controllers in a difficult reach-to-pick-and-place task conducted without direct vision. Results from 40 non-amputee participants in this between-subjects study indicated that vibrotactile feedback combined with synthetic reflexes proved significantly more advantageous than the standard prosthesis in several of the task milestones. In addition, vibrotactile feedback and synthetic reflexes improved grasp placement compared to only synthetic reflexes or pressure feedback combined with synthetic reflexes. These results indicate that both autonomous controllers and haptic feedback facilitate success in dexterous tasks without vision, and that the type of haptic display matters.

cs.RO↗

arXiv2021

A novel testbed for investigating the impact of teleoperator dynamics on perceived environment dynamics

Human-in-the-loop telerobotic systems (HiLTS) are robotic tools designed to extend and in some circumstances improve the dexterous capabilities of the human operator in virtual and remote environments. Dexterous manipulation, however, depends on how well the telerobot is incorporated into the operator's sensorimotor control scheme. Empirical evidence suggests that haptic feedback can lead to improved dexterity. Unfortunately, haptic feedback can also introduce dynamics between the leader and follower of the telerobot that affect both stability and device performance. While concerted research effort has focused on masking these device dynamics or bypassing them altogether, it is not well understood how human operators incorporate these dynamics into their control scheme. We believe that to advance dexterous telerobotic manipulation, it is crucial to understand the process by which humans operators incorporate teleoperator dynamics and distinguish them from the dynamics of the environment. Key to this knowledge is an understanding of how advanced telerobotic architectures compare to the gold standard, the rigid mechanical teleoperators first introduced in the 1950's. In this manuscript, we present a teleoperator testbed that has reconfigurable transmissions between the leader and follower to change its dynamic behavior. The intent of this testbed is to investigate the effect of the teleoperator's dynamics on perception of and task performance in the remote/virtual environment. We describe the hardware and software components of the testbed and then demonstrate how the different teleoperator transmissions can lead to differences, sometimes significant, in the dynamics that would be felt by the operator when exploring the same environment.

cs.RO↗

arXiv2021

Towards an Understanding of the Role Operator Limb Dynamics Plays in Haptic Perception of Stiffness

Creating haptic interfaces capable of rendering the rich sensation needed for dexterous manipulation is crucial for the advancement of human-in-the-loop telerobotic systems (HiLTS). One limiting factor has been the absence of detailed knowledge of the effect of operator limb dynamics and haptic exploration dynamics on haptic perception. We propose to begin investigations of these effects with single-joint haptic exploration and feedback of physical and virtual environments. Here, we present our experimental apparatus, a 1-DoF rotational kinesthetic haptic device and electromyography (EMG) system, along with preliminary findings from our efforts to investigate the change in stiffness discrimination thresholds for differing exploration velocities. Result trends indicate a possible relationship between exploration velocity and discrimination thresholds, as well as a complex interaction between muscle activation, exploration velocity, and haptic feedback.

cs.RO↗

arXiv2021

A novel teleoperator testbed to understand the effects of master-slave dynamics on embodiment and kinesthetic perception *

With the rising popularity of telerobotic systems, the focus on transparency with regards to haptic perception is also increasing. Transparency, however, represents a theoretical ideal as most bilateral force-reflecting telerobots introduce dynamics (stiffness and damping) between the operator and the environment. To achieve true dexterity, it will be essential to understand how humans embody the dynamics of these telerobots and thereby distinguish them from the environment they are exploring. In this short manuscript, we introduce a novel single degree-of-freedom testbed designed to perform psychophysical and task performance assessments of kinesthetic perception during telerobotic exploration. The system is capable of being configured as a rigid mechanical teleoperator, a dynamic mechanical teleoperator, and an electromechanicaal teleoperator. We performed prefatory system identification and found that the system is capable of simulating telerobotic exploration necessary to understand the impact of master-slave dynamics on kinesthetic perception.

cs.RO↗

arXiv2021

Towards an understanding of how humans perceive stiffness during bimanual exploration

In this paper, an experimental testbed and associated psychophysical paradigm are presented for understanding how people discriminate torsional stiffness using wrist rotation about their forearm. Featured in the testbed are two 1-DoF rotary kinesthetic haptic devices. An adaptive staircase was used to evaluate JNDs for a stiffness discrimination task where participants explored virtual torsion springs by rotating their forearms. The JNDs were evaluated across seven different conditions, under four different exploration modes: bimanual, unimanual, bimanual feedback for unimanual displacement, and unimanual feedback for bimanual displacement. The discrimination results will inform future investigation into understanding how stiffness percepts vary.

cs.HC↗

arXiv2021

Investigating the effects of exploration dynamics on stiffness perception

The utility of Human-in-the-loop telerobotic systems (HiLTS) is driven in part by the quality of feedback it can provide to the operator. While the dynamic interaction between robot and environment can often be sensed or modeled, the dynamic coupling of the human-robot interface is often overlooked. Enabling dexterous manipulation through HiLTS however, will require careful consideration of human haptic perception as it relates to the human's changing limb impedance at the human-robot interface. In this manuscript, we present results from a stiffness perception task run on a simple 1-DoF rotational kinesthetic device at three different angular velocities, based on participant's natural exploration strategy. We evaluated performance effects of exploration velocity as a proxy measurement for limb impedance and the results indicate the need to further investigate how the human body incorporates its knowledge of the body dynamics in kinesthetic perception under active exploration.

cs.RO↗

arXiv2021

Preliminary investigation into how limb choice affects kinesthetic perception

We have a limited understanding of how we integrate haptic information in real-time from our upper limbs to perform complex bimanual tasks, an ability that humans routinely employ to perform tasks of varying levels of difficulty. In order to understand how information from both limbs is used to create a unified percept, it is important to study both the limbs separately first. Prevalent theories highlighting the role of central nervous system (CNS) in accounting for internal body dynamics seem to suggest that both upper limbs should be equally sensitive to external stimuli. However, there is empirical proof demonstrating a perceptual difference in our upper limbs for tasks like shape discrimination, prompting the need to study effects of limb choice on kinesthetic perception. In this manuscript, we start evaluating Just Noticeable Difference (JND) for stiffness for both forearms separately. Early results validate the need for a more thorough investigation of limb choice on kinesthetic perception.

cs.RO↗

arXiv2021

Sensorimotor-inspired Tactile Feedback and Control Improve Consistency of Prosthesis Manipulation in the Absence of Direct Vision

The lack of haptically aware upper-limb prostheses forces amputees to rely largely on visual cues to complete activities of daily living. In contrast, able-bodied individuals inherently rely on conscious haptic perception and automatic tactile reflexes to govern volitional actions in situations that do not allow for constant visual attention. We therefore propose a myoelectric prosthesis system that reflects these concepts to aid manipulation performance without direct vision. To implement this design, we built two fabric-based tactile sensors that measure contact location along the palmar and dorsal sides of the prosthetic fingers and grasp pressure at the tip of the prosthetic thumb. Inspired by the natural sensorimotor system, we use the measurements from these sensors to provide vibrotactile feedback of contact location and implement a tactile grasp controller that uses automatic reflexes to prevent over-grasping and object slip. We compare this system to a standard myoelectric prosthesis in a challenging reach-to-pick-and-place task conducted without direct vision; 17 able-bodied adults took part in this single-session between-subjects study. Participants in the tactile group achieved more consistent high performance compared to participants in the standard group. These results indicate that the addition of contact-location feedback and reflex control increases the consistency with which objects can be grasped and moved without direct vision in upper-limb prosthetics.

cs.RO↗