arXiv subjects

Ilana Nisky

Publications and source records attributed to Ilana Nisky.

18 recordsLinked to original sources

arXiv2026

Fine Wrist Control as a Marker of Surgical Teleoperation Expertise

Unlike most intensely physical pursuits, surgical robotic teleoperation training focuses primarily on task outcomes rather than surgeon body posture or biomechanics during task completion. Toward the question of the role of biomechanics in surgical expertise, we sought to characterize the articular motion of expert teleoperators as compared to novice users. Twenty-seven novices and nine experts completed a non-medical cylinder-on-peg transfer task while their upper limb biomechanics were recorded via motion trackers. During more difficult motions, experts stabilized their wrist motion more than novices, while maintaining adequate range of motion in their shoulder and elbow and completing the task significantly faster than novices. This marker of expertise suggests the importance of attention to user biomechanics during teleoperation of surgical robots.

cs.RO↗

arXiv2026

Video-Based Detection and Analysis of Errors in Robotic Surgical Training

Robot-assisted minimally invasive surgeries offer many advantages but require complex motor tasks that take surgeons years to master. There is currently a lack of knowledge on how surgeons acquire these robotic surgical skills. Toward bridging this gap, a previous study followed surgical residents learning complex surgical dry lab tasks on a surgical robot over six months. Errors are an important measure for training and skill evaluation, but unlike in virtual simulations, in dry lab training, errors are difficult to monitor automatically. Here, we analyzed errors in the ring tower transfer task, in which surgical residents moved a ring along a curved wire as quickly and accurately as possible. We developed an image-processing algorithm using color and size thresholds, optical flow and short time Fourier transforms to detect collision errors and achieved a detection accuracy of approximately 95%. Using the detected errors and task completion time, we found that the residents reduced their completion time and number of errors over the six months, while the percentage of task time spent making errors remained relatively constant on average. This analysis sheds light on the learning process of the residents and can serve as a step towards providing error-related feedback to robotic surgeons.

cs.RO↗

arXiv2025

Effect of Performance Feedback Timing on Motor Learning for a Surgical Training Task

Objective: Robot-assisted minimally invasive surgery (RMIS) has become the gold standard for a variety of surgical procedures, but the optimal method of training surgeons for RMIS is unknown. We hypothesized that real-time, rather than post-task, error feedback would better increase learning speed and reduce errors. Methods: Forty-two surgical novices learned a virtual version of the ring-on-wire task, a canonical task in RMIS training. We investigated the impact of feedback timing with multi-sensory (haptic and visual) cues in three groups: (1) real-time error feedback, (2) trial replay with error feedback, and (3) no error feedback. Results: Participant performance was evaluated based on the accuracy of ring position and orientation during the task. Participants who received real-time feedback outperformed other groups in ring orientation. Additionally, participants who received feedback in replay outperformed participants who did not receive any error feedback on ring orientation during long, straight path sections. There were no significant differences between groups for ring position overall, but participants who received real-time feedback outperformed the other groups in positional accuracy on tightly curved path sections. Conclusion: The addition of real-time haptic and visual error feedback improves learning outcomes in a virtual surgical task over error feedback in replay or no error feedback at all. Significance: This work demonstrates that multi-sensory error feedback delivered in real time leads to better training outcomes as compared to the same feedback delivered after task completion. This novel method of training may enable surgical trainees to develop skills with greater speed and accuracy.

cs.RO↗

arXiv2025

Advances in Compliance Detection: Novel Models Using Vision-Based Tactile Sensors

Compliance is a critical parameter for describing objects in engineering, agriculture, and biomedical applications. Traditional compliance detection methods are limited by their lack of portability and scalability, rely on specialized, often expensive equipment, and are unsuitable for robotic applications. Moreover, existing neural network-based approaches using vision-based tactile sensors still suffer from insufficient prediction accuracy. In this paper, we propose two models based on Long-term Recurrent Convolutional Networks (LRCNs) and Transformer architectures that leverage RGB tactile images and other information captured by the vision-based sensor GelSight to predict compliance metrics accurately. We validate the performance of these models using multiple metrics and demonstrate their effectiveness in accurately estimating compliance. The proposed models exhibit significant performance improvement over the baseline. Additionally, we investigated the correlation between sensor compliance and object compliance estimation, which revealed that objects that are harder than the sensor are more challenging to estimate.

cs.CV↗

arXiv2025

Dataset and Analysis of Long-Term Skill Acquisition in Robot-Assisted Minimally Invasive Surgery

Objective: We aim to investigate long-term robotic surgical skill acquisition among surgical residents and the effects of training intervals and fatigue on performance. Methods: For six months, surgical residents participated in three training sessions once a month, surrounding a single 26-hour hospital shift. In each shift, they participated in training sessions scheduled before, during, and after the shift. In each training session, they performed three dry-lab training tasks: Ring Tower Transfer, Knot-Tying, and Suturing. We collected a comprehensive dataset, including videos synchronized with kinematic data, activity tracking, and scans of the suturing pads. Results: We collected a dataset of 972 trials performed by 18 residents of different surgical specializations. Participants demonstrated consistent performance improvement across all tasks. In addition, we found variations in between-shift learning and forgetting across metrics and tasks, and hints for possible effects of fatigue. Conclusion: The findings from our first analysis shed light on the long-term learning processes of robotic surgical skills with extended intervals and varying levels of fatigue. Significance: This study lays the groundwork for future research aimed at optimizing training protocols and enhancing AI applications in surgery, ultimately contributing to improved patient outcomes. The dataset will be made available upon acceptance of our journal submission.

cs.RO↗

arXiv2025

Bowel Incision Closure with a Semi-Automated Robot-Assisted Laser Tissue Soldering System

Traditional methods for closing gastrointestinal (GI) surgery incisions, like suturing and stapling, present significant challenges, including potentially life-threatening leaks. These techniques, especially in robot-assisted minimally invasive surgery (RAMIS), require advanced manual skills. While their repetitive and time-consuming nature makes them suitable candidates for automation, the automation process is complicated by the need for extensive contact with the tissue. Addressing this, we demonstrate a semi-autonomous contactless surgical procedure using our novel Robot-assisted Laser Tissue Soldering (RLTS) system on a live porcine bowel. Towards this in-vivo demonstration, we optimized soldering protocols and system parameters in ex-vivo experiments on porcine bowels and a porcine cadaver. To assess the RLTS system performance, we compared the pressure at which the anastomosis leaked between our robotic soldering and manual suturing. With the best setup, we advanced to an in-vivo Heineke Mikulicz closure on small bowel incision in live pigs and evaluated their healing for two weeks. All pigs successfully completing the procedure (N=5) survived without leaks and the histology indicated mucosal regeneration and fibrous tissue adhesion. This marks the first in-vivo semi-automated contactless incision closure, paving the way for automating GI surgery incision closure which has the potential to become an alternative to traditional methods.

q-bio.QM↗

arXiv2023

Haptic Guidance and Haptic Error Amplification in a Virtual Surgical Robotic Training Environment

Teleoperated robotic systems have introduced more intuitive control for minimally invasive surgery, but the optimal method for training remains unknown. Recent motor learning studies have demonstrated that exaggeration of errors helps trainees learn to perform tasks with greater speed and accuracy. We hypothesized that training in a force field that pushes the operator away from a desired path would improve their performance on a virtual reality ring-on-wire task. Forty surgical novices trained under a no-force, guidance, or error-amplifying force field over five days. Completion time, translational and rotational path error, and combined error-time were evaluated under no force field on the final day. The groups significantly differed in combined error-time, with the guidance group performing the worst. Error-amplifying field participants showed the most improvement and did not plateau in their performance during training, suggesting that learning was still ongoing. Guidance field participants had the worst performance on the final day, confirming the guidance hypothesis. Participants with high initial path error benefited more from guidance. Participants with high initial combined error-time benefited more from guidance and error-amplifying force field training. Our results suggest that error-amplifying and error-reducing haptic training for robot-assisted telesurgery benefits trainees of different abilities differently.

cs.RO↗

arXiv2023

Design and Assessment of a Bimanual Haptic Epidural Needle Insertion Simulator

The case experience of anesthesiologists is one of the leading causes of accidental dural punctures and failed epidurals - the most common complications of epidural analgesia used for pain relief during delivery. We designed a bimanual haptic simulator to train anesthesiologists and optimize epidural analgesia skill acquisition. We present an assessment study conducted with 22 anesthesiologists of different competency levels from several Israeli hospitals. Our simulator emulates the forces applied to the epidural (Touhy) needle, held by one hand, and those applied to the Loss of Resistance (LOR) syringe, held by the other one. The resistance is calculated based on a model of the epidural region layers parameterized by the weight of the patient. We measured the movements of both haptic devices and quantified the results' rate (success, failed epidurals, and dural punctures), insertion strategies, and the participants' answers to questionnaires about their perception of the simulation realism. We demonstrated good construct validity by showing that the simulator can distinguish between real-life novices and experts. Face and content validity were examined by studying users' impressions regarding the simulator's realism and fulfillment of purpose. We found differences in strategies between different level anesthesiologists, and suggest trainee-based instruction in advanced training stages.

cs.HC↗

arXiv2023

Using LOR Syringe Probes as a Method to Reduce Errors in Epidural Analgesia -- a Robotic Simulation Study

Epidural analgesia involves injection of anesthetics into the epidural space, using a Touhy needle to proceed through the layers in the epidural region and a "loss of resistance" (LOR) syringe to sense the environment stiffness. The anesthesiologist's case experience is one of the leading causes of accidental dural puncture and failed epidural - the two most common complications of epidural analgesia. Robotic simulation is an appealing solution to help train novices in this task. Another benefit of it is the ability to record the kinematic information throughout the procedure. In this work, we used a haptic bimanual simulator, that we designed and validated in previous work, to explore the effect LOR probing strategies had on procedure outcomes. Our results indicate that most participants probed more with the LOR syringe in successful trials, compared to unsuccessful trials. Furthermore, this result was more prominent in the three layers preceding the epidural space. Our findings can assist in creating better instructions for training novices in the task of epidural analgesia. We posit that instructing anesthesia residents to use the LOR syringe more extensively and educating them to do so more when they are in proximity to the epidural space can help improve skill acquisition in this task.

cs.RO↗

arXiv2022

Task Dynamics of Prior Training Influence Visual Force Estimation Ability During Teleoperation

The lack of haptic feedback in Robot-assisted Minimally Invasive Surgery (RMIS) is a potential barrier to safe tissue handling during surgery. Bayesian modeling theory suggests that surgeons with experience in open or laparoscopic surgery can develop priors of tissue stiffness that translate to better force estimation abilities during RMIS compared to surgeons with no experience. To test if prior haptic experience leads to improved force estimation ability in teleoperation, 33 participants were assigned to one of three training conditions: manual manipulation, teleoperation with force feedback, or teleoperation without force feedback, and learned to tension a silicone sample to a set of force values. They were then asked to perform the tension task, and a previously unencountered palpation task, to a different set of force values under teleoperation without force feedback. Compared to the teleoperation groups, the manual group had higher force error in the tension task outside the range of forces they had trained on, but showed better speed-accuracy functions in the palpation task at low force levels. This suggests that the dynamics of the training modality affect force estimation ability during teleoperation, with the prior haptic experience accessible if formed under the same dynamics as the task.

cs.RO↗

arXiv2022

Predicting the Timing of Camera Movements From the Kinematics of Instruments in Robotic-Assisted Surgery Using Artificial Neural Networks

Robotic-assisted surgeries benefit both surgeons and patients, however, surgeons frequently need to adjust the endoscopic camera to achieve good viewpoints. Simultaneously controlling the camera and the surgical instruments is impossible, and consequentially, these camera adjustments repeatedly interrupt the surgery. Autonomous camera control could help overcome this challenge, but most existing systems are reactive, e.g., by having the camera follow the surgical instruments. We propose a predictive approach for anticipating when camera movements will occur using artificial neural networks. We used the kinematic data of the surgical instruments, which were recorded during robotic-assisted surgical training on porcine models. We split the data into segments, and labeled each either as a segment that immediately precedes a camera movement, or one that does not. Due to the large class imbalance, we trained an ensemble of networks, each on a balanced sub-set of the training data. We found that the instruments' kinematic data can be used to predict when camera movements will occur, and evaluated the performance on different segment durations and ensemble sizes. We also studied how much in advance an upcoming camera movement can be predicted, and found that predicting a camera movement 0.25, 0.5, and 1 second before they occurred achieved 98%, 94%, and 84% accuracy relative to the prediction of an imminent camera movement. This indicates that camera movement events can be predicted early enough to leave time for computing and executing an autonomous camera movement and suggests that an autonomous camera controller for RAMIS may one day be feasible.

cs.LG↗

arXiv2021

Robot-Assisted Surgical Training Over Several Days in a Virtual Surgical Environment with Divergent and Convergent Force Fields

Surgical procedures require a high level of technical skill to ensure efficiency and patient safety. Due to the direct effect of surgeon skill on patient outcomes, the development of cost-effective and realistic training methods is imperative to accelerate skill acquisition. Teleoperated robotic devices allow for intuitive ergonomic control, but the learning curve for these systems remains steep. Recent studies in motor learning have shown that visual or physical exaggeration of errors helps trainees to learn to perform tasks faster and more accurately. In this study, we extended the work from two previous studies to investigate the performance of subjects in different force field training conditions, including convergent (assistive), divergent (resistive), and no force field (null).

cs.RO↗

arXiv2021

Combining Time-Dependent Force Perturbations in Robot-Assisted Surgery Training

Teleoperated robot-assisted minimally-invasive surgery (RAMIS) offers many advantages over open surgery. However, there are still no guidelines for training skills in RAMIS. Motor learning theories have the potential to improve the design of RAMIS training but they are based on simple movements that do not resemble the complex movements required in surgery. To fill this gap, we designed an experiment to investigate the effect of time-dependent force perturbations on the learning of a pattern-cutting surgical task. Thirty participants took part in the experiment: (1) a control group that trained without perturbations, and (2) a 1Hz group that trained with 1Hz periodic force perturbations that pushed each participant's hand inwards and outwards in the radial direction. We monitored their learning using four objective metrics and found that participants in the 1Hz group learned how to overcome the perturbations and improved their performances during training without impairing their performances after the perturbations were removed. Our results present an important step toward understanding the effect of adding perturbations to RAMIS training protocols and improving RAMIS training for the benefit of surgeons and patients.

cs.RO↗

arXiv2021

Rate of Orientation Change as a New Metric for Robot-Assisted and Open Surgical Skill Evaluation

Surgeons' technical skill directly impacts patient outcomes. To date, the angular motion of the instruments has been largely overlooked in objective skill evaluation. To fill this gap, we have developed metrics for surgical skill evaluation that are based on the orientation of surgical instruments. We tested our new metrics on two datasets with different conditions: (1) a dataset of experienced robotic surgeons and nonmedical users performing needle-driving on a dry lab model, and (2) a small dataset of suturing movements performed by surgeons training on a porcine model. We evaluated the performance of our new metrics (angular displacement and the rate of orientation change) alongside the performances of classical metrics (task time and path length). We calculated each metric on different segments of the movement. Our results highlighted the importance of segmentation rather than calculating the metrics on the entire movement. Our new metric, the rate of orientation change, showed statistically significant differences between experienced surgeons and nonmedical users / novice surgeons, which were consistent with the classical task time metric. The rate of orientation change captures technical aspects that are taught during surgeons' training, and together with classical metrics can lead to a more comprehensive discrimination of skills.

cs.RO↗

arXiv2020

A Novel Grip Force Measurement Concept for Tactile Stimulation Mechanisms -- Design, Validation, and User Study

We developed a new grip force measurement concept that allows for embedding tactile stimulation mechanisms in a gripper. This concept is based on a single force sensor to measure the force applied on each side of the gripper, and it substantially reduces artifacts of force measurement caused by tactor motion. To test the feasibility of this new concept, we built a device that measures control of grip force in response to a tactile stimulation from a moving tactor. First, we used a custom designed testing setup with a second force sensor to calibrate our device over a range of 0 to 20 N without movement of the tactors. Second, we tested the effect of tactor movement on the measured grip force and measured artifacts of 1% of the measured force. Third, we demonstrated that during the application of dynamically changing grip forces, the average errors were 2.9% and 3.7% for the left and right sides of the gripper, respectively. Finally, we conducted a user study and found that in response to tactor movement, participants increased their grip force, and that the increase was larger for a smaller target force and depended on the amount of tactile stimulation.

cs.RO↗

arXiv2018

Human-centered transparency of grasping via a robot-assisted minimally invasive surgery system

We investigate grasping of rigid objects in unilateral robot-assisted minimally invasive surgery (RAMIS) in this paper. We define a human-centered transparency that quantifies natural action and perception in RAMIS. We demonstrate this human-centered transparency analysis for different values of gripper scaling - the scaling between the grasp aperture of the surgeon-side manipulator and the aperture of the surgical instrument grasper. Thirty-one participants performed teleoperated grasping and perceptual assessment of rigid objects in one of three gripper scaling conditions (fine, normal, and quick, trading off precision and responsiveness). Psychophysical analysis of the variability of maximal grasping aperture during prehension and of the reported size of the object revealed that in normal and quick (but not in the fine) gripper scaling conditions, teleoperated grasping with our system was similar to natural grasping, and therefore, human-centered transparent. We anticipate that using motor control and psychophysics for human-centered optimizing of teleoperation control will eventually improve the usability of RAMIS.

cs.HC↗

arXiv2018

Stochastic Optimal Control for Modeling Reaching Movements in the Presence of Obstacles: Theory and Simulation

In many human-in-the-loop robotic applications such as robot-assisted surgery and remote teleoperation, predicting the intended motion of the human operator may be useful for successful implementation of shared control, guidance virtual fixtures, and predictive control. Developing computational models of human movements is a critical foundation for such motion prediction frameworks. With this motivation, we present a computational framework for modeling reaching movements in the presence of obstacles. We propose a stochastic optimal control framework that consists of probabilistic collision avoidance constraints and a cost function that trades-off between effort and end-state variance in the presence of a signal-dependent noise. First, we present a series of reformulations to convert the original non-linear and non-convex optimal control into a parametric quadratic programming problem. We show that the parameters can be tuned to model various collision avoidance strategies, thereby capturing the quintessential variability associated with human motion. Then, we present a simulation study that demonstrates the complex interaction between avoidance strategies, control cost, and the probability of collision avoidance. The proposed framework can benefit a variety of applications that require teleoperation in cluttered spaces, including robot-assisted surgery. In addition, it can also be viewed as a new optimizer which produces smooth and probabilistically-safe trajectories under signal dependent noise.

cs.RO↗

arXiv2017

What Can Spatiotemporal Characteristics of Movements in RAMIS Tell Us?

Quantitative characterization of surgical movements can improve the quality of patient care by informing the development of new training protocols for surgeons, and the design and control of surgical robots. Here, we present a novel characterization of open and teleoperated suturing movements that is based on principles from computational motor control. We focus on the extensively-studied relationship between the speed of movement and its geometry. In three-dimensional movements, this relationship is defined by the one-sixth power law that relates between the speed, the curvature, and the torsion of movement trajectories. We fitted the parameters of the one-sixth power law to suturing movements of participants with different levels of surgical experience in open (using sensorized forceps) and teleoperated (using the da Vinci Research Kit / da Vinci Surgical System) conditions from two different datasets. We found that teleoperation significantly affected the parameters of the power law, and that there were large differences between different stages of movement. These results open a new avenue for studying the effect of teleoperation on the spatiotemporal characteristics of the movements of surgeons, and lay the foundation for the development of new algorithms for automatic segmentation of surgical tasks.

cs.RO↗