arXiv subjects

Arndt F. Schilling

Publications and source records attributed to Arndt F. Schilling.

5 recordsLinked to original sources

arXiv2025

Continual Learning from Simulated Interactions via Multitask Prospective Rehearsal for Bionic Limb Behavior Modeling

Lower limb amputations and neuromuscular impairments severely restrict mobility, necessitating advancements beyond conventional prosthetics. While motorized bionic limbs show promise, their effectiveness depends on replicating the dynamic coordination of human movement across diverse environments. In this paper, we introduce a model for human behavior in the context of bionic prosthesis control. Our approach leverages human locomotion demonstrations to learn the synergistic coupling of the lower limbs, enabling the prediction of the kinematic behavior of a missing limb during tasks such as walking, climbing inclines, and stairs. We propose a multitasking, continually adaptive model that anticipates and refines movements over time. At the core of our method is a technique called multitask prospective rehearsal, that anticipates and synthesizes future movements based on the previous prediction and employs a corrective mechanism for subsequent predictions. Our evolving architecture merges lightweight, task-specific modules on a shared backbone, ensuring both specificity and scalability. We validate our model through experiments on real-world human gait datasets, including transtibial amputees, across a wide range of locomotion tasks. Results demonstrate that our approach consistently outperforms baseline models, particularly in scenarios with distributional shifts, adversarial perturbations, and noise.

cs.LG↗

arXiv2024

Embracing Large Language and Multimodal Models for Prosthetic Technologies

This article presents a vision for the future of prosthetic devices, leveraging the advancements in large language models (LLMs) and Large Multimodal Models (LMMs) to revolutionize the interaction between humans and assistive technologies. Unlike traditional prostheses, which rely on limited and predefined commands, this approach aims to develop intelligent prostheses that understand and respond to users' needs through natural language and multimodal inputs. The realization of this vision involves developing a control system capable of understanding and translating a wide array of natural language and multimodal inputs into actionable commands for prosthetic devices. This includes the creation of models that can extract and interpret features from both textual and multimodal data, ensuring devices not only follow user commands but also respond intelligently to the environment and user intent, thus marking a significant leap forward in prosthetic technology.

cs.RO↗

arXiv2022

PrePARE: Predictive Proprioception for Agile Failure Event Detection in Robotic Exploration of Extreme Terrains

Legged robots can traverse a wide variety of terrains, some of which may be challenging for wheeled robots, such as stairs or highly uneven surfaces. However, quadruped robots face stability challenges on slippery surfaces. This can be resolved by adjusting the robot's locomotion by switching to more conservative and stable locomotion modes, such as crawl mode (where three feet are in contact with the ground always) or amble mode (where one foot touches down at a time) to prevent potential falls. To tackle these challenges, we propose an approach to learn a model from past robot experience for predictive detection of potential failures. Accordingly, we trigger gait switching merely based on proprioceptive sensory information. To learn this predictive model, we propose a semi-supervised process for detecting and annotating ground truth slip events in two stages: We first detect abnormal occurrences in the time series sequences of the gait data using an unsupervised anomaly detector, and then, the anomalies are verified with expert human knowledge in a replay simulation to assert the event of a slip. These annotated slip events are then used as ground truth examples to train an ensemble decision learner for predicting slip probabilities across terrains for traversability. We analyze our model on data recorded by a legged robot on multiple sites with slippery terrain. We demonstrate that a potential slip event can be predicted up to 720 ms ahead of a potential fall with an average precision greater than 0.95 and an average F-score of 0.82. Finally, we validate our approach in real-time by deploying it on a legged robot and switching its gait mode based on slip event detection.

cs.RO↗

arXiv2021

Learning a Shared Model for Motorized Prosthetic Joints to Predict Ankle-Joint Motion

Control strategies for active prostheses or orthoses use sensor inputs to recognize the user's locomotive intention and generate corresponding control commands for producing the desired locomotion. In this paper, we propose a learning-based shared model for predicting ankle-joint motion for different locomotion modes like level-ground walking, stair ascent, stair descent, slope ascent, and slope descent without the need to classify between them. Features extracted from hip and knee joint angular motion are used to continuously predict the ankle angles and moments using a Feed-Forward Neural Network-based shared model. We show that the shared model is adequate for predicting the ankle angles and moments for different locomotion modes without explicitly classifying between the modes. The proposed strategy shows the potential for devising a high-level controller for an intelligent prosthetic ankle that can adapt to different locomotion modes.

cs.RO↗

arXiv2021

A hybrid approach for dynamically training a torque prediction model for devising a human-machine interface control strategy

Human-machine interfaces (HMI) play a pivotal role in the rehabilitation and daily assistance of lower-limb amputees. The brain of such interfaces is a control model that detects the user's intention using sensor input and generates corresponding output (control commands). With recent advances in technology, AI-based policies have gained attention as control models for HMIs. However, supervised learning techniques require affluent amounts of labeled training data from the user, which is challenging in the context of lower-limb rehabilitation. Moreover, a static pre-trained model does not take the temporal variations in the motion of the amputee (e.g., due to speed, terrain) into account. In this study, we aimed to address both of these issues by creating an incremental training approach for a torque prediction model using incomplete user-specific training data and biologically inspired temporal patterns of human gait. To reach this goal, we created a hybrid of two distinct approaches, a generic inter-individual and an adapting individual-specific model that exploits the inter-limb synergistic coupling during human gait to learn a function that predicts the torque at the ankle joint continuously based on the kinematic sequences of the hip, knee, and shank. An inter-individual generic base model learns temporal patterns of gait from a set of able-bodied individuals and predicts the gait patterns for a new individual, while the individual-specific adaptation model learns and predicts the temporal patterns of gait specific to a particular individual. The iterative training using the hybrid model was validated on eight able-bodied and five transtibial amputee subjects. It was found that, with the addition of estimators fitted to individual-specific data, the accuracy significantly increased from the baseline inter-individual model and plateaued within two to three iterations.

cs.CE↗