arXiv subjects

Francisco J. Valero-Cuevas

Publications and source records attributed to Francisco J. Valero-Cuevas.

11 recordsLinked to original sources

arXiv2025

Hardware Implementation of a Zero-Prior-Knowledge Approach to Lifelong Learning in Kinematic Control of Tendon-Driven Quadrupeds

Like mammals, robots must rapidly learn to control their bodies and interact with their environment despite incomplete knowledge of their body structure and surroundings. They must also adapt to continuous changes in both. This work presents a bio-inspired learning algorithm, General-to-Particular (G2P), applied to a tendon-driven quadruped robotic system developed and fabricated in-house. Our quadruped robot undergoes an initial five-minute phase of generalized motor babbling, followed by 15 refinement trials (each lasting 20 seconds) to achieve specific cyclical movements. This process mirrors the exploration-exploitation paradigm observed in mammals. With each refinement, the robot progressively improves upon its initial "good enough" solution. Our results serve as a proof-of-concept, demonstrating the hardware-in-the-loop system's ability to learn the control of a tendon-driven quadruped with redundancies in just a few minutes to achieve functional and adaptive cyclical non-convex movements. By advancing autonomous control in robotic locomotion, our approach paves the way for robots capable of dynamically adjusting to new environments, ensuring sustained adaptability and performance.

cs.RO↗

arXiv2024

Brain-Body-Task Co-Adaptation can Improve Autonomous Learning and Speed of Bipedal Walking

Inspired by animals that co-adapt their brain and body to interact with the environment, we present a tendon-driven and over-actuated (i.e., n joint, n+1 actuators) bipedal robot that (i) exploits its backdrivable mechanical properties to manage body-environment interactions without explicit control, and (ii) uses a simple 3-layer neural network to learn to walk after only 2 minutes of 'natural' motor babbling (i.e., an exploration strategy that is compatible with leg and task dynamics; akin to childsplay). This brain-body collaboration first learns to produce feet cyclical movements 'in air' and, without further tuning, can produce locomotion when the biped is lowered to be in slight contact with the ground. In contrast, training with 2 minutes of 'naive' motor babbling (i.e., an exploration strategy that ignores leg task dynamics), does not produce consistent cyclical movements 'in air', and produces erratic movements and no locomotion when in slight contact with the ground. When further lowering the biped and making the desired leg trajectories reach 1cm below ground (causing the desired-vs-obtained trajectories error to be unavoidable), cyclical movements based on either natural or naive babbling presented almost equally persistent trends, and locomotion emerged with naive babbling. Therefore, we show how continual learning of walking in unforeseen circumstances can be driven by continual physical adaptation rooted in the backdrivable properties of the plant and enhanced by exploration strategies that exploit plant dynamics. Our studies also demonstrate that the bio-inspired codesign and co-adaptations of limbs and control strategies can produce locomotion without explicit control of trajectory errors.

cs.RO↗

arXiv2020

The utility of tactile force to autonomous learning of in-hand manipulation is task-dependent

Tactile sensors provide information that can be used to learn and execute manipulation tasks. Different tasks, however, might require different levels of sensory information; which in turn likely affect learning rates and performance. This paper evaluates the role of tactile information on autonomous learning of manipulation with a simulated 3-finger tendon-driven hand. We compare the ability of the same learning algorithm (Proximal Policy Optimization, PPO) to learn two manipulation tasks (rolling a ball about the horizontal axis with and without rotational stiffness) with three levels of tactile sensing: no sensing, 1D normal force, and 3D force vector. Surprisingly, and contrary to recent work on manipulation, adding 1D force-sensing did not always improve learning rates compared to no sensing---likely due to whether or not normal force is relevant to the task. Nonetheless, even though 3D force-sensing increases the dimensionality of the sensory input---which would in general hamper algorithm convergence---it resulted in faster learning rates and better performance. We conclude that, in general, sensory input is useful to learning only when it is relevant to the task---as is the case of 3D force-sensing for in-hand manipulation against gravity. Moreover, the utility of 3D force-sensing can even offset the added computational cost of learning with higher-dimensional sensory input.

cs.RO↗

arXiv2019

Simple Kinematic Feedback Enhances Autonomous Learning in Bio-Inspired Tendon-Driven Systems

Error feedback is known to improve performance by correcting control signals in response to perturbations. Here we show how adding simple error feedback can also accelerate and robustify autonomous learning in a tendon-driven robot. We implemented two versions of the General-to-Particular (G2P) autonomous learning algorithm to produce multiple movement tasks using a tendon-driven leg with two joints and three tendons: one with and one without kinematic feedback. As expected, feedback improved performance in simulation and hardware. However, we see these improvements even in the presence of sensory delays of up to 100 ms and when experiencing substantial contact collisions. Importantly, feedback accelerates learning and enhances G2P's continual refinement of the initial inverse map by providing the system with more relevant data to train on. This allows the system to perform well even after only 60 seconds of initial motor babbling.

cs.RO↗

arXiv2019

Autonomous Control of a Tendon-driven Robotic Limb with Elastic Elements Reveals that Added Elasticity can Enhance Learning

Passive elastic elements can contribute to stability, energetic efficiency, and impact absorption in both biological and robotic systems. They also add dynamical complexity which makes them more challenging to model and control. The impact of this added complexity to autonomous learning has not been thoroughly explored. This is especially relevant to tendon-driven limbs whose cables and tendons are inevitably elastic. Here, we explored the efficacy of autonomous learning and control on a simulated bio-plausible tendon-driven leg across different tendon stiffness values. We demonstrate that increasing stiffness of the simulated muscles can require more iterations for the inverse map to converge but can then perform more accurately, especially in discrete tasks. Moreover, the system is robust to subsequent changes in muscle stiffnesses and can adapt on-the-go within 5 attempts. Lastly, we test the system for the functional task of locomotion, and found similar effects of muscle stiffness to learning and performance. Given that a range of stiffness values led to improved learning and maximized performance, we conclude the robot bodies and autonomous controllers---at least for tendon-driven systems---can be co-developed to take advantage of elastic elements. Importantly, this opens also the door to development efforts that recapitulate the beneficial aspects of the co-evolution of brains and bodies in vertebrates.

cs.RO↗

arXiv2018

Autonomous Functional Locomotion in a Tendon-Driven Limb via Limited Experience

Robots will become ubiquitously useful only when they can use few attempts to teach themselves to perform different tasks, even with complex bodies and in dynamical environments. Vertebrates, in fact, successfully use trial-and-error to learn multiple tasks in spite of their intricate tendon-driven anatomies. Roboticists find such tendon-driven systems particularly hard to control because they are simultaneously nonlinear, under-determined (many tendon tensions combine to produce few net joint torques), and over-determined (few joint rotations define how many tendons need to be reeled-in/payed-out). We demonstrate---for the first time in simulation and in hardware---how a model-free approach allows few-shot autonomous learning to produce effective locomotion in a 3-tendon/2-joint tendon-driven leg. Initially, an artificial neural network fed by sparsely sampled data collected using motor babbling creates an inverse map from limb kinematics to motor activations, which is analogous to juvenile vertebrates playing during development. Thereafter, iterative reward-driven exploration of candidate motor activations simultaneously refines the inverse map and finds a functional locomotor limit-cycle autonomously. This biologically-inspired algorithm, which we call G2P (General to Particular), enables versatile adaptation of robots to changes in the target task, mechanics of their bodies, and environment. Moreover, this work empowers future studies of few-shot autonomous learning in biological systems, which is the foundation of their enviable functional versatility.

cs.RO↗

arXiv2018

Quantifying and attenuating pathologic tremor in virtual reality

We present a virtual reality (VR) experience that creates a research-grade benchmark in assessing patients with active upper-limb tremor, while simultaneously offering the opportunity for patients to engage with VR experiences without their pathologic tremor. Accurate and precise use of handheld motion controllers in VR gaming applications may be limited for patients with upper limb tremor. In parallel, objective tools measuring tremor are not in widespread, routine clinical use. We used a commercially available VR system and designed a challenging virtual-balloon-popping test mimicking a common nose-to-target pointing task used by medical practitioners to subjectively evaluate tremor in the exam room. Within our VR experience, we offer a software mode which uses a low-pass filter to adjust hand position and pointing orientation over a series of past data points. This digital filter creates a smoothing function for hand movement which effectively removes the patient's tremor in the VR representation. While the patient completes trials of the reaching task, quantitative data on the pathologic tremor is digitally recorded. With speed, accuracy, and the tremor components computed across three axes of movement, patients can be evaluated for their tremor amplitudes in a quantitative, replicable, and enjoyable manner. Removal of tremor in digital space may allow patients having significant upper limb tremor to have both an objective clinical measurement of symptoms while providing patients positive feedback and interaction.

q-bio.QM↗

arXiv2017

Physiological Tremor Increases when Skeletal Muscle is Shortened: Implications for Fusimotor Control

The involuntary force fluctuations associated with physiological (as distinct from pathological) tremor are an unavoidable component of human motor control. While the origins of the physiological tremor are known to depend on muscle afferentation, it is possible that the mechanical properties of muscle-tendon systems also affect its generation, amplification and maintenance. In this paper, we investigated the dependence of physiological tremor during tonic, isometric plantarflextion torque at 30% of maximum at three ankle angles. The amplitude of physiological tremor increased as calf muscles shortened in contrast to the stretch reflex whose amplitude decreases as muscle shortens. We used a closed-loop simulation model of afferented muscle to explore the mechanisms responsible for this behavior. We demonstrate that changing muscle lengths does not suffice to explain our experimental findings. Rather, the model consistently required the modulation of gamme-static fusimotor drive to produce increases in physiological tremor with muscle shortening--while successfully replicating the concomitant reduction in stretch reflex amplitude. This need to control gamma-static fusimotor drive explicitly as a function of muscle length has important implication. First, it permits the amplitudes of physiological tremor and stretch reflex to be decoupled. Second, it postulates neuromechanical interaction that require length-dependent gamma drive modulation to be independent from alpha drive to the parent muscle. Lastly, it suggests that physiological tremor can be used as a simple, non-invasive measure of the afferent mechanisms underlying healthy motor function, and their disruption in neurological conditions.

q-bio.NC↗

arXiv2017

Shapechanger: Environments for Transfer Learning

We present Shapechanger, a library for transfer reinforcement learning specifically designed for robotic tasks. We consider three types of knowledge transfer---from simulation to simulation, from simulation to real, and from real to real---and a wide range of tasks with continuous states and actions. Shapechanger is under active development and open-sourced at: https://github.com/seba-1511/shapechanger/.

cs.LG↗

arXiv2015

Slow Limb Movements Require Precise Control of Muscle Stiffness

Slow and accurate finger and limb movements are essential to daily activities, but their neural control and governing mechanics are relatively unexplored. We consider neuromechanical systems where slow movements are produced by neural commands that modulate muscle stiffness. This formulation based on strain-energy equilibria is in agreement with prior work on neural control of muscle and limb impedance. Slow limb movements are driftless in the sense that movement stops when neural commands stop. We demonstrate, in the context of two planar tendon-driven systems representing a finger and a leg, that the control of muscle stiffness suffices to produce stable and accurate limb postures and quasi-static (slow) transitions among them. We prove, however, that stable postures are achievable only when muscles are pre-tensioned, as is the case for natural muscle tone. Our results further indicate, in accordance with experimental findings, that slow movements are non-smooth. The non-smoothness arises because the precision with which individual muscle stiffnesses need to be controlled changes substantially throughout the limb's motion. These results underscore the fundamental roles of muscle tone and accurate neural control of muscle stiffness in producing stable limb postures and slow movements.

q-bio.TO↗

arXiv2006

Estimating degrees of freedom in motor systems

Studies of the degrees of freedom or "synergies" in musculoskeletal systems rely critically on algorithms to estimate the "dimension" of kinematic or neural data. Linear algorithms such as principal component analysis (PCA) are used almost exclusively for this purpose. However, biological systems tend to possess nonlinearities and operate at multiple spatial and temporal scales so that the set of reachable system states typically does not lie close to a single linear subspace. We compare the performance of PCA to two alternative nonlinear algorithms (Isomap and our novel pointwise dimension estimation (PD-E)) using synthetic and motion capture data from a robotic arm with known kinematic dimensions, as well as motion capture data from human hands. We find that consideration of the spectral properties of the singular value decomposition in PCA can lead to more accurate dimension estimates than the dominant practice of using a fixed variance capture threshold. We investigate methods for identifying a single integer dimension using PCA and Isomap. In contrast, PD-E provides a range of estimates of fractal dimension. This helps to identify heterogeneous geometric structure of data sets such as unions of manifolds of differing dimensions, to which Isomap is less sensitive. Contrary to common opinion regarding fractal dimension methods, PD-E yielded reasonable results with reasonable amounts of data. We conclude that it is necessary and feasible to complement PCA with other methods that take into consideration the nonlinear properties of biological systems for a more robust estimation of their degrees of freedom.

q-bio.QM↗