arXiv subjects

Matei Ciocarlie

Publications and source records attributed to Matei Ciocarlie.

At least 37 records · Page 2Linked to original sources

arXiv2024

An Investigation of Multi-feature Extraction and Super-resolution with Fast Microphone Arrays

In this work, we use MEMS microphones as vibration sensors to simultaneously classify texture and estimate contact position and velocity. Vibration sensors are an important facet of both human and robotic tactile sensing, providing fast detection of contact and onset of slip. Microphones are an attractive option for implementing vibration sensing as they offer a fast response and can be sampled quickly, are affordable, and occupy a very small footprint. Our prototype sensor uses only a sparse array (8-9 mm spacing) of distributed MEMS microphones (<$1, 3.76 x 2.95 x 1.10 mm) embedded under an elastomer. We use transformer-based architectures for data analysis, taking advantage of the microphones' high sampling rate to run our models on time-series data as opposed to individual snapshots. This approach allows us to obtain 77.3% average accuracy on 4-class texture classification (84.2% when excluding the slowest drag velocity), 1.8 mm mean error on contact localization, and 5.6 mm/s mean error on contact velocity. We show that the learned texture and localization models are robust to varying velocity and generalize to unseen velocities. We also report that our sensor provides fast contact detection, an important advantage of fast transducers. This investigation illustrates the capabilities one can achieve with a MEMS microphone array alone, leaving valuable sensor real estate available for integration with complementary tactile sensing modalities.

cs.RO↗

arXiv2024

Grasp Force Assistance via Throttle-based Wrist Angle Control on a Robotic Hand Orthosis for C6-C7 Spinal Cord Injury

Individuals with hand paralysis resulting from C6-C7 spinal cord injuries frequently rely on tenodesis for grasping. However, tenodesis generates limited grasping force and demands constant exertion to maintain a grasp, leading to fatigue and sometimes pain. We introduce the MyHand-SCI, a wearable robot that provides grasping assistance through motorized exotendons. Our user-driven device enables independent, ipsilateral operation via a novel Throttle-based Wrist Angle control method, which allows users to maintain grasps without continued wrist extension. A pilot case study with a person with C6 spinal cord injury shows an improvement in functional grasping and grasping force, as well as a preserved ability to modulate grasping force while using our device, thus improving their ability to manipulate everyday objects. This research is a step towards developing effective and intuitive wearable assistive devices for individuals with spinal cord injury.

cs.RO↗

arXiv2024

R$\times$R: Rapid eXploration for Reinforcement Learning via Sampling-based Reset Distributions and Imitation Pre-training

We present a method for enabling Reinforcement Learning of motor control policies for complex skills such as dexterous manipulation. We posit that a key difficulty for training such policies is the difficulty of exploring the problem state space, as the accessible and useful regions of this space form a complex structure along manifolds of the original high-dimensional state space. This work presents a method to enable and support exploration with Sampling-based Planning. We use a generally applicable non-holonomic Rapidly-exploring Random Trees algorithm and present multiple methods to use the resulting structure to bootstrap model-free Reinforcement Learning. Our method is effective at learning various challenging dexterous motor control skills of higher difficulty than previously shown. In particular, we achieve dexterous in-hand manipulation of complex objects while simultaneously securing the object without the use of passive support surfaces. These policies also transfer effectively to real robots. A number of example videos can also be found on the project website: https://sbrl.cs.columbia.edu

cs.RO↗

arXiv2023

Towards Tenodesis-Modulated Control of an Assistive Hand Exoskeleton for SCI

Restoration of hand function is one of the highest priorities for SCI populations. In this work, we present a prototype of a robotic assistive orthosis capable of implementing tenodesis user control. The underactuated device provides active grasping assistance while preserving free wrist mobility through the use of Bowden cables. This device enables force modulation during grasping, which was effectively leveraged by a participant with C6 SCI to demonstrate improved grasping abilities using the orthosis, scoring 11 on the Grasp and Release Test using the device compared to 1 without it.

cs.RO↗

arXiv2023

MORPH: Design Co-optimization with Reinforcement Learning via a Differentiable Hardware Model Proxy

We introduce MORPH, a method for co-optimization of hardware design parameters and control policies in simulation using reinforcement learning. Like most co-optimization methods, MORPH relies on a model of the hardware being optimized, usually simulated based on the laws of physics. However, such a model is often difficult to integrate into an effective optimization routine. To address this, we introduce a proxy hardware model, which is always differentiable and enables efficient co-optimization alongside a long-horizon control policy using RL. MORPH is designed to ensure that the optimized hardware proxy remains as close as possible to its realistic counterpart, while still enabling task completion. We demonstrate our approach on simulated 2D reaching and 3D multi-fingered manipulation tasks.

cs.RO↗

arXiv2023

Dexterous In-hand Manipulation by Guiding Exploration with Simple Sub-skill Controllers

Recently, reinforcement learning has led to dexterous manipulation skills of increasing complexity. Nonetheless, learning these skills in simulation still exhibits poor sample-efficiency which stems from the fact these skills are learned from scratch without the benefit of any domain expertise. In this work, we aim to improve the sample efficiency of learning dexterous in-hand manipulation skills using controllers available via domain knowledge. To this end, we design simple sub-skill controllers and demonstrate improved sample efficiency using a framework that guides exploration toward relevant state space by following actions from these controllers. We are the first to demonstrate learning hard-to-explore finger-gaiting in-hand manipulation skills without the use of an exploratory reset distribution. Video results can be found at https://roamlab.github.io/vge

cs.RO↗

arXiv2023

Sampling-based Exploration for Reinforcement Learning of Dexterous Manipulation

In this paper, we present a novel method for achieving dexterous manipulation of complex objects, while simultaneously securing the object without the use of passive support surfaces. We posit that a key difficulty for training such policies in a Reinforcement Learning framework is the difficulty of exploring the problem state space, as the accessible regions of this space form a complex structure along manifolds of a high-dimensional space. To address this challenge, we use two versions of the non-holonomic Rapidly-Exploring Random Trees algorithm; one version is more general, but requires explicit use of the environment's transition function, while the second version uses manipulation-specific kinematic constraints to attain better sample efficiency. In both cases, we use states found via sampling-based exploration to generate reset distributions that enable training control policies under full dynamic constraints via model-free Reinforcement Learning. We show that these policies are effective at manipulation problems of higher difficulty than previously shown, and also transfer effectively to real robots. Videos of the real-hand demonstrations can be found on the project website: https://sbrl.cs.columbia.edu/

cs.RO↗

arXiv2023

Decision Making for Human-in-the-loop Robotic Agents via Uncertainty-Aware Reinforcement Learning

In a Human-in-the-Loop paradigm, a robotic agent is able to act mostly autonomously in solving a task, but can request help from an external expert when needed. However, knowing when to request such assistance is critical: too few requests can lead to the robot making mistakes, but too many requests can overload the expert. In this paper, we present a Reinforcement Learning based approach to this problem, where a semi-autonomous agent asks for external assistance when it has low confidence in the eventual success of the task. The confidence level is computed by estimating the variance of the return from the current state. We show that this estimate can be iteratively improved during training using a Bellman-like recursion. On discrete navigation problems with both fully- and partially-observable state information, we show that our method makes effective use of a limited budget of expert calls at run-time, despite having no access to the expert at training time.

cs.RO↗

arXiv2023

TANDEM3D: Active Tactile Exploration for 3D Object Recognition

Tactile recognition of 3D objects remains a challenging task. Compared to 2D shapes, the complex geometry of 3D surfaces requires richer tactile signals, more dexterous actions, and more advanced encoding techniques. In this work, we propose TANDEM3D, a method that applies a co-training framework for exploration and decision making to 3D object recognition with tactile signals. Starting with our previous work, which introduced a co-training paradigm for 2D recognition problems, we introduce a number of advances that enable us to scale up to 3D. TANDEM3D is based on a novel encoder that builds 3D object representation from contact positions and normals using PointNet++. Furthermore, by enabling 6DOF movement, TANDEM3D explores and collects discriminative touch information with high efficiency. Our method is trained entirely in simulation and validated with real-world experiments. Compared to state-of-the-art baselines, TANDEM3D achieves higher accuracy and a lower number of actions in recognizing 3D objects and is also shown to be more robust to different types and amounts of sensor noise. Video is available at https://jxu.ai/tandem3d.

cs.CV↗

arXiv2022

TANDEM: Learning Joint Exploration and Decision Making with Tactile Sensors

Inspired by the human ability to perform complex manipulation in the complete absence of vision (like retrieving an object from a pocket), the robotic manipulation field is motivated to develop new methods for tactile-based object interaction. However, tactile sensing presents the challenge of being an active sensing modality: a touch sensor provides sparse, local data, and must be used in conjunction with effective exploration strategies in order to collect information. In this work, we focus on the process of guiding tactile exploration, and its interplay with task-related decision making. We propose TANDEM (TActile exploration aNd DEcision Making), an architecture to learn efficient exploration strategies in conjunction with decision making. Our approach is based on separate but co-trained modules for exploration and discrimination. We demonstrate this method on a tactile object recognition task, where a robot equipped with a touch sensor must explore and identify an object from a known set based on binary contact signals alone. TANDEM achieves higher accuracy with fewer actions than alternative methods and is also shown to be more robust to sensor noise.

cs.RO↗

arXiv2022

Discovering Synergies for Robot Manipulation with Multi-Task Reinforcement Learning

Controlling robotic manipulators with high-dimensional action spaces for dexterous tasks is a challenging problem. Inspired by human manipulation, researchers have studied generating and using postural synergies for robot hands to accomplish manipulation tasks, leveraging the lower dimensional nature of synergistic action spaces. However, many of these works require pre-collected data from an existing controller in order to derive such a subspace by means of dimensionality reduction. In this paper, we present a framework that simultaneously discovers a synergy space and a multi-task policy that operates on this low-dimensional action space to accomplish diverse manipulation tasks. We demonstrate that our end-to-end method is able to perform multiple tasks using few synergies, and outperforms sequential methods that apply dimensionality reduction to independently collected data. We also show that deriving synergies using multiple tasks can lead to a subspace that enables robots to efficiently learn new manipulation tasks and interactions with new objects.

cs.RO↗

arXiv2022

Design of Spiral-Cable Forearm Exoskeleton to Assist Supination for Hemiparetic Stroke Subjects

We present the development of a cable-based passive forearm exoskeleton that is designed to assist supination for hemiparetic stroke survivors. Our device uniquely provides torque sufficient for counteracting spasticity within a below-elbow apparatus. The mechanism consists of a spiral single-tendon routing embedded in a rigid forearm brace and terminated at the hand and upper-forearm. A spool with an internal releasable-ratchet mechanism allows the user to manually retract the tendon and rotate the hand to counteract involuntary pronation synergies due to stroke. We characterize the mechanism with benchtop testing and five healthy subjects, and perform a preliminary assessment of the exoskeleton with a single chronic stroke subject having minimal supination ability. The mechanism can be integrated into an existing active hand-opening orthosis to enable supination support during grasping tasks, and also allows for a future actuated supination strategy.

cs.RO↗

arXiv2022

Thumb Stabilization and Assistance in a Robotic Hand Orthosis for Post-Stroke Hemiparesis

We propose a dual-cable method of stabilizing the thumb in the context of a hand orthosis designed for individuals with upper extremity hemiparesis after stroke. This cable network adds opposition/reposition capabilities to the thumb, and increases the likelihood of forming a hand pose that can successfully manipulate objects. In addition to a passive-thumb version (where both cables are of fixed length), our approach also allows for a single-actuator active-thumb version (where the extension cable is actuated while the abductor remains passive), which allows a range of motion intended to facilitate creating and maintaining grasps. We performed experiments with five chronic stroke survivors consisting of unimanual resistive-pull tasks and bimanual twisting tasks with simulated real-world objects; these explored the effects of thumb assistance on grasp stability and functional range of motion. Our results show that both active- and passive-thumb versions achieved similar performance in terms of improving grasp force generation over a no-device baseline, but active thumb stabilization enabled users to maintain grasps for longer durations.

cs.RO↗

arXiv2022

On the Feasibility of Learning Finger-gaiting In-hand Manipulation with Intrinsic Sensing

Finger-gaiting manipulation is an important skill to achieve large-angle in-hand re-orientation of objects. However, achieving these gaits with arbitrary orientations of the hand is challenging due to the unstable nature of the task. In this work, we use model-free reinforcement learning (RL) to learn finger-gaiting only via precision grasps and demonstrate finger-gaiting for rotation about an axis purely using on-board proprioceptive and tactile feedback. To tackle the inherent instability of precision grasping, we propose the use of initial state distributions that enable effective exploration of the state space. Our method can learn finger-gaiting with significantly improved sample complexity than the state-of-the-art. The policies we obtain are robust and also transfer to novel objects. Videos can be found at https://roamlab.github.io/learnfg/

cs.RO↗

arXiv2022

Adaptive Semi-Supervised Intent Inferral to Control a Powered Hand Orthosis for Stroke

In order to provide therapy in a functional context, controls for wearable robotic orthoses need to be robust and intuitive. We have previously introduced an intuitive, user-driven, EMG-based method to operate a robotic hand orthosis, but the process of training a control that is robust to concept drift (changes in the input signal) places a substantial burden on the user. In this paper, we explore semi-supervised learning as a paradigm for controlling a powered hand orthosis for stroke subjects. To the best of our knowledge, this is the first use of semi-supervised learning for an orthotic application. Specifically, we propose a disagreement-based semi-supervision algorithm for handling intrasession concept drift based on multimodal ipsilateral sensing. We evaluate the performance of our algorithm on data collected from five stroke subjects. Our results show that the proposed algorithm helps the device adapt to intrasession drift using unlabeled data and reduces the training burden placed on the user. We also validate the feasibility of our proposed algorithm with a functional task; in these experiments, two subjects successfully completed multiple instances of a pick-and-handover task.

cs.RO↗

arXiv2021

Design Paradigms Based on Spring Agonists for Underactuated Robot Hands: Concepts and Application

In this paper, we focus on a rarely used paradigm in the design of underactuated robot hands: the use of springs as agonists and tendons as antagonists. We formalize this approach in a design matrix also considering its interplay with the underactuation method used (one tendon for multiple joints vs. multiple tendons on one motor shaft). We then show how different cells in this design matrix can be combined in order to facilitate the implementation of desired postural synergies with a single motor. Furthermore, we show that when agonist and antagonist tendons are combined on the same motor shaft, the resulting spring force cancellation can be leveraged to produce multiple desirable behaviors, which we demonstrate in a physical prototype.

cs.RO↗

arXiv2020

Hardware as Policy: Mechanical and Computational Co-Optimization using Deep Reinforcement Learning

Deep Reinforcement Learning (RL) has shown great success in learning complex control policies for a variety of applications in robotics. However, in most such cases, the hardware of the robot has been considered immutable, modeled as part of the environment. In this study, we explore the problem of learning hardware and control parameters together in a unified RL framework. To achieve this, we propose to model the robot body as a "hardware policy", analogous to and optimized jointly with its computational counterpart. We show that, by modeling such hardware policies as auto-differentiable computational graphs, the ensuing optimization problem can be solved efficiently by gradient-based algorithms from the Policy Optimization family. We present two such design examples: a toy mass-spring problem, and a real-world problem of designing an underactuated hand. We compare our method against traditional co-optimization approaches, and also demonstrate its effectiveness by building a physical prototype based on the learned hardware parameters. Videos and more details are available at https://roamlab.github.io/hwasp/ .

cs.RO↗

arXiv2020

A Continuous Teleoperation Subspace with Empirical and Algorithmic Mapping Algorithms for Non-Anthropomorphic Hands

Teleoperation is a valuable tool for robotic manipulators in highly unstructured environments. However, finding an intuitive mapping between a human hand and a non-anthropomorphic robot hand can be difficult, due to the hands' dissimilar kinematics. In this paper, we seek to create a mapping between the human hand and a fully actuated, non-anthropomorphic robot hand that is intuitive enough to enable effective real-time teleoperation, even for novice users. To accomplish this, we propose a low-dimensional teleoperation subspace which can be used as an intermediary for mapping between hand pose spaces. We present two different methods to define the teleoperation subspace: an empirical definition, which requires a person to define hand motions in an intuitive, hand-specific way, and an algorithmic definition, which is kinematically independent, and uses objects to define the subspace. We use each of these definitions to create a teleoperation mapping for different hands. One of the main contributions of this paper is the validation of both the empirical and algorithmic mappings with teleoperation experiments controlled by ten novices and performed on two kinematically distinct hands. The experiments show that the proposed subspace is relevant to teleoperation, intuitive enough to enable control by novices, and can generalize to non-anthropomorphic hands with different kinematics.

cs.RO↗