arXiv subjects

Tapomayukh Bhattacharjee

Publications and source records attributed to Tapomayukh Bhattacharjee.

47 records · Page 3Linked to original sources

arXiv2021

Leveraging Post Hoc Context for Faster Learning in Bandit Settings with Applications in Robot-Assisted Feeding

Autonomous robot-assisted feeding requires the ability to acquire a wide variety of food items. However, it is impossible for such a system to be trained on all types of food in existence. Therefore, a key challenge is choosing a manipulation strategy for a previously unseen food item. Previous work showed that the problem can be represented as a linear bandit with visual context. However, food has a wide variety of multi-modal properties relevant to manipulation that can be hard to distinguish visually. Our key insight is that we can leverage the haptic context we collect during and after manipulation (i.e., "post hoc") to learn some of these properties and more quickly adapt our visual model to previously unseen food. In general, we propose a modified linear contextual bandit framework augmented with post hoc context observed after action selection to empirically increase learning speed and reduce cumulative regret. Experiments on synthetic data demonstrate that this effect is more pronounced when the dimensionality of the context is large relative to the post hoc context or when the post hoc context model is particularly easy to learn. Finally, we apply this framework to the bite acquisition problem and demonstrate the acquisition of 8 previously unseen types of food with 21% fewer failures across 64 attempts.

cs.RO↗

arXiv2020

Material Recognition via Heat Transfer Given Ambiguous Initial Conditions

Humans and robots can recognize materials with distinct thermal effusivities by making physical contact and observing temperatures during heat transfer. This works well with room temperature materials and humans and robots at human body temperatures. Past research has shown that cooling or heating a material can result in temperatures that are similar to contact with another material. To thoroughly investigate this perceptual ambiguity, we designed a psychophysical experiment in which a participant discriminates between two materials given ambiguous initial conditions. We conducted a study with 32 human participants and a robot. Humans and the robot confused the materials. We also found that robots can overcome this ambiguity using two temperature sensors with different temperatures prior to contact. We support this conclusion based on a mathematical proof using a heat transfer model and empirical results in which a robot achieved 100% accuracy compared to 5% human accuracy. Our results also indicate that robots can use subtle cues to distinguish thermally ambiguous materials with a single temperature sensor. Overall, our work provides insights into challenging conditions for material recognition via heat transfer, and suggests methods by which robots can overcome these challenges to outperform humans.

cs.RO↗

arXiv2020

Grasping with Chopsticks: Combating Covariate Shift in Model-free Imitation Learning for Fine Manipulation

Billions of people use chopsticks, a simple yet versatile tool, for fine manipulation of everyday objects. The small, curved, and slippery tips of chopsticks pose a challenge for picking up small objects, making them a suitably complex test case. This paper leverages human demonstrations to develop an autonomous chopsticks-equipped robotic manipulator. Due to the lack of accurate models for fine manipulation, we explore model-free imitation learning, which traditionally suffers from the covariate shift phenomenon that causes poor generalization. We propose two approaches to reduce covariate shift, neither of which requires access to an interactive expert or a model, unlike previous approaches. First, we alleviate single-step prediction errors by applying an invariant operator to increase the data support at critical steps for grasping. Second, we generate synthetic corrective labels by adding bounded noise and combining parametric and non-parametric methods to prevent error accumulation. We demonstrate our methods on a real chopstick-equipped robot that we built, and observe the agent's success rate increase from 37.3% to 80%, which is comparable to the human expert performance of 82.6%.

cs.RO↗

arXiv2020

Adaptive Robot-Assisted Feeding: An Online Learning Framework for Acquiring Previously Unseen Food Items

A successful robot-assisted feeding system requires bite acquisition of a wide variety of food items. It must adapt to changing user food preferences under uncertain visual and physical environments. Different food items in different environmental conditions require different manipulation strategies for successful bite acquisition. Therefore, a key challenge is how to handle previously unseen food items with very different success rate distributions over strategy. Combining low-level controllers and planners into discrete action trajectories, we show that the problem can be represented using a linear contextual bandit setting. We construct a simulated environment using a doubly robust loss estimate from previously seen food items, which we use to tune the parameters of off-the-shelf contextual bandit algorithms. Finally, we demonstrate empirically on a robot-assisted feeding system that, even starting with a model trained on thousands of skewering attempts on dissimilar previously seen food items, $ε$-greedy and LinUCB algorithms can quickly converge to the most successful manipulation strategy.

cs.RO↗

arXiv2020

Telemanipulation with Chopsticks: Analyzing Human Factors in User Demonstrations

Chopsticks constitute a simple yet versatile tool that humans have used for thousands of years to perform a variety of challenging tasks ranging from food manipulation to surgery. Applying such a simple tool in a diverse repertoire of scenarios requires significant adaptability. Towards developing autonomous manipulators with comparable adaptability to humans, we study chopsticks-based manipulation to gain insights into human manipulation strategies. We conduct a within-subjects user study with 25 participants, evaluating three different data-collection methods: normal chopsticks, motion-captured chopsticks, and a novel chopstick telemanipulation interface. We analyze factors governing human performance across a variety of challenging chopstick-based grasping tasks. Although participants rated teleoperation as the least comfortable and most difficult-to-use method, teleoperation enabled users to achieve the highest success rates on three out of five objects considered. Further, we notice that subjects quickly learned and adapted to the teleoperation interface. Finally, while motion-captured chopsticks could provide a better reflection of how humans use chopsticks, the teleoperation interface can produce quality on-hardware demonstrations from which the robot can directly learn.

cs.RO↗

arXiv2019

Robot-Assisted Feeding: Generalizing Skewering Strategies across Food Items on a Realistic Plate

A robot-assisted feeding system must successfully acquire many different food items. A key challenge is the wide variation in the physical properties of food, demanding diverse acquisition strategies that are also capable of adapting to previously unseen items. Our key insight is that items with similar physical properties will exhibit similar success rates across an action space, allowing the robot to generalize its actions to previously unseen items. To better understand which skewering strategy works best for each food item, we collected a dataset of 2450 robot bite acquisition trials for 16 food items with varying properties. Analyzing the dataset provided insights into how the food items' surrounding environment, fork pitch, and fork roll angles affect bite acquisition success. We then developed a bite acquisition framework that takes the image of a full plate as an input, segments it into food items, and then applies our Skewering-Position-Action network (SPANet) to choose a target food item and a corresponding action so that the bite acquisition success rate is maximized. SPANet also uses the surrounding environment features of food items to predict action success rates. We used this framework to perform multiple experiments on uncluttered and cluttered plates. Results indicate that our integrated system can successfully generalize skewering strategies to many previously unseen food items.

cs.RO↗

arXiv2019

Towards Robotic Feeding: Role of Haptics in Fork-based Food Manipulation

Autonomous feeding is challenging because it requires manipulation of food items with various compliance, sizes, and shapes. To understand how humans manipulate food items during feeding and to explore ways to adapt their strategies to robots, we collected a rich dataset of human trajectories by asking them to pick up food and feed it to a mannequin. From the analysis of the collected haptic and motion signals, we demonstrate that humans adapt their control policies to accommodate to the compliance and shape of the food item being acquired. We propose a taxonomy of manipulation strategies for feeding to highlight such policies. As a first step to generate compliance-dependent policies, we propose a set of classifiers for compliance-based food categorization from haptic and motion signals. We compare these human manipulation strategies with fixed position-control policies via a robot. Our analysis of success and failure cases of human and robot policies further highlights the importance of adapting the policy to the compliance of a food item.

cs.RO↗

arXiv2017

Inferring Object Properties with a Tactile Sensing Array Given Varying Joint Stiffness and Velocity

Whole-arm tactile sensing enables a robot to sense contact and infer contact properties across its entire arm. Within this paper, we demonstrate that using data-driven methods, a humanoid robot can infer mechanical properties of objects from contact with its forearm during a simple reaching motion. A key issue is the extent to which the performance of data-driven methods can generalize to robot actions that differ from those used during training. To investigate this, we developed an idealized physics-based lumped element model of a robot with a compliant joint making contact with an object. Using this physics-based model, we performed experiments with varied robot, object and environment parameters. We also collected data from a tactile-sensing forearm on a real robot as it made contact with various objects during a simple reaching motion with varied arm velocities and joint stiffnesses. The robot used one nearest neighbor classifiers (1-NN), hidden Markov models (HMMs), and long short-term memory (LSTM) networks to infer two object properties (hard vs. soft and moved vs. unmoved) based on features of time-varying tactile sensor data (maximum force, contact area, and contact motion). We found that, in contrast to 1-NN, the performance of LSTMs (with sufficient data availability) and multivariate HMMs successfully generalized to new robot motions with distinct velocities and joint stiffnesses. Compared to single features, using multiple features gave the best results for both experiments with physics-based models and a real-robot.

cs.RO↗

arXiv2015

A Handheld Device for the In Situ Acquisition of Multimodal Tactile Sensing Data

Multimodal tactile sensing could potentially enable robots to improve their performance at manipulation tasks by rapidly discriminating between task-relevant objects. Data-driven approaches to this tactile perception problem show promise, but there is a dearth of suitable training data. In this two-page paper, we present a portable handheld device for the efficient acquisition of multimodal tactile sensing data from objects in their natural settings, such as homes. The multimodal tactile sensor on the device integrates a fabric-based force sensor, a contact microphone, an accelerometer, temperature sensors, and a heating element. We briefly introduce our approach, describe the device, and demonstrate feasibility through an evaluation with a small data set that we captured by making contact with 7 task-relevant objects in a bathroom of a person's home.

cs.RO↗

arXiv2013

Validation of a Control Algorithm for Human-like Reaching Motion using 7-DOF Arm and 19-DOF Hand-Arm Systems

This technical report gives an overview of our work on control algorithms dealing with redundant robot systems for achieving human-like motion characteristics. Previously, we developed a novel control law to exhibit human-motion characteristics in redundant robot arm systems as well as arm-trunk systems for reaching tasks [1], [2]. This newly developed method nullifies the need for the computation of pseudo-inverse of Jacobian while the formulation and optimization of any artificial performance index is not necessary. The time-varying properties of the muscle stiffness and damping as well as the low-pass filter characteristics of human muscles have been modeled by the proposed control law to generate human-motion characteristics for reaching motion like quasi-straight line trajectory of the end-effector and symmetric bell shaped velocity profile. This report focuses on the experiments performed using a 7-DOF redundant robot-arm system which proved the effectiveness of this algorithm in imitating human-like motion characteristics. In addition, we extended this algorithm to a 19-DOF Hand-Arm System for a reach-to-grasp task. Simulations using the 19-DOF Hand-Arm System show the effectiveness of the proposed scheme for effective human-like hand-arm coordination in reach-to-grasp tasks for pinch and envelope grasps on objects of different shapes such as a box, a cylinder, and a sphere.

cs.RO↗

arXiv2013

Non-linear Task-Space Disturbance Observer for Position Regulation of Redundant Robot Arms against Perturbations in 3D Environments

Many day-to-day activities require the dexterous manipulation of a redundant humanoid arm in complex 3D environments. However, position regulation of such robot arm systems becomes very difficult in presence of non-linear uncertainties in the system. Also, perturbations exist due to various unwanted interactions with obstacles for clumsy environments in which obstacle avoidance is not possible, and this makes position regulation even more difficult. This report proposes a non-linear task-space disturbance observer by virtue of which position regulation of such robotic systems can be achieved in spite of such perturbations and uncertainties. Simulations are conducted using a 7-DOF redundant robot arm system to show the effectiveness of the proposed method. These results are then compared with the case of a conventional mass-damper based task-space disturbance observer to show the enhancement in performance using the developed concept. This proposed method is then applied to a controller which exhibits human-like motion characteristics for reaching a target. Arbitrary perturbations in the form of interactions with obstacles are introduced in its path. Results show that the robot end-effector successfully continues to move in its path of a human-like quasi-straight trajectory even if the joint trajectories deviated by a considerable amount due to the perturbations. These results are also compared with that of the unperturbed motion of the robot which further prove the significance of the developed scheme.

cs.RO↗