Searcharxiv⌕ Search

arXiv subjects

Masayuki Inaba

Publications and source records attributed to Masayuki Inaba.

At least 91 records · Page 5Linked to original sources
arXiv2024

Continuous Jumping of a Parallel Wire-Driven Monopedal Robot RAMIEL Using Reinforcement Learning

We have developed a parallel wire-driven monopedal robot, RAMIEL, which has both speed and power due to the parallel wire mechanism and a long acceleration distance. RAMIEL is capable of jumping high and continuously, and so has high performance in traveling. On the other hand, one of the drawbacks of a minimal parallel wire-driven robot without joint encoders is that the current joint velocities estimated from the wire lengths oscillate due to the elongation of the wires, making the values unreliable. Therefore, despite its high performance, the control of the robot is unstable, and in 10 out of 16 jumps, the robot could only jump up to two times continuously. In this study, we propose a method to realize a continuous jumping motion by reinforcement learning in simulation, and its application to the actual robot. Because the joint velocities oscillate with the elongation of the wires, they are not used directly, but instead are inferred from the time series of joint angles. At the same time, noise that imitates the vibration caused by the elongation of the wires is added for transfer to the actual robot. The results show that the system can be applied to the actual robot RAMIEL as well as to the stable continuous jumping motion in simulation.

cs.RO↗
arXiv2024

Continuous Object State Recognition for Cooking Robots Using Pre-Trained Vision-Language Models and Black-box Optimization

The state recognition of the environment and objects by robots is generally based on the judgement of the current state as a classification problem. On the other hand, state changes of food in cooking happen continuously and need to be captured not only at a certain time point but also continuously over time. In addition, the state changes of food are complex and cannot be easily described by manual programming. Therefore, we propose a method to recognize the continuous state changes of food for cooking robots through the spoken language using pre-trained large-scale vision-language models. By using models that can compute the similarity between images and texts continuously over time, we can capture the state changes of food while cooking. We also show that by adjusting the weighting of each text prompt based on fitting the similarity changes to a sigmoid function and then performing black-box optimization, more accurate and robust continuous state recognition can be achieved. We demonstrate the effectiveness and limitations of this method by performing the recognition of water boiling, butter melting, egg cooking, and onion stir-frying.

cs.RO↗
arXiv2024

Design and Control of a Small Humanoid Equipped with Flight Unit and Wheels for Multimodal Locomotion

Humanoids are versatile robotic platforms owing to their limbs with multiple degrees of freedom. Although humanoids can walk like humans, they are relatively slow, and cannot run over large barriers. To address these limitations, we aim to achieve rapid terrestrial locomotion ability and simultaneously expand the locomotion domain to the air by utilizing thrust for propulsion. In this paper, we first describe an optimized construction method for a humanoid robot equipped with wheels and a flight unit to achieve these abilities. Then, we describe the integrated control framework of the proposed flying humanoid for each locomotion mode: aerial, legged, and wheeled locomotion. Finally, we achieved multimodal locomotion and aerial manipulation experiments using the proposed robot platform. To the best of our knowledge, this is the first time that a single humanoid has simultaneously achieved three different types of locomotion, including flight.

cs.RO↗
arXiv2024

SAQIEL: Ultra-Light and Safe Manipulator with Passive 3D Wire Alignment Mechanism

Improving the safety of collaborative manipulators necessitates the reduction of inertia in the moving part. Within this paper, we introduce a novel approach in the form of a passive 3D wire aligner, serving as a lightweight and low-friction power transmission mechanism, thus achieving the desired low inertia in the manipulator's operation. Through the utilization of this innovation, the consolidation of hefty actuators onto the root link becomes feasible, consequently enabling a supple drive characterized by minimal friction. To demonstrate the efficacy of this device, we fabricate an ultralight 7 degrees of freedom (DoF) manipulator named SAQIEL, boasting a mere 1.5 kg weight for its moving components. Notably, to mitigate friction within SAQIEL's actuation system, we employ a distinctive mechanism that directly winds wires using motors, obviating the need for traditional gear or belt-based speed reduction mechanisms. Through a series of empirical trials, we substantiate that SAQIEL adeptly strikes balance between lightweight design, substantial payload capacity, elevated velocity, precision, and adaptability.

cs.RO↗
arXiv2024

Design Optimization of Wire Arrangement with Variable Relay Points in Numerical Simulation for Tendon-driven Robots

One of the most important features of tendon-driven robots is the ease of wire arrangement and the degree of freedom it affords, enabling the construction of a body that satisfies the desired characteristics by modifying the wire arrangement. Various wire arrangement optimization methods have been proposed, but they have simplified the configuration by assuming that the moment arm of wires to joints are constant, or by disregarding wire arrangements that span multiple joints and include relay points. In this study, we formulate a more flexible wire arrangement optimization problem in which each wire is represented by a start point, multiple relay points, and an end point, and achieve the desired physical performance based on black-box optimization. We consider a multi-objective optimization which simultaneously takes into account both the feasible operational force space and velocity space, and discuss the optimization results obtained from various configurations.

cs.RO↗
arXiv2023

Daily Assistive View Control Learning of Low-Cost Low-Rigidity Robot via Large-Scale Vision-Language Model

In this study, we develop a simple daily assistive robot that controls its own vision according to linguistic instructions. The robot performs several daily tasks such as recording a user's face, hands, or screen, and remotely capturing images of desired locations. To construct such a robot, we combine a pre-trained large-scale vision-language model with a low-cost low-rigidity robot arm. The correlation between the robot's physical and visual information is learned probabilistically using a neural network, and changes in the probability distribution based on changes in time and environment are considered by parametric bias, which is a learnable network input variable. We demonstrate the effectiveness of this learning method by open-vocabulary view control experiments with an actual robot arm, MyCobot.

cs.RO↗
arXiv2023

RAMIEL: A Parallel-Wire Driven Monopedal Robot for High and Continuous Jumping

Legged robots with high locomotive performance have been extensively studied, and various leg structures have been proposed. Especially, a leg structure that can achieve both continuous and high jumps is advantageous for moving around in a three-dimensional environment. In this study, we propose a parallel wire-driven leg structure, which has one DoF of linear motion and two DoFs of rotation and is controlled by six wires, as a structure that can achieve both continuous jumping and high jumping. The proposed structure can simultaneously achieve high controllability on each DoF, long acceleration distance and high power required for jumping. In order to verify the jumping performance of the parallel wire-driven leg structure, we have developed a parallel wire-driven monopedal robot, RAMIEL. RAMIEL is equipped with quasi-direct drive, high power wire winding mechanisms and a lightweight leg, and can achieve a maximum jumping height of 1.6 m and a maximum of seven continuous jumps.

cs.RO↗
arXiv2023

Binary State Recognition by Robots using Visual Question Answering of Pre-Trained Vision-Language Model

Recognition of the current state is indispensable for the operation of a robot. There are various states to be recognized, such as whether an elevator door is open or closed, whether an object has been grasped correctly, and whether the TV is turned on or off. Until now, these states have been recognized by programmatically describing the state of a point cloud or raw image, by annotating and learning images, by using special sensors, etc. In contrast to these methods, we apply Visual Question Answering (VQA) from a Pre-Trained Vision-Language Model (PTVLM) trained on a large-scale dataset, to such binary state recognition. This idea allows us to intuitively describe state recognition in language without any re-training, thereby improving the recognition ability of robots in a simple and general way. We summarize various techniques in questioning methods and image processing, and clarify their properties through experiments.

cs.RO↗
arXiv2023

Robotic Applications of Pre-Trained Vision-Language Models to Various Recognition Behaviors

In recent years, a number of models that learn the relations between vision and language from large datasets have been released. These models perform a variety of tasks, such as answering questions about images, retrieving sentences that best correspond to images, and finding regions in images that correspond to phrases. Although there are some examples, the connection between these pre-trained vision-language models and robotics is still weak. If they are directly connected to robot motions, they lose their versatility due to the embodiment of the robot and the difficulty of data collection, and become inapplicable to a wide range of bodies and situations. Therefore, in this study, we categorize and summarize the methods to utilize the pre-trained vision-language models flexibly and easily in a way that the robot can understand, without directly connecting them to robot motions. We discuss how to use these models for robot motion selection and motion planning without re-training the models. We consider five types of methods to extract information understandable for robots, and show the results of state recognition, object recognition, affordance recognition, relation recognition, and anomaly detection based on the combination of these five methods. We expect that this study will add flexibility and ease-of-use, as well as new applications, to the recognition behavior of existing robots.

cs.RO↗
arXiv2023

Semantic Scene Difference Detection in Daily Life Patroling by Mobile Robots using Pre-Trained Large-Scale Vision-Language Model

It is important for daily life support robots to detect changes in their environment and perform tasks. In the field of anomaly detection in computer vision, probabilistic and deep learning methods have been used to calculate the image distance. These methods calculate distances by focusing on image pixels. In contrast, this study aims to detect semantic changes in the daily life environment using the current development of large-scale vision-language models. Using its Visual Question Answering (VQA) model, we propose a method to detect semantic changes by applying multiple questions to a reference image and a current image and obtaining answers in the form of sentences. Unlike deep learning-based methods in anomaly detection, this method does not require any training or fine-tuning, is not affected by noise, and is sensitive to semantic state changes in the real world. In our experiments, we demonstrated the effectiveness of this method by applying it to a patrol task in a real-life environment using a mobile robot, Fetch Mobile Manipulator. In the future, it may be possible to add explanatory power to changes in the daily life environment through spoken language.

cs.RO↗
arXiv2023

Development of a Whole-body Work Imitation Learning System by a Biped and Bi-armed Humanoid

Imitation learning has been actively studied in recent years. In particular, skill acquisition by a robot with a fixed body, whose root link position and posture and camera angle of view do not change, has been realized in many cases. On the other hand, imitation of the behavior of robots with floating links, such as humanoid robots, is still a difficult task. In this study, we develop an imitation learning system using a biped robot with a floating link. There are two main problems in developing such a system. The first is a teleoperation device for humanoids, and the second is a control system that can withstand heavy workloads and long-term data collection. For the first point, we use the whole body control device TABLIS. It can control not only the arms but also the legs and can perform bilateral control with the robot. By connecting this TABLIS with the high-power humanoid robot JAXON, we construct a control system for imitation learning. For the second point, we will build a system that can collect long-term data based on posture optimization, and can simultaneously move the robot's limbs. We combine high-cycle posture generation with posture optimization methods, including whole-body joint torque minimization and contact force optimization. We designed an integrated system with the above two features to achieve various tasks through imitation learning. Finally, we demonstrate the effectiveness of this system by experiments of manipulating flexible fabrics such that not only the hands but also the head and waist move simultaneously, manipulating objects using legs characteristic of humanoids, and lifting heavy objects that require large forces.

cs.RO↗
arXiv2023

Daily Assistive Modular Robot Design Based on Multi-Objective Black-Box Optimization

The range of robot activities is expanding from industries with fixed environments to diverse and changing environments, such as nursing care support and daily life support. In particular, autonomous construction of robots that are personalized for each user and task is required. Therefore, we develop an actuator module that can be reconfigured to various link configurations, can carry heavy objects using a locking mechanism, and can be easily operated by human teaching using a releasing mechanism. Given multiple target coordinates, a modular robot configuration that satisfies these coordinates and minimizes the required torque is automatically generated by Tree-structured Parzen Estimator (TPE), a type of black-box optimization. Based on the obtained results, we show that the robot can be reconfigured to perform various functions such as moving monitors and lights, serving food, and so on.

cs.RO↗
arXiv2023

Recognition of Heat-Induced Food State Changes by Time-Series Use of Vision-Language Model for Cooking Robot

Cooking tasks are characterized by large changes in the state of the food, which is one of the major challenges in robot execution of cooking tasks. In particular, cooking using a stove to apply heat to the foodstuff causes many special state changes that are not seen in other tasks, making it difficult to design a recognizer. In this study, we propose a unified method for recognizing changes in the cooking state of robots by using the vision-language model that can discriminate open-vocabulary objects in a time-series manner. We collected data on four typical state changes in cooking using a real robot and confirmed the effectiveness of the proposed method. We also compared the conditions and discussed the types of natural language prompts and the image regions that are suitable for recognizing the state changes.

cs.RO↗
arXiv2023

Automatic Diary Generation System including Information on Joint Experiences between Humans and Robots

In this study, we propose an automatic diary generation system that uses information from past joint experiences with the aim of increasing the favorability for robots through shared experiences between humans and robots. For the verbalization of the robot's memory, the system applies a large-scale language model, which is a rapidly developing field. Since this model does not have memories of experiences, it generates a diary by receiving information from joint experiences. As an experiment, a robot and a human went for a walk and generated a diary with interaction and dialogue history. The proposed diary achieved high scores in comfort and performance in the evaluation of the robot's impression. In the survey of diaries giving more favorable impressions, diaries with information on joint experiences were selected higher than diaries without such information, because diaries with information on joint experiences showed more cooperation between the robot and the human and more intimacy from the robot.

cs.RO↗
arXiv2023

A method for Selecting Scenes and Emotion-based Descriptions for a Robot's Diary

In this study, we examined scene selection methods and emotion-based descriptions for a robot's daily diary. We proposed a scene selection method and an emotion description method that take into account semantic and affective information, and created several types of diaries. Experiments were conducted to examine the change in sentiment values and preference of each diary, and it was found that the robot's feelings and impressions changed more from date to date when scenes were selected using the affective captions. Furthermore, we found that the robot's emotion generally improves the preference of the robot's diary regardless of the scene it describes. However, presenting negative or mixed emotions at once may decrease the preference of the diary or reduce the robot's robot-likeness, and thus the method of presenting emotions still needs further investigation.

cs.RO↗
arXiv2023

Foundation Model based Open Vocabulary Task Planning and Executive System for General Purpose Service Robots

This paper describes a strategy for implementing a robotic system capable of performing General Purpose Service Robot (GPSR) tasks in robocup@home. The GPSR task is that a real robot hears a variety of commands in spoken language and executes a task in a daily life environment. To achieve the task, we integrate foundation models based inference system and a state machine task executable. The foundation models plan the task and detect objects with open vocabulary, and a state machine task executable manages each robot's actions. This system works stable, and we took first place in the RoboCup@home Japan Open 2022's GPSR with 130 points, more than 85 points ahead of the other teams.

cs.RO↗
arXiv2023

Online Estimation of Self-Body Deflection With Various Sensor Data Based on Directional Statistics

In this paper, we propose a method for online estimation of the robot's posture. Our method uses von Mises and Bingham distributions as probability distributions of joint angles and 3D orientation, which are used in directional statistics. We constructed a particle filter using these distributions and configured a system to estimate the robot's posture from various sensor information (e.g., joint encoders, IMU sensors, and cameras). Furthermore, unlike tangent space approximations, these distributions can handle global features and represent sensor characteristics as observation noises. As an application, we show that the yaw drift of a 6-axis IMU sensor can be represented probabilistically to prevent adverse effects on attitude estimation. For the estimation, we used an approximate model that assumes the actual robot posture can be reproduced by correcting the joint angles of a rigid body model. In the experiment part, we tested the estimator's effectiveness by examining that the joint angles generated with the approximate model can be estimated using the link pose of the same model. We then applied the estimator to the actual robot and confirmed that the gripper position could be estimated, thereby verifying the validity of the approximate model in our situation.

cs.RO↗
arXiv2023

VQA-based Robotic State Recognition Optimized with Genetic Algorithm

State recognition of objects and environment in robots has been conducted in various ways. In most cases, this is executed by processing point clouds, learning images with annotations, and using specialized sensors. In contrast, in this study, we propose a state recognition method that applies Visual Question Answering (VQA) in a Pre-Trained Vision-Language Model (PTVLM) trained from a large-scale dataset. By using VQA, it is possible to intuitively describe robotic state recognition in the spoken language. On the other hand, there are various possible ways to ask about the same event, and the performance of state recognition differs depending on the question. Therefore, in order to improve the performance of state recognition using VQA, we search for an appropriate combination of questions using a genetic algorithm. We show that our system can recognize not only the open/closed of a refrigerator door and the on/off of a display, but also the open/closed of a transparent door and the state of water, which have been difficult to recognize.

cs.RO↗