arXiv subjects

Robotics

Explore arXiv robotics papers and cs.RO metadata. Search for robot learning, motion planning and control, then check experiments in the source manuscript.

Follow this topic with RSS

At least 1,279 records · Page 71Linked to original sources

arXiv2025

Uncertainty-Aware Planning for Heterogeneous Robot Teams using Dynamic Topological Graphs and Mixed-Integer Programming

Multi-robot planning and coordination in uncertain environments is a fundamental computational challenge, since the belief space increases exponentially with the number of robots. In this paper, we address the problem of planning in uncertain environments with a heterogeneous robot team of fast scout vehicles for information gathering and more risk-averse carrier robots from which the scouts vehicles are deployed. To overcome the computational challenges, we represent the environment and operational scenario using a topological graph, where the parameters of the edge weight distributions vary with the state of the robot team on the graph, and we formulate a computationally efficient mixed-integer program which removes the dependence on the number of robots from its decision space. Our formulation results in the capability to generate optimal multi-robot, long-horizon plans in seconds that could otherwise be computationally intractable. Ultimately our approach enables real-time re-planning, since the computation time is significantly faster than the time to execute one step. We evaluate our algorithm in a scenario where the robot team must traverse an environment while minimizing detection by observers in positions that are uncertain to the robot team. We demonstrate that our method is computationally tractable, can improve performance in the presence of imperfect information, and can be adjusted for different risk profiles.

cs.RO↗

arXiv2025

Open X-Embodiment: Robotic Learning Datasets and RT-X Models

Large, high-capacity models trained on diverse datasets have shown remarkable successes on efficiently tackling downstream applications. In domains from NLP to Computer Vision, this has led to a consolidation of pretrained models, with general pretrained backbones serving as a starting point for many applications. Can such a consolidation happen in robotics? Conventionally, robotic learning methods train a separate model for every application, every robot, and even every environment. Can we instead train generalist X-robot policy that can be adapted efficiently to new robots, tasks, and environments? In this paper, we provide datasets in standardized data formats and models to make it possible to explore this possibility in the context of robotic manipulation, alongside experimental results that provide an example of effective X-robot policies. We assemble a dataset from 22 different robots collected through a collaboration between 21 institutions, demonstrating 527 skills (160266 tasks). We show that a high-capacity model trained on this data, which we call RT-X, exhibits positive transfer and improves the capabilities of multiple robots by leveraging experience from other platforms. More details can be found on the project website https://robotics-transformer-x.github.io.

cs.RO↗

arXiv2024

ManyQuadrupeds: Learning a Single Locomotion Policy for Diverse Quadruped Robots

Learning a locomotion policy for quadruped robots has traditionally been constrained to a specific robot morphology, mass, and size. The learning process must usually be repeated for every new robot, where hyperparameters and reward function weights must be re-tuned to maximize performance for each new system. Alternatively, attempting to train a single policy to accommodate different robot sizes, while maintaining the same degrees of freedom (DoF) and morphology, requires either complex learning frameworks, or mass, inertia, and dimension randomization, which leads to prolonged training periods. In our study, we show that drawing inspiration from animal motor control allows us to effectively train a single locomotion policy capable of controlling a diverse range of quadruped robots. The robot differences encompass: a variable number of DoFs, (i.e. 12 or 16 joints), three distinct morphologies, a broad mass range spanning from 2 kg to 200 kg, and nominal standing heights ranging from 18 cm to 100 cm. Our policy modulates a representation of the Central Pattern Generator (CPG) in the spinal cord, effectively coordinating both frequencies and amplitudes of the CPG to produce rhythmic output (Rhythm Generation), which is then mapped to a Pattern Formation (PF) layer. Across different robots, the only varying component is the PF layer, which adjusts the scaling parameters for the stride height and length. Subsequently, we evaluate the sim-to-real transfer by testing the single policy on both the Unitree Go1 and A1 robots. Remarkably, we observe robust performance, even when adding a 15 kg load, equivalent to 125% of the A1 robot's nominal mass.

cs.RO↗

arXiv2023

Habitat 3.0: A Co-Habitat for Humans, Avatars and Robots

We present Habitat 3.0: a simulation platform for studying collaborative human-robot tasks in home environments. Habitat 3.0 offers contributions across three dimensions: (1) Accurate humanoid simulation: addressing challenges in modeling complex deformable bodies and diversity in appearance and motion, all while ensuring high simulation speed. (2) Human-in-the-loop infrastructure: enabling real human interaction with simulated robots via mouse/keyboard or a VR interface, facilitating evaluation of robot policies with human input. (3) Collaborative tasks: studying two collaborative tasks, Social Navigation and Social Rearrangement. Social Navigation investigates a robot's ability to locate and follow humanoid avatars in unseen environments, whereas Social Rearrangement addresses collaboration between a humanoid and robot while rearranging a scene. These contributions allow us to study end-to-end learned and heuristic baselines for human-robot collaboration in-depth, as well as evaluate them with humans in the loop. Our experiments demonstrate that learned robot policies lead to efficient task completion when collaborating with unseen humanoid agents and human partners that might exhibit behaviors that the robot has not seen before. Additionally, we observe emergent behaviors during collaborative task execution, such as the robot yielding space when obstructing a humanoid agent, thereby allowing the effective completion of the task by the humanoid agent. Furthermore, our experiments using the human-in-the-loop tool demonstrate that our automated evaluation with humanoids can provide an indication of the relative ordering of different policies when evaluated with real human collaborators. Habitat 3.0 unlocks interesting new features in simulators for Embodied AI, and we hope it paves the way for a new frontier of embodied human-AI interaction capabilities.

cs.HC↗

arXiv2024

Mechanically-Inflatable Bio-Inspired Locomotion for Robotic Pipeline Inspection

Pipelines, vital for fluid transport, pose an important yet challenging inspection task, particularly in small, flexible biological systems, that robots have yet to master. In this study, we explored the development of an innovative robot inspired by the ovipositor of parasitic wasps to navigate and inspect pipelines. The robot features a flexible locomotion system that adapts to different tube sizes and shapes through a mechanical inflation technique. The flexible locomotion system employs a reciprocating motion, in which groups of three sliders extend and retract in a cyclic fashion. In a proof-of-principle experiment, the robot locomotion efficiency demonstrated positive linear correlation (r=0.6434) with the diameter ratio (ratio of robot diameter to tube diameter). The robot showcased a remarkable ability to traverse tubes of different sizes, shapes and payloads with an average of (70%) locomotion efficiency across all testing conditions, at varying diameter ratios (0.7-1.5). Furthermore, the mechanical inflation mechanism displayed substantial load-carrying capacity, producing considerable holding force of (13 N), equivalent to carrying a payload of approximately (5.8 Kg) inclusive the robot weight. This novel soft robotic system shows promise for inspection and navigation within tubular confined spaces, particularly in scenarios requiring adaptability to different tube shapes, sizes, and load-carrying capacities. This novel design serves as a foundation for a new class of pipeline inspection robots that exhibit versatility across various pipeline environments, potentially including biological systems.

cs.RO↗

arXiv2023

NewsGPT: ChatGPT Integration for Robot-Reporter

The integration of large language models (LLMs) with social robots has emerged as a promising avenue for enhancing human-robot interactions at a time when news reports generated by artificial intelligence (AI) are gaining in credibility. This integration is expected to intensify and become a more productive resource for journalism, media, communication, and education. In this paper a novel system is proposed that integrates AI's generative pretrained transformer (GPT) model with the Pepper robot, with the aim of improving the robot's natural language understanding and response generation capabilities for enhanced social interactions. By leveraging GPT's powerful language processing capabilities, this system offers a comprehensive pipeline that incorporates voice input recording, speech-to-text transcription, context analysis, and text-to-speech synthesis action generation. The Pepper robot is enabled to comprehend user queries, generate informative responses with general knowledge, maintain contextually relevant conversations, and act as a more domain-oriented news reporter. It is also linked with a news resource and powered with a Google search capability. To evaluate the performance of the framework, experiments were conducted involving a set of diverse questions. The robot's responses were assessed on the basis of eight criteria, including relevance, context, and fluency. Despite some identified limitations, this system contributes to the field of journalism and human-robot interaction by showcasing the potential of integrating LLMs with social robots. The proposed framework opens up opportunities for improving the conversational capabilities of robots, enabling interactions that are smoother, more engaging, and more context aware.

cs.RO↗

arXiv2023

Asymmetric Design of Control Barrier Function for Multiagent Autonomous Robotic Systems

In this paper, we propose a method to avoid "no-solution" situations of the control barrier function (CBF) for distributed collision avoidance in a multiagent autonomous robotic system (MARS). MARS, which is composed of distributed autonomous mobile robots, is expected to effectively perform cooperative tasks such as searching in a certain area. Therefore, collision avoidance must be considered when implementing MARS in the real world. The CBF is effective for solving collision-avoidance problems. However, in extreme conditions where many robots congregate at one location, the CBF constraints that ensure a safe distance between robots may be violated. We theoretically demonstrate that this problem can occur in certain situations, and introduce an asymmetric design for the inequality constraints of CBF. We asymmetrically decentralized inequality constraints with weight functions using the absolute speed of the robot so that other robots can take over the constraints of the robot in severe condition. We demonstrate the effectiveness of the proposed method in a two-dimensional situation wherein multiple robots congregate at one location. We implement the proposed method on real robots and the confirmed the effectiveness of this theory.

cs.RO↗

arXiv2025

Scalable Multi-Robot Motion Planning Using Workspace Guidance-Informed Hypergraphs

In this work, we propose a method for multiple mobile robot motion planning that efficiently plans for robot teams up to 128 robots (an order of magnitude larger than existing state-of-the-art methods) in congested settings with narrow passages in the environment. We achieve this improvement in scalability by extending the state-of-the-art Decomposable State Space Hypergraph (DaSH) multi-robot planning framework to support mobile robot motion planning in congested environments. This is a problem that DaSH cannot be directly applied to because it lacks a highly structured, easily discretizable task space and features kinodynamic constraints. We accomplish this by exploiting knowledge about the workspace topology to limit exploration of the planning space and through modifying DaSH's conflict resolution scheme. This guidance captures when coordination between robots is necessary, allowing us to decompose the intractably large multi-robot search space while limiting risk of inter-robot conflicts by composing relevant robot groups together while planning.

cs.RO↗

arXiv2024

Towards Proactive Safe Human-Robot Collaborations via Data-Efficient Conditional Behavior Prediction

We focus on the problem of how we can enable a robot to collaborate seamlessly with a human partner, specifically in scenarios where preexisting data is sparse. Much prior work in human-robot collaboration uses observational models of humans (i.e. models that treat the robot purely as an observer) to choose the robot's behavior, but such models do not account for the influence the robot has on the human's actions, which may lead to inefficient interactions. We instead formulate the problem of optimally choosing a collaborative robot's behavior based on a conditional model of the human that depends on the robot's future behavior. First, we propose a novel model-based formulation of conditional behavior prediction that allows the robot to infer the human's intentions based on its future plan in data-sparse environments. We then show how to utilize a conditional model for proactive goal selection and safe trajectory generation around human collaborators. Finally, we use our proposed proactive controller in a collaborative task with real users to show that it can improve users' interactions with a robot collaborator quantitatively and qualitatively.

cs.RO↗

arXiv2023

Robot Learning in the Era of Foundation Models: A Survey

The proliferation of Large Language Models (LLMs) has s fueled a shift in robot learning from automation towards general embodied Artificial Intelligence (AI). Adopting foundation models together with traditional learning methods to robot learning has increasingly gained recent interest research community and showed potential for real-life application. However, there are few literatures comprehensively reviewing the relatively new technologies combined with robotics. The purpose of this review is to systematically assess the state-of-the-art foundation model techniques in the robot learning and to identify future potential areas. Specifically, we first summarized the technical evolution of robot learning and identified the necessary preliminary preparations for foundation models including the simulators, datasets, foundation model framework. In addition, we focused on the following four mainstream areas of robot learning including manipulation, navigation, planning, and reasoning and demonstrated how the foundation model techniques can be adopted in the above scenarios. Furthermore, critical issues which are neglected in the current literatures including robot hardware and software decoupling, dynamic data, generalization performance with the presence of human, etc. were discussed. This review highlights the state-of-the-art progress of foundation models in robot learning and future research should focus on multimodal interaction especially dynamics data, exclusive foundation models for robots, and AI alignment, etc.

cs.RO↗

arXiv2023

Measuring the perception of the personalized activities with CloudIA robot

Socially Assistive Robots represent a valid solution for improving the quality of life and the mood of older adults. In this context, this work presents the CloudIA robot, a non-human-like robot intended to promote sociality and well-being among older adults. The design of the robot and of the provided services were carried out by a multidisciplinary team of designers and technology developers in tandem with professional caregivers. The capabilities of the robot were implemented according to the received guidelines and tested in two nursing facilities by 15 older people. Qualitative and quantitative metrics were used to investigate the engagement of the participants during the interaction with the robot, and to investigate any differences in the interaction during the proposed activities. The results highlighted the general tendency of humanizing the robotic platform and demonstrated the feasibility of introducing the CloudIA robot in support of the professional caregivers' work. From this pilot test, further ideas on improving the personalization of the robotic platform emerged.

cs.RO↗

arXiv2023

Codesign of Humanoid Robots for Ergonomy Collaboration with Multiple Humans via Genetic Algorithms and Nonlinear Optimization

Ergonomics is a key factor to consider when designing control architectures for effective physical collaborations between humans and humanoid robots. In contrast, ergonomic indexes are often overlooked in the robot design phase, which leads to suboptimal performance in physical human-robot interaction tasks. This paper proposes a novel methodology for optimizing the design of humanoid robots with respect to ergonomic indicators associated with the interaction of multiple agents. Our approach leverages a dynamic and kinematic parameterization of the robot link and motor specifications to seek for optimal robot designs using a bilevel optimization approach. Specifically, a genetic algorithm first generates robot designs by selecting the link and motor characteristics. Then, we use nonlinear optimization to evaluate interaction ergonomy indexes during collaborative payload lifting with different humans and weights. To assess the effectiveness of our approach, we compare the optimal design obtained using bilevel optimization against the design obtained using nonlinear optimization. Our results show that the proposed approach significantly improves ergonomics in terms of energy expenditure calculated in two reference scenarios involving static and dynamic robot motions. We plan to apply our methodology to drive the design of the ergoCub2 robot, a humanoid intended for optimal physical collaboration with humans in diverse environments

cs.RO↗

arXiv2025

Double Oracle Algorithm for Game-Theoretic Robot Allocation on Graphs

We study the problem of game-theoretic robot allocation where two players strategically allocate robots to compete for multiple sites of interest. Robots possess offensive or defensive capabilities to interfere and weaken their opponents to take over a competing site. This problem belongs to the conventional Colonel Blotto Game. Considering the robots' heterogeneous capabilities and environmental factors, we generalize the conventional Blotto game by incorporating heterogeneous robot types and graph constraints that capture the robot transitions between sites. Then we employ the Double Oracle Algorithm (DOA) to solve for the Nash equilibrium of the generalized Blotto game. Particularly, for cyclic-dominance-heterogeneous (CDH) robots that inhibit each other, we define a new transformation rule between any two robot types. Building on the transformation, we design a novel utility function to measure the game's outcome quantitatively. Moreover, we rigorously prove the correctness of the designed utility function. Finally, we conduct extensive simulations to demonstrate the effectiveness of DOA on computing Nash equilibrium for homogeneous, linear heterogeneous, and CDH robot allocation on graphs.

cs.RO↗

arXiv2025

QUAR-VLA: Vision-Language-Action Model for Quadruped Robots

The important manifestation of robot intelligence is the ability to naturally interact and autonomously make decisions. Traditional approaches to robot control often compartmentalize perception, planning, and decision-making, simplifying system design but limiting the synergy between different information streams. This compartmentalization poses challenges in achieving seamless autonomous reasoning, decision-making, and action execution. To address these limitations, a novel paradigm, named Vision-Language-Action tasks for QUAdruped Robots (QUAR-VLA), has been introduced in this paper. This approach tightly integrates visual information and instructions to generate executable actions, effectively merging perception, planning, and decision-making. The central idea is to elevate the overall intelligence of the robot. Within this framework, a notable challenge lies in aligning fine-grained instructions with visual perception information. This emphasizes the complexity involved in ensuring that the robot accurately interprets and acts upon detailed instructions in harmony with its visual observations. Consequently, we propose QUAdruped Robotic Transformer (QUART), a family of VLA models to integrate visual information and instructions from diverse modalities as input and generates executable actions for real-world robots and present QUAdruped Robot Dataset (QUARD), a large-scale multi-task dataset including navigation, complex terrain locomotion, and whole-body manipulation tasks for training QUART models. Our extensive evaluation (4000 evaluation trials) shows that our approach leads to performant robotic policies and enables QUART to obtain a range of emergent capabilities.

cs.RO↗

arXiv2024

"Oh, Sorry, I Think I Interrupted You'': Designing Repair Strategies for Robotic Longitudinal Well-being Coaching

Robotic well-being coaches have been shown to successfully promote people's mental well-being. To provide successful coaching, a robotic coach should have the capability to repair the mistakes it makes. Past investigations of robot mistakes are limited to game or task-based, one-off and in-lab studies. This paper presents a 4-phase design process to design repair strategies for robotic longitudinal well-being coaching with the involvement of real-world stakeholders: 1) designing repair strategies with a professional well-being coach; 2) a longitudinal study with the involvement of experienced users (i.e., who had already interacted with a robotic coach) to investigate the repair strategies defined in (1); 3) a design workshop with users from the study in (2) to gather their perspectives on the robotic coach's repair strategies; 4) discussing the results obtained in (2) and (3) with the mental well-being professional to reflect on how to design repair strategies for robotic coaching. Our results show that users have different expectations for a robotic coach than a human coach, which influences how repair strategies should be designed. We show that different repair strategies (e.g., apologizing, explaining, or repairing empathically) are appropriate in different scenarios, and that preferences for repair strategies change during longitudinal interactions with the robotic coach.

cs.RO↗

arXiv2024

CLIP feature-based randomized control using images and text for multiple tasks and robots

This study presents a control framework leveraging vision language models (VLMs) for multiple tasks and robots. Notably, existing control methods using VLMs have achieved high performance in various tasks and robots in the training environment. However, these methods incur high costs for learning control policies for tasks and robots other than those in the training environment. Considering the application of industrial and household robots, learning in novel environments where robots are introduced is challenging. To address this issue, we propose a control framework that does not require learning control policies. Our framework combines the vision-language CLIP model with a randomized control. CLIP computes the similarity between images and texts by embedding them in the feature space. This study employs CLIP to compute the similarity between camera images and text representing the target state. In our method, the robot is controlled by a randomized controller that simultaneously explores and increases the similarity gradients. Moreover, we fine-tune the CLIP to improve the performance of the proposed method. Consequently, we confirm the effectiveness of our approach through a multitask simulation and a real robot experiment using a two-wheeled robot and robot arm.

cs.RO↗

arXiv2024

G.O.G: A Versatile Gripper-On-Gripper Design for Bimanual Cloth Manipulation with a Single Robotic Arm

The manipulation of garments poses research challenges due to their deformable nature and the extensive variability in shapes and sizes. Despite numerous attempts by researchers to address these via approaches involving robot perception and control, there has been a relatively limited interest in resolving it through the co-development of robot hardware. Consequently, the majority of studies employ off-the-shelf grippers in conjunction with dual robot arms to enable bimanual manipulation and high dexterity. However, this dual-arm system increases the overall cost of the robotic system as well as its control complexity in order to tackle robot collisions and other robot coordination issues. As an alternative approach, we propose to enable bimanual cloth manipulation using a single robot arm via novel end effector design -- sharing dexterity skills between manipulator and gripper rather than relying entirely on robot arm coordination. To this end, we introduce a new gripper, called G.O.G., based on a gripper-on-gripper structure where the first gripper independently regulates the span, up to 500mm, between its fingers which are in turn also grippers. These finger grippers consist of a variable friction module that enables two grasping modes: firm and sliding grasps. Household item and cloth object benchmarks are employed to evaluate the performance of the proposed design, encompassing both experiments on the gripper design itself and on cloth manipulation. Experimental results demonstrate the potential of the introduced ideas to undertake a range of bimanual cloth manipulation tasks with a single robot arm. Supplementary material is available at https://sites.google.com/view/gripperongripper.

cs.RO↗

arXiv2024

ARMCHAIR: integrated inverse reinforcement learning and model predictive control for human-robot collaboration

One of the key issues in human-robot collaboration is the development of computational models that allow robots to predict and adapt to human behavior. Much progress has been achieved in developing such models, as well as control techniques that address the autonomy problems of motion planning and decision-making in robotics. However, the integration of computational models of human behavior with such control techniques still poses a major challenge, resulting in a bottleneck for efficient collaborative human-robot teams. In this context, we present a novel architecture for human-robot collaboration: Adaptive Robot Motion for Collaboration with Humans using Adversarial Inverse Reinforcement learning (ARMCHAIR). Our solution leverages adversarial inverse reinforcement learning and model predictive control to compute optimal trajectories and decisions for a mobile multi-robot system that collaborates with a human in an exploration task. During the mission, ARMCHAIR operates without human intervention, autonomously identifying the necessity to support and acting accordingly. Our approach also explicitly addresses the network connectivity requirement of the human-robot team. Extensive simulation-based evaluations demonstrate that ARMCHAIR allows a group of robots to safely support a simulated human in an exploration scenario, preventing collisions and network disconnections, and improving the overall performance of the task.

cs.RO↗