arXiv subjects

Roya Firoozi

Publications and source records attributed to Roya Firoozi.

12 recordsLinked to original sources

arXiv2025

E2Map: Experience-and-Emotion Map for Self-Reflective Robot Navigation with Language Models

Large language models (LLMs) have shown significant potential in guiding embodied agents to execute language instructions across a range of tasks, including robotic manipulation and navigation. However, existing methods are primarily designed for static environments and do not leverage the agent's own experiences to refine its initial plans. Given that real-world environments are inherently stochastic, initial plans based solely on LLMs' general knowledge may fail to achieve their objectives, unlike in static scenarios. To address this limitation, this study introduces the Experience-and-Emotion Map (E2Map), which integrates not only LLM knowledge but also the agent's real-world experiences, drawing inspiration from human emotional responses. The proposed methodology enables one-shot behavior adjustments by updating the E2Map based on the agent's experiences. Our evaluation in stochastic navigation environments, including both simulations and real-world scenarios, demonstrates that the proposed method significantly enhances performance in stochastic environments compared to existing LLM-based approaches. Code and supplementary materials are available at https://e2map.github.io/.

cs.RO↗

arXiv2024

OA-MPC: Occlusion-Aware MPC for Guaranteed Safe Robot Navigation with Unseen Dynamic Obstacles

For safe navigation in dynamic uncertain environments, robotic systems rely on the perception and prediction of other agents. Particularly, in occluded areas where cameras and LiDAR give no data, the robot must be able to reason about potential movements of invisible dynamic agents. This work presents a provably safe motion planning scheme for real-time navigation in an a priori unmapped environment, where occluded dynamic agents are present. Safety guarantees are provided based on reachability analysis. Forward reachable sets associated with potential occluded agents, such as pedestrians, are computed and incorporated into planning. An iterative optimization-based planner is presented that alternates between two optimizations: nonlinear Model Predictive Control (NMPC) and collision avoidance. Recursive feasibility of the MPC is guaranteed by introducing a terminal stopping constraint. The effectiveness of the proposed algorithm is demonstrated through simulation studies and hardware experiments with a TurtleBot robot. A video of experimental results is available at \url{https://youtu.be/OUnkB5Feyuk}.

cs.RO↗

arXiv2024

Splat-MOVER: Multi-Stage, Open-Vocabulary Robotic Manipulation via Editable Gaussian Splatting

We present Splat-MOVER, a modular robotics stack for open-vocabulary robotic manipulation, which leverages the editability of Gaussian Splatting (GSplat) scene representations to enable multi-stage manipulation tasks. Splat-MOVER consists of: (i) ASK-Splat, a GSplat representation that distills semantic and grasp affordance features into the 3D scene. ASK-Splat enables geometric, semantic, and affordance understanding of 3D scenes, which is critical in many robotics tasks; (ii) SEE-Splat, a real-time scene-editing module using 3D semantic masking and infilling to visualize the motions of objects that result from robot interactions in the real-world. SEE-Splat creates a "digital twin" of the evolving environment throughout the manipulation task; and (iii) Grasp-Splat, a grasp generation module that uses ASK-Splat and SEE-Splat to propose affordance-aligned candidate grasps for open-world objects. ASK-Splat is trained in real-time from RGB images in a brief scanning phase prior to operation, while SEE-Splat and Grasp-Splat run in real-time during operation. We demonstrate the superior performance of Splat-MOVER in hardware experiments on a Kinova robot compared to two recent baselines in four single-stage, open-vocabulary manipulation tasks and in four multi-stage manipulation tasks, using the edited scene to reflect changes due to prior manipulation stages, which is not possible with existing baselines. Video demonstrations and the code for the project are available at https://splatmover.github.io.

cs.RO↗

arXiv2024

A Distributed Multi-Vehicle Coordination Algorithm for Navigation in Tight Environments

This work presents a distributed method for multi-vehicle coordination based on nonlinear model predictive control (NMPC) and dual decomposition. Our approach allows the vehicles to coordinate in tight spaces (e.g., busy highway lanes or parking lots) by using a polytopic description of each vehicle's shape and formulating collision avoidance as a dual optimization problem. Our method accommodates heterogeneous teams of vehicles (i.e., vehicles with different polytopic shapes and dynamic models can be part of the same team). Our method allows the vehicles to share their intentions in a distributed fashion without relying on a central coordinator and efficiently provides collision-free trajectories for the vehicles. In addition, our method decouples the individual-vehicles' trajectory optimization from their collision-avoidance objectives enhancing the scalability of the method and allowing one to exploit parallel hardware architectures. All these features are particularly important for vehicular applications, where the systems operate at high-frequency rates in dynamic environments. To validate our method, we apply it in a vehicular application, that is, the autonomous lane-merging of a team of connected vehicles to form a platoon. We compare our design with the centralized NMPC design to show the computational benefits of the proposed distributed algorithm.

cs.RO↗

arXiv2023

Foundation Models in Robotics: Applications, Challenges, and the Future

We survey applications of pretrained foundation models in robotics. Traditional deep learning models in robotics are trained on small datasets tailored for specific tasks, which limits their adaptability across diverse applications. In contrast, foundation models pretrained on internet-scale data appear to have superior generalization capabilities, and in some instances display an emergent ability to find zero-shot solutions to problems that are not present in the training data. Foundation models may hold the potential to enhance various components of the robot autonomy stack, from perception to decision-making and control. For example, large language models can generate code or provide common sense reasoning, while vision-language models enable open-vocabulary visual recognition. However, significant open research challenges remain, particularly around the scarcity of robot-relevant training data, safety guarantees and uncertainty quantification, and real-time execution. In this survey, we study recent papers that have used or built foundation models to solve robotics problems. We explore how foundation models contribute to improving robot capabilities in the domains of perception, decision-making, and control. We discuss the challenges hindering the adoption of foundation models in robot autonomy and provide opportunities and potential pathways for future advancements. The GitHub project corresponding to this paper (Preliminary release. We are committed to further enhancing and updating this work to ensure its quality and relevance) can be found here: https://github.com/robotics-survey/Awesome-Robotics-Foundation-Models

cs.RO↗

arXiv2023

Local Non-Cooperative Games with Principled Player Selection for Scalable Motion Planning

Game-theoretic motion planners are a powerful tool for the control of interactive multi-agent robot systems. Indeed, contrary to predict-then-plan paradigms, game-theoretic planners do not ignore the interactive nature of the problem, and simultaneously predict the behaviour of other agents while considering change in one's policy. This, however, comes at the expense of computational complexity, especially as the number of agents considered grows. In fact, planning with more than a handful of agents can quickly become intractable, disqualifying game-theoretic planners as possible candidates for large scale planning. In this paper, we propose a planning algorithm enabling the use of game-theoretic planners in robot systems with a large number of agents. Our planner is based on the reality of locality of information and thus deploys local games with a selected subset of agents in a receding horizon fashion to plan collision avoiding trajectories. We propose five different principled schemes for selecting game participants and compare their collision avoidance performance. We observe that the use of Control Barrier Functions for priority ranking is a potent solution to the player selection problem for motion planning.

cs.RO↗

arXiv2022

Intention Communication and Hypothesis Likelihood in Game-Theoretic Motion Planning

Game-theoretic motion planners are a potent solution for controlling systems of multiple highly interactive robots. Most existing game-theoretic planners unrealistically assume a priori objective function knowledge is available to all agents. To address this, we propose a fault-tolerant receding horizon game-theoretic motion planner that leverages inter-agent communication with intention hypothesis likelihood. Specifically, robots communicate their objective function incorporating their intentions. A discrete Bayesian filter is designed to infer the objectives in real-time based on the discrepancy between observed trajectories and the ones from communicated intentions. In simulation, we consider three safety-critical autonomous driving scenarios of overtaking, lane-merging and intersection crossing, to demonstrate our planner's ability to capitalize on alternative intention hypotheses to generate safe trajectories in the presence of faulty transmissions in the communication network.

cs.RO↗

arXiv2021

Cylindrical Battery Fault Detection under Extreme Fast Charging: A Physics-based Learning Approach

High power operation in extreme fast charging significantly increases the risk of internal faults in Electric Vehicle batteries which can lead to accelerated battery failure. Early detection of these faults is crucial for battery safety and widespread deployment of fast charging. In this setting, we propose a real-time {detection} framework for battery voltage and thermal faults. A major challenge in battery fault detection arises from the effect of uncertainties originating from sensor inaccuracies, nominal aging, or unmodelled dynamics. Inspired by physics-based learning, we explore a detection paradigm that combines physics-based models, model-based detection observers, and data-driven learning techniques to address this challenge. Specifically, we construct the {detection} observers based on an experimentally identified electrochemical-thermal model, and subsequently design the observer tuning parameters following Lyapunov's stability theory. Furthermore, we utilize Gaussian Process Regression technique to learn the model and measurement uncertainties which in turn aid the {detection} observers in distinguishing faults and uncertainties. Such uncertainty learning essentially helps suppressing their effects, potentially enabling early detection of faults. We perform simulation and experimental case studies on the proposed fault {detection} scheme verifying the potential of physics-based learning in early detection of battery faults.

eess.SY↗

arXiv2020

Formation and Reconfiguration of Tight Multi-Lane Platoons

Advances in vehicular communication technologies are expected to facilitate cooperative driving. Connected and Automated Vehicles (CAVs) are able to collaboratively plan and execute driving maneuvers by sharing their perceptual knowledge and future plans. In this paper, an architecture for autonomous navigation of tight multi-lane platoons travelling on public roads is presented. Using the proposed approach, CAVs are able to form single or multi-lane platoons of various geometrical configurations. They are able to reshape and adjust their configurations according to changes in the environment. The proposed architecture consists of two main components: an offline motion planner system and an online hierarchical control system. The motion planner uses an optimization-based approach for cooperative formation and reconfiguration in tight spaces. A constrained optimization scheme is used to plan smooth, dynamically feasible and collision-free trajectories for all the vehicles within the platoon. The paper addresses online computation limitations by employing a family of maneuvers precomputed offline and stored on a look-up table on the vehicles. The online hierarchical control system is composed of three levels: a traffic operation system (TOS), a decision-maker, and a path-follower. The TOS determines the desired platoon reconfiguration. The decision-maker checks the feasibility of the reconfiguration plan. The reconfiguration maneuver is executed by a low-level path-following feedback controller in real-time. The effectiveness of the approach is demonstrated through simulations of three case studies: 1) formation reconfiguration 2) obstacle avoidance, and 3) benchmarking against behavior-based planning in which the desired formation is achieved using a sequence of motion primitives. Videos and software can be found online https://github.com/RoyaFiroozi/Centralized-Planning.

cs.RO↗

arXiv2020

Improving Urban Traffic Throughput with Vehicle Platooning: Theory and Experiments

In this paper we present a model-predictive control (MPC) based approach for vehicle platooning in an urban traffic setting. Our primary goal is to demonstrate that vehicle platooning has the potential to significantly increase throughput at intersections, which can create bottlenecks in the traffic flow. To do so, our approach relies on vehicle connectivity: vehicle-to-vehicle (V2V) and vehicle-to-infrastructure (V2I) communication. In particular, we introduce a customized V2V message set which features a velocity forecast, i.e. a prediction on the future velocity trajectory, which enables platooning vehicles to accurately maintain short following distances, thereby increasing throughput. Furthermore, V2I communication allows platoons to react immediately to changes in the state of nearby traffic lights, e.g. when the traffic phase becomes green, enabling additional gains in traffic efficiency. We present our design of the vehicle platooning system, and then evaluate performance by estimating the potential gains in terms of throughput using our results from simulation, as well as experiments conducted with real test vehicles on a closed track. Lastly, we briefly overview our demonstration of vehicle platooning on public roadways in Arcadia, CA.

eess.SY↗

arXiv2018

Safe Adaptive Cruise Control with Road Grade Preview and V2V Communication

We present the design of a safe Adaptive Cruise Control (ACC) which uses road grade and lead vehicle motion preview. The ACC controller is designed by using a Model Predictive Control (MPC) framework to optimize comfort, safety, energy-efficiency and speed tracking accuracy. Safety is achieved by computing a robust invariant terminal set. The paper presents a novel approach to compute such set which is less conservative than existing methods. The proposed controller ensures safe inter-vehicle spacing at all times despite changes in the road grade and uncertainty in the predicted motion of the lead vehicle. Simulation results compare the proposed controller with a controller that does not incorporate prior grade knowledge on two scenarios including car-following and autonomous intersection crossing. The results demonstrate the effectiveness of the proposed control algorithm.

eess.SY↗

arXiv2018

Vehicle Localization and Control on Roads with Prior Grade Map

We propose a map-aided vehicle localization method for GPS-denied environments. This approach exploits prior knowledge of the road grade map and vehicle on-board sensor measurements to accurately estimate the longitudinal position of the vehicle. Real-time localization is crucial to systems that utilize position-dependent information for planning and control. We validate the effectiveness of the localization method on a hierarchical control system. The higher level planner optimizes the vehicle velocity to minimize the energy consumption for a given route by employing traffic condition and road grade data. The lower level is a cruise control system that tracks the position-dependent optimal reference velocity. Performance of the proposed localization algorithm is evaluated using both simulations and experiments.

cs.RO↗