arXiv subjects

Jorge Peña Queralta

Publications and source records attributed to Jorge Peña Queralta.

At least 37 records · Page 2Linked to original sources

arXiv2022

Distributed Robotic Systems in the Edge-Cloud Continuum with ROS 2: a Review on Novel Architectures and Technology Readiness

Robotic systems are more connected, networked, and distributed than ever. New architectures that comply with the \textit{de facto} robotics middleware standard, ROS\,2, have recently emerged to fill the gap in terms of hybrid systems deployed from edge to cloud. This paper reviews new architectures and technologies that enable containerized robotic applications to seamlessly run at the edge or in the cloud. We also overview systems that include solutions from extension to ROS\,2 tooling to the integration of Kubernetes and ROS\,2. Another important trend is robot learning, and how new simulators and cloud simulations are enabling, e.g., large-scale reinforcement learning or distributed federated learning solutions. This has also enabled deeper integration of continuous interaction and continuous deployment (CI/CD) pipelines for robotic systems development, going beyond standard software unit tests with simulated tests to build and validate code automatically. We discuss the current technology readiness and list the potential new application scenarios that are becoming available. Finally, we discuss the current challenges in distributed robotic systems and list open research questions in the field.

cs.RO↗

arXiv2022

Vision-based GNSS-Free Localization for UAVs in the Wild

Considering the accelerated development of Unmanned Aerial Vehicles (UAVs) applications in both industrial and research scenarios, there is an increasing need for localizing these aerial systems in non-urban environments, using GNSS-Free, vision-based methods. Our paper proposes a vision-based localization algorithm that utilizes deep features to compute geographical coordinates of a UAV flying in the wild. The method is based on matching salient features of RGB photographs captured by the drone camera and sections of a pre-built map consisting of georeferenced open-source satellite images. Experimental results prove that vision-based localization has comparable accuracy with traditional GNSS-based methods, which serve as ground truth. Compared to state-of-the-art Visual Odometry (VO) approaches, our solution is designed for long-distance, high-altitude UAV flights. Code and datasets are available at https://github.com/TIERS/wildnav.

cs.RO↗

arXiv2022

Decentralized Vision-Based Byzantine Agent Detection in Multi-Robot Systems with IOTA Smart Contracts

Multiple opportunities lie at the intersection of multi-robot systems and distributed ledger technologies (DLTs). In this work, we investigate the potential of new DLT solutions such as IOTA, for detecting anomalies and byzantine agents in multi-robot systems in a decentralized manner. Traditional blockchain approaches are not applicable to real-world networked and decentralized robotic systems where connectivity conditions are not ideal. To address this, we leverage recent advances in partition-tolerant and byzantine-tolerant collaborative decision-making processes with IOTA smart contracts. We show how our work in vision-based anomaly and change detection can be applied to detecting byzantine agents within multiple robots operating in the same environment. We show that IOTA smart contracts add a low computational overhead while allowing to build trust within the multi-robot system. The proposed approach effectively enables byzantine robot detection based on the comparison of images submitted by the different robots and detection of anomalies and changes between them.

cs.RO↗

arXiv2022

A Benchmark for Multi-Modal Lidar SLAM with Ground Truth in GNSS-Denied Environments

Lidar-based simultaneous localization and mapping (SLAM) approaches have obtained considerable success in autonomous robotic systems. This is in part owing to the high-accuracy of robust SLAM algorithms and the emergence of new and lower-cost lidar products. This study benchmarks current state-of-the-art lidar SLAM algorithms with a multi-modal lidar sensor setup showcasing diverse scanning modalities (spinning and solid-state) and sensing technologies, and lidar cameras, mounted on a mobile sensing and computing platform. We extend our previous multi-modal multi-lidar dataset with additional sequences and new sources of ground truth data. Specifically, we propose a new multi-modal multi-lidar SLAM-assisted and ICP-based sensor fusion method for generating ground truth maps. With these maps, we then match real-time pointcloud data using a natural distribution transform (NDT) method to obtain the ground truth with full 6 DOF pose estimation. This novel ground truth data leverages high-resolution spinning and solid-state lidars. We also include new open road sequences with GNSS-RTK data and additional indoor sequences with motion capture (MOCAP) ground truth, complementing the previous forest sequences with MOCAP data. We perform an analysis of the positioning accuracy achieved with ten different SLAM algorithm and lidar combinations. We also report the resource utilization in four different computational platforms and a total of five settings (Intel and Jetson ARM CPUs). Our experimental results show that current state-of-the-art lidar SLAM algorithms perform very differently for different types of sensors. More results, code, and the dataset can be found at: \href{https://github.com/TIERS/tiers-lidars-dataset-enhanced}{github.com/TIERS/tiers-lidars-dataset-enhanced.

cs.RO↗

arXiv2022

Partition-Tolerant and Byzantine-Tolerant Decision-Making for Distributed Robotic Systems with IOTA and ROS 2

With the increasing ubiquity of autonomous robotic solutions, the interest in their connectivity and in the cooperation within multi-robot systems is rising. Two aspects that are a matter of current research are robot security and secure multi-robot collaboration robust to byzantine agents. Blockchain and other distributed ledger technologies (DLTs) have been proposed to address the challenges in both domains. Nonetheless, some key challenges include scalability and deployment within real-world networks. This paper presents an approach to integrating IOTA and ROS 2 for more scalable DLT-based robotic systems while allowing for network partition tolerance after deployment. This is, to the best of our knowledge, the first implementation of IOTA smart contracts for robotic systems, and the first integrated design with ROS 2. This is in comparison to the vast majority of the literature which relies on Ethereum. We present a general IOTA+ROS 2 architecture leading to partition-tolerant decision-making processes that also inherit byzantine tolerance properties from the embedded blockchain structures. We demonstrate the effectiveness of the proposed framework for a cooperative mapping application in a system with intermittent network connectivity. We show both superior performance with respect to Ethereum in the presence of network partitions, and a low impact in terms of computational resource utilization. These results open the path for wider integration of blockchain solutions in distributed robotic systems with less stringent connectivity and computational requirements.

cs.RO↗

arXiv2022

UWB Role Allocation with Distributed Ledger Technologies for Scalable Relative Localization in Multi-Robot Systems

Systems for relative localization in multi-robot systems based on ultra-wideband (UWB) ranging have recently emerged as robust solutions for GNSS-denied environments. Scalability remains one of the key challenges, particularly in ad-hoc deployments. Recent solutions include dynamic allocation of active and passive localization modes for different robots or nodes in the system. With larger-scale systems becoming more distributed, key research questions arise in the areas of security and trustability of such localization systems. This paper studies the potential integration of collaborative-decision making processes with distributed ledger technologies. Specifically, we investigate the design and implementation of a methodology for running an UWB role allocation algorithm within smart contracts in a blockchain. In previous works, we have separately studied the integration of ROS2 with the Hyperledger Fabric blockchain, and introduced a new algorithm for scalable UWB-based localization. In this paper, we extend these works by (i) running experiments with larger number of mobile robots switching between different spatial configurations and (ii) integrating the dynamic UWB role allocation algorithm into Fabric smart contracts for distributed decision-making in a system of multiple mobile robots. This enables us to deliver the same functionality within a secure and trustable process, with enhanced identity and data access management. Our results show the effectiveness of the UWB role allocation for continuously varying spatial formations of six autonomous mobile robots, while demonstrating a low impact on latency and computational resources of adding the blockchain layer that does not affect the localization process.

cs.RO↗

arXiv2022

Self-Calibrating Anomaly and Change Detection for Autonomous Inspection Robots

Automatic detection of visual anomalies and changes in the environment has been a topic of recurrent attention in the fields of machine learning and computer vision over the past decades. A visual anomaly or change detection algorithm identifies regions of an image that differ from a reference image or dataset. The majority of existing approaches focus on anomaly or fault detection in a specific class of images or environments, while general purpose visual anomaly detection algorithms are more scarce in the literature. In this paper, we propose a comprehensive deep learning framework for detecting anomalies and changes in a priori unknown environments after a reference dataset is gathered, and without need for retraining the model. We use the SuperPoint and SuperGlue feature extraction and matching methods to detect anomalies based on reference images taken from a similar location and with partial overlapping of the field of view. We also introduce a self-calibrating method for the proposed model in order to address the problem of sensitivity to feature matching thresholds and environmental conditions. To evaluate the proposed framework, we have used a ground robot system for the purpose of reference and query data collection. We show that high accuracy can be obtained using the proposed method. We also show that the calibration process enhances changes and foreign object detection performance

cs.CV↗

arXiv2022

Hyperledger Fabric Blockchain and ROS 2 Integration for Autonomous Mobile Robots

In industrial applications, security and trust in the system are requirements for widespread adoption. Blockchain technologies have emerged as a potential solution to address identity management and secure data aggregation and control. However, the vast majority of works to date utilize Ethereum and smart contracts that are not scalable or well suited for industrial applications. This paper presents what is, to the best of our knowledge, the first integration of ROS 2 with the Hyperledger Fabric blockchain. With a framework that leverages Fabric smart contracts and ROS 2 through a Go application, we delve into the potential of using blockchain for controlling robots, and gathering and processing their data. We demonstrate the applicability of the proposed framework to an inventory management use-case where different robots are used to detect objects of interest in a given area. Designed to meet the requirements of distributed robotic systems, we show that the performance of the robots is not impacted significantly by the blockchain layer. At the same time, we provide examples for developing other applications that integrate Fabric smart contracts with ROS 2. Our results pave the way for further adoption of blockchain technologies in autonomous robotic systems for building trustable data sharing.

cs.RO↗

arXiv2022

Secure Heterogeneous Multi-Robot Collaboration and Docking with Hyperledger Fabric Blockchain

In recent years, multi-robot systems have received increasing attention from both industry and academia. Besides the need of accurate and robust estimation of relative localization, security and trust in the system are essential to enable wider adoption. In this paper, we propose a framework using Hyperledger Fabric for multi-robot collaboration in industrial applications. We rely on blockchain identities for the interaction of ground and aerial robots, and use smart contracts for collaborative decision making. The use of ultra-wideband (UWB) localization for both autonomous navigation and robot collaboration extends our previous work in Fabric-based fleet management. We focus on an inventory management application which uses a ground robot and an aerial robot to inspect a warehouse-like environment and store information about the found objects in the blockchain. We measure the impact of adding the blockchain layer, analyze the transaction commit latency and compare the resource utilization of blockchain-related processes to the already running data processing modules.

cs.RO↗

arXiv2022

Federated Learning for Vision-based Obstacle Avoidance in the Internet of Robotic Things

Deep learning methods have revolutionized mobile robotics, from advanced perception models for an enhanced situational awareness to novel control approaches through reinforcement learning. This paper explores the potential of federated learning for distributed systems of mobile robots enabling collaboration on the Internet of Robotic Things. To demonstrate the effectiveness of such an approach, we deploy wheeled robots in different indoor environments. We analyze the performance of a federated learning approach and compare it to a traditional centralized training process with a priori aggregated data. We show the benefits of collaborative learning across heterogeneous environments and the potential for sim-to-real knowledge transfer. Our results demonstrate significant performance benefits of FL and sim-to-real transfer for vision-based navigation, in addition to the inherent privacy-preserving nature of FL by keeping computation at the edge. This is, to the best of our knowledge, the first work to leverage FL for vision-based navigation that also tests results in real-world settings.

cs.RO↗

arXiv2022

VIO-UWB-Based Collaborative Localization and Dense Scene Reconstruction within Heterogeneous Multi-Robot Systems

Effective collaboration in multi-robot systems requires accurate and robust estimation of relative localization: from cooperative manipulation to collaborative sensing, and including cooperative exploration or cooperative transportation. This paper introduces a novel approach to collaborative localization for dense scene reconstruction in heterogeneous multi-robot systems comprising ground robots and micro-aerial vehicles (MAVs). We solve the problem of full relative pose estimation without sliding time windows by relying on UWB-based ranging and Visual Inertial Odometry (VIO)-based egomotion estimation for localization, while exploiting lidars onboard the ground robots for full relative pose estimation in a single reference frame. During operation, the rigidity eigenvalue provides feedback to the system. To tackle the challenge of path planning and obstacle avoidance of MAVs in GNSS-denied environments, we maintain line-of-sight between ground robots and MAVs. Because lidars capable of dense reconstruction have limited FoV, this introduces new constraints to the system. Therefore, we propose a novel formulation with a variant of the Dubins multiple traveling salesman problem with neighborhoods (DMTSPN) where we include constraints related to the limited FoV of the ground robots. Our approach is validated with simulations and experiments with real robots for the different parts of the system.

cs.RO↗

arXiv2022

Secure Encoded Instruction Graphs for End-to-End Data Validation in Autonomous Robots

As autonomous robots are becoming more widespread, more attention is being paid to the security of robotic operation. Autonomous robots can be seen as cyber-physical systems: they can operate in virtual, physical, and human realms. Therefore, securing the operations of autonomous robots requires not only securing their data (e.g., sensor inputs and mission instructions) but securing their interactions with their environment. There is currently a deficiency of methods that would allow robots to securely ensure their sensors and actuators are operating correctly without external feedback. This paper introduces an encoding method and end-to-end validation framework for the missions of autonomous robots. In particular, we present a proof of concept of a map encoding method, which allows robots to navigate realistic environments and validate operational instructions with almost zero {\it a priori} knowledge. We demonstrate our framework using two different encoded maps in experiments with simulated and real robots. Our encoded maps have the same advantages as typical landmark-based navigation, but with the added benefit of cryptographic hashes that enable end-to-end information validation. Our method is applicable to any aspect of robotic operation in which there is a predefined set of actions or instructions given to the robot.

cs.RO↗

arXiv2022

Towards Large-Scale Relative Localization in Multi-Robot Systems with Dynamic UWB Role Allocation

Ultra-wideband (UWB) ranging has emerged as a key radio technology for robot positioning and relative localization in multi-robot systems. Multiple works are now advancing towards more scalable systems, but challenges still remain. This paper proposes a novel approach to relative localization in multi-robot systems where the roles of the UWB nodes are dynamically allocated between active nodes (using time-of-flight for ranging estimation to other active nodes) and passive nodes (using time-difference-of-arrival for estimating range differences with respect to pairs of active nodes). We adaptively update UWB roles based on the location of the robots with respect to the convex envelope defined by active nodes, and introducing constraints in the form of localization frequency and accuracy requirements. We demonstrate the applicability of the proposed approach and show that the localization errors remain comparable to fixed-role systems. Then, we show how the navigation of an autonomous drone is affected by the changes in the localization system, obtaining significantly better trajectory tracking accuracy than when relying in passive localization only. Our results pave the way for UWB-based localization in large-scale multi-robot deployments, for either relative positioning or for applications in GNSS-denied environments.

cs.RO↗

arXiv2022

Analyzing General-Purpose Deep-Learning Detection and Segmentation Models with Images from a Lidar as a Camera Sensor

Over the last decade, robotic perception algorithms have significantly benefited from the rapid advances in deep learning (DL). Indeed, a significant amount of the autonomy stack of different commercial and research platforms relies on DL for situational awareness, especially vision sensors. This work explores the potential of general-purpose DL perception algorithms, specifically detection and segmentation neural networks, for processing image-like outputs of advanced lidar sensors. Rather than processing the three-dimensional point cloud data, this is, to the best of our knowledge, the first work to focus on low-resolution images with 360\textdegree field of view obtained with lidar sensors by encoding either depth, reflectivity, or near-infrared light in the image pixels. We show that with adequate preprocessing, general-purpose DL models can process these images, opening the door to their usage in environmental conditions where vision sensors present inherent limitations. We provide both a qualitative and quantitative analysis of the performance of a variety of neural network architectures. We believe that using DL models built for visual cameras offers significant advantages due to the much wider availability and maturity compared to point cloud-based perception.

cs.RO↗

arXiv2022

Multi-Modal Lidar Dataset for Benchmarking General-Purpose Localization and Mapping Algorithms

Lidar technology has evolved significantly over the last decade, with higher resolution, better accuracy, and lower cost devices available today. In addition, new scanning modalities and novel sensor technologies have emerged in recent years. Public datasets have enabled benchmarking of algorithms and have set standards for the cutting edge technology. However, existing datasets are not representative of the technological landscape, with only a reduced number of lidars available. This inherently limits the development and comparison of general-purpose algorithms in the evolving landscape. This paper presents a novel multi-modal lidar dataset with sensors showcasing different scanning modalities (spinning and solid-state), sensing technologies, and lidar cameras. The focus of the dataset is on low-drift odometry, with ground truth data available in both indoors and outdoors environment with sub-millimeter accuracy from a motion capture (MOCAP) system. For comparison in longer distances, we also include data recorded in larger spaces indoors and outdoors. The dataset contains point cloud data from spinning lidars and solid-state lidars. Also, it provides range images from high resolution spinning lidars, RGB and depth images from a lidar camera, and inertial data from built-in IMUs. This is, to the best of our knowledge, the lidar dataset with the most variety of sensors and environments where ground truth data is available. This dataset can be widely used in multiple research areas, such as 3D LiDAR simultaneous localization and mapping (SLAM), performance comparison between multi-modal lidars, appearance recognition and loop closure detection. The datasets are available at: https://github.com/TIERS/tiers-lidars-dataset.

cs.RO↗

arXiv2021

Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey

Deep reinforcement learning has recently seen huge success across multiple areas in the robotics domain. Owing to the limitations of gathering real-world data, i.e., sample inefficiency and the cost of collecting it, simulation environments are utilized for training the different agents. This not only aids in providing a potentially infinite data source, but also alleviates safety concerns with real robots. Nonetheless, the gap between the simulated and real worlds degrades the performance of the policies once the models are transferred into real robots. Multiple research efforts are therefore now being directed towards closing this sim-to-real gap and accomplish more efficient policy transfer. Recent years have seen the emergence of multiple methods applicable to different domains, but there is a lack, to the best of our knowledge, of a comprehensive review summarizing and putting into context the different methods. In this survey paper, we cover the fundamental background behind sim-to-real transfer in deep reinforcement learning and overview the main methods being utilized at the moment: domain randomization, domain adaptation, imitation learning, meta-learning and knowledge distillation. We categorize some of the most relevant recent works, and outline the main application scenarios. Finally, we discuss the main opportunities and challenges of the different approaches and point to the most promising directions.

cs.LG↗

arXiv2021

Adaptive Lidar Scan Frame Integration: Tracking Known MAVs in 3D Point Clouds

Micro-aerial vehicles (MAVs) are becoming ubiquitous across multiple industries and application domains. Lightweight MAVs with only an onboard flight controller and a minimal sensor suite (e.g., IMU, vision, and vertical ranging sensors) have potential as mobile and easily deployable sensing platforms. When deployed from a ground robot, a key parameter is a relative localization between the ground robot and the MAV. This paper proposes a novel method for tracking MAVs in lidar point clouds. In lidar point clouds, we consider the speed and distance of the MAV to actively adapt the lidar's frame integration time and, in essence, the density and size of the point cloud to be processed. We show that this method enables more persistent and robust tracking when the speed of the MAV or its distance to the tracking sensor changes. In addition, we propose a multi-modal tracking method that relies on high-frequency scans for accurate state estimation, lower-frequency scans for robust and persistent tracking, and sub-Hz processing for trajectory and object identification. These three integration and processing modalities allow for an overall accurate and robust MAV tracking while ensuring the object being tracked meets shape and size constraints.

cs.RO↗

arXiv2021

An Overview of Federated Learning at the Edge and Distributed Ledger Technologies for Robotic and Autonomous Systems

Autonomous systems are becoming inherently ubiquitous with the advancements of computing and communication solutions enabling low-latency offloading and real-time collaboration of distributed devices. Decentralized technologies with blockchain and distributed ledger technologies (DLTs) are playing a key role. At the same time, advances in deep learning (DL) have significantly raised the degree of autonomy and level of intelligence of robotic and autonomous systems. While these technological revolutions were taking place, raising concerns in terms of data security and end-user privacy has become an inescapable research consideration. Federated learning (FL) is a promising solution to privacy-preserving DL at the edge, with an inherently distributed nature by learning on isolated data islands and communicating only model updates. However, FL by itself does not provide the levels of security and robustness required by today's standards in distributed autonomous systems. This survey covers applications of FL to autonomous robots, analyzes the role of DLT and FL for these systems, and introduces the key background concepts and considerations in current research.

cs.RO↗