arXiv subjects

Ram Vasudevan

Publications and source records attributed to Ram Vasudevan.

At least 73 records · Page 4Linked to original sources

arXiv2021

Reachability-based Trajectory Safeguard (RTS): A Safe and Fast Reinforcement Learning Safety Layer for Continuous Control

Reinforcement Learning (RL) algorithms have achieved remarkable performance in decision making and control tasks due to their ability to reason about long-term, cumulative reward using trial and error. However, during RL training, applying this trial-and-error approach to real-world robots operating in safety critical environment may lead to collisions. To address this challenge, this paper proposes a Reachability-based Trajectory Safeguard (RTS), which leverages reachability analysis to ensure safety during training and operation. Given a known (but uncertain) model of a robot, RTS precomputes a Forward Reachable Set of the robot tracking a continuum of parameterized trajectories. At runtime, the RL agent selects from this continuum in a receding-horizon way to control the robot; the FRS is used to identify if the agent's choice is safe or not, and to adjust unsafe choices. The efficacy of this method is illustrated on three nonlinear robot models, including a 12-D quadrotor drone, in simulation and in comparison with state-of-the-art safe motion planning methods.

cs.RO↗

arXiv2021

Learning Rotation-Invariant Representations of Point Clouds Using Aligned Edge Convolutional Neural Networks

Point cloud analysis is an area of increasing interest due to the development of 3D sensors that are able to rapidly measure the depth of scenes accurately. Unfortunately, applying deep learning techniques to perform point cloud analysis is non-trivial due to the inability of these methods to generalize to unseen rotations. To address this limitation, one usually has to augment the training data, which can lead to extra computation and require larger model complexity. This paper proposes a new neural network called the Aligned Edge Convolutional Neural Network (AECNN) that learns a feature representation of point clouds relative to Local Reference Frames (LRFs) to ensure invariance to rotation. In particular, features are learned locally and aligned with respect to the LRF of an automatically computed reference point. The proposed approach is evaluated on point cloud classification and part segmentation tasks. This paper illustrates that the proposed technique outperforms a variety of state of the art approaches (even those trained on augmented datasets) in terms of robustness to rotation without requiring any additional data augmentation.

cs.CV↗

arXiv2021

Point Set Voting for Partial Point Cloud Analysis

The continual improvement of 3D sensors has driven the development of algorithms to perform point cloud analysis. In fact, techniques for point cloud classification and segmentation have in recent years achieved incredible performance driven in part by leveraging large synthetic datasets. Unfortunately these same state-of-the-art approaches perform poorly when applied to incomplete point clouds. This limitation of existing algorithms is particularly concerning since point clouds generated by 3D sensors in the real world are usually incomplete due to perspective view or occlusion by other objects. This paper proposes a general model for partial point clouds analysis wherein the latent feature encoding a complete point clouds is inferred by applying a local point set voting strategy. In particular, each local point set constructs a vote that corresponds to a distribution in the latent space, and the optimal latent feature is the one with the highest probability. This approach ensures that any subsequent point cloud analysis is robust to partial observation while simultaneously guaranteeing that the proposed model is able to output multiple possible results. This paper illustrates that this proposed method achieves state-of-the-art performance on shape classification, part segmentation and point cloud completion.

cs.CV↗

arXiv2020

BiTraP: Bi-directional Pedestrian Trajectory Prediction with Multi-modal Goal Estimation

Pedestrian trajectory prediction is an essential task in robotic applications such as autonomous driving and robot navigation. State-of-the-art trajectory predictors use a conditional variational autoencoder (CVAE) with recurrent neural networks (RNNs) to encode observed trajectories and decode multi-modal future trajectories. This process can suffer from accumulated errors over long prediction horizons (>=2 seconds). This paper presents BiTraP, a goal-conditioned bi-directional multi-modal trajectory prediction method based on the CVAE. BiTraP estimates the goal (end-point) of trajectories and introduces a novel bi-directional decoder to improve longer-term trajectory prediction accuracy. Extensive experiments show that BiTraP generalizes to both first-person view (FPV) and bird's-eye view (BEV) scenarios and outperforms state-of-the-art results by ~10-50%. We also show that different choices of non-parametric versus parametric target models in the CVAE directly influence the predicted multi-modal trajectory distributions. These results provide guidance on trajectory predictor design for robotic applications such as collision avoidance and navigation systems.

cs.CV↗

arXiv2020

Advantages of Bilinear Koopman Realizations for the Modeling and Control of Systems with Unknown Dynamics

Nonlinear dynamical systems can be made easier to control by lifting them into the space of observable functions, where their evolution is described by the linear Koopman operator. This paper describes how the Koopman operator can be used to generate approximate linear, bilinear, and nonlinear model realizations from data, and argues in favor of bilinear realizations for characterizing systems with unknown dynamics. Necessary and sufficient conditions for a dynamical system to have a valid linear or bilinear realization over a given set of observable functions are presented and used to show that every control-affine system admits an infinite-dimensional bilinear realization, but does not necessarily admit a linear one. Therefore, approximate bilinear realizations constructed from generic sets of basis functions tend to improve as the number of basis functions increases, whereas approximate linear realizations may not. To demonstrate the advantages of bilinear Koopman realizations for control, a linear, bilinear, and nonlinear Koopman model realization of a simulated robot arm are constructed from data. In a trajectory following task, the bilinear realization exceeds the prediction accuracy of the linear realization and the computational efficiency of the nonlinear realization when incorporated into a model predictive control framework.

cs.RO↗

arXiv2020

Trip Recovery in Lower-Limb Prostheses using Reachable Sets of Predicted Human Motion

People with lower-limb loss, the majority of which use passive prostheses, exhibit a high incidence of falls each year. Powered lower-limb prostheses have the potential to reduce fall rates by actively helping the user recover from a stumble, but the unpredictability of the human response makes it difficult to design controllers that ensure a successful recovery. This paper presents a method called TRIP-RTD (Trip Recovery in Prostheses via Reachability-based Trajectory Design) for online trajectory planning in a knee prosthesis during and after a stumble that can accommodate a set of possible predictions of human behavior. Using this predicted set of human behavior, the proposed method computes a parameterized reachable set of trajectories for the human-prosthesis system. To ensure safety at run-time, TRIP-RTD selects a trajectory for the prosthesis that guarantees that all possible states of the human-prosthesis system at touchdown arrive in the basin of attraction of the nominal behavior of the system. In simulated stumble experiments where a nominal phase-based controller was unable to help the system recover, TRIP-RTD produced trajectories in under 101 ms that led to successful recoveries for all feasible solutions found.

cs.RO↗

arXiv2020

Reachable Sets for Safe, Real-Time Manipulator Trajectory Design

For robotic arms to operate in arbitrary environments, especially near people, it is critical to certify the safety of their motion planning algorithms. However, there is often a trade-off between safety and real-time performance; one can either carefully design safe plans, or rapidly generate potentially-unsafe plans. This work presents a receding-horizon, real-time trajectory planner with safety guarantees, called ARMTD (Autonomous Reachability-based Manipulator Trajectory Design). The method first computes (offline) a reachable set of parameterized trajectories for each joint of an arm. Each trajectory includes a fail-safe maneuver (braking to a stop). At runtime, in each receding-horizon planning iteration, ARMTD constructs a parameterized reachable set of the full arm in workspace and intersects it with obstacles to generate sub-differentiable, provably-conservative collision-avoidance constraints on the trajectory parameters. ARMTD then performs trajectory optimization over the parameters, subject to these constraints. On a 6 degree-of-freedom arm, ARMTD outperforms CHOMP in simulation, never crashes, and completes a variety of real-time planning tasks on hardware.

cs.RO↗

arXiv2020

LiStereo: Generate Dense Depth Maps from LIDAR and Stereo Imagery

An accurate depth map of the environment is critical to the safe operation of autonomous robots and vehicles. Currently, either light detection and ranging (LIDAR) or stereo matching algorithms are used to acquire such depth information. However, a high-resolution LIDAR is expensive and produces sparse depth map at large range; stereo matching algorithms are able to generate denser depth maps but are typically less accurate than LIDAR at long range. This paper combines these approaches together to generate high-quality dense depth maps. Unlike previous approaches that are trained using ground-truth labels, the proposed model adopts a self-supervised training process. Experiments show that the proposed method is able to generate high-quality dense depth maps and performs robustly even with low-resolution inputs. This shows the potential to reduce the cost by using LIDARs with lower resolution in concert with stereo systems while maintaining high resolution.

cs.RO↗

arXiv2020

Pixel-Wise Motion Deblurring of Thermal Videos

Uncooled microbolometers can enable robots to see in the absence of visible illumination by imaging the "heat" radiated from the scene. Despite this ability to see in the dark, these sensors suffer from significant motion blur. This has limited their application on robotic systems. As described in this paper, this motion blur arises due to the thermal inertia of each pixel. This has meant that traditional motion deblurring techniques, which rely on identifying an appropriate spatial blur kernel to perform spatial deconvolution, are unable to reliably perform motion deblurring on thermal camera images. To address this problem, this paper formulates reversing the effect of thermal inertia at a single pixel as a Least Absolute Shrinkage and Selection Operator (LASSO) problem which we can solve rapidly using a quadratic programming solver. By leveraging sparsity and a high frame rate, this pixel-wise LASSO formulation is able to recover motion deblurred frames of thermal videos without using any spatial information. To compare its quality against state-of-the-art visible camera based deblurring methods, this paper evaluated the performance of a family of pre-trained object detectors on a set of images restored by different deblurring algorithms. All evaluated object detectors performed systematically better on images restored by the proposed algorithm rather than any other tested, state-of-the-art methods.

cs.CV↗

arXiv2020

Off The Beaten Sidewalk: Pedestrian Prediction In Shared Spaces For Autonomous Vehicles

Pedestrians and drivers interact closely in a wide range of environments. Autonomous vehicles (AVs) correspondingly face the need to predict pedestrians' future trajectories in these same environments. Traditional model-based prediction methods have been limited to making predictions in highly structured scenes with signalized intersections, marked crosswalks, or curbs. Deep learning methods have instead leveraged datasets to learn predictive features that generalize across scenes, at the cost of model interpretability. This paper aims to achieve both widely applicable and interpretable predictions by proposing a risk-based attention mechanism to learn when pedestrians yield, and a model of vehicle influence to learn how yielding affects motion. A novel probabilistic method, Off the Sidewalk Predictions (OSP), uses these to achieve accurate predictions in both shared spaces and traditional scenes. Experiments on urban datasets demonstrate that the realtime method achieves state-of-the-art performance.

cs.RO↗

arXiv2020

Bridging the Gap Between Safety and Real-Time Performance in Receding-Horizon Trajectory Design for Mobile Robots

To operate with limited sensor horizons in unpredictable environments, autonomous robots use a receding-horizon strategy to plan trajectories, wherein they execute a short plan while creating the next plan. However, creating safe, dynamically-feasible trajectories in real time is challenging; and, planners must ensure persistent feasibility, meaning a new trajectory is always available before the previous one has finished executing. Existing approaches make a tradeoff between model complexity and planning speed, which can require sacrificing guarantees of safety and dynamic feasibility. This work presents the Reachability-based Trajectory Design (RTD) method for trajectory planning. RTD begins with an offline Forward Reachable Set (FRS) computation of a robot's motion when tracking parameterized trajectories; the FRS provably bounds tracking error. At runtime, the FRS is used to map obstacles to parameterized trajectories, allowing RTD to select a safe trajectory at every planning iteration. RTD prescribes an obstacle representation to ensure that obstacle constraints can be created and evaluated in real time while maintaining safety. Persistent feasibility is achieved by prescribing a minimum sensor horizon and a minimum duration for the planned trajectories. A system decomposition approach is used to improve the tractability of computing the FRS, allowing RTD to create more complex plans at runtime. RTD is compared in simulation with Rapidly-Exploring Random Trees and Nonlinear Model-Predictive Control. RTD is also demonstrated in randomly-crafted environments on two hardware platforms: a differential-drive Segway, and a car-like Rover. The proposed method is safe and persistently feasible across thousands of simulations and dozens of real-world hardware demos.

cs.RO↗

arXiv2020

Covariance-Robust Dynamic Watermarking

Attack detection and mitigation strategies for cyberphysical systems (CPS) are an active area of research, and researchers have developed a variety of attack-detection tools such as dynamic watermarking. However, such methods often make assumptions that are difficult to guarantee, such as exact knowledge of the distribution of measurement noise. Here, we develop a new dynamic watermarking method that we call covariance-robust dynamic watermarking, which is able to handle uncertainties in the covariance of measurement noise. Specifically, we consider two cases. In the first this covariance is fixed but unknown, and in the second this covariance is slowly-varying. For our tests, we only require knowledge of a set within which the covariance lies. Furthermore, we connect this problem to that of algorithmic fairness and the nascent field of fair hypothesis testing, and we show that our tests satisfy some notions of fairness. Finally, we exhibit the efficacy of our tests on empirical examples chosen to reflect values observed in a standard simulation model of autonomous vehicles.

eess.SY↗

arXiv2020

Safe, Optimal, Real-time Trajectory Planning with a Parallel Constrained Bernstein Algorithm

To move through the world, mobile robots typically use a receding-horizon strategy, wherein they execute an old plan while computing a new plan to incorporate new sensor information. A plan should be dynamically feasible, meaning it obeys constraints like the robot's dynamics and obstacle avoidance; it should have liveness, meaning the robot does not stop to plan so frequently that it cannot accomplish tasks; and it should be optimal, meaning that the robot tries to satisfy a user-specified cost function such as reaching a goal location as quickly as possible. Reachability-based Trajectory Design (RTD) is a planning method that can generate provably dynamically-feasible plans. However, RTD solves a nonlinear polynmial optimization program at each planning iteration, preventing optimality guarantees; furthermore, RTD can struggle with liveness because the robot must brake to a stop when the solver finds local minima or cannot find a feasible solution. This paper proposes RTD*, which certifiably finds the globally optimal plan (if such a plan exists) at each planning iteration. This method is enabled by a novel Parallelized Constrained Bernstein Algorithm (PCBA), which is a branch-and-bound method for polynomial optimization. The contributions of this paper are: the implementation of PCBA; proofs of bounds on the time and memory usage of PCBA; a comparison of PCBA to state of the art solvers; and the demonstration of PCBA/RTD* on a mobile robot. RTD* outperforms RTD in terms of optimality and liveness for real-time planning in a variety of environments with randomly-placed obstacles.

math.OC↗

arXiv2020

On-Demand Trajectory Predictions for Interaction Aware Highway Driving

Highway driving places significant demands on human drivers and autonomous vehicles (AVs) alike due to high speeds and the complex interactions in dense traffic. Merging onto the highway poses additional challenges by limiting the amount of time available for decision-making. Predicting others' trajectories accurately and quickly is crucial to safely executing maneuvers. Many existing prediction methods based on neural networks have focused on modeling interactions to achieve better accuracy while assuming the existence of observation windows over 3s long. This paper proposes a novel probabilistic model for trajectory prediction that performs competitively with as little as 400ms of observations. The proposed model extends a deterministic car-following model to the probabilistic setting by treating model parameters as unknown random variables and introducing regularization terms. A realtime inference procedure is derived to estimate the parameters from observations in this new model. Experiments on dense traffic in the NGSIM dataset demonstrate that the proposed method achieves state-of-the-art performance with both highly constrained and more traditional observation windows.

cs.RO↗

arXiv2020

Emulating duration and curvature of coral snake anti-predator thrashing behaviors using a soft-robotic platform

This paper presents a soft-robotic platform for exploring the ecological relevance of non-locomotory movements via animal-robot interactions. Coral snakes (genus Micrurus) and their mimics use vigorous, non-locomotory, and arrhythmic thrashing to deter predation. There is variation across snake species in the duration and curvature of anti-predator thrashes, and it is unclear how these aspects of motion interact to contribute to snake survival. In this work, soft robots composed of fiber-reinforced elastomeric enclosures (FREEs) are developed to emulate the anti-predator behaviors of three genera of snake. Curvature and duration of motion are estimated for both live snakes and robots, providing a quantitative assessment of the robots' ability to emulate snake poses. The curvature values of the fabricated soft-robotic head, midsection, and tail segments are found to overlap with those exhibited by live snakes. Soft robot motion durations were less than or equal to those of snakes for all three genera. Additionally, combinations of segments were selected to emulate three specific snake genera with distinct anti-predatory behavior, producing curvature values that aligned well with live snake observations.

cs.RO↗

arXiv2020

Koopman-based Control of a Soft Continuum Manipulator Under Variable Loading Conditions

Controlling soft continuum manipulator arms is difficult due to their infinite degrees of freedom, nonlinear material properties, and large deflections under loading. This paper presents a data-driven approach to identifying soft manipulator models that enables consistent control under variable loading conditions. This is achieved by incorporating loads into a linear Koopman operator model as states and estimating their values online via an observer within the control loop. Using this approach, real-time, fully autonomous control of a pneumatically actuated soft continuum manipulator is achieved. In several trajectory following experiments, this controller is shown to be more accurate and precise than controllers based on models that are unable to explicitly account for loading. The manipulator also successfully performs pick and place of objects with unknown mass, demonstrating the efficacy of this approach in executing real-world manipulation tasks.

cs.RO↗

arXiv2020

Detecting Deception Attacks on Autonomous Vehicles via Linear Time-Varying Dynamic Watermarking

Cyber-physical systems (CPS) such as autonomous vehicles rely on both on-board sensors and external communications to estimate their state. Unfortunately, these communications render the system vulnerable to cyber-attacks. While many attack detection methods have begun to address these concerns, they are limited to linear time-invariant (LTI) systems. Though LTI system models provide accurate approximations for CPS such as autonomous vehicles at constant speed and turning radii, they are inaccurate for more complex motions such as lane changes, turns, and changes in velocity. Since these more complex motions are more suitably described by linear time-varying (LTV) system models rather than LTI models, Dynamic Watermarking, which adds a private excitation to the input signal to validate measurements, has recently been extended to LTV systems. However, this extension does not allow for LTV systems that require several steps before the effect of a given control input can be seen in the measurement signal. Additionally, there is no consideration for the time-varying effects of auto-correlation. Furthermore, a proof of concept was only provided using simulations of a simplified model. This paper relaxes the requirement for inputs to be visible in a single step and constructs an auto-correlation normalizing factor to remove the effects of auto-correlation. In addition, Dynamic Watermarking is applied to a high-fidelity vehicle model in carsim and a 1/10 scale autonomous rover to further reinforce the proof of concept for realistic systems. In each case, the vehicle follows a predefined path with time-varying velocity and turning radii. A replay attack, which replays previously recorded measurements, is shown to be detectable using LTV Dynamic Watermarking in a quick and repeatable manner.

math.OC↗

arXiv2019

Shadow Transfer: Single Image Relighting For Urban Road Scenes

Illumination effects in images, specifically cast shadows and shading, have been shown to decrease the performance of deep neural networks on a large number of vision-based detection, recognition and segmentation tasks in urban driving scenes. A key factor that contributes to this performance gap is the lack of `time-of-day' diversity within real, labeled datasets. There have been impressive advances in the realm of image to image translation in transferring previously unseen visual effects into a dataset, specifically in day to night translation. However, it is not easy to constrain what visual effects, let alone illumination effects, are transferred from one dataset to another during the training process. To address this problem, we propose deep learning framework, called Shadow Transfer, that can relight complex outdoor scenes by transferring realistic shadow, shading, and other lighting effects onto a single image. The novelty of the proposed framework is that it is both self-supervised, and is designed to operate on sensor and label information that is easily available in autonomous vehicle datasets. We show the effectiveness of this method on both synthetic and real datasets, and we provide experiments that demonstrate that the proposed method produces images of higher visual quality than state of the art image to image translation methods.

cs.CV↗