arXiv subjects

Matthew Johnson-Roberson

Publications and source records attributed to Matthew Johnson-Roberson.

At least 73 records · Page 4Linked to original sources

arXiv2019

Leveraging the Template and Anchor Framework for Safe, Online Robotic Gait Design

Online control design using a high-fidelity, full-order model for a bipedal robot can be challenging due to the size of the state space of the model. A commonly adopted solution to overcome this challenge is to approximate the full-order model (anchor) with a simplified, reduced-order model (template), while performing control synthesis. Unfortunately it is challenging to make formal guarantees about the safety of an anchor model using a controller designed in an online fashion using a template model. To address this problem, this paper proposes a method to generate safety-preserving controllers for anchor models by performing reachability analysis on template models while bounding the modeling error. This paper describes how this reachable set can be incorporated into a Model Predictive Control framework to select controllers that result in safe walking on the anchor model in an online fashion. The method is illustrated on a 5-link RABBIT model, and is shown to allow the robot to walk safely while utilizing controllers designed in an online fashion.

cs.RO↗

arXiv2019

Risk Assessment and Planning with Bidirectional Reachability for Autonomous Driving

Knowing and predicting dangerous factors within a scene are two key components during autonomous driving, especially in a crowded urban environment. To navigate safely in environments, risk assessment is needed to quantify and associate the risk of taking a certain action. Risk assessment and planning is usually done by first tracking and predicting trajectories of other agents, such as vehicles and pedestrians, and then choosing an action to avoid collision in the future. However, few existing risk assessment algorithms handle occlusion and other sensory limitations effectively. This paper explores the possibility of efficient risk assessment under occlusion via both forward and backward reachability. The proposed algorithm can not only identify where the risk-induced factors are, but also be used for motion planning by executing low-level commands, such as throttle. The proposed method is evaluated on various four-way highly occluded intersections with up to five other vehicles in the scene. Compared with other risk assessment algorithms, the proposed method shows better efficiency, meaning that the ego vehicle reaches the goal at a higher speed. In addition, it also lowers the median collision rate by 7.5x.

cs.RO↗

arXiv2019

Detecting Generalized Replay Attacks via Time-Varying Dynamic Watermarking

Cyber-physical systems (CPS) often rely on external communication for supervisory control or sensing. Unfortunately, these communications render the system vulnerable to cyber-attacks. Attacks that alter messages, such as replay attacks that record measurement signals and then play them back to the system, can cause devastating effects. Dynamic Watermarking methods, which inject a private excitation into control inputs to secure resulting measurement signals, have begun addressing the challenges of detecting these attacks, but have been restricted to linear time invariant (LTI) systems. Though LTI models are sufficient for some applications, other CPS, such as autonomous vehicles, require more complex models. This paper develops a linear time-varying (LTV) extension to previous Dynamic Watermarking methods by designing a matrix normalization factor to accommodate the temporal changes in the system. Implementable tests are provided with considerations for real-world systems. The proposed method is then shown to be able to detect generalized replay attacks both in theory and in simulation using a LTV vehicle model.

math.OC↗

arXiv2019

Bio-LSTM: A Biomechanically Inspired Recurrent Neural Network for 3D Pedestrian Pose and Gait Prediction

In applications such as autonomous driving, it is important to understand, infer, and anticipate the intention and future behavior of pedestrians. This ability allows vehicles to avoid collisions and improve ride safety and quality. This paper proposes a biomechanically inspired recurrent neural network (Bio-LSTM) that can predict the location and 3D articulated body pose of pedestrians in a global coordinate frame, given 3D poses and locations estimated in prior frames with inaccuracy. The proposed network is able to predict poses and global locations for multiple pedestrians simultaneously, for pedestrians up to 45 meters from the cameras (urban intersection scale). The outputs of the proposed network are full-body 3D meshes represented in Skinned Multi-Person Linear (SMPL) model parameters. The proposed approach relies on a novel objective function that incorporates the periodicity of human walking (gait), the mirror symmetry of the human body, and the change of ground reaction forces in a human gait cycle. This paper presents prediction results on the PedX dataset, a large-scale, in-the-wild data set collected at real urban intersections with heavy pedestrian traffic. Results show that the proposed network can successfully learn the characteristics of pedestrian gait and produce accurate and consistent 3D pose predictions.

cs.RO↗

arXiv2019

Occlusion-Aware Risk Assessment for Autonomous Driving in Urban Environments

Navigating safely in urban environments remains a challenging problem for autonomous vehicles. Occlusion and limited sensor range can pose significant challenges to safely navigate among pedestrians and other vehicles in the environment. Enabling vehicles to quantify the risk posed by unseen regions allows them to anticipate future possibilities, resulting in increased safety and ride comfort. This paper proposes an algorithm that takes advantage of the known road layouts to forecast, quantify, and aggregate risk associated with occlusions and limited sensor range. This allows us to make predictions of risk induced by unobserved vehicles even in heavily occluded urban environments. The risk can then be used either by a low-level planning algorithm to generate better trajectories, or by a high-level one to plan a better route. The proposed algorithm is evaluated on intersection layouts from real-world map data with up to five other vehicles in the scene, and verified to reduce collision rates by 4.8x comparing to a baseline method while improving driving comfort.

cs.RO↗

arXiv2019

Energy Management for Autonomous Underwater Vehicles Using Economic Model Predictive Control

This paper investigates the problem of energy-optimal control for autonomous underwater vehicles (AUVs). To improve the endurance of AUVs, we propose a novel energy-optimal control scheme based on the economic model predictive control (MPC) framework. We first formulate a cost function that computes the energy spent for vehicle operation over a finite-time prediction horizon. Then, to account for the energy consumption beyond the prediction horizon, a terminal cost that approximates the energy to reach the goal (energy-to-go) is incorporated into the MPC cost function.

eess.SY↗

arXiv2019

Stochastic Sampling Simulation for Pedestrian Trajectory Prediction

Urban environments pose a significant challenge for autonomous vehicles (AVs) as they must safely navigate while in close proximity to many pedestrians. It is crucial for the AV to correctly understand and predict the future trajectories of pedestrians to avoid collision and plan a safe path. Deep neural networks (DNNs) have shown promising results in accurately predicting pedestrian trajectories, relying on large amounts of annotated real-world data to learn pedestrian behavior. However, collecting and annotating these large real-world pedestrian datasets is costly in both time and labor. This paper describes a novel method using a stochastic sampling-based simulation to train DNNs for pedestrian trajectory prediction with social interaction. Our novel simulation method can generate vast amounts of automatically-annotated, realistic, and naturalistic synthetic pedestrian trajectories based on small amounts of real annotation. We then use such synthetic trajectories to train an off-the-shelf state-of-the-art deep learning approach Social GAN (Generative Adversarial Network) to perform pedestrian trajectory prediction. Our proposed architecture, trained only using synthetic trajectories, achieves better prediction results compared to those trained on human-annotated real-world data using the same network. Our work demonstrates the effectiveness and potential of using simulation as a substitution for human annotation efforts to train high-performing prediction algorithms such as the DNNs.

cs.RO↗

arXiv2019

Towards Provably Not-at-Fault Control of Autonomous Robots in Arbitrary Dynamic Environments

As autonomous robots increasingly become part of daily life, they will often encounter dynamic environments while only having limited information about their surroundings. Unfortunately, due to the possible presence of malicious dynamic actors, it is infeasible to develop an algorithm that can guarantee collision-free operation. Instead, one can attempt to design a control technique that guarantees the robot is not-at-fault in any collision. In the literature, making such guarantees in real time has been restricted to static environments or specific dynamic models. To ensure not-at-fault behavior, a robot must first correctly sense and predict the world around it within some sufficiently large sensor horizon (the prediction problem), then correctly control relative to the predictions (the control problem). This paper addresses the control problem by proposing Reachability-based Trajectory Design for Dynamic environments (RTD-D), which guarantees that a robot with an arbitrary nonlinear dynamic model correctly responds to predictions in arbitrary dynamic environments. RTD-D first computes a Forward Reachable Set (FRS) offline of the robot tracking parameterized desired trajectories that include fail-safe maneuvers. Then, for online receding-horizon planning, the method provides a way to discretize predictions of an arbitrary dynamic environment to enable real-time collision checking. The FRS is used to map these discretized predictions to trajectories that the robot can track while provably not-at-fault. One such trajectory is chosen at each iteration, or the robot executes the fail-safe maneuver from its previous trajectory which is guaranteed to be not at fault. RTD-D is shown to produce not-at-fault behavior over thousands of simulations and several real-world hardware demonstrations on two robots: a Segway, and a small electric vehicle.

cs.RO↗

arXiv2019

Guaranteed Safe Reachability-based Trajectory Design for a High-Fidelity Model of an Autonomous Passenger Vehicle

Trajectory planning is challenging for autonomous cars since they operate in unpredictable environments with limited sensor horizons. To incorporate new information as it is sensed, planning is done in a loop, with the next plan being computed as the previous plan is executed. The recent Reachability-based Trajectory Design (RTD) is a provably safe, real-time algorithm for trajectory planning. RTD consists of an offline component, where a Forward Reachable Set (FRS) is computed for the vehicle tracking parameterized trajectories; and an online part, where the FRS is used to map obstacles to constraints for trajectory optimization in a provably-safe way. In the literature, RTD has only been applied to small mobile robots. The contribution of this work is applying RTD to a passenger vehicle in CarSim, with a full powertrain model, chassis and tire dynamics. RTD produces safe trajectory plans with the vehicle traveling up to 15 m/s on a two-lane road, with randomly-placed obstacles only known to the vehicle when detected within its sensor horizon. RTD is compared with a Nonlinear Model-Predictive Control (NMPC) and a Rapidly-exploring Random Tree (RRT) approach. The experiment demonstrates RTD's ability to plan safe trajectories in real time, in contrast to the existing state-of-the-art approaches.

eess.SY↗

arXiv2019

DispSegNet: Leveraging Semantics for End-to-End Learning of Disparity Estimation from Stereo Imagery

Recent work has shown that convolutional neural networks (CNNs) can be applied successfully in disparity estimation, but these methods still suffer from errors in regions of low-texture, occlusions and reflections. Concurrently, deep learning for semantic segmentation has shown great progress in recent years. In this paper, we design a CNN architecture that combines these two tasks to improve the quality and accuracy of disparity estimation with the help of semantic segmentation. Specifically, we propose a network structure in which these two tasks are highly coupled. One key novelty of this approach is the two-stage refinement process. Initial disparity estimates are refined with an embedding learned from the semantic segmentation branch of the network. The proposed model is trained using an unsupervised approach, in which images from one half of the stereo pair are warped and compared against images from the other camera. Another key advantage of the proposed approach is that a single network is capable of outputting disparity estimates and semantic labels. These outputs are of great use in autonomous vehicle operation; with real-time constraints being key, such performance improvements increase the viability of driving applications. Experiments on KITTI and Cityscapes datasets show that our model can achieve state-of-the-art results and that leveraging embedding learned from semantic segmentation improves the performance of disparity estimation.

cs.CV↗

arXiv2019

Sensor Transfer: Learning Optimal Sensor Effect Image Augmentation for Sim-to-Real Domain Adaptation

Performance on benchmark datasets has drastically improved with advances in deep learning. Still, cross-dataset generalization performance remains relatively low due to the domain shift that can occur between two different datasets. This domain shift is especially exaggerated between synthetic and real datasets. Significant research has been done to reduce this gap, specifically via modeling variation in the spatial layout of a scene, such as occlusions, and scene environmental factors, such as time of day and weather effects. However, few works have addressed modeling the variation in the sensor domain as a means of reducing the synthetic to real domain gap. The camera or sensor used to capture a dataset introduces artifacts into the image data that are unique to the sensor model, suggesting that sensor effects may also contribute to domain shift. To address this, we propose a learned augmentation network composed of physically-based augmentation functions. Our proposed augmentation pipeline transfers specific effects of the sensor model -- chromatic aberration, blur, exposure, noise, and color temperature -- from a real dataset to a synthetic dataset. We provide experiments that demonstrate that augmenting synthetic training datasets with the proposed learned augmentation framework reduces the domain gap between synthetic and real domains for object detection in urban driving scenes.

cs.CV↗

arXiv2018

Simulation and Real-World Evaluation of Attack Detection Schemes

A variety of anomaly detection schemes have been proposed to detect malicious attacks to Cyber-Physical Systems. Among these schemes, Dynamic Watermarking methods have been proven highly effective at detecting a wide range of attacks. Unfortunately, in contrast to other anomaly detectors, no method has been presented to design a Dynamic Watermarking detector to achieve a user-specified false alarm rate, or subsequently evaluate the capabilities of an attacker under such a selection. This paper describes methods to measure the capability of an attacker, to numerically approximate this metric, and to design a Dynamic Watermarking detector that can achieve a user-specified rate of false alarms. The performance of the Dynamic Watermarking detector is compared to three classical anomaly detectors in simulation and on a real-world platform. These experiments illustrate that the attack capability under the Dynamic Watermarking detector is comparable to those of classic anomaly detectors. Importantly, these experiments also make clear that the Dynamic Watermarking detector is consistently able to detect attacks that the other class of detectors are unable to identify.

eess.SY↗

arXiv2018

A constrained control-planning strategy for redundant manipulators

This paper presents an interconnected control-planning strategy for redundant manipulators, subject to system and environmental constraints. The method incorporates low-level control characteristics and high-level planning components into a robust strategy for manipulators acting in complex environments, subject to joint limits. This strategy is formulated using an adaptive control rule, the estimated dynamic model of the robotic system and the nullspace of the linearized constraints. A path is generated that takes into account the capabilities of the platform. The proposed method is computationally efficient, enabling its implementation on a real multi-body robotic system. Through experimental results with a 7 DOF manipulator, we demonstrate the performance of the method in real-world scenarios.

cs.RO↗

arXiv2018

Modeling Camera Effects to Improve Visual Learning from Synthetic Data

Recent work has focused on generating synthetic imagery to increase the size and variability of training data for learning visual tasks in urban scenes. This includes increasing the occurrence of occlusions or varying environmental and weather effects. However, few have addressed modeling variation in the sensor domain. Sensor effects can degrade real images, limiting generalizability of network performance on visual tasks trained on synthetic data and tested in real environments. This paper proposes an efficient, automatic, physically-based augmentation pipeline to vary sensor effects --chromatic aberration, blur, exposure, noise, and color cast-- for synthetic imagery. In particular, this paper illustrates that augmenting synthetic training datasets with the proposed pipeline reduces the domain gap between synthetic and real domains for the task of object detection in urban driving scenes.

cs.CV↗

arXiv2018

Real-Time Model Predictive Control for Energy Management in Autonomous Underwater Vehicle

Improving endurance is crucial for extending the spatial and temporal operation range of autonomous underwater vehicles (AUVs). Considering the hardware constraints and the performance requirements, an intelligent energy management system is required to extend the operation range of AUVs. This paper presents a novel model predictive control (MPC) framework for energy-optimal point-to-point motion control of an AUV. In this scheme, the energy management problem of an AUV is reformulated as a surge motion optimization problem in two stages. First, a system-level energy minimization problem is solved by managing the trade-off between the energies required for overcoming the positive buoyancy and surge drag force in static optimization. Next, an MPC with a special cost function formulation is proposed to deal with transients and system dynamics. A switching logic for handling the transition between the static and dynamic stages is incorporated to reduce the computational efforts. Simulation results show that the proposed method is able to achieve near-optimal energy consumption with considerable lower computational complexity.

eess.SY↗

arXiv2018

PedX: Benchmark Dataset for Metric 3D Pose Estimation of Pedestrians in Complex Urban Intersections

This paper presents a novel dataset titled PedX, a large-scale multimodal collection of pedestrians at complex urban intersections. PedX consists of more than 5,000 pairs of high-resolution (12MP) stereo images and LiDAR data along with providing 2D and 3D labels of pedestrians. We also present a novel 3D model fitting algorithm for automatic 3D labeling harnessing constraints across different modalities and novel shape and temporal priors. All annotated 3D pedestrians are localized into the real-world metric space, and the generated 3D models are validated using a mocap system configured in a controlled outdoor environment to simulate pedestrians in urban intersections. We also show that the manual 2D labels can be replaced by state-of-the-art automated labeling approaches, thereby facilitating automatic generation of large scale datasets.

cs.CV↗

arXiv2018

Failing to Learn: Autonomously Identifying Perception Failures for Self-driving Cars

One of the major open challenges in self-driving cars is the ability to detect cars and pedestrians to safely navigate in the world. Deep learning-based object detector approaches have enabled great advances in using camera imagery to detect and classify objects. But for a safety critical application, such as autonomous driving, the error rates of the current state of the art are still too high to enable safe operation. Moreover, the characterization of object detector performance is primarily limited to testing on prerecorded datasets. Errors that occur on novel data go undetected without additional human labels. In this letter, we propose an automated method to identify mistakes made by object detectors without ground truth labels. We show that inconsistencies in the object detector output between a pair of similar images can be used as hypotheses for false negatives (e.g., missed detections) and using a novel set of features for each hypothesis, an off-the-shelf binary classifier can be used to find valid errors. In particular, we study two distinct cues - temporal and stereo inconsistencies - using data that are readily available on most autonomous vehicles. Our method can be used with any camera-based object detector and we illustrate the technique on several sets of real world data. We show that a state-of-the-art detector, tracker, and our classifier trained only on synthetic data can identify valid errors on KITTI tracking dataset with an average precision of 0.94. We also release a new tracking dataset with 104 sequences totaling 80,655 labeled pairs of stereo images along with ground truth disparity from a game engine to facilitate further research. The dataset and code are available at https://fcav.engin.umich.edu/research/failing-to-learn

cs.CV↗

arXiv2018

A Matrix Representation of the Multiple Vehicle Routing Problem for Pickup and Delivery

This paper develops a computationally efficient algorithm for the Multiple Vehicle Pickup and Delivery Problem (MVPDP) with the objective of minimizing the tour cost incurred while completing the task of pickup and delivery of customers. To this end, this paper constructs a novel 0-1 Integer Quadratic Programming (IQP) problem to exactly solve the MVPDP. Compared to the state-of-the-art Mixed Integer Linear Programming (MILP) formulation of the problem, the one presented here requires fewer constraints and decision variables. To ensure that this IQP formulation of the MVPDP can be solved in a computationally efficient manner, this paper devises a set of sufficient conditions to ensure convexity of this formulation when the integer variables are relaxed. In addition, this paper describes a transformation to map any non-convex IQP formulation of the MVPDP into an equivalent convex one. The superior computational efficacy of this convex IQP method when compared to the state-of-the-art MILP formulation is demonstrated through extensive simulated and real-world experiments.

math.OC↗