arXiv subjects

Aaron D. Ames

Publications and source records attributed to Aaron D. Ames.

At least 217 records · Page 12Linked to original sources

arXiv2021

Preference-Based Learning for User-Guided HZD Gait Generation on Bipedal Walking Robots

This paper presents a framework that leverages both control theory and machine learning to obtain stable and robust bipedal locomotion without the need for manual parameter tuning. Traditionally, gaits are generated through trajectory optimization methods and then realized experimentally -- a process that often requires extensive tuning due to differences between the models and hardware. In this work, the process of gait realization via hybrid zero dynamics (HZD) based optimization is formally combined with preference-based learning to systematically realize dynamically stable walking. Importantly, this learning approach does not require a carefully constructed reward function, but instead utilizes human pairwise preferences. The power of the proposed approach is demonstrated through two experiments on a planar biped AMBER-3M: the first with rigid point-feet, and the second with induced model uncertainty through the addition of springs where the added compliance was not accounted for in the gait generation or in the controller. In both experiments, the framework achieves stable, robust, efficient, and natural walking in fewer than 50 iterations with no reliance on a simulation environment. These results demonstrate a promising step in the unification of control theory and learning.

cs.RO↗

arXiv2021

Constrained Risk-Averse Markov Decision Processes

We consider the problem of designing policies for Markov decision processes (MDPs) with dynamic coherent risk objectives and constraints. We begin by formulating the problem in a Lagrangian framework. Under the assumption that the risk objectives and constraints can be represented by a Markov risk transition mapping, we propose an optimization-based method to synthesize Markovian policies that lower-bound the constrained risk-averse problem. We demonstrate that the formulated optimization problems are in the form of difference convex programs (DCPs) and can be solved by the disciplined convex-concave programming (DCCP) framework. We show that these results generalize linear programs for constrained MDPs with total discounted expected costs and constraints. Finally, we illustrate the effectiveness of the proposed method with numerical experiments on a rover navigation problem involving conditional-value-at-risk (CVaR) and entropic-value-at-risk (EVaR) coherent risk measures.

cs.AI↗

arXiv2021

Risk-Averse Stochastic Shortest Path Planning

We consider the stochastic shortest path planning problem in MDPs, i.e., the problem of designing policies that ensure reaching a goal state from a given initial state with minimum accrued cost. In order to account for rare but important realizations of the system, we consider a nested dynamic coherent risk total cost functional rather than the conventional risk-neutral total expected cost. Under some assumptions, we show that optimal, stationary, Markovian policies exist and can be found via a special Bellman's equation. We propose a computational technique based on difference convex programs (DCPs) to find the associated value functions and therefore the risk-averse policies. A rover navigation MDP is used to illustrate the proposed methodology with conditional-value-at-risk (CVaR) and entropic-value-at-risk (EVaR) coherent risk measures.

eess.SY↗

arXiv2021

Risk-Averse Planning via CVaR Barrier Functions: Application to Bipedal Robot Locomotion

Enforcing safety in the presence of stochastic uncertainty is a challenging problem. Traditionally, researchers have proposed safety in the statistical mean as a safety measure in this case. However, ensuring safety in the statistical mean is only reasonable if system's safe behavior in the large number of runs is of interest, which precludes the use of mean safety in practical scenarios. In this paper, we propose a risk sensitive notion of safety called conditional-value-at-risk (CVaR) safety, which is concerned with safe performance in the worst case realizations. We introduce CVaR barrier functions as a tool to enforce CVaR-safety and propose conditions for their Boolean compositions. Given a legacy controller, we show that we can design a minimally interfering CVaR-safe controller via solving difference convex programs. We elucidate the proposed method by applying it to a bipedal robot locomotion case study.

cs.RO↗

arXiv2021

Sampled-Data Stabilization with Control Lyapunov Functions via Quadratically Constrained Quadratic Programs

Controller design for nonlinear systems with Control Lyapunov Function (CLF) based quadratic programs has recently been successfully applied to a diverse set of difficult control tasks. These existing formulations do not address the gap between design with continuous time models and the discrete time sampled implementation of the resulting controllers, often leading to poor performance on hardware platforms. We propose an approach to close this gap by synthesizing sampled-data counterparts to these CLF-based controllers, specified as quadratically constrained quadratic programs (QCQPs). Assuming feedback linearizability and stable zero-dynamics of a system's continuous time model, we derive practical stability guarantees for the resulting sampled-data system. We demonstrate improved performance of the proposed approach over continuous time counterparts in simulation.

eess.SY↗

arXiv2021

Multi-rate Control Design under Input Constraints via Fixed-Time Barrier Functions

In this paper, we introduce the notion of periodic safety, which requires that the system trajectories periodically visit a subset of a forward-invariant safe set, and utilize it in a multi-rate framework where a high-level planner generates a reference trajectory that is tracked by a low-level controller under input constraints. We introduce the notion of fixed-time barrier functions which is leveraged by the proposed low-level controller in a quadratic programming framework. Then, we design a model predictive control policy for high-level planning with a bound on the rate of change for the reference trajectory to guarantee that periodic safety is achieved. We demonstrate the effectiveness of the proposed strategy on a simulation example, where the proposed fixed-time stabilizing low-level controller shows successful satisfaction of control objectives, whereas an exponentially stabilizing low-level controller fails.

eess.SY↗

arXiv2021

Time-Optimal Navigation in Uncertain Environments with High-Level Specifications

Mixed observable Markov decision processes (MOMDPs) are a modeling framework for autonomous systems described by both fully and partially observable states. In this work, we study the problem of synthesizing a control policy for MOMDPs that minimizes the expected time to complete the control task while satisfying syntactically co-safe Linear Temporal Logic (scLTL) specifications. First, we present an exact dynamic programming update to compute the value function. Afterwards, we propose a point-based approximation, which allows us to compute a lower bound of the closed-loop probability of satisfying the specifications. The effectiveness of the proposed approach and comparisons with standard strategies are shown on high-fidelity navigation tasks with partially observable static obstacles.

eess.SY↗

arXiv2020

Reactive motion planning with probabilistic safety guarantees

Motion planning in environments with multiple agents is critical to many important autonomous applications such as autonomous vehicles and assistive robots. This paper considers the problem of motion planning, where the controlled agent shares the environment with multiple uncontrolled agents. First, a predictive model of the uncontrolled agents is trained to predict all possible trajectories within a short horizon based on the scenario. The prediction is then fed to a motion planning module based on model predictive control. We proved generalization bound for the predictive model using three different methods, post-bloating, support vector machine (SVM), and conformal analysis, all capable of generating stochastic guarantees of the correctness of the predictor. The proposed approach is demonstrated in simulation in a scenario emulating autonomous highway driving.

cs.RO↗

arXiv2020

Online decentralized decision making with inequality constraints: an ADMM approach

We discuss an online decentralized decision making problem where the agents are coupled with affine inequality constraints. Alternating Direction Method of Multipliers (ADMM) is used as the computation engine and we discuss the convergence of the algorithm in an online setting. To be specific, when decisions have to be made sequentially with a fixed time step, there might not be enough time for the ADMM to converge before the scenario changes and the decision needs to be updated. In this case, a suboptimal solution is employed and we analyze the optimality gap given the convergence condition. Moreover, in many cases, the decision making problem changes gradually over time. We propose a warm-start scheme to accelerate the convergence of ADMM and analyze the benefit of the warm-start. The proposed method is demonstrated in a decentralized multiagent control barrier function problem with simulation.

eess.SY↗

arXiv2020

Decentralized Task and Path Planning for Multi-Robot Systems

We consider a multi-robot system with a team of collaborative robots and multiple tasks that emerges over time. We propose a fully decentralized task and path planning (DTPP) framework consisting of a task allocation module and a localized path planning module. Each task is modeled as a Markov Decision Process (MDP) or a Mixed Observed Markov Decision Process (MOMDP) depending on whether full states or partial states are observable. The task allocation module then aims at maximizing the expected pure reward (reward minus cost) of the robotic team. We fuse the Markov model into a factor graph formulation so that the task allocation can be decentrally solved using the max-sum algorithm. Each robot agent follows the optimal policy synthesized for the Markov model and we propose a localized forward dynamic programming scheme that resolves conflicts between agents and avoids collisions. The proposed framework is demonstrated with high fidelity ROS simulations and experiments with multiple ground robots.

cs.RO↗

arXiv2020

Lidar-based exploration and discretization for mobile robot planning

In robotic applications, the control, and actuation deal with a continuous description of the system and environment, while high-level planning usually works with a discrete description. This paper considers the problem of bridging the low-level control and high-level planning for robotic systems via sensor data. In particular, we propose a discretization algorithm that identifies free polytopes via lidar point cloud data. A transition graph is then constructed where each node corresponds to a free polytope and two nodes are connected with an edge if the two corresponding free polytopes intersect. Furthermore, a distance measure is associated with each edge, which allows for the assessment of quality (or cost) of the transition for high-level planning. For the low-level control, the free polytopes act as a convenient encoding of the environment and allow for the planning of collision-free trajectories that realizes the high-level plan. The results are demonstrated in high-fidelity ROS simulations and experiments with a drone and a Segway.

cs.RO↗

arXiv2020

Model-Dependent Prosthesis Control with Interaction Force Estimation

Current prosthesis control methods are primarily model-independent - lacking formal guarantees of stability, relying largely on heuristic tuning parameters for good performance, and neglecting use of the natural dynamics of the system. Model-dependence for prosthesis controllers is difficult to achieve due to the unknown human dynamics. We build upon previous work which synthesized provably stable prosthesis walking through the use of rapidly exponentially stabilizing control Lyapunov functions (RES-CLFs). This paper utilizes RES-CLFs together with force estimation to construct model-based optimization-based controllers for the prosthesis. These are experimentally realized on hardware with onboard sensing and computation. This hardware demonstration has formal guarantees of stability, utilizes the natural dynamics of the system, and achieves superior tracking to other prosthesis trajectory tracking control methods.

eess.SY↗

arXiv2020

Guaranteeing Safety of Learned Perception Modules via Measurement-Robust Control Barrier Functions

Modern nonlinear control theory seeks to develop feedback controllers that endow systems with properties such as safety and stability. The guarantees ensured by these controllers often rely on accurate estimates of the system state for determining control actions. In practice, measurement model uncertainty can lead to error in state estimates that degrades these guarantees. In this paper, we seek to unify techniques from control theory and machine learning to synthesize controllers that achieve safety in the presence of measurement model uncertainty. We define the notion of a Measurement-Robust Control Barrier Function (MR-CBF) as a tool for determining safe control inputs when facing measurement model uncertainty. Furthermore, MR-CBFs are used to inform sampling methodologies for learning-based perception systems and quantify tolerable error in the resulting learned models. We demonstrate the efficacy of MR-CBFs in achieving safety with measurement model uncertainty on a simulated Segway system.

eess.SY↗

arXiv2020

Inverse Dynamics Control of Compliant Hybrid Zero Dynamic Walking

We present a trajectory planning and control architecture for bipedal locomotion at a variety of speeds on a highly underactuated and compliant bipedal robot. A library of compliant walking trajectories are planned offline, and stored as compact arrays of polynomial coefficients for tracking online. The control implementation uses a floating-base inverse dynamics controller which generates dynamically consistent feedforward torques to realize walking using information obtained from the trajectory optimization. The effectiveness of the controller is demonstrated in simulation and on hardware for walking both indoors on flat terrain and over unplanned disturbances outdoors. Additionally, both the controller and optimization source code are made available on GitHub.

cs.RO↗

arXiv2020

Dynamic Walking: Toward Agile and Efficient Bipedal Robots

Dynamic walking on bipedal robots has evolved from an idea in science fiction to a practical reality. This is due to continued progress in three key areas: a mathematical understanding of locomotion, the computational ability to encode this mathematics through optimization, and the hardware capable of realizing this understanding in practice. In this context, this review article outlines the end-to-end process of methods which have proven effective in the literature for achieving dynamic walking on bipedal robots. We begin by introducing mathematical models of locomotion, from reduced order models that capture essential walking behaviors to hybrid dynamical systems that encode the full order continuous dynamics along with discrete footstrike dynamics. These models form the basis for gait generation via (nonlinear) optimization problems. Finally, models and their generated gaits merge in the context of real-time control, wherein walking behaviors are translated to hardware. The concepts presented are illustrated throughout in simulation, and experimental instantiation on multiple walking platforms are highlighted to demonstrate the ability to realize dynamic walking on bipedal robots that is agile and efficient.

cs.RO↗

arXiv2020

Formal Verification of Safety Critical Autonomous Systems via Bayesian Optimization

As control systems become increasingly more complex, there exists a pressing need to find systematic ways of verifying them. To address this concern, there has been significant work in developing test generation schemes for black-box control architectures. These schemes test a black-box control architecture's ability to satisfy its control objectives, when these objectives are expressed as operational specifications through temporal logic formulae. Our work extends these prior, model based results by lower bounding the probability by which the black-box system will satisfy its operational specification, when subject to a pre-specified set of environmental phenomena. We do so by systematically generating tests to minimize a Lipschitz continuous robustness measure for the operational specification. We demonstrate our method with experimental results, wherein we show that our framework can reasonably lower bound the probability of specification satisfaction.

eess.SY↗

arXiv2020

Safe Multi-Agent Interaction through Robust Control Barrier Functions with Learned Uncertainties

Robots operating in real world settings must navigate and maintain safety while interacting with many heterogeneous agents and obstacles. Multi-Agent Control Barrier Functions (CBF) have emerged as a computationally efficient tool to guarantee safety in multi-agent environments, but they assume perfect knowledge of both the robot dynamics and other agents' dynamics. While knowledge of the robot's dynamics might be reasonably well known, the heterogeneity of agents in real-world environments means there will always be considerable uncertainty in our prediction of other agents' dynamics. This work aims to learn high-confidence bounds for these dynamic uncertainties using Matrix-Variate Gaussian Process models, and incorporates them into a robust multi-agent CBF framework. We transform the resulting min-max robust CBF into a quadratic program, which can be efficiently solved in real time. We verify via simulation results that the nominal multi-agent CBF is often violated during agent interactions, whereas our robust formulation maintains safety with a much higher probability and adapts to learned uncertainties

cs.RO↗

arXiv2020

Safety-Critical Kinematic Control of Robotic Systems

Over the decades, kinematic controllers have proven to be practically useful for applications like set-point and trajectory tracking in robotic systems. To this end, we formulate a novel safety-critical paradigm for kinematic control in this paper. In particular, we extend the methodology of control barrier functions (CBFs) to kinematic equations governing robotic systems. We demonstrate a purely kinematic implementation of a velocity-based CBF, and subsequently introduce a formulation that guarantees safety at the level of dynamics. This is achieved through a new form CBFs that incorporate kinetic energy with the classical forms, thereby minimizing model dependence and conservativeness. The approach is then extended to underactuated systems. This method and the purely kinematic implementation are demonstrated in simulation on two robotic platforms: a 6-DOF robotic manipulator, and a cart-pole system.

eess.SY↗