arXiv subjects

Melanie N. Zeilinger

Publications and source records attributed to Melanie N. Zeilinger.

At least 127 records · Page 7Linked to original sources

arXiv2020

Cautious Bayesian Optimization for Efficient and Scalable Policy Search

Sample efficiency is one of the key factors when applying policy search to real-world problems. In recent years, Bayesian Optimization (BO) has become prominent in the field of robotics due to its sample efficiency and little prior knowledge needed. However, one drawback of BO is its poor performance on high-dimensional search spaces as it focuses on global search. In the policy search setting, local optimization is typically sufficient as initial policies are often available, e.g., via meta-learning, kinesthetic demonstrations or sim-to-real approaches. In this paper, we propose to constrain the policy search space to a sublevel-set of the Bayesian surrogate model's predictive uncertainty. This simple yet effective way of constraining the policy update enables BO to scale to high-dimensional spaces (>100) as well as reduces the risk of damaging the system. We demonstrate the effectiveness of our approach on a wide range of problems, including a motor skills task, adapting deep RL agents to new reward signals and a sim-to-real task for an inverted pendulum system.

cs.RO↗

arXiv2020

Volume Control of Low-Cost Ventilator with Automatic Set-Point Adaptation

This paper considers the control design for a low-cost ventilator that is based on a manual resuscitator bag (also known as AmbuBag) to pump air into the lungs of a patient who is physically unable to breathe. First, it experimentally shows that for accurately tracking tidal volumes, the controller needs to be adapted to the individual patient and the different configurations, e.g., hardware or operation modes. Second, it proposes a set-point adaptation algorithm that uses sensor measurements of a flow meter to automatically adapt the controller to the setup at hand. Third, it experimentally shows that such an adaptive solution improves the performance of the ventilator for various setups. One objective of this paper is to increase awareness of the need for feedback control using sensor measurements in low-cost ventilator solutions in order to automatically adapt to the specific scenario.

eess.SY↗

arXiv2020

Distributed Safe Learning using an Invariance-based Safety Framework

In large-scale networks of uncertain dynamical systems, where communication is limited and there is a strong interaction among subsystems, learning local models and control policies offers great potential for designing high-performance controllers. At the same time, the lack of safety guarantees, here considered in the form of constraint satisfaction, prevents the use of data-driven techniques to safety-critical distributed systems. This paper presents a safety framework that guarantees constraint satisfaction for uncertain distributed systems while learning. The framework considers linear systems with coupling in the dynamics and subject to bounded parametric uncertainty, and makes use of robust invariance to guarantee safety. In particular, a robust non-convex invariant set, given by the union of multiple ellipsoidal invariant sets, and a nonlinear backup control law, given by the combination of multiple stabilizing linear feedbacks, are computed offline. In presence of unsafe inputs, the safety framework applies the backup control law, preventing the system to violate the constraints. As the robust invariant set and the backup stabilizing controller are computed offline, the online operations reduce to simple function evaluations, which enables the use of the proposed framework on systems with limited computational resources. The capabilities of the safety framework are illustrated by three numerical examples.

eess.SY↗

arXiv2020

Bayesian model predictive control: Efficient model exploration and regret bounds using posterior sampling

Tight performance specifications in combination with operational constraints make model predictive control (MPC) the method of choice in various industries. As the performance of an MPC controller depends on a sufficiently accurate objective and prediction model of the process, a significant effort in the MPC design procedure is dedicated to modeling and identification. Driven by the increasing amount of available system data and advances in the field of machine learning, data-driven MPC techniques have been developed to facilitate the MPC controller design. While these methods are able to leverage available data, they typically do not provide principled mechanisms to automatically trade off exploitation of available data and exploration to improve and update the objective and prediction model. To this end, we present a learning-based MPC formulation using posterior sampling techniques, which provides finite-time regret bounds on the learning performance while being simple to implement using off-the-shelf MPC software and algorithms. The performance analysis of the method is based on posterior sampling theory and its practical efficiency is illustrated using a numerical example of a highly nonlinear dynamical car-trailer system.

eess.SY↗

arXiv2020

Scalable synthesis of safety certificates from data with application to learning-based control

The control of complex systems faces a trade-off between high performance and safety guarantees, which in particular restricts the application of learning-based methods to safety-critical systems. A recently proposed framework to address this issue is the use of a safety controller, which guarantees to keep the system within a safe region of the state space. This paper introduces efficient techniques for the synthesis of a safe set and control law, which offer improved scalability properties by relying on approximations based on convex optimization problems. The first proposed method requires only an approximate linear system model and Lipschitz continuity of the unknown nonlinear dynamics. The second method extends the results by showing how a Gaussian process prior on the unknown system dynamics can be used in order to reduce conservatism of the resulting safe set. We demonstrate the results with numerical examples, including an autonomous convoy of vehicles.

eess.SY↗

arXiv2020

Maximum Likelihood Methods for Inverse Learning of Optimal Controllers

This paper presents a framework for inverse learning of objective functions for constrained optimal control problems, which is based on the Karush-Kuhn-Tucker (KKT) conditions. We discuss three variants corresponding to different model assumptions and computational complexities. The first method uses a convex relaxation of the KKT conditions and serves as the benchmark. The main contribution of this paper is the proposition of two learning methods that combine the KKT conditions with maximum likelihood estimation. The key benefit of this combination is the systematic treatment of constraints for learning from noisy data with a branch-and-bound algorithm using likelihood arguments. This paper discusses theoretic properties of the learning methods and presents simulation results that highlight the advantages of using the maximum likelihood formulation for learning objective functions.

eess.SY↗

arXiv2020

On Simulation and Trajectory Prediction with Gaussian Process Dynamics

Established techniques for simulation and prediction with Gaussian process (GP) dynamics often implicitly make use of an independence assumption on successive function evaluations of the dynamics model. This can result in significant error and underestimation of the prediction uncertainty, potentially leading to failures in safety-critical applications. This paper discusses methods that explicitly take the correlation of successive function evaluations into account. We first describe two sampling-based techniques; one approach provides samples of the true trajectory distribution, suitable for `ground truth' simulations, while the other draws function samples from basis function approximations of the GP. Second, we propose a linearization-based technique that directly provides approximations of the trajectory distribution, taking correlations explicitly into account. We demonstrate the procedures in simple numerical examples, contrasting the results with established methods.

cs.LG↗

arXiv2020

Noisy-Input Entropy Search for Efficient Robust Bayesian Optimization

We consider the problem of robust optimization within the well-established Bayesian optimization (BO) framework. While BO is intrinsically robust to noisy evaluations of the objective function, standard approaches do not consider the case of uncertainty about the input parameters. In this paper, we propose Noisy-Input Entropy Search (NES), a novel information-theoretic acquisition function that is designed to find robust optima for problems with both input and measurement noise. NES is based on the key insight that the robust objective in many cases can be modeled as a Gaussian process, however, it cannot be observed directly. We evaluate NES on several benchmark problems from the optimization literature and from engineering. The results show that NES reliably finds robust optima, outperforming existing methods from the literature on all benchmarks.

stat.ML↗

arXiv2020

Bayesian Optimization for Policy Search in High-Dimensional Systems via Automatic Domain Selection

Bayesian Optimization (BO) is an effective method for optimizing expensive-to-evaluate black-box functions with a wide range of applications for example in robotics, system design and parameter optimization. However, scaling BO to problems with large input dimensions (>10) remains an open challenge. In this paper, we propose to leverage results from optimal control to scale BO to higher dimensional control tasks and to reduce the need for manually selecting the optimization domain. The contributions of this paper are twofold: 1) We show how we can make use of a learned dynamics model in combination with a model-based controller to simplify the BO problem by focusing onto the most relevant regions of the optimization domain. 2) Based on (1) we present a method to find an embedding in parameter space that reduces the effective dimensionality of the optimization problem. To evaluate the effectiveness of the proposed approach, we present an experimental evaluation on real hardware, as well as simulated tasks including a 48-dimensional policy for a quadcopter.

eess.SY↗

arXiv2020

Using Human Ratings for Feedback Control: A Supervised Learning Approach with Application to Rehabilitation Robotics

This paper presents a method for tailoring a parametric controller based on human ratings. The method leverages supervised learning concepts in order to train a reward model from data. It is applied to a gait rehabilitation robot with the goal of teaching the robot how to walk patients physiologically. In this context, the reward model judges the physiology of the gait cycle (instead of therapists) using sensor measurements provided by the robot and the automatic feedback controller chooses the input settings of the robot to maximize the reward. The key advantage of the proposed method is that only a few input adaptations are necessary to achieve a physiological gait cycle. Experiments with nondisabled subjects show that the proposed method permits the incorporation of human expertise into a control law and to automatically walk patients physiologically.

cs.RO↗

arXiv2019

Cautious Model Predictive Control using Gaussian Process Regression

Gaussian process (GP) regression has been widely used in supervised machine learning due to its flexibility and inherent ability to describe uncertainty in function estimation. In the context of control, it is seeing increasing use for modeling of nonlinear dynamical systems from data, as it allows the direct assessment of residual model uncertainty. We present a model predictive control (MPC) approach that integrates a nominal system with an additive nonlinear part of the dynamics modeled as a GP. Approximation techniques for propagating the state distribution are reviewed and we describe a principled way of formulating the chance constrained MPC problem, which takes into account residual uncertainties provided by the GP model to enable cautious control. Using additional approximations for efficient computation, we finally demonstrate the approach in a simulation example, as well as in a hardware implementation for autonomous racing of remote controlled race cars, highlighting improvements with regard to both performance and safety over a nominal controller.

eess.SY↗

arXiv2019

Dual Stochastic MPC for Systems with Parametric and Structural Uncertainty

Designing controllers for systems affected by model uncertainty can prove to be a challenge, especially when seeking the optimal compromise between the conflicting goals of identification and control. This trade-off is explicitly taken into account in the dual control problem, for which the exact solution is provided by stochastic dynamic programming. Due to its computational intractability, we propose a sampling-based approximation for systems affected by both parametric and structural model uncertainty. The approach proposed in this paper separates the prediction horizon in a dual and an exploitation part. The dual part is formulated as a scenario tree that actively discriminates among a set of potential models while learning unknown parameters. In the exploitation part, achieved information is fixed for each scenario, and open-loop control sequences are computed for the remainder of the horizon. As a result, we solve one optimization problem over a collection of control sequences for the entire horizon, explicitly considering the knowledge gained in each scenario, leading to a dual model predictive control formulation.

eess.SY↗

arXiv2019

Constrained Inverse Optimal Control with Application to a Human Manipulation Task

This paper presents an inverse optimal control methodology and its application to training a predictive model of human motor control from a manipulation task. It introduces a convex formulation for learning both objective function and constraints of an infinite-horizon constrained optimal control problem with nonlinear system dynamics. The inverse approach utilizes Bellman's principle of optimality to formulate the infinite-horizon optimal control problem as a shortest path problem and Lagrange multipliers to identify constraints. We highlight the key benefit of using the shortest path formulation, i.e., the possibility of training the predictive model with short and selected trajectory segments. The method is applied to training a predictive model of movements of a human subject from a manipulation task. The study indicates that individual human movements can be predicted with low error using an infinite-horizon optimal control problem with constraints on shoulder movement.

eess.SY↗

arXiv2019

An Approximate Dynamic Programming Approach for Dual Stochastic Model Predictive Control

Dual control explicitly addresses the problem of trading off active exploration and exploitation in the optimal control of partially unknown systems. While the problem can be cast in the framework of stochastic dynamic programming, exact solutions are only tractable for discrete state and action spaces of very small dimension due to a series of nested minimization and expectation operations. We propose an approximate dual control method for systems with continuous state and input domain based on a rollout dynamic programming approach, splitting the control horizon into a dual and an exploitation part. The dual part is approximated using a scenario tree generated by sampling the process noise and the unknown system parameters, for which the underlying distribution is updated via Bayesian estimation along the horizon. In the exploitation part, we fix the resulting parameter estimate of each scenario branch and compute an open-loop control sequence for the remainder of the horizon. The key benefit of the proposed sampling-based approximation is that it enables the formulation as one optimization problem that computes a collection of control sequences over the scenario tree, leading to a dual model predictive control formulation.

eess.SY↗

arXiv2019

Knee Compliance Reduces Peak Swing Phase Collision Forces in a Lower-Limb Exoskeleton Leg: A Test Bench Evaluation

Powered lower limb exoskeletons are a viable solution for people with a spinal cord injury to regain mobility for their daily activities. However, the commonly employed rigid actuation and pre-programmed trajectories increase the risk of falling in case of collisions with external objects. Compliant actuation may reduce forces during collisions, thus protecting hardware and user. However, experimental data of collisions specific to lower limb exoskeletons are not available. In this work, we investigated how a variable stiffness actuator at the knee joint influences collision forces transmitted to the user via the exoskeleton. In a test bench experiment, we compared three configurations of an exoskeleton leg with a variable stiffness knee actuator in (i) compliant or (ii) stiff configurations, and with (iii) a rigid actuator. The peak torque observed at the pelvis was reduced from 260.2 Nm to 116.2 Nm as stiffness decreased. In addition, the mechanical impulse was reduced by a factor of three. These results indicate that compliance in the knee joint of an exoskeleton can be favorable in case of collision and should be considered when designing powered lower limb exoskeletons. Overall, this could decrease the effort necessary to maintain balance after a collision and improved collision handling in exoskeletons could result in safer use and benefit their usefulness in daily life.

physics.med-ph↗

arXiv2019

Linear model predictive safety certification for learning-based control

While it has been repeatedly shown that learning-based controllers can provide superior performance, they often lack of safety guarantees. This paper aims at addressing this problem by introducing a model predictive safety certification (MPSC) scheme for polytopic linear systems with additive disturbances. The scheme verifies safety of a proposed learning-based input and modifies it as little as necessary in order to keep the system within a given set of constraints. Safety is thereby related to the existence of a model predictive controller (MPC) providing a feasible trajectory towards a safe target set. A robust MPC formulation accounts for the fact that the model is generally uncertain in the context of learning, which allows proving constraint satisfaction at all times under the proposed MPSC strategy. The MPSC scheme can be used in order to expand any potentially conservative set of safe states for learning and we prove an iterative technique for enlarging the safe set. Finally, a practical data-based design procedure for MPSC is proposed using scenario optimization.

eess.SY↗

arXiv2019

Recursively Feasible Stochastic Model Predictive Control using Indirect Feedback

We present a stochastic model predictive control (MPC) method for linear discrete-time systems subject to possibly unbounded and correlated additive stochastic disturbance sequences. Chance constraints are treated in analogy to robust MPC using the concept of probabilistic reachable sets for constraint tightening. We introduce an initialization of each MPC iteration which is always recursively feasibility and thereby allows that chance constraint satisfaction for the closed-loop system can readily be shown. Under an i.i.d. zero mean assumption on the additive disturbance, we furthermore provide an average asymptotic performance bound. Two examples illustrate the approach, highlighting feedback properties of the novel initialization scheme, as well as the inclusion of time-varying, correlated disturbances in a building control setting.

eess.SY↗

arXiv2018

Cautious NMPC with Gaussian Process Dynamics for Autonomous Miniature Race Cars

This paper presents an adaptive high performance control method for autonomous miniature race cars. Racing dynamics are notoriously hard to model from first principles, which is addressed by means of a cautious nonlinear model predictive control (NMPC) approach that learns to improve its dynamics model from data and safely increases racing performance. The approach makes use of a Gaussian Process (GP) and takes residual model uncertainty into account through a chance constrained formulation. We present a sparse GP approximation with dynamically adjusting inducing inputs, enabling a real-time implementable controller. The formulation is demonstrated in simulations, which show significant improvement with respect to both lap time and constraint satisfaction compared to an NMPC without model learning.

eess.SY↗