arXiv subjects

Mario Zanon

Publications and source records attributed to Mario Zanon.

At least 37 records · Page 2Linked to original sources

arXiv2023

Experimental Validation of Safe MPC for Autonomous Driving in Uncertain Environments

The full deployment of autonomous driving systems on a worldwide scale requires that the self-driving vehicle be operated in a provably safe manner, i.e., the vehicle must be able to avoid collisions in any possible traffic situation. In this paper, we propose a framework based on Model Predictive Control (MPC) that endows the self-driving vehicle with the necessary safety guarantees. In particular, our framework ensures constraint satisfaction at all times, while tracking the reference trajectory as close as obstacles allow, resulting in a safe and comfortable driving behavior. To discuss the performance and real-time capability of our framework, we provide first an illustrative simulation example, and then we demonstrate the effectiveness of our framework in experiments with a real test vehicle.

cs.RO↗

arXiv2023

Equivalence of Optimality Criteria for Markov Decision Process and Model Predictive Control

This paper shows that the optimal policy and value functions of a Markov Decision Process (MDP), either discounted or not, can be captured by a finite-horizon undiscounted Optimal Control Problem (OCP), even if based on an inexact model. This can be achieved by selecting a proper stage cost and terminal cost for the OCP. A very useful particular case of OCP is a Model Predictive Control (MPC) scheme where a deterministic (possibly nonlinear) model is used to reduce the computational complexity. This observation leads us to parameterize an MPC scheme fully, including the cost function. In practice, Reinforcement Learning algorithms can then be used to tune the parameterized MPC scheme. We verify the developed theorems analytically in an LQR case and we investigate some other nonlinear examples in simulations.

eess.SY↗

arXiv2022

Model Predictive Control Tuning by Monte Carlo Simulation and Controller Matching

This paper presents a systematic method for the selection of the Model Predictive Control (MPC) stage cost. We match the MPC feedback law to a proportional-integral (PI) controller, which we efficiently tune by high-performance Monte Carlo (MC) simulation. The PI tuning offers a wide range of tuning possibilities that is then inherited by the MPC design. The MC simulation tuning of the PI controller is based on the minimization of two different objectives; 1) the 2-norm tracking error, and 2) a bi-objective consisting of the 2-norm tracking error and a 2-norm input rate of movement penalty. We apply the method to design MPC for an exothermic chemical reaction conducted in an adiabatic continuous stirred tank reactor (CSTR). The process is of interest as the nonlinear dynamics result in a desired operating point very close to a constraint. Our MPC design includes stage costs automatically designed to match the tuned PI controllers, hard input constraints, and a soft output constraint. Stochastic simulation results show that both the PI controller and the MPC can track the desired operating point. However, the MPC shows reduced output constraint violation compared to the PI controller. As such, the MPC design method successfully combines the efficient tuning of the PI controller with the constraint handling properties of MPC.

eess.SY↗

arXiv2022

Learning for MPC with Stability & Safety Guarantees

The combination of learning methods with Model Predictive Control (MPC) has attracted a significant amount of attention in the recent literature. The hope of this combination is to reduce the reliance of MPC schemes on accurate models, and to tap into the fast developing machine learning and reinforcement learning tools to exploit the growing amount of data available for many systems. In particular, the combination of reinforcement learning and MPC has been proposed as a viable and theoretically justified approach to introduce explainable, safe and stable policies in reinforcement learning. However, a formal theory detailing how the safety and stability of an MPC-based policy can be maintained through the parameter updates delivered by the learning tools is still lacking. This paper addresses this gap. The theory is developed for the generic Robust MPC case, and applied in simulation in the robust tube-based linear MPC case, where the theory is fairly easy to deploy in practice. The paper focuses on Reinforcement Learning as a learning tool, but it applies to any learning method that updates the MPC parameters online.

cs.LG↗

arXiv2022

Economic MPC of Markov Decision Processes: Dissipativity in Undiscounted Infinite-Horizon Optimal Control

Economic Model Predictive Control (MPC) dissipativity theory is central to discussing the stability of policies resulting from minimizing economic stage costs. In its current form, the dissipativity theory for economic MPC applies to problems based on deterministic dynamics or to very specific classes of stochastic problems, and does not readily extend to generic Markov Decision Processes. In this paper, we clarify the core reason for this difficulty, and propose a generalization of the economic MPC dissipativity theory that circumvents it. This generalization focuses on undiscounted infinite-horizon problems and is based on nonlinear stage cost functionals, allowing one to discuss the Lyapunov asymptotic stability of policies for Markov Decision Processes in terms of the probability measures underlying their stochastic dynamics. This theory is illustrated for the stochastic Linear Quadratic Regulator with Gaussian process noise, for which a storage functional can be provided explicitly. For the sake of brevity, we limit our discussion to undiscounted Markov Decision Processes.

eess.SY↗

arXiv2022

A New Dissipativity Condition for Asymptotic Stability of Discounted Economic MPC

Economic Model Predictive Control has recently gained popularity due to its ability to directly optimize a given performance criterion, while enforcing constraint satisfaction for nonlinear systems. Recent research has developed both numerical algorithms and stability analysis for the undiscounted case. The introduction of a discount factor in the cost, however, can be desirable in some cases of interest, e.g., economics, stochastically terminating processes, Markov decision processes, etc. Unfortunately, the stability theory in this case is still not fully developed. In this paper we propose a new dissipativity condition to prove asymptotic stability in the infinite horizon case and we connect our results with existing ones in the literature on discounted economic optimal control. Numerical examples are provided to illustrate the theoretical results.

eess.SY↗

arXiv2021

Model Predictive Control with Environment Adaptation for Legged Locomotion

Re-planning in legged locomotion is crucial to track the desired user velocity while adapting to the terrain and rejecting external disturbances. In this work, we propose and test in experiments a real-time Nonlinear Model Predictive Control (NMPC) tailored to a legged robot for achieving dynamic locomotion on a variety of terrains. We introduce a mobility-based criterion to define an NMPC cost that enhances the locomotion of quadruped robots while maximizing leg mobility and improves adaptation to the terrain features. Our NMPC is based on the real-time iteration scheme that allows us to re-plan online at $25\,\mathrm{Hz}$ with a prediction horizon of $2$ seconds. We use the single rigid body dynamic model defined in the center of mass frame in order to increase the computational efficiency. In simulations, the NMPC is tested to traverse a set of pallets of different sizes, to walk into a V-shaped chimney,and to locomote over rough terrain. In real experiments, we demonstrate the effectiveness of our NMPC with the mobility feature that allowed IIT's $87\, \mathrm{kg}$ quadruped robot HyQ to achieve an omni-directional walk on flat terrain, to traverse a static pallet, and to adapt to a repositioned pallet during a walk.

cs.RO↗

arXiv2021

A Semi-Distributed Interior Point Algorithm for Optimal Coordination of Automated Vehicles at Intersections

In this paper, we consider the optimal coordination of automated vehicles at intersections under fixed crossing orders. We formulate the problem using direct optimal control and exploit the structure to construct a semi-distributed primal-dual interior-point algorithm to solve it by parallelizing most of the computations. Differently from standard distributed optimization algorithms, where the optimization problem is split, in our approach we split the linear algebra steps, such that the algorithm takes the same steps as a fully centralized one, while still performing computations in a distributed fashion. We analyze the communication requirements of the algorithm, and propose an approximation scheme which can significantly reduce the data exchange. We demonstrate the effectiveness of the algorithm in hard but realistic scenarios, which show that the approximation leads to reductions in communicated data of almost 99\% of the exact formulation, at the expense of less than 1\% suboptimality.

eess.SY↗

arXiv2021

Data-driven synthesis of Robust Invariant Sets and Controllers

This paper presents a method to identify an uncertain linear time-invariant (LTI) prediction model for tube-based Robust Model Predictive Control (RMPC). The uncertain model is determined from a given state-input dataset by formulating and solving a Semidefinite Programming problem (SDP), that also determines a static linear feedback gain and corresponding invariant sets satisfying the inclusions required to guarantee recursive feasibility and stability of the RMPC scheme, while minimizing an identification criterion. As demonstrated through an example, the proposed concurrent approach provides less conservative invariant sets than a sequential approach.

eess.SY↗

arXiv2021

Computation of Input Disturbance Sets for Constrained Output Reachability

Linear models with additive unknown-but-bounded input disturbances are extensively used to model uncertainty in robust control systems design. Typically, the disturbance set is either assumed to be known a priori or estimated from data through set-membership identification. However, the problem of computing a suitable input disturbance set in case the set of possible output values is assigned a priori has received relatively little attention. This problem arises in many contexts, such as in supervisory control, actuator design, decentralized control, and others. In this paper, we propose a method to compute input disturbance sets (and the corresponding set of states) such that the resulting set of outputs matches as closely as possible a given set of outputs, while additionally satisfying strict (inner or outer) inclusion constraints. We formulate the problem as an optimization problem by relying on the concept of robust invariance. The effectiveness of the approach is demonstrated in numerical examples that illustrate how to solve safe reference set and input-constraint set computation problems.

math.OC↗

arXiv2021

Constrained Controller and Observer Design by Inverse Optimality

Model Predictive Control (MPC) is often tuned by trial and error. When a baseline linear controller exists that is already well tuned in the absence of constraints and MPC is introduced to enforce them, one would like to avoid altering the original linear feedback law whenever they are not active. We formulate this problem as a controller matching similar to [1]-[3], which we extend to a more general framework. We prove that a positive-definite stage cost matrix yielding this matching property can be computed for all stabilizing linear controllers. Additionally, we prove that the constrained estimation problem can also be solved similarly, by matching a linear observer with a Moving Horizon Estimator (MHE). Finally, we discuss various aspects of the practical implementation of the proposed technique in some examples.

eess.SY↗

arXiv2021

Model Predictive Control with Infeasible Reference Trajectories

Model Predictive Control (MPC) formulations are typically built on the requirement that a feasible reference trajectory is available. In practical settings, however, references that are infeasible with respect to the system dynamics are used for convenience. In this paper, we prove under which conditions an MPC formulation is Input-to-State Stable~(ISS) in closed-loop when an infeasible reference is used, and that with proper terminal conditions, asymptotic stability towards an optimal reference may be achieved. We illustrate the theoretical results with a four-dimensional robotic joint example.

eess.SY↗

arXiv2021

Fast and scalable likelihood maximization for Exponential Random Graph Models with local constraints

Exponential Random Graph Models (ERGMs) have gained increasing popularity over the years. Rooted into statistical physics, the ERGMs framework has been successfully employed for reconstructing networks, detecting statistically significant patterns in graphs, counting networked configurations with given properties. From a technical point of view, the ERGMs workflow is defined by two subsequent optimization steps: the first one concerns the maximization of Shannon entropy and leads to identify the functional form of the ensemble probability distribution that is maximally non-committal with respect to the missing information; the second one concerns the maximization of the likelihood function induced by this probability distribution and leads to its numerical determination. This second step translates into the resolution of a system of $O(N)$ non-linear, coupled equations (with $N$ being the total number of nodes of the network under analysis), a problem that is affected by three main issues, i.e. accuracy, speed and scalability. The present paper aims at addressing these problems by comparing the performance of three algorithms (i.e. Newton's method, a quasi-Newton method and a recently-proposed fixed-point recipe) in solving several ERGMs, defined by binary and weighted constraints in both a directed and an undirected fashion. While Newton's method performs best for relatively little networks, the fixed-point recipe is to be preferred when large configurations are considered, as it ensures convergence to the solution within seconds for networks with hundreds of thousands of nodes (e.g. the Internet, Bitcoin). We attach to the paper a Python code implementing the three aforementioned algorithms on all the ERGMs considered in the present work.

physics.data-an↗

arXiv2021

Safe Trajectory Tracking in Uncertain Environments

In Model Predictive Control (MPC) formulations of trajectory tracking problems, infeasible reference trajectories and a-priori unknown constraints can lead to cumbersome designs, aggressive tracking, and loss of recursive feasibility. This is the case, for example, in trajectory tracking applications for mobile systems in the presence of constraints which are not fully known a-priori. In this paper, we propose a new framework called Model Predictive Flexible trajectory Tracking Control (MPFTC), which relaxes the trajectory tracking requirement. Additionally, we accommodate recursive feasibility in the presence of a-priori unknown constraints, which might render the reference trajectory infeasible. In the proposed framework, constraint satisfaction is guaranteed at all times while the reference trajectory is tracked as good as constraint satisfaction allows, thus simplifying the controller design and reducing possibly aggressive tracking behavior. The proposed framework is illustrated with three numerical examples.

eess.SY↗

arXiv2021

Stability-Constrained Markov Decision Processes Using MPC

In this paper, we consider solving discounted Markov Decision Processes (MDPs) under the constraint that the resulting policy is stabilizing. In practice MDPs are solved based on some form of policy approximation. We will leverage recent results proposing to use Model Predictive Control (MPC) as a structured policy in the context of Reinforcement Learning to make it possible to introduce stability requirements directly inside the MPC-based policy. This will restrict the solution of the MDP to stabilizing policies by construction. The stability theory for MPC is most mature for the undiscounted MPC case. Hence, we will first show in this paper that stable discounted MDPs can be reformulated as undiscounted ones. This observation will entail that the MPC-based policy with stability requirements will produce the optimal policy for the discounted MDP if it is stable, and the best stabilizing policy otherwise.

cs.LG↗

arXiv2020

A Gauss-Newton-Like Hessian Approximation for Economic NMPC

Economic Model Predictive Control (EMPC) has recently become popular because of its ability to control constrained nonlinear systems while explicitly optimizing a prescribed performance criterion. Large performance gains have been reported for many applications and closed-loop stability has been recently investigated. However, computational performance still remains an open issue and only few contributions have proposed real-time algorithms tailored to EMPC. We perform a step towards computationally cheap algorithms for EMPC by proposing a new positive-definite Hessian approximation which does not hinder fast convergence and is suitable for being used within the real-time iteration (RTI) scheme. We provide two simulation examples to demonstrate the effectiveness of RTI-based EMPC relying on the proposed Hessian approximation.

eess.SY↗

arXiv2020

Safe Reinforcement Learning Using Robust MPC

Reinforcement Learning (RL) has recently impressed the world with stunning results in various applications. While the potential of RL is now well-established, many critical aspects still need to be tackled, including safety and stability issues. These issues, while partially neglected by the RL community, are central to the control community which has been widely investigating them. Model Predictive Control (MPC) is one of the most successful control techniques because, among others, of its ability to provide such guarantees even for uncertain constrained systems. Since MPC is an optimization-based technique, optimality has also often been claimed. Unfortunately, the performance of MPC is highly dependent on the accuracy of the model used for predictions. In this paper, we propose to combine RL and MPC in order to exploit the advantages of both and, therefore, obtain a controller which is optimal and safe. We illustrate the results with a numerical example in simulations.

eess.SY↗

arXiv2020

Primal or Dual Terminal Constraints in Economic MPC? -- Comparison and Insights

This chapter compares different formulations for Economic nonlinear Model Predictive Control (EMPC) which are all based on an established dissipativity assumption on the underlying Optimal Control Problem (OCP). This includes schemes with and without stabilizing terminal constraints, respectively, or with stabilizing terminal costs. We recall that a recently proposed approach based on gradient correcting terminal penalties implies a terminal constraint on the adjoints of the OCP. We analyze the feasibility implications of these dual/adjoint terminal constraints and we compare our findings to approaches with and without primal terminal constraints. Moreover, we suggest a conceptual framework for approximation of the minimal stabilizing horizon length. Finally, we illustrate our findings considering a chemical reactor as an example.

eess.SY↗