arXiv subjects

Milan Korda

Publications and source records attributed to Milan Korda.

At least 55 records · Page 3Linked to original sources

arXiv2020

Computing controlled invariant sets from data using convex optimization

This work presents a data-driven method for approximation of the maximum positively invariant (MPI) set and the maximum controlled invariant (MCI) set for nonlinear dynamical systems. The method only requires the knowledge of a finite collection of one-step transitions of the discrete-time dynamics, without the requirement of segments of trajectories or the control inputs that effected the transitions to be available. The approach uses a novel characterization of the MPI and MCI sets as the solution to an infinite-dimensional linear programming (LP) problem in the space of continuous functions, with the optimum being attained by a (Lipschitz) continuous function under mild assumptions. The infinite-dimensional LP is then approximated by restricting the decision variable to a finite-dimensional subspace and by imposing the non-negativity constraint of this LP only on the available data samples. This leads to a single finite-dimensional LP that can be easily solved using off-the-shelf solvers. We analyze the convergence rate and sample complexity, proving probabilistic as well as hard guarantees on the volume error of the approximations. The approach is very general, requiring minimal underlying assumptions. In particular, the dynamics is not required to be polynomial or even continuous (forgoing some of the theoretical results). Detailed numerical examples up to state-space dimension ten with code available online demonstrate the method.

math.OC↗

arXiv2020

Convex computation of extremal invariant measures of nonlinear dynamical systems and Markov processes

We propose a convex-optimization-based framework for computation of invariant measures of polynomial dynamical systems and Markov processes, in discrete and continuous time. The set of all invariant measures is characterized as the feasible set of an infinite-dimensional linear program (LP). The objective functional of this LP is then used to single-out a specific measure (or a class of measures) extremal with respect to the selected functional such as physical measures, ergodic measures, atomic measures (corresponding to, e.g., periodic orbits) or measures absolutely continuous w.r.t. to a given measure. The infinite-dimensional LP is then approximated using a standard hierarchy of finite-dimensional semidefinite programming problems (SDPs), the solutions of which are truncated moment sequences, which are then used to reconstruct the measure. In particular, we show how to approximate the support of the measure as well as how to construct a sequence of weakly converging absolutely continuous approximations. The presented framework, where a convex functional is minimized or maximized among all invariant measures, can be seen as a generalization of and a computational method to carry out the so called ergodic optimization, where linear functionals are optimized over the set of invariant measures. Finally, we also describe how the presented framework can be adapted to compute eigenmeasures of the Perron-Frobenius operator.

math.OC↗

arXiv2020

Optimal construction of Koopman eigenfunctions for prediction and control

This work presents a novel data-driven framework for constructing eigenfunctions of the Koopman operator geared toward prediction and control. The method leverages the richness of the spectrum of the Koopman operator away from attractors to construct a rich set of eigenfunctions such that the state (or any other observable quantity of interest) is in the span of these eigenfunctions and hence predictable in a linear fashion. The eigenfunction construction is optimization-based with no dictionary selection required. Once a predictor for the uncontrolled part of the system is obtained in this way, the incorporation of control is done through a multi-step prediction error minimization, carried out by a simple linear least-squares regression. The predictor so obtained is in the form of a linear controlled dynamical system and can be readily applied within the Koopman model predictive control framework of [12] to control nonlinear dynamical systems using linear model predictive control tools. The method is entirely data-driven and based purely on convex optimization, with no reliance on neural networks or other non-convex machine learning tools. The novel eigenfunction construction method is also analyzed theoretically, proving rigorously that the family of eigenfunctions obtained is rich enough to span the space of all continuous functions. In addition, the method is extended to construct generalized eigenfunctions that also give rise Koopman invariant subspaces and hence can be used for linear prediction. Detailed numerical examples with code available online demonstrate the approach, both for prediction and feedback control.

math.OC↗

arXiv2018

Data-driven spectral analysis of the Koopman operator

Starting from measured data, we develop a method to compute the fine structure of the spectrum of the Koopman operator with rigorous convergence guarantees. The method is based on the observation that, in the measure-preserving ergodic setting, the moments of the spectral measure associated to a given observable are computable from a single trajectory of this observable. Having finitely many moments available, we use the classical Christoffel-Darboux kernel to separate the atomic and absolutely continuous parts of the spectrum, supported by convergence guarantees as the number of moments tends to infinity. In addition, we propose a technique to detect the singular continuous part of the spectrum as well as two methods to approximate the spectral measure with guaranteed convergence in the weak topology, irrespective of whether the singular continuous part is present or not. The proposed method is simple to implement and readily applicable to large-scale systems since the computational complexity is dominated by inverting an $N\times N$ Hermitian positive-definite Toeplitz matrix, where $N$ is the number of moments, for which efficient and numerically stable algorithms exist; in particular, the complexity of the approach is independent of the dimension of the underlying state-space. We also show how to compute, from measured data, the spectral projection on a given segment of the unit circle, allowing us to obtain a finite-dimensional approximation of the operator that explicitly takes into account the point and continuous parts of the spectrum. Finally, we describe a relationship between the proposed method and the so-called Hankel Dynamic Mode Decomposition, providing new insights into the behavior of the eigenvalues of the Hankel DMD operator. A number of numerical examples illustrate the approach, including a study of the spectrum of the lid-driven two-dimensional cavity flow.

math.DS↗

arXiv2018

A data-driven Koopman model predictive control framework for nonlinear flows

The Koopman operator theory is an increasingly popular formalism of dynamical systems theory which enables analysis and prediction of the nonlinear dynamics from measurement data. Building on the recent development of the Koopman model predictive control framework (Korda and Mezic 2016), we propose a methodology for closed-loop feedback control of nonlinear flows in a fully data-driven and model-free manner. In the first step, we compute a Koopman-linear representation of the control system using a variation of the extended dynamic mode decomposition algorithm and then we apply model predictive control to the constructed linear model. Our methodology handles both full-state and sparse measurement; in the latter case, it incorporates the delay-embedding of the available data into the identification and control processes. We illustrate the application of this methodology on the periodic Burgers' equation and the boundary control of a cavity flow governed by the two-dimensional incompressible Navier-Stokes equations. In both examples the proposed methodology is successful in accomplishing the control tasks with sub-millisecond computation time required for evaluation of the control input in closed-loop, thereby allowing for a real-time deployment.

physics.flu-dyn↗

arXiv2018

Moments and convex optimization for analysis and control of nonlinear partial differential equations

This work presents a convex-optimization-based framework for analysis and control of nonlinear partial differential equations. The approach uses a particular weak embedding of the nonlinear PDE, resulting in a linear equation in the space of Borel measures. This equation is then used as a constraint of an infinite-dimensional linear programming problem (LP). This LP is then approximated by a hierarchy of convex, finite-dimensional, semidefinite programming problems (SDPs). In the case of analysis of uncontrolled PDEs, the solutions to these SDPs provide bounds on a specified, possibly nonlinear, functional of the solutions to the PDE; in the case of PDE control, the solutions to these SDPs provide bounds on the optimal value of a given optimal control problem as well as suboptimal feedback controllers. The entire approach is based purely on convex optimization and does not rely on spatio-temporal gridding, even though the PDE addressed can be fully nonlinear. The approach is applicable to a very broad class nonlinear PDEs with polynomial data. Computational complexity is analyzed and several complexity reduction procedures are described. Numerical examples demonstrate the approach.

math.OC↗

arXiv2018

Power grid transient stabilization using Koopman model predictive control

This work addresses the problem of transient stabilization of a power grid, following a destabilizing disturbance. The model considered is the cascade interconnection of seven New England test models with the disturbance (e.g., a powerline failure) occurring in the first grid and propagating forward, emulating a wide-area blackout. We consider a data-driven control framework based on the Koopman operator theory, where a linear predictor, evolving on a higher dimensional (embedded) state-space, is built from observed data and subsequently used within a model predictive control (MPC) framework, allowing for the use of efficient computational tools of linear MPC to control this highly nonlinear dynamical system.

math.OC↗

arXiv2018

Linear predictors for nonlinear dynamical systems: Koopman operator meets model predictive control

This paper presents a class of linear predictors for nonlinear controlled dynamical systems. The basic idea is to lift the nonlinear dynamics into a higher dimensional space where its evolution is approximately linear. In an uncontrolled setting, this procedure amounts to a numerical approximation of the Koopman operator associated to the nonlinear dynamics. In this work, we extend the Koopman operator to controlled dynamical systems and compute a finite-dimensional approximation of the operator in such a way that this approximation has the form a linear controlled dynamical system. In numerical examples, the linear predictors obtained in this way exhibit a performance superior to existing linear predictors such as those based on local linearization or the so-called Carleman linearization. Importantly, the procedure to construct these linear predictors is completely data-driven and extremely simple -- it boils down to a nonlinear transformation of the data (the lifting) and a linear least squares problem in the lifted space that can be readily solved for large data sets. These linear predictors can be readily used to design controllers for the nonlinear dynamical system using linear controller design methodologies. We focus in particular on model predictive control (MPC) and show that MPC controllers designed in this way enjoy computational complexity of the underlying optimization problem comparable to that of MPC for a linear dynamical system of the same size. Importantly, linear inequality constraints on the state and control inputs as well as nonlinear constraints on the state can be imposed in a linear fashion in the proposed MPC scheme. Similarly, cost functions nonlinear in the state variable can be handled in a linear fashion. Numerical examples (including a high-dimensional nonlinear PDE control) demonstrate the approach with the source code available online.

math.OC↗

arXiv2017

On Convergence of Extended Dynamic Mode Decomposition to the Koopman Operator

Extended Dynamic Mode Decomposition (EDMD) is an algorithm that approximates the action of the Koopman operator on an $N$-dimensional subspace of the space of observables by sampling at $M$ points in the state space. Assuming that the samples are drawn either independently or ergodically from some measure $μ$, it was shown that, in the limit as $M\rightarrow\infty$, the EDMD operator $\mathcal{K}_{N,M}$ converges to $\mathcal{K}_N$, where $\mathcal{K}_N$ is the $L_2(μ)$-orthogonal projection of the action of the Koopman operator on the finite-dimensional subspace of observables. In this work, we show that, as $N \rightarrow \infty$, the operator $\mathcal{K}_N$ converges in the strong operator topology to the Koopman operator. This in particular implies convergence of the predictions of future values of a given observable over any finite time horizon, a fact important for practical applications such as forecasting, estimation and control. In addition, we show that accumulation points of the spectra of $\mathcal{K}_N$ correspond to the eigenvalues of the Koopman operator with the associated eigenfunctions converging weakly to an eigenfunction of the Koopman operator, provided that the weak limit of eigenfunctions is nonzero. As a by-product, we propose an analytic version of the EDMD algorithm which, under some assumptions, allows one to construct $\mathcal{K}_N$ directly, without the use of sampling. Finally, under additional assumptions, we analyze convergence of $\mathcal{K}_{N,N}$ (i.e., $M=N$), proving convergence, along a subsequence, to weak eigenfunctions (or eigendistributions) related to the eigenmeasures of the Perron-Frobenius operator. No assumptions on the observables belonging to a finite-dimensional invariant subspace of the Koopman operator are required throughout.

math.OC↗

arXiv2017

On Turnpike and Dissipativity Properties of Continuous-Time Optimal Control Problems

This paper investigates the relations between three different properties, which are of importance in optimal control problems: dissipativity of the underlying dynamics with respect to a specific supply rate, optimal operation at steady state, and the turnpike property. We show in a continuous-time setting that if along optimal trajectories a strict dissipation inequality is satisfied, then this implies optimal operation at this steady state and the existence of a turnpike at the same steady state. Finally, we establish novel converse turnpike results, i.e., we show that the existence of a turnpike at a steady state implies optimal operation at this steady state and dissipativity with respect to this steady state. We draw upon a numerical example to illustrate our findings.

eess.SY↗

arXiv2016

Convergence rates of moment-sum-of-squares hierarchies for volume approximation of semialgebraic sets

Moment-sum-of-squares hierarchies of semidefinite programs can be used to approximate the volume of a given compact basic semialgebraic set K. The idea consists of approximating from above the indicator function of K with a sequence of polynomials of increasing degree d, so that the integrals of these polynomials generate a convergence sequence of upper bounds on the volume of K. We show that the asymptotic rate of this convergence is at least O(1/ log log d).

math.OC↗

arXiv2016

Stability and Performance Verification of Optimization-based Controllers

This paper presents a method to verify closed-loop properties of optimization-based controllers for deterministic and stochastic constrained polynomial discrete-time dynamical systems. The closed-loop properties amenable to the proposed technique include global and local stability, performance with respect to a given cost function (both in a deterministic and stochastic setting) and the $\mathcal{L}_2$ gain. The method applies to a wide range of practical control problems: For instance, a dynamical controller (e.g., a PID) plus input saturation, model predictive control with state estimation, inexact model and soft constraints, or a general optimization-based controller where the underlying problem is solved with a fixed number of iterations of a first-order method are all amenable to the proposed approach. The approach is based on the observation that the control input generated by an optimization-based controller satisfies the associated Karush-Kuhn-Tucker (KKT) conditions which, provided all data is polynomial, are a system of polynomial equalities and inequalities. The closed-loop properties can then be analyzed using sum-of-squares (SOS) programming.

math.OC↗

arXiv2016

Convergence rates of moment-sum-of-squares hierarchies for optimal control problems

We study the convergence rate of moment-sum-of-squares hierarchies of semidefinite programs for optimal control problems with polynomial data. It is known that these hierarchies generate polynomial under-approximations to the value function of the optimal control problem and that these under-approximations converge in the L1 norm to the value function as their degree d tends to infinity. We show that the rate of this convergence is O(1/ log log d). We treat in detail the continuous-time infinite-horizon discounted problem and describe in brief how the same rate can be obtained for the finite-horizon continuous-time problem and for the discrete-time counterparts of both problems.

math.OC↗

arXiv2015

Controller design and value function approximation for nonlinear dynamical systems

This work considers the infinite-time discounted optimal control problem for continuous time input-affine polynomial dynamical systems subject to polynomial state and box input constraints. We propose a sequence of sum-of-squares (SOS) approximations of this problem obtained by first lifting the original problem into the space of measures with continuous densities and then restricting these densities to polynomials. These approximations are tightenings, rather than relaxations, of the original problem and provide a sequence of rational controllers with value functions associated to these controllers converging (under some technical assumptions) to the value function of the original problem. In addition, we describe a method to obtain polynomial approximations from above and from below to the value function of the extracted rational controllers, and a method to obtain approximations from below to the optimal value function of the original problem, thereby obtaining a sequence of asymptotically optimal rational controllers with explicit estimates of suboptimality. Numerical examples demonstrate the approach.

math.OC↗

arXiv2015

Solving the Infinite-horizon Constrained LQR Problem using Accelerated Dual Proximal Methods

This work presents an algorithmic scheme for solving the infinite-time constrained linear quadratic regulation problem. We employ an accelerated version of a popular proximal gradient scheme, commonly known as the Forward-Backward Splitting (FBS), and prove its convergence to the optimal solution in our infinite-dimensional setting. Each iteration of the algorithm requires only finite memory, is computationally cheap, and makes no use of terminal invariant sets; hence, the algorithm can be applied to systems of very large dimensions. The acceleration brings in optimal convergence rates O(1/k^2) for function values and O(1/k) for primal iterates and renders the proposed method a practical alternative to model predictive control schemes for setpoint tracking. In addition, for the case when the true system is subject to disturbances or modelling errors, we propose an efficient warm-starting procedure, which significantly reduces the number of iterations when the algorithm is applied in closed-loop. Numerical examples demonstrate the approach.

math.OC↗

arXiv2014

Controller design and region of attraction estimation for nonlinear dynamical systems

This work presents a method to obtain inner and outer approximations of the region of attraction of a given target set as well as an admissible controller generating the inner approximation. The method is applicable to constrained polynomial dynamical systems and extends to trigonometric and rational systems. The method consists of three steps: compute outer approximations, extract a polynomial controller while guaranteeing the satisfaction of the input constraints, compute inner approximations with respect to the closed-loop system with this controller. Each step of the method is a convex optimization problem, in fact a semidefinite program consisting of minimizing a linear function subject to linear matrix inequality (LMI) constraints. The inner approximations are positively invariant provided that the target set is included in the inner approximation and/or is itself invariant. %The approach readily extends to trigonometric dynamics and/or constraints.

math.OC↗

arXiv2013

Convex computation of the region of attraction of polynomial control systems

We address the long-standing problem of computing the region of attraction (ROA) of a target set (e.g., a neighborhood of an equilibrium point) of a controlled nonlinear system with polynomial dynamics and semialgebraic state and input constraints. We show that the ROA can be computed by solving an infinite-dimensional convex linear programming (LP) problem over the space of measures. In turn, this problem can be solved approximately via a classical converging hierarchy of convex finite-dimensional linear matrix inequalities (LMIs). Our approach is genuinely primal in the sense that convexity of the problem of computing the ROA is an outcome of optimizing directly over system trajectories. The dual infinite-dimensional LP on nonnegative continuous functions (approximated by polynomial sum-of-squares) allows us to generate a hierarchy of semialgebraic outer approximations of the ROA at the price of solving a sequence of LMI problems with asymptotically vanishing conservatism. This sharply contrasts with the existing literature which follows an exclusively dual Lyapunov approach yielding either nonconvex bilinear matrix inequalities or conservative LMI conditions. The approach is simple and readily applicable as the outer approximations are the outcome of a single semidefinite program with no additional data required besides the problem description.

math.OC↗

arXiv2013

Convex computation of the maximum controlled invariant set for polynomial control systems

We characterize the maximum controlled invariant (MCI) set for discrete- as well as continuous-time nonlinear dynamical systems as the solution of an infinite-dimensional linear programming problem. For systems with polynomial dynamics and compact semialgebraic state and control constraints, we describe a hierarchy of finite-dimensional linear matrix inequality (LMI) relaxations whose optimal values converge to the volume of the MCI set; dual to these LMI relaxations are sum-of-squares (SOS) problems providing a converging sequence of outer approximations to the MCI set. The approach is simple and readily applicable in the sense that the approximations are the outcome of a single semidefinite program with no additional input apart from the problem description. A number of numerical examples illustrate the approach.

math.OC↗