arXiv subjects

John Lygeros

Publications and source records attributed to John Lygeros.

At least 163 records · Page 9Linked to original sources

arXiv2021

On the Synthesis of Bellman Inequalities for Data-Driven Optimal Control

In the context of the linear programming (LP) approach to data-driven control, one assumes that the dynamical system is unknown but can be observed indirectly through data on its evolution. Both theoretical and empirical evidence suggest that a desired suboptimality gap is often only achieved with massive exploration of the state-space. In case of linear systems, we discuss how a relatively small but sufficiently rich dataset can be exploited to generate new constraints offline and without observing the corresponding transitions. Moreover, we show how to reconstruct the associated unknown stage-costs and, when the system is stochastic, we offer insights on the related problem of estimating the expected value in the Bellman operator without re-initializing the dynamics in the same state-input pairs.

math.OC↗

arXiv2021

Sampled-Data Online Feedback Equilibrium Seeking: Stability and Tracking

This paper proposes a general framework for constructing feedback controllers that drive complex dynamical systems to "efficient" steady-state (or slowly varying) operating points. Efficiency is encoded using generalized equations which can model a broad spectrum of useful objectives, such as optimality or equilibria (e.g. Nash, Wardrop, etc.) in noncooperative games. The core idea of the proposed approach is to directly implement iterative solution (or equilibrium seeking) algorithms in closed loop with physical systems. Sufficient conditions for closed-loop stability and robustness are derived; these also serve as the first closed-loop stability results for sampled-data feedback-based optimization. Numerical simulations of smart building automation and game-theoretic robotic swarm coordination support the theoretical results.

math.OC↗

arXiv2021

Discrete-Time Linear-Quadratic Regulation via Optimal Transport

In this paper, we consider a discrete-time stochastic control problem with uncertain initial and target states. We first discuss the connection between optimal transport and stochastic control problems of this form. Next, we formulate a linear-quadratic regulator problem where the initial and terminal states are distributed according to specified probability densities. A closed-form solution for the optimal transport map in the case of linear-time varying systems is derived, along with an algorithm for computing the optimal map. Two numerical examples pertaining to swarm deployment demonstrate the practical applicability of the model, and performance of the numerical method.

math.OC↗

arXiv2021

Safety-Aware Cascade Controller Tuning Using Constrained Bayesian Optimization

This paper presents an automated, model-free, data-driven method for the safe tuning of PID cascade controller gains based on Bayesian optimization. The optimization objective is composed of data-driven performance metrics and modeled using Gaussian processes. We further introduce a data-driven constraint that captures the stability requirements from system data. Numerical evaluation shows that the proposed approach outperforms relay feedback autotuning and quickly converges to the global optimum, thanks to a tailored stopping criterion. We demonstrate the performance of the method in simulations and experiments. For experimental implementation, in addition to the introduced safety constraint, we integrate a method for automatic detection of the critical gains and extend the optimization objective with a penalty depending on the proximity of the current candidate points to the critical gains. The resulting automated tuning method optimizes system performance while ensuring stability and standardization

eess.SY↗

arXiv2021

Distributionally Robust Chance Constrained Data-enabled Predictive Control

We study the problem of finite-time constrained optimal control of unknown stochastic linear time-invariant systems, which is the key ingredient of a predictive control algorithm -- albeit typically having access to a model. We propose a novel distributionally robust data-enabled predictive control (DeePC) algorithm which uses noise-corrupted input/output data to predict future trajectories and compute optimal control inputs while satisfying output chance constraints. The algorithm is based on (i) a non-parametric representation of the subspace spanning the system behaviour, where past trajectories are sorted in Page or Hankel matrices; and (ii) a distributionally robust optimization formulation which gives rise to strong probabilistic performance guarantees. We show that for certain objective functions, DeePC exhibits strong out-of-sample performance, and at the same time respects constraints with high probability. The algorithm provides an end-to-end approach to control design for unknown stochastic linear time-invariant systems. We illustrate the closed-loop performance of the DeePC in an aerial robotics case study.

math.OC↗

arXiv2021

Passivity-based Decentralized Control for Discrete-time Large-scale Systems

Passivity theory has recently contributed to developing decentralized control schemes for large-scale systems. Many decentralized passivity-based control schemes are designed in continuous-time. It is well-known, however, that the passivity properties of continuous-time systems may be lost under discretization. In this work, we present a novel stabilizing decentralized control scheme by ensuring passivity for discrete-time systems directly and thus avoiding the issue of passivity preservation. The controller is synthesized by locally solving a semidefinite program offline for each subsystem in a decentralized fashion. This program comprises local conditions ensuring that the corresponding subsystem is locally passive. Passivity is ensured with respect to a local virtual output which is different from the local actual output. The program also comprises local conditions ensuring that the local passivity of all subsystems implies the asymptotic stability of the whole system. The performance of the proposed controller is evaluated on a case study in DC microgrids.

eess.SY↗

arXiv2021

Decentralized trajectory optimization for multi-agent exploration

Autonomous exploration is an application of growing importance in robotics. A promising strategy is ergodic trajectory planning, whereby an agent spends in each area a fraction of time which is proportional to its probability information density function. In this paper, a decentralized ergodic multi-agent trajectory planning algorithm featuring limited communication constraints is proposed. The agents' trajectories are designed by optimizing a weighted cost encompassing ergodicity, control energy and close-distance operation objectives. To solve the underlying optimal control problem, a second-order descent iterative method coupled with a projection operator in the form of an optimal feedback controller is used. Exhaustive numerical analyses show that the multi-agent solution allows a much more efficient exploration in terms of completion task time and control energy distribution by leveraging collaboration among agents.

math.OC↗

arXiv2021

Decentralized decision making for networks of uncertain systems

Distributed model predictive control (MPC) has been proven a successful method in regulating the operation of large-scale networks of constrained dynamical systems. This paper is concerned with cooperative distributed MPC in which the decision actions of the systems are usually derived by the solution of a system-wide optimization problem. However, formulating and solving such large-scale optimization problems is often a hard task which requires extensive information communication among the individual systems and fails to address privacy concerns in the network. Hence, the main challenge is to design decision policies with a prescribed structure so that the resulting system-wide optimization problem to admit a loosely coupled structure and be amendable to distributed computation algorithms. In this paper, we propose a decentralized problem synthesis scheme which only requires each system to communicate sets which bound its states evolution to neighboring systems. The proposed method alleviates concerns on privacy since this limited communication scheme does not reveal the exact characteristics of the dynamics within each system. In addition, it enables a distributed computation of the solution, making our method highly scalable. We demonstrate in a number of numerical studies, inspired by engineering and finance, the efficacy of the proposed approach which leads to solutions that closely approximate those obtained by the centralized formulation only at a fraction of the computational effort.

math.OC↗

arXiv2021

Decentralized Data-Enabled Predictive Control for Power System Oscillation Damping

We employ a novel data-enabled predictive control (DeePC) algorithm in voltage source converter (VSC) based high-voltage DC (HVDC) stations to perform safe and optimal wide-area control for power system oscillation damping. Conventional optimal wide-area control is model-based. However, in practice detailed and accurate parametric power system models are rarely available. In contrast, the DeePC algorithm uses only input/output data measured from the unknown system to predict the future trajectories and calculate the optimal control policy. We showcase that the DeePC algorithm can effectively attenuate inter-area oscillations even in the presence of measurement noise, communication delays, nonlinear loads and uncertain load fluctuations. We investigate the performance under different matrix structures as data-driven predictors. Furthermore, we derive a novel Min-Max DeePC algorithm to be applied independently in multiple VSC-HVDC stations to mitigate inter-area oscillations, which enables decentralized and robust optimal wide-area control. Further, we discuss how to relieve the computational burden of the Min-Max DeePC by reducing the dimension of prediction uncertainty and how to leverage disturbance feedback to reduce the conservativeness of robustification. We illustrate our results with high-fidelity, nonlinear, and noisy simulations of a four-area test system.

eess.SY↗

arXiv2021

Learning $Q$-function approximations for hybrid control problems

The main challenge in controlling hybrid systems arises from having to consider an exponential number of sequences of future modes to make good long-term decisions. Model predictive control (MPC) computes a control action through a finite-horizon optimisation problem. A key ingredient in this problem is a terminal cost, to account for the system's evolution beyond the chosen horizon. A good terminal cost can reduce the horizon length required for good control action and is often tuned empirically by observing performance. We build on the idea of using $N$-step $Q$-functions $(\mathcal{Q}^{(N)})$ in the MPC objective to avoid having to choose a terminal cost. We present a formulation incorporating the system dynamics and constraints to approximate the optimal $\mathcal{Q}^{(N)}$-function and algorithms to train the approximation parameters through an exploration of the state space. We test the control policy derived from the trained approximations on two benchmark problems through simulations and observe that our algorithms are able to learn good $\mathcal{Q}^{(N)}$-approximations for high dimensional hybrid systems based on a relatively small dataset. Finally, we compare our controller's performance against that of Hybrid MPC in terms of computation time and closed-loop cost.

math.OC↗

arXiv2021

Robust Data-Enabled Predictive Control: Tractable Formulations and Performance Guarantees

We introduce a general framework for robust data-enabled predictive control (DeePC) for linear time-invariant (LTI) systems. The proposed framework enables us to obtain model-free optimal control for LTI systems based on noisy input/output data. More specifically, robust DeePC solves a min-max optimization problem to compute the optimal control sequence that is resilient to all possible realizations of the uncertainties in the input/output data within a prescribed uncertainty set. We present computationally tractable reformulations of the min-max problem with various uncertainty sets. Furthermore, we show that even though an accurate prediction of the future behavior is unattainable in practice due to inaccessibility of the perfect input/output data, the obtained robust optimal control sequence provides performance guarantees for the actually realized input/output cost. We further show that the robust DeePC generalizes and robustifies the regularized DeePC (with quadratic regularization or 1-norm regularization) proposed in the literature. Finally, we demonstrate the performance of the proposed robust DeePC algorithm on high-fidelity, nonlinear, and noisy simulations of a grid-connected power converter system.

eess.SY↗

arXiv2021

Quadratic Regularization of Data-Enabled Predictive Control: Theory and Application to Power Converter Experiments

Data-driven control that circumvents the process of system identification by providing optimal control inputs directly from system data has attracted renewed attention in recent years. In this paper, we focus on understanding the effects of the regularization on the data-enabled predictive control (DeePC) algorithm. We provide theoretical motivation and interpretation for including a quadratic regularization term. Our analysis shows that the quadratic regularization term leads to robust and optimal solutions with regards to disturbances affecting the data. Moreover, when the input/output constraints are inactive, the quadratic regularization leads to a closed-form solution of the DeePC algorithm and thus enables fast calculations. On this basis, we propose a framework for data-driven synchronization and power regulations of power converters, which is tested by high-fidelity simulations and experiments.

eess.SY↗

arXiv2021

Learning from Simulation, Racing in Reality

We present a reinforcement learning-based solution to autonomously race on a miniature race car platform. We show that a policy that is trained purely in simulation using a relatively simple vehicle model, including model randomization, can be successfully transferred to the real robotic setup. We achieve this by using novel policy output regularization approach and a lifted action space which enables smooth actions but still aggressive race car driving. We show that this regularized policy does outperform the Soft Actor Critic (SAC) baseline method, both in simulation and on the real car, but it is still outperformed by a Model Predictive Controller (MPC) state of the art method. The refinement of the policy with three hours of real-world interaction data allows the reinforcement learning policy to achieve lap times similar to the MPC controller while reducing track constraint violations by 50%.

cs.RO↗

arXiv2021

Safe and Efficient Model-free Adaptive Control via Bayesian Optimization

Adaptive control approaches yield high-performance controllers when a precise system model or suitable parametrizations of the controller are available. Existing data-driven approaches for adaptive control mostly augment standard model-based methods with additional information about uncertainties in the dynamics or about disturbances. In this work, we propose a purely data-driven, model-free approach for adaptive control. Tuning low-level controllers based solely on system data raises concerns on the underlying algorithm safety and computational performance. Thus, our approach builds on GoOSE, an algorithm for safe and sample-efficient Bayesian optimization. We introduce several computational and algorithmic modifications in GoOSE that enable its practical use on a rotational motion system. We numerically demonstrate for several types of disturbances that our approach is sample efficient, outperforms constrained Bayesian optimization in terms of safety, and achieves the performance optima computed by grid evaluation. We further demonstrate the proposed adaptive control approach experimentally on a rotational motion system.

eess.SY↗

arXiv2020

On the Asymptotic Behavior of the Douglas-Rachford and Proximal-Point Algorithms for Convex Optimization

The authors in (Banjac et al., 2019) recently showed that the Douglas-Rachford algorithm provides certificates of infeasibility for a class of convex optimization problems. In particular, they showed that the difference between consecutive iterates generated by the algorithm converges to certificates of primal and dual strong infeasibility. Their result was shown in a finite dimensional Euclidean setting and for a particular structure of the constraint set. In this paper, we extend the result to real Hilbert spaces and a general nonempty closed convex set. Moreover, we show that the proximal-point algorithm applied to the set of optimality conditions of the problem generates similar infeasibility certificates.

math.OC↗

arXiv2020

Constrained Optimal Tracking Control of Unknown Systems: A Multi-Step Linear Programming Approach

We study the problem of optimal state-feedback tracking control for unknown discrete-time deterministic systems with input constraints. To handle input constraints, state-of-art methods utilize a certain nonquadratic stage cost function, which is sometimes limiting real systems. Furthermore, it is well known that Policy Iteration (PI) and Value Iteration (VI), two widely used algorithms in data-driven control, offer complementary strengths and weaknesses. In this work, a two-step transformation is employed, which converts the constrained-input optimal tracking problem to an unconstrained augmented optimal regulation problem, and allows the consideration of general stage cost functions. Then, a novel multi-step VI algorithm based on Q-learning and linear programming is derived. The proposed algorithm improves the convergence speed of VI, avoids the requirement for an initial stabilizing control policy of PI, and computes a constrained optimal feedback controller without the knowledge of a system model and stage cost function. Simulation studies demonstrate the reliability and performance of the proposed approach.

eess.SY↗

arXiv2020

Improving Tractability of Real-Time Control Schemes via Simplified $\mathcal{S}$-Lemma

Various control schemes rely on a solution of a convex optimization problem involving a particular robust quadratic constraint, which can be reformulated as a linear matrix inequality using the well-known $\mathcal{S}$-lemma. However, the computational effort required to solve the resulting semidefinite program may be prohibitively large for real-time applications requiring a repeated solution of such a problem. We use some recent advances in robust optimization that allow us to reformulate such a robust constraint as a set of linear and second-order cone constraints, which are computationally better suited to real-time applications. A numerical example demonstrates a huge speedup that can be obtained using the proposed reformulation.

math.OC↗

arXiv2020

Data-driven optimal control with a relaxed linear program

The linear programming (LP) approach has a long history in the theory of approximate dynamic programming. When it comes to computation, however, the LP approach often suffers from poor scalability. In this work, we introduce a relaxed version of the Bellman operator for q-functions and prove that it is still a monotone contraction mapping with a unique fixed point. In the spirit of the LP approach, we exploit the new operator to build a relaxed linear program (RLP). Compared to the standard LP formulation, our RLP has only one family of constraints and half the decision variables, making it more scalable and computationally efficient. For deterministic systems, the RLP trivially returns the correct q-function. For stochastic linear systems in continuous spaces, the solution to the RLP preserves the minimizer of the optimal q-function, hence retrieves the optimal policy. Theoretical results are backed up in simulation where we solve sampled versions of the LPs with data collected by interacting with the environment. For general nonlinear systems, we observe that the RLP again tends to preserve the minimizers of the solution to the LP, though the relative performance is influenced by the specific geometry of the problem.

eess.SY↗