arXiv subjects

John Lygeros

Publications and source records attributed to John Lygeros.

At least 199 records · Page 11Linked to original sources

arXiv2019

Data-Enabled Predictive Control for Grid-Connected Power Converters

We apply a novel data-enabled predictive control (DeePC) algorithm in grid-connected power converters to perform safe and optimal control. Rather than a model, the DeePC algorithm solely needs input/output data measured from the unknown system to predict future trajectories. We show that the DeePC can eliminate undesired oscillations in a grid-connected power converter and stabilize an unstable system. However, the DeePC algorithm may suffer from poor scalability when applied in high-order systems. To this end, we present a finite-horizon output-based model predictive control (MPC) for grid-connected power converters, which uses an N-step auto-regressive-moving-average (ARMA) model for system representation. The ARMA model is identified via an N-step prediction error method (PEM) in a recursive way. We investigate the connection between the DeePC and the concatenated PEM-MPC method, and then analytically and numerically compare their closed-loop performance. Moreover, the PEM-MPC is applied in a voltage source converter based HVDC station which is connected to a two-area power system so as to eliminate low-frequency oscillations. All of our results are illustrated with high-fidelity, nonlinear, and noisy simulations.

eess.SY↗

arXiv2019

Data-Enabled Predictive Control: In the Shallows of the DeePC

We consider the problem of optimal trajectory tracking for unknown systems. A novel data-enabled predictive control (DeePC) algorithm is presented that computes optimal and safe control policies using real-time feedback driving the unknown system along a desired trajectory while satisfying system constraints. Using a finite number of data samples from the unknown system, our proposed algorithm uses a behavioural systems theory approach to learn a non-parametric system model used to predict future trajectories. The DeePC algorithm is shown to be equivalent to the classical and widely adopted Model Predictive Control (MPC) algorithm in the case of deterministic linear time-invariant systems. In the case of nonlinear stochastic systems, we propose regularizations to the DeePC algorithm. Simulations are provided to illustrate performance and compare the algorithm with other methods.

math.OC↗

arXiv2019

Accelerated Point-wise Maximum Approach to Approximate Dynamic Programming

We describe an approximate dynamic programming approach to compute lower bounds on the optimal value function for a discrete time, continuous space, infinite horizon setting. The approach iteratively constructs a family of lower bounding approximate value functions by using the so-called Bellman inequality. The novelty of our approach is that, at each iteration, we aim to compute an approximate value function that maximizes the point-wise maximum taken with the family of approximate value functions computed thus far. This leads to a non-convex objective, and we propose a gradient ascent algorithm to find stationary points by solving a sequence of convex optimization problems. We provide convergence guarantees for our algorithm and an interpretation for how the gradient computation relates to the state relevance weighting parameter appearing in related approximate dynamic programming approaches. We demonstrate through numerical examples that, when compared to existing approaches, the algorithm we propose computes tighter sub-optimality bounds with less computation time.

eess.SY↗

arXiv2019

Decentralized Resource Allocation via Dual Consensus ADMM

We consider a resource allocation problem over an undirected network of agents, where edges of the network define communication links. The goal is to minimize the sum of agent-specific convex objective functions, while the agents' decisions are coupled via a convex conic constraint. We derive two methods by applying the alternating direction method of multipliers (ADMM) for decentralized consensus optimization to the dual of our resource allocation problem. Both methods are fully parallelizable and decentralized in the sense that each agent exchanges information only with its neighbors in the network and requires only its own data for updating its decision. We prove convergence of the proposed methods and demonstrate their effectiveness with a numerical example.

math.OC↗

arXiv2019

Nonlinear Control of Quadcopters via Approximate Dynamic Programming

While Approximate Dynamic Programming has successfully been used in many applications involving discrete states and inputs such as playing the games of Tetris or chess, it has not been used in many continuous state and input space applications. In this paper, we combine Approximate Dynamic Programming techniques and apply them to the continuous, non-linear and high dimensional dynamics of a quadcopter vehicle. We use a polynomial approximation of the dynamics and sum-of-squares programming techniques to compute a family of polynomial value function approximations for different tuning parameters. The resulting approximations to the optimal value function are combined in a point-wise maximum approach, which is used to compute the online policy. The success of the method is demonstrated in both simulations and experiments on a quadcopter. The control performance is compared to a linear time-varying Model Predictive Controller. The two methods are then combined to keep the computational benefits of a short horizon MPC and the long term performance benefits of the Approximate Dynamic Programming value function as the terminal cost.

eess.SY↗

arXiv2019

A Non-Cooperative Game Approach to Autonomous Racing

We consider autonomous racing of two cars and present an approach to formulate racing decisions as a non-cooperative non-zero-sum game. We design three different games where the players aim to fulfill static track constraints as well as avoid collision with each other; the latter constraint depends on the combined actions of the two players. The difference between the games are the collision constraints and the payoff. In the first game collision avoidance is only considered by the follower, and each player maximizes their own progress towards the finish line. We show that, thanks to the sequential structure of this game, equilibria can be computed through an efficient sequential maximization approach. Further, we show these actions, if feasible, are also a Stackelberg and Nash equilibrium in pure strategies of our second game where both players consider the collision constraints. The payoff of our third game is designed to promote blocking, by additionally rewarding the cars for staying ahead at the end of the horizon. We show that this changes the Stackelberg equilibrium, but has a minor influence on the Nash equilibria. For online implementation, we propose to play the games in a moving horizon fashion, and discuss two methods for guaranteeing feasibility of the resulting coupled repeated games. Finally, we study the performance of the proposed approaches in simulation for a set-up that replicates the miniature race car tested at the Automatic Control Laboratory of ETH Zurich. The simulation study shows that the presented games can successfully model different racing behaviors and generate interesting racing situations.

math.OC↗

arXiv2018

Inference of the three-dimensional chromatin structure and its temporal behavior

Understanding the three-dimensional (3D) structure of the genome is essential for elucidating vital biological processes and their links to human disease. To determine how the genome folds within the nucleus, chromosome conformation capture methods such as HiC have recently been employed. However, computational methods that exploit the resulting high-throughput, high-resolution data are still suffering from important limitations. In this work, we explore the idea of manifold learning for the 3D chromatin structure inference and present a novel method, REcurrent Autoencoders for CHromatin 3D structure prediction (REACH-3D). Our framework employs autoencoders with recurrent neural units to reconstruct the chromatin structure. In comparison to existing methods, REACH-3D makes no transfer function assumption and permits dynamic analysis. Evaluating REACH-3D on synthetic data indicated high agreement with the ground truth. When tested on real experimental HiC data, REACH-3D recovered most faithfully the expected biological properties and obtained the highest correlation coefficient with microscopy measurements. Last, REACH-3D was applied to dynamic HiC data, where it successfully modeled chromatin conformation during the cell cycle.

q-bio.GN↗

arXiv2018

Strengthening the Group: Aggregated Frequency Reserve Bidding with ADMM

In a power grid, the electricity supply and demand must be balanced at all times to maintain the system's frequency. In practice, the grid operator achieves this balance by procuring frequency reserves in an ahead-of-time market setting. During runtime, these reserves are then dispatched whenever there is an imbalance in the grid. Recently, there has been an increasing interest in engaging electricity consumers, such as plug-in electric vehicles or buildings, to offer such frequency reserves by exploiting their flexibility in power consumption. In this work, we focus on an aggregation of buildings that places a joint bid on a reserve market. The resulting shared decision is modeled as a large-scale optimization problem. Our main contribution is to show that the aggregation can make its decision in a computationally efficient and conceptually meaningful way, using the alternating direction method of multipliers (ADMM). The proposed approach exhibits several attractive features that include (i) the computational burden is distributed between the buildings; (ii) the setup naturally provides privacy and flexibility; (iii) the iterative algorithm can be stopped at any time, providing a feasible (though suboptimal) solution; and (iv) the algorithm provides the foundation for a reward distribution scheme that strengthens the group.

math.OC↗

arXiv2018

Data-Driven Chance Constrained Optimization under Wasserstein Ambiguity Sets

We present a data-driven approach for distributionally robust chance constrained optimization problems (DRCCPs). We consider the case where the decision maker has access to a finite number of samples or realizations of the uncertainty. The chance constraint is then required to hold for all distributions that are close to the empirical distribution constructed from the samples (where the distance between two distributions is defined via the Wasserstein metric). We first reformulate DRCCPs under data-driven Wasserstein ambiguity sets and a general class of constraint functions. When the feasibility set of the chance constraint program is replaced by its convex inner approximation, we present a convex reformulation of the program and show its tractability when the constraint function is affine in both the decision variable and the uncertainty. For constraint functions concave in the uncertainty, we show that a cutting-surface algorithm converges to an approximate solution of the convex inner approximation of DRCCPs. Finally, for constraint functions convex in the uncertainty, we compare the feasibility set with other sample-based approaches for chance constrained programs.

math.OC↗

arXiv2018

Generalized Dual Dynamic Programming for Infinite Horizon Problems in Continuous State and Action Spaces

We describe a nonlinear generalization of dual dynamic programming theory and its application to value function estimation for deterministic control problems over continuous state and action spaces, in a discrete-time infinite horizon setting. We prove, using a Benders-type argument leveraging the monotonicity of the Bellman operator, that the result of a one-stage policy evaluation can be used to produce nonlinear lower bounds on the optimal value function that are valid over the entire state space. These bounds contain terms reflecting the functional form of the system's costs, dynamics, and constraints. We provide an iterative algorithm that produces successively better approximations of the optimal value function, and prove under certain assumptions that it achieves an arbitrarily low desired Bellman optimality tolerance at pre-selected points in the state space, in a finite number of iterations. We also describe means of certifying the quality of the value function generated. We demonstrate the efficacy of the approach on systems whose dimensions are too large for conventional dynamic programming approaches to be practical.

math.OC↗

arXiv2018

Performance guarantees for model-based Approximate Dynamic Programming in continuous spaces

We study both the value function and Q-function formulation of the Linear Programming approach to Approximate Dynamic Programming. The approach is model-based and optimizes over a restricted function space to approximate the value function or Q-function. Working in the discrete time, continuous space setting, we provide guarantees for the fitting error and online performance of the policy. In particular, the online performance guarantee is obtained by analyzing an iterated version of the greedy policy, and the fitting error guarantee by analyzing an iterated version of the Bellman inequality. These guarantees complement the existing bounds that appear in the literature. The Q-function formulation offers benefits, for example, in decentralized controller design, however it can lead to computationally demanding optimization problems. To alleviate this drawback, we provide a condition that simplifies the formulation, resulting in improved computational times.

eess.SY↗

arXiv2018

A Two-Stage Polynomial Approach to Stochastic Optimization of District Heating Networks

In this paper, we use stochastic polynomial optimization to derive high-performance operating strategies for heating networks with uncertain or variable demand. The heat flow in district heating networks can be regulated by varying the supply temperature, the mass flow rate, or both simultaneously, leading to different operating strategies. The task of choosing the set-points within each strategy that minimize the network losses for a range of demand conditions can be cast as a two-stage stochastic optimization problem with polynomial objective and polynomial constraints. We derive a generalized moment problem (GMP) equivalent to such a two-stage stochastic optimization problem, and describe a hierarchy of moment relaxations approximating the optimal solution of the GMP. Under various network design parameters, we use the method to compute (approximately) optimal strategies when one or both of the mass flow rate and supply temperature for a benchmark heat network. We report that the performance of an optimally-parameterized fixed-temperature variable-mass-flow strategy can approach that of a fully variable strategy.

math.OC↗

arXiv2018

A Moment and Sum-of-Squares Extension of Dual Dynamic Programming with Application to Nonlinear Energy Storage Problems

We present a finite-horizon optimization algorithm that extends the established concept of Dual Dynamic Programming (DDP) in two ways. First, in contrast to the linear costs, dynamics, and constraints of standard DDP, we consider problems in which all of these can be polynomial functions. Second, we allow the state trajectory to be described by probability distributions rather than point values, and return approximate value functions fitted to these. The algorithm is in part an adaptation of sum-of-squares techniques used in the approximate dynamic programming literature. It alternates between a forward simulation through the horizon, in which the moments of the state distribution are propagated through a succession of single-stage problems, and a backward recursion, in which a new polynomial function is derived for each stage using the moments of the state as fixed data. The value function approximation returned for a given stage is the point-wise maximum of all polynomials derived for that stage. This contrasts with the piecewise affine functions derived in conventional DDP. We prove key convergence properties of the new algorithm, and validate it in simulation on two case studies related to the optimal operation of energy storage devices with nonlinear characteristics. The first is a small borehole storage problem, for which multiple value function approximations can be compared. The second is a larger problem, for which conventional discretized dynamic programming is intractable.

math.OC↗

arXiv2018

Unlocking the Potential of Flexible Energy Resources to Help Balance the Power Grid

Flexible energy resources can help balance the power grid by providing different types of ancillary services. However, the balancing potential of most types of resources is restricted by physical constraints such as the size of their energy buffer, limits on power-ramp rates, or control delays. Using the example of Secondary Frequency Regulation, this paper shows how the flexibility of various resources can be exploited more efficiently by considering multiple resources with complementary physical properties and controlling them in a coordinated way. To this end, optimal adjustable control policies are computed based on robust optimization. Our problem formulation takes into account power ramp-rate constraints explicitly, and accurately models the different timescales and lead times of the energy and reserve markets. Simulations demonstrate that aggregations of select resources can offer significantly more regulation capacity than the resources could provide individually.

eess.SY↗

arXiv2018

An Exact Convex Relaxation of the Freeway Network Control Problem with Controlled Merging Junctions

We consider the freeway network control problem where the aim is to optimize the operation of traffic networks modeled by the Cell Transmission Model via ramp metering and partial mainline demand control. Optimal control problems using the Cell Transmission Model are usually non-convex, due to the nonlinear fundamental diagram, but a convex relaxation in which demand and supply constraints are relaxed is often used. Previous works have established conditions under which solutions of the relaxation can be made feasible with respect to the original constraints. In this work, we generalize these conditions and show that the control of flows into merging junctions is sufficient to do so if the objective is to minimize the total time spent in traffic. We derive this result by introducing an alternative system representation. In the new representation, the system dynamics are concave and state-monotone. We show that exactness of the convex relaxation of finite horizon optimal control problems follows from these properties. Deriving the main result via a characterization of the system dynamics allows one to treat arbitrary monotone, concave fundamental diagrams and several types of control for merging junctions in a uniform manner. The derivation also suggests a straightforward method to verify if the results continue to hold for extensions or modifications of the models studied in this work.

math.OC↗

arXiv2018

On monotonicity of FIFO-diverging junctions

This technical note concerns the dynamics of FIFO-diverging junctions in compartmental models for traffic networks. Many strong results on the dynamical behavior of such traffic networks rely on monotonicity of the underlying dynamics. In road traffic modeling, a common model for diverging junctions is based on the First-in, first-out principle. These type of junctions pose a problem in the analysis of traffic dynamics, since their dynamics are not monotone with respect to the positive orthant. However, this technical note demonstrates that they are in fact monotone with respect to the partial order induced by a particular, polyhedral cone.

math.DS↗

arXiv2018

Optimal Linearizations of Power Systems with Uncertain Supply and Demand

Linearized models of power systems are often desirable to formulate tractable control and optimization problems that still reflect real-world physics adequately under various operating conditions. In this paper, we propose an approach that can make use of known data concerning the distribution of demand, and/or intermittent supply, to minimize expected model inconsistency with respect to the original non-linear model. The optimal linearization is obtained by approximating a generalized moment problem with a hierarchy of sparse semi-definite relaxations. The output is a linearization point that minimizes the expected absolute constraint violation with respect to the uncertain supply and demand. Numerical results for different power systems networks demonstrate the accuracy and scalability of our linearization method.

math.OC↗

arXiv2018

On the Efficiency of Nash Equilibria in Aggregative Charging Games

Several works have recently suggested to model the problem of coordinating the charging needs of a fleet of electric vehicles as a game, and have proposed distributed algorithms to coordinate the vehicles towards a Nash equilibrium of such game. However, Nash equilibria have been shown to posses desirable system-level properties only in simplified cases. In this work, we use the concept of price of anarchy to analyze the inefficiency of Nash equilibria when compared to the social optimum solution. More precisely, we show that i) for linear price functions depending on all the charging instants, the price of anarchy converges to one as the population of vehicles grows; ii) for price functions that depend only on the instantaneous demand, the price of anarchy converges to one if the price function takes the form of a positive pure monomial; iii) for general classes of price functions, the asymptotic price of anarchy can be bounded. For finite populations, we additionaly provide a bound on the price of anarchy as a function of the number vehicles in the system. We support the theoretical findings by means of numerical simulations.

eess.SY↗