Searcharxiv⌕ Search

arXiv subjects

John Lygeros

Publications and source records attributed to John Lygeros.

At least 253 records · Page 14Linked to original sources

A scenario approach for non-convex control design

Randomized optimization is an established tool for control design with modulated robustness. While for uncertain convex programs there exist randomized approaches with efficient sampling, this is not the case for non-convex problems. Approaches based on statistical learning theory are applicable to non-convex problems, but they usually are conservative in terms of performance and require high sample complexity to achieve the desired probabilistic guarantees. In this paper, we derive a novel scenario approach for a wide class of random non-convex programs, with a sample complexity similar to that of uncertain convex programs and with probabilistic guarantees that hold not only for the optimal solution of the scenario program, but for all feasible solutions inside a set of a-priori chosen complexity. We also address measure-theoretic issues for uncertain convex and non-convex programs. Among the family of non-convex control- design problems that can be addressed via randomization, we apply our scenario approach to randomized Model Predictive Control for chance-constrained nonlinear control-affine systems.

eess.SY↗

Performance Bounds for the Scenario Approach and an Extension to a Class of Non-convex Programs

We consider the Scenario Convex Program (SCP) for two classes of optimization problems that are not tractable in general: Robust Convex Programs (RCPs) and Chance-Constrained Programs (CCPs). We establish a probabilistic bridge from the optimal value of SCP to the optimal values of RCP and CCP in which the uncertainty takes values in a general, possibly infinite dimensional, metric space. We then extend our results to a certain class of non-convex problems that includes, for example, binary decision variables. In the process, we also settle a measurability issue for a general class of scenario programs, which to date has been addressed by an assumption. Finally, we demonstrate the applicability of our results on a benchmark problem and a problem in fault detection and isolation.

math.OC↗

Stability and performance of stochastic predictive control

This article is concerned with stability and performance of controlled stochastic processes under receding horizon policies. We carry out a systematic study of methods to guarantee stability under receding horizon policies via appropriate selections of cost functions in the underlying finite-horizon optimal control problem. We also obtain quantitative bounds on the performance of the system under receding horizon policies as measured by the long-run expected average cost. The results are illustrated with the help of several simple examples.

eess.SY↗

Designing Experiments to Understand the Variability in Biochemical Reaction Networks

Exploiting the information provided by the molecular noise of a biological process has proven to be valuable in extracting knowledge about the underlying kinetic parameters and sources of variability from single cell measurements. However, quantifying this additional information a priori, to decide whether a single cell experiment might be beneficial, is currently only possibly in very simple systems where either the chemical master equation is computationally tractable or a Gaussian approximation is appropriate. Here we show how the information provided by distribution measurements can be approximated from the first four moments of the underlying process. The derived formulas are generally valid for any stochastic kinetic model including models that comprise both intrinsic and extrinsic noise. This allows us to propose an optimal experimental design framework for heterogeneous cell populations which we employ to compare the utility of dual reporter and perturbation experiments for separating extrinsic and intrinsic noise in a simple model of gene expression. Subsequently, we compare the information content of different experiments which have been performed in an engineered light-switch gene expression system in yeast and show that well chosen gene induction patterns may allow one to identify features of the system which remain hidden in unplanned experiments.

q-bio.QM↗

Symbolic control of stochastic systems via approximately bisimilar finite abstractions

Symbolic approaches to the control design over complex systems employ the construction of finite-state models that are related to the original control systems, then use techniques from finite-state synthesis to compute controllers satisfying specifications given in a temporal logic, and finally translate the synthesized schemes back as controllers for the concrete complex systems. Such approaches have been successfully developed and implemented for the synthesis of controllers over non-probabilistic control systems. In this paper, we extend the technique to probabilistic control systems modeled by controlled stochastic differential equations. We show that for every stochastic control system satisfying a probabilistic variant of incremental input-to-state stability, and for every given precision $\varepsilon>0$, a finite-state transition system can be constructed, which is $\varepsilon$-approximately bisimilar (in the sense of moments) to the original stochastic control system. Moreover, we provide results relating stochastic control systems to their corresponding finite-state transition systems in terms of probabilistic bisimulation relations known in the literature. We demonstrate the effectiveness of the construction by synthesizing controllers for stochastic control systems over rich specifications expressed in linear temporal logic. The discussed technique enables a new, automated, correct-by-construction controller synthesis approach for stochastic control systems, which are common mathematical models employed in many safety critical systems subject to structured uncertainty and are thus relevant for cyber-physical applications.

math.OC↗

Approximate Dynamic Programming via Sum of Squares Programming

We describe an approximate dynamic programming method for stochastic control problems on infinite state and input spaces. The optimal value function is approximated by a linear combination of basis functions with coefficients as decision variables. By relaxing the Bellman equation to an inequality, one obtains a linear program in the basis coefficients with an infinite set of constraints. We show that a recently introduced method, which obtains convex quadratic value function approximations, can be extended to higher order polynomial approximations via sum of squares programming techniques. An approximate value function can then be computed offline by solving a semidefinite program, without having to sample the infinite constraint. The policy is evaluated online by solving a polynomial optimization problem, which also turns out to be convex in some cases. We experimentally validate the method on an autonomous helicopter testbed using a 10-dimensional helicopter model.

math.OC↗

Distributed Model Predictive Consensus via the Alternating Direction Method of Multipliers

We propose a distributed optimization method for solving a distributed model predictive consensus problem. The goal is to design a distributed controller for a network of dynamical systems to optimize a coupled objective function while respecting state and input constraints. The distributed optimization method is an augmented Lagrangian method called the Alternating Direction Method of Multipliers (ADMM), which was introduced in the 1970s but has seen a recent resurgence in the context of dramatic increases in computing power and the development of widely available distributed computing platforms. The method is applied to position and velocity consensus in a network of double integrators. We find that a few tens of ADMM iterations yield closed-loop performance near what is achieved by solving the optimization problem centrally. Furthermore, the use of recent code generation techniques for solving local subproblems yields fast overall computation times.

math.OC↗

Isospectral flows on a class of finite-dimensional Jacobi matrices

We present a new matrix-valued isospectral ordinary differential equation that asymptotically block-diagonalizes $n\times n$ zero-diagonal Jacobi matrices employed as its initial condition. This o.d.e.\ features a right-hand side with a nested commutator of matrices, and structurally resembles the double-bracket o.d.e.\ studied by R.W.\ Brockett in 1991. We prove that its solutions converge asymptotically, that the limit is block-diagonal, and above all, that the limit matrix is defined uniquely as follows: For $n$ even, a block-diagonal matrix containing $2\times 2$ blocks, such that the super-diagonal entries are sorted by strictly increasing absolute value. Furthermore, the off-diagonal entries in these $2\times 2$ blocks have the same sign as the respective entries in the matrix employed as initial condition. For $n$ odd, there is one additional $1\times 1$ block containing a zero that is the top left entry of the limit matrix. The results presented here extend some early work of Kac and van Moerbeke.

math.DS↗

Uniform moment bounds of multi-dimensional functions of discrete-time stochastic processes

We establish conditions for uniform $r$-th moment bound of certain $\R^d$-valued functions of a discrete-time stochastic process taking values in a general metric space. The conditions include an appropriate negative drift together with a uniform $L_p$ bound on the jumps of the process for $p > r + 1$. Applications of the result are given in connection to iterated function systems and biochemical reaction networks.

math.PR↗

Stochastic receding horizon control with output feedback and bounded control inputs

We provide a solution to the problem of receding horizon control for stochastic discrete-time systems with bounded control inputs and imperfect state measurements. For a suitable choice of control policies, we show that the finite-horizon optimization problem to be solved on-line is convex and successively feasible. Due to the inherent nonlinearity of the feedback loop, a slight extension of the Kalman filter is exploited to estimate the state optimally in mean-square sense. We show that the receding horizon implementation of the resulting control policies renders the state of the overall system mean-square bounded under mild assumptions. Finally, we discuss how some of the quantities required by the finite-horizon optimization problem can be computed off-line, reducing the on-line computation, and present some numerical examples.

math.OC↗

Mean-square boundedness of stochastic networked control systems with bounded control inputs

We consider the problem of controlling marginally stable linear systems using bounded control inputs for networked control settings in which the communication channel between the remote controller and the system is unreliable. We assume that the states are perfectly observed, but the control inputs are transmitted over a noisy communication channel. Under mild hypotheses on the noise introduced by the control communication channel and large enough control authority, we construct a control policy that renders the state of the closed-loop system mean-square bounded.

math.OC↗

Attaining mean square boundedness of a marginally stable noisy linear system with a bounded control input

We construct control policies that ensure bounded variance of a noisy marginally stable linear system in closed-loop. It is assumed that the noise sequence is a mutually independent sequence of random vectors, enters the dynamics affinely, and has bounded fourth moment. The magnitude of the control is required to be of the order of the first moment of the noise, and the policies we obtain are simple and computable.

math.OC↗

Maximizing the probability of attaining a target prior to extinction

We present a dynamic programming-based solution to the problem of maximizing the probability of attaining a target set before hitting a cemetery set for a discrete-time Markov control process. Under mild hypotheses we establish that there exists a deterministic stationary policy that achieves the maximum value of this probability. We demonstrate how the maximization of this probability can be computed through the maximization of an expected total reward until the first hitting time to either the target or the cemetery set. Martingale characterizations of thrifty, equalizing, and optimal policies in the context of our problem are also established.

math.OC↗

Hamilton-Jacobi formulation for reach-avoid differential games

A new framework for formulating reachability problems with competing inputs, nonlinear dynamics and state constraints as optimal control problems is developed. Such reach-avoid problems arise in, among others, the study of safety problems in hybrid systems. Earlier approaches to reach-avoid computations are either restricted to linear systems, or face numerical difficulties due to possible discontinuities in the Hamiltonian of the optimal control problem. The main advantage of the approach proposed in this paper is that it can be applied to a general class of target hitting continuous dynamic games with nonlinear dynamics, and has very good properties in terms of its numerical solution, since the value function and the Hamiltonian of the system are both continuous. The performance of the proposed method is demonstrated by applying it to a two aircraft collision avoidance scenario under target window constraints and in the presence of wind disturbance. Target Windows are a novel concept in air traffic management, and represent spatial and temporal constraints, that the aircraft have to respect to meet their schedule.

math.OC↗

On the connections between PCTL and Dynamic Programming

Probabilistic Computation Tree Logic (PCTL) is a well-known modal logic which has become a standard for expressing temporal properties of finite-state Markov chains in the context of automated model checking. In this paper, we give a definition of PCTL for noncountable-space Markov chains, and we show that there is a substantial affinity between certain of its operators and problems of Dynamic Programming. After proving some uniqueness properties of the solutions to the latter, we conclude the paper with two examples to show that some recovery strategies in practical applications, which are naturally stated as reach-avoid problems, can be actually viewed as particular cases of PCTL formulas.

math.OC↗

Stochastic model predictive control with bounded control inputs: a vector space approach

We design receding horizon control strategies for stochastic discrete-time linear systems with additive (possibly) unbounded disturbances, while obeying hard bounds on the control inputs. We pose the problem of selecting an appropriate optimal controller on vector spaces of functions and show that the resulting optimization problem has a tractable convex solution. Under the assumption that the zero-input and zero-noise system is asymptotically stable, we show that the variance of the state is bounded when enforcing hard bounds on the control inputs, for any receding horizon implementation. Throughout the article we provide several examples that illustrate how quantities needed in the formulation of the resulting optimization problems can be calculated off-line, as well as comparative examples that illustrate the effectiveness of our control strategies.

math.OC↗

Stochastic control up to a hitting time: optimality and rolling-horizon implementation

We present a dynamic programming-based solution to a stochastic optimal control problem up to a hitting time for a discrete-time Markov control process. Firstly, we determine an optimal control policy to steer the process toward a compact target set while simultaneously minimizing an expected discounted cost. We then provide a rolling-horizon strategy for approximating the optimal policy, together with quantitative characterization of its sub-optimality with respect to the optimal policy. Finally, we address related issues of asymptotic discount-optimality of the value-iteration policy. Both the state and action spaces are assumed to be Polish.

math.OC↗