Searcharxiv⌕ Search

arXiv subjects

Peter E. Caines

Publications and source records attributed to Peter E. Caines.

32 records · Page 2Linked to original sources
arXiv2019

Graphon Control of Large-scale Networks of Linear Systems

To achieve control objectives for extremely large-scale complex networks using standard methods is essentially intractable. In this work a theory of the approximate control of complex network systems is proposed and developed by the use of graphon theory and the theory of infinite dimensional systems. First, graphon dynamical system models are formulated in an appropriate infinite dimensional space in order to represent arbitrary-size networks of linear dynamical systems, and to define the convergence of sequences of network systems with limits in the space. Exact controllability and approximate controllability of graphon dynamical systems are then investigated. Second, the minimum energy state-to-state control problem and the linear quadratic regulator problem for systems on complex networks are considered. The control problem for graphon limit systems is solved in each case and approximations are defined which yield control laws for the original control problems. Furthermore, convergence properties of the approximation schemes are established. A systematic control design methodology is developed within this framework. Finally, numerical examples of networks with randomly sampled weightings are presented to illustrate the effectiveness of the graphon control methodology.

math.OC↗
arXiv2019

Optimal and Approximate Solutions to Linear Quadratic Regulation of a Class of Graphon Dynamical Systems

In this paper we study the linear quadratic regulation (LQR) problem for dynamical systems coupled over large-scale networks and obtain locally computable low-complexity solutions. The underlying large or even infinite networks are represented by graphons and the couplings appear in both the dynamics and the quadratic cost. The optimal solution is obtained first for graphon dynamical systems for the special case where the graphons are exactly characterized by finite spectral summands. The complexity of generating these control solutions involves solving d+1 scalar Riccati equations where d is the number of non-zero eigenvalues in the spectral representation. Based on this, we provide a suboptimal low-complexity solution for problems with general graphon couplings via spectral approximations and analyze the performance under the approximate control. Finally, a numerical example is given to illustrate the explicit solution and demonstrate the simplicity of the solution.

math.OC↗
arXiv2019

Spectral Representations of Graphons in Very Large Network Systems Control

Graphon-based control has recently been proposed and developed to solve control problems for dynamical systems on networks which are very large or growing without bound (see Gao and Caines, CDC 2017, CDC 2018). In this paper, spectral representations, eigenfunctions and approximations of graphons, and their applications to graphon-based control are studied. First, spectral properties of graphons are presented and then approximations based on Fourier approximated eigenfunctions are analyzed. Within this framework, two classes of graphons with simple spectral representations are given. Applications to graphon-based control analysis are next presented; in particular, the controllability of systems distributed over very large networks is expressed in terms of the properties of the corresponding graphon dynamical systems. Moreover, spectral analysis based upon real-world network data is presented, which demonstrates that low-dimensional spectral approximations of networks are possible. Finally, an initial, exploratory investigation of the utility of the spectral analysis methodology in graphon systems control to study the control of epidemic spread is presented.

eess.SY↗
arXiv2019

Mean Field Game Systems with Common Noise and Markovian Latent Processes

In many stochastic games stemming from financial models, the environment evolves with latent factors and there may be common noise across agents' states. Two classic examples are: (i) multi-agent trading on electronic exchanges, and (ii) systemic risk induced through inter-bank lending/borrowing. Moreover, agents' actions often affect the environment, and some agent's may be small while others large. Hence sub-population of agents may act as minor agents, while another class may act as major agents. To capture the essence of such problems, here, we introduce a general class of non-cooperative heterogeneous stochastic games with one major agent and a large population of minor agents where agents interact with an observed common process impacted by the mean field. A latent Markov chain and a latent Wiener process (common noise) modulate the common process, and agents cannot observe them. We use filtering techniques coupled with a convex analysis approach to (i) solve the mean field game limit of the problem, (ii) demonstrate that the best response strategies generate an $ε$-Nash equilibrium for finite populations, and (iii) obtain explicit characterisations of the best response strategies.

math.OC↗
arXiv2019

Mean Field Game Theory for Agents with Individual-State Partial Observations

Subject to reasonable conditions, in large population stochastic dynamics games, where the agents are coupled by the system's mean field (i.e. the state distribution of the generic agent) through their nonlinear dynamics and their nonlinear cost functions, it can be shown that a best response control action for each agent exists which (i) depends only upon the individual agent's state observations and the mean field, and (ii) achieves a $ε$-Nash equilibrium for the system. In this work we formulate a class of problems where each agent has only partial observations on its individual state. We employ nonlinear filtering theory and the Separation Principle in order to analyze the game in the asymptotically infinite population limit. The main result is that the $ε$-Nash equilibrium property holds where the best response control action of each agent depends upon the conditional density of its own state generated by a nonlinear filter, together with the system's mean field. Finally, comparing this MFG problem with state estimation to that found in the literature with a major agent whose partially observed state process is independent of the control action of any individual agent, it is seen that, in contrast, the partially observed state process of any agent in this work depends upon that agent's control action.

math.OC↗
arXiv2018

The density evolution of the killed Mckean-Vlasov process

The study of the density evolution naturally arises in Mean Field Game theory for the estimation of the density of the large population dynamics. In this paper, we study the density evolution of McKean-Vlasov stochastic differential equations in the presence of an absorbing boundary, where the solution to such equations corresponds to the dynamics of partially killed large populations. By using a fixed point theorem, we show that the density evolution is characterized as the unique solution of an integro-differential Fokker-Planck equation with Cauchy-Dirichlet data.

math.PR↗
arXiv2018

On the Hybrid Minimum Principle

The Hybrid Minimum Principle (HMP) is established for the optimal control of deterministic hybrid systems with both autonomous and controlled switchings and jumps where state jumps at the switching instants are permitted to be accompanied by changes in the dimension of the state space. First order variational analysis is performed via the needle variation methodology and the necessary optimality conditions are established in the form of the HMP. A feature of special interest in this work is the explicit presentations of boundary conditions on the Hamiltonians and the adjoint processes before and after switchings and jumps. In addition to an analytic example, the HMP results are illustrated for the optimal control of an electric vehicle with transmission, where the modelling of the powertrain requires the consideration of both autonomous and controlled switchings accompanied by dimension changes.

math.OC↗
arXiv2016

On the Control of Affine Systems with Safety Constraints: Relaxed In-Block Controllability

We consider affine systems defined on polytopes and study the cases where the systems are not in-block controllable with respect to the given polytopes. That are the cases in which we cannot fully control the affine systems within the interior of a given polytope, representing the intersection of given safety constraints. Instead, we introduce in this paper the notion of relaxed in-block controllability (RIBC), which can be useful for the cases where one can distinguish between soft and hard safety constraints. In particular, we study whether all the states in the interior of a given polytope, formed by the intersection of soft safety constraints, are mutually accessible through the interior of a given bigger polytope, formed by the intersection of hard safety constraints, by applying uniformly bounded control inputs. By exploring the geometry of the problem, we provide necessary conditions for RIBC. We then show when these conditions are also sufficient. Several illustrative examples are also given to clarify the main results.

eess.SY↗
arXiv2016

On the Relation between the Minimum Principle and Dynamic Programming for Classical and Hybrid Control Systems

Hybrid optimal control problems are studied for a general class of hybrid systems where autonomous and controlled state jumps are allowed at the switching instants and in addition to terminal and running costs switching between discrete states incurs costs. The statements of the Hybrid Minimum Principle and Hybrid Dynamic Programming are presented in this framework and it is shown that under certain assumptions the adjoint process in the Hybrid Minimum Principle and the gradient of the value function in Hybrid Dynamic Programming are governed by the same set of differential equations and have the same boundary conditions and hence are almost everywhere identical to each other along optimal trajectories. Analytic examples are provided to illustrate the results and, in particular, a Riccati formalism for linear quadratic hybrid tracking problems is presented.

math.OC↗
arXiv2014

On the Hybrid Minimum Principle On Lie Groups and the Exponential Gradient HMP Algorithm

This paper provides a geometrical derivation of the Hybrid Minimum Principle (HMP) for autonomous hybrid systems whose state manifolds constitute Lie groups $(G,\star)$ which are left invariant under the controlled dynamics of the system, and whose switching manifolds are defined as smooth embedded time invariant submanifolds of $G$. The analysis is expressed in terms of extremal (i.e. optimal) trajectories on the cotangent bundle of the state manifold $G$. The Hybrid Maximum Principle (HMP) algorithm introduced in \cite{Shaikh} is extended to the so-called Exponential Gradient algorithm. The convergence analysis for the algorithm is based upon the LaSalle Invariance Principle and simulation results illustrate their efficacy.

math.OC↗
arXiv2013

ε-Nash Mean Field Game Theory for Nonlinear Stochastic Dynamical Systems with Major and Minor Agents

This paper studies a large population dynamic game involving nonlinear stochastic dynamical systems with agents of the following mixed types: (i) a major agent, and (ii) a population of $N$ minor agents where $N$ is very large. The major and minor (MM) agents are coupled via both: (i) their individual nonlinear stochastic dynamics, and (ii) their individual finite time horizon nonlinear cost functions. This problem is approached by the so-called $ε$-Nash Mean Field Game ($ε$-NMFG) theory. A distinct feature of the mixed agent MFG problem is that even asymptotically (as the population size $N$ approaches infinity) the noise process of the major agent causes random fluctuation of the mean field behaviour of the minor agents. To deal with this, the overall asymptotic ($N \rightarrow \infty$) mean field game problem is decomposed into: (i) two non-standard stochastic optimal control problems with random coefficient processes which yield forward adapted stochastic best response control processes determined from the solution of (backward in time) stochastic Hamilton-Jacobi-Bellman (SHJB) equations, and (ii) two stochastic coefficient McKean-Vlasov (SMV) equations which characterize the state of the major agent and the measure determining the mean field behaviour of the minor agents. Existence and uniqueness of the solution to the Stochastic Mean Field Game (SMFG) system (SHJB and SMV equations) is established by a fixed point argument in the Wasserstein space of random probability measures. In the case that minor agents are coupled to the major agent only through their cost functions, the $ε_N$-Nash equilibrium property of the SMFG best responses is shown for a finite $N$ population system where $ε_N=O(1/\sqrt N)$.

math.OC↗
arXiv2012

Mean Field Stochastic Adaptive Control

For noncooperative games the mean field (MF) methodology provides decentralized strategies which yield Nash equilibria for large population systems in the asymptotic limit of an infinite (mass) population. The MF control laws use only the local information of each agent on its own state and own dynamical parameters, while the mass effect is calculated offline using the distribution function of (i) the population's dynamical parameters, and (ii) the population's cost function parameters, for the infinite population case. These laws yield approximate equilibria when applied in the finite population. In this paper, these a priori information conditions are relaxed, and incrementally the cases are considered where, first, the agents estimate their own dynamical parameters, and, second, estimate the distribution parameter in (i) and (ii) above. An MF stochastic adaptive control (SAC) law in which each agent observes a random subset of the population of agents is specified, where the ratio of the cardinality of the observed set to that of the number of agents decays to zero as the population size tends to infinity. Each agent estimates its own dynamical parameters via the recursive weighted least squares (RWLS) algorithm and the distribution of the population's dynamical parameters via maximum likelihood estimation (MLE). Under reasonable conditions on the population dynamical parameter distribution, the MF-SAC Law applied by each agent results in (i) the strong consistency of the self parameter estimates and the strong consistency of the population distribution function parameters; (ii) the long run average stability of all agent systems; (iii) a (strong) e-Nash equilibrium for the population of agents; and (iv) the a.s. equality of the long run average cost and the non-adaptive cost in the population limit.

math.OC↗
arXiv2012

On the Optimal Control of Impulsive Hybrid Systems On Riemannian Manifolds

This paper provides a geometrical derivation of the Hybrid Minimum Principle (HMP) for autonomous impulsive hybrid systems on Riemannian manifolds, i.e. systems where the manifold valued component of the hybrid state trajectory may have a jump discontinuity when the discrete component changes value. The analysis is expressed in terms of extremal trajectories on the cotangent bundle of the manifold state space. In the case of autonomous hybrid systems, switching manifolds are defined as smooth embedded submanifolds of the state manifold and the jump function is defined as a smooth map on the switching manifold. The HMP results are obtained in the case of time invariant switching manifolds and state jumps on Riemannian manifolds.

math.OC↗