SearcharxivSearch

arXiv subjects

Peter E. Caines

Publications and source records attributed to Peter E. Caines.

At least 19 recordsLinked to original sources

Mean Field Games and Control on Large Expander Graphs

This paper investigates mean field games and control on sparse networks. In the case of large expander graphs, the limit topologies are analyzed using the graphexon framework, which characterizes both dense network limits and sparse connections. We prove that the sequence of empirical graphexon measures defined on finite graphs converges weakly to a limit graphexon measure on a continuous state space. Furthermore, the associated sequence of discrete averaging operators converges strongly to a continuous operator. These properties enable the formulation of a linear-quadratic mean field game in which each agent is identified by a spatial network label $α\in X$ and only interacts with the neighborhood average defined by the operator $\mathcal{G}$ characterized by large expander graphs. In Section 5, algebraic conditions for the global asymptotic stability of the closed-loop system are established. The analysis identifies parameter thresholds that gives rise to a Turing-type topological instability, where the homogeneous mean state remains stable while the spatial deviation field diverges over the continuous spectrum of the limit operator.

math.OC

Transmission Neural Networks: Inhibitory and Excitatory Connections

This paper extends the Transmission Neural Network model proposed by Gao and Caines in [1]-[3] to incorporate inhibitory connections and neurotransmitter populations. The extended network model contains binary neuronal states, transmission dynamics, and inhibitory and excitatory connections. Under technical assumptions, we establish the characterization of the firing probabilities of neurons, and show that such a characterization considering inhibitions can be equivalently represented by a neural network where each neuron has a continuous state of dimension 2. Moreover, we incorporated neurotransmitter populations into the modeling and establish the limit network model when the number of neurotransmitters at all synaptic connections go to infinity. Finally, sufficient conditions for stability and contraction properties of the limit network model are established.

cs.SI

Concentration of Cumulative Reward in Markov Decision Processes

In this paper, we investigate the concentration properties of cumulative reward in Markov Decision Processes (MDPs), focusing on both asymptotic and non-asymptotic settings. We introduce a unified approach to characterize reward concentration in MDPs, covering both infinite-horizon settings (i.e., average and discounted reward frameworks) and finite-horizon setting. Our asymptotic results include the law of large numbers, the central limit theorem, and the law of iterated logarithms, while our non-asymptotic bounds include Azuma-Hoeffding-type inequalities and a non-asymptotic version of the law of iterated logarithms. Additionally, we explore two key implications of our results. First, we analyze the sample path behavior of the difference in rewards between any two stationary policies. Second, we show that two alternative definitions of regret for learning policies proposed in the literature are rate-equivalent. Our proof techniques rely on a martingale decomposition of cumulative reward, properties of the solution to the policy evaluation fixed-point equation, and both asymptotic and non-asymptotic concentration results for martingale difference sequences.

cs.LG

Transmission Neural Networks: Approximate Receding Horizon Control for Virus Spread on Networks

Transmission Neural Networks (TransNNs) proposed by Gao and Caines (2022) serve as both virus spread models over networks and neural network models with tuneable activation functions. This paper establishes that TransNNs provide upper bounds on the infection probability generated from the associated Markovian stochastic Susceptible-Infected-Susceptible (SIS) model with 2^n state configurations where n is the number of nodes in the network, and can be employed as an approximate model for the latter. Based on such an approximation, a TransNN-based receding horizon control approach for mitigating virus spread is proposed and we demonstrate that it allows significant computational savings compared to the dynamic programming solution to Markovian SIS model with 2^n state configurations, as well as providing less conservative control actions compared to the TransNN-based optimal control. Finally, numerical comparisons among (a) dynamic programming solutions for the Markovian SIS model, (b) TransNN-based optimal control and (c) the proposed TransNN-based receding horizon control are presented.

eess.SY

Transmission Neural Networks: Approximation and Optimal Control

Transmission Neural Networks (TransNNs) introduced by Gao and Caines (2022) connect virus spread models over networks and neural networks with tuneable activation functions. This paper presents the approximation technique and the underlying assumptions employed by TransNNs in relation to the corresponding Markovian Susceptible-Infected-Susceptible (SIS) model with 2^n states, where n is the number of nodes in the network. The underlying infection paths are assumed to be stochastic with heterogeneous and time-varying transmission probabilities. We obtain the conditional probability of infection in the stochastic 2^n-state SIS epidemic model corresponding to each state configuration under mild assumptions, which enables control solutions based on Markov decision processes (MDP). Finally, MDP control with 2^n-state SIS epidemic models and optimal control with TransNNs are compared in terms of mitigating virus spread over networks through vaccination, and it is shown that TranNNs enable the generation of control laws with significant computational savings, albeit with more conservative control actions.

eess.SY

Comparative Analysis of Learning-Based Methods for Transient Stability Assessment

Transient stability and critical clearing time (CCT) are important concepts in power system protection and control. This paper explores and compares various learning-based methods for predicting CCT under uncertainties arising from renewable generation, loads, and contingencies. Specially, we introduce new definitions of transient stability (B-stablilty) and CCT from an engineering perspective. For training the models, only the initial values of system variables and contingency cases are used as features, enabling the provision of protection information based on these initial values. To enhance efficiency, a hybrid feature selection strategy combining the maximal information coefficient (MIC) and Spearman's Correlation Coefficient (SCC) is employed to reduce the feature dimension. The performance of different learning-based models is evaluated on a WSCC 9-bus system.

eess.SY

Quadratic Optimal Control of Graphon Q-noise Linear Systems

The modelling of linear quadratic Gaussian optimal control problems on large complex networks is intractable computationally. Graphon theory provides an approach to overcome these issues by defining limit objects for infinite sequences of graphs permitting one to approximate arbitrarily large networks by infinite dimensional operators. This is extended to stochastic systems by the use of Q-noise, a generalization of Wiener processes in finite dimensional spaces to processes in function spaces. The optimal control of linear quadratic problems on graphon systems with Q-noise disturbances are defined and shown to be the limit of the corresponding finite graph optimal control problem. The theory is extended to low rank systems, and a fully worked special case is presented. In addition, the worst-case long-range average and infinite horizon discounted optimal control performance with respect to Q-noise distribution are computed for a small set of standard graphon limits.

math.OC

Strong Consistency and Rate of Convergence of Switched Least Squares System Identification for Autonomous Markov Jump Linear Systems

In this paper, we investigate the problem of system identification for autonomous Markov jump linear systems (MJS) with complete state observations. We propose switched least squares method for identification of MJS, show that this method is strongly consistent, and derive data-dependent and data-independent rates of convergence. In particular, our data-independent rate of convergence shows that, almost surely, the system identification error is $\mathcal{O}\big(\sqrt{\log(T)/T} \big)$ where $T$ is the time horizon. These results show that switched least squares method for MJS has the same rate of convergence as least squares method for autonomous linear systems. We derive our results by imposing a general stability assumption on the model called stability in the average sense. We show that stability in the average sense is a weaker form of stability compared to the stability assumptions commonly imposed in the literature. We present numerical examples to illustrate the performance of the proposed method.

cs.LG

Transmission Neural Networks: From Virus Spread Models to Neural Networks

This work connects models for virus spread on networks with their equivalent neural network representations. Based on this connection, we propose a new neural network architecture, called Transmission Neural Networks (TransNNs) where activation functions are primarily associated with links and are allowed to have different activation levels. Furthermore, this connection leads to the discovery and the derivation of three new activation functions with tunable or trainable parameters. Moreover, we prove that TransNNs with a single hidden layer and a fixed non-zero bias term are universal function approximators. Finally, we present new fundamental derivations of continuous time epidemic network models based on TransNNs.

cs.LG

Stationary Cost Nodes in Infinite Horizon LQG-GMFGs

An analysis of infinite horizon linear quadratic Gaussian (LQG) Mean Field Games is given within the general framework of Graphon Mean Field Games (GMFG) on dense infinite graphs (or networks) introduced in Caines and Huang (2018). For a class of LQG-GMFGs, analytical expressions are derived for the infinite horizon Nash values at the nodes of the infinite graph. Furthermore, under specific conditions on the network and the initial population means, it is shown that the nodes with strict local maximal infinite network degree are also nodes with strict local minimal cost at equilibrium.

eess.SY

A Class of Hybrid LQG Mean Field Games with State-Invariant Switching and Stopping Strategies

A novel framework is presented that combines Mean Field Game (MFG) theory and Hybrid Optimal Control (HOC) theory to obtain a unique $ε$-Nash equilibrium for a non-cooperative game with switching and stopping times. We consider the case where there exists one major agent with a significant influence on the system together with a large number of minor agents constituting two subpopulations, each agent with individually asymptotically negligible effect on the whole system. Each agent has stochastic linear dynamics with quadratic costs, and the agents are coupled in their dynamics and costs by the average state of minor agents (i.e. the empirical mean field). It is shown that for a class of Hybrid LQG MFGs, the optimal switching and stopping times are state-invariant and only depend on the dynamical parameters of each agent. Accordingly, a hybrid systems formulation of the game is presented via the indexing by discrete events: (i) the switching of the major agent between alternative dynamics or (ii) the termination of the agents' trajectories in one or both of the subpopulations of minor agents. Optimal switchings and stopping time strategies together with best response control actions for, respectively, the major agent and all minor agents are established with respect to their individual cost criteria by an application of Hybrid LQG MFG theory.

eess.SY

Graphon Mean Field Games and the GMFG Equations

The emergence of the graphon theory of large networks and their infinite limits has enabled the formulation of a theory of the centralized control of dynamical systems distributed on asymptotically infinite networks (Gao and Caines, IEEE CDC 2017, 2018). Furthermore, the study of the decentralized control of such systems was initiated in (Caines and Huang, IEEE CDC 2018, 2019), where Graphon Mean Field Games (GMFG) and the GMFG equations were formulated for the analysis of non-cooperative dynamic games on unbounded networks. In that work, existence and uniqueness results were introduced for the GMFG equations, together with an epsilon-Nash theory for GMFG systems which relates infinite population equilibria on infinite networks to finite population equilibria on finite networks. Those results are rigorously established in this paper.

math.OC

LQG Graphon Mean Field Games: Analysis via Graphon Invariant Subspaces

This paper studies approximate solutions to large-scale linear quadratic stochastic games with homogeneous nodal dynamics parameters and heterogeneous network couplings within the graphon mean field game framework in [2]-[4]. A graphon time-varying dynamical system model is first formulated to study the finite and then limit problems of linear quadratic Gaussian graphon mean field games (LQG-GMFG). The Nash equilibrium of the limit problem is then characterized by two coupled graphon time-varying dynamical systems. Sufficient conditions are established for the existence of a unique solution to the limit LQG-GMFG problem. For the computation of LQG-GMFG solutions two methods are established and employed where one is based on fixed point iterations and the other on a decoupling operator Riccati equation; furthermore, two corresponding sets of solutions are established based on spectral decompositions. Finally, a set of numerical simulations on networks associated with different types of graphons are presented.

eess.SY

Subspace Decomposition for Graphon LQR: Applications to VLSNs of Harmonic Oscillators

Graphon control has been proposed and developed in [1]-[3] to approximately solve control problems for very large-scale networks (VLSNs) of linear dynamical systems based on graphon limits. This paper provides a solution method based on invariant subspace decompositions for a class of graphon linear quadratic regulation (LQR) problems where the local dynamics share homogeneous parameters but the graphon couplings may be heterogeneous among the coupled agents. Graphon couplings in this paper appear in states, controls and costs, and these couplings may be represented by different graphons. By exploring a common invariant subspace of the couplings, the original problem is decomposed into a network coupled LQR problem of finite dimension and a decoupled infinite dimensional LQR problem. A centralized optimal solution, and a nodal collaborative optimal control solution where each agent computes its part of the optimal solution locally, are established. The application of these solutions to finite network LQR problems may be via (i) the graphon control methodology [3], or (ii) the representation of finite LQR problems as special cases of graphon LQR problems. The complexity of these solutions involves solving one nd X nd dimensional Riccati equation and one n X n Riccati equation, where n is the dimension of each nodal agent state and d is the dimension of the nontrivial common invariant subspace of the coupling operators, whereas a direct approach involves solving an nN X nN dimensional Riccati equation, where N is the size of the network. For situations where the graphon couplings do not admit exact low-rank representations, approximate control is developed based on low-rank approximations. Finally, an application to the regulation of harmonic oscillators coupled over large networks with uncertainties is demonstrated.

eess.SY

Linear Quadratic Graphon Field Games

Linear quadratic graphon field games (LQ-GFGs) are defined to be LQ games which involve a large number of agents that are weakly coupled via a weighted undirected graph on which each node represents an agent. The links of the graph correspond to couplings between the agents' dynamics, as well as between the individual cost functions, which each agent attempts to minimize. We formulate limit LQ-GFG problems based on the assumption that these graphs lie in a sequence which converges to a limit graphon. First, under a finite-rank assumption on the limit graphon, the existence and uniqueness of solutions to the formulated limit LQ-GFG problem is established. Second, based upon the solutions to the limit LQ-GFG problem, epsilon-Nash equilibria are constructed for the corresponding game problems with a very large but finite number of players. This result is then generalized to the case with random initial conditions. It is to be noted that LQ-GFG problems are distinct from the class of graphon mean field game (GMFG) problems where a population is hypothesized to be associated with each node of the graph [Caines and Huang CDC 2018, 2019].

eess.SY

$ε$-Nash Equilibria for Major Minor LQG Mean Field Games with Partial Observations of All Agents

The partially observed major minor LQG and nonlinear mean field game (PO MM LQG MFG) systems where it is assumed the major agent's state is partially observed by each minor agent, and the major agent completely observes its own state have been analysed in the literature. In this paper, PO MM LQG MFG problems with general information patterns are studied where (i) the major agent has partial observations of its own state, and (ii) each minor agent has partial observations of its own state and the major agent's state. The assumption of partial observations by all agents leads to a new situation involving the recursive estimation by each minor agent of the major agent's estimate of its own state. For a general case of indefinite LQG MFG systems, the existence of $ε$-Nash equilibria together with the individual agents' control laws yielding the equilibria are established via the Separation Principle.

math.OC

Convex Analysis for LQG Systems with Applications to Major Minor LQG Mean-Field Game Systems

We develop a convex analysis approach for solving LQG optimal control problems and apply it to major-minor (MM) LQG mean-field game (MFG) systems. The approach retrieves the best response strategies for the major agent and all minor agents that attain an $ε$-Nash equilibrium. An important and distinctive advantage to this approach is that unlike the classical approach in the literature, we are able to avoid imposing assumptions on the evolution of the mean-field. In particular, this provides a tool for dealing with complex and non-standard systems.

eess.SY