arXiv subjects

Jalal Arabneydi

Publications and source records attributed to Jalal Arabneydi.

24 records · Page 2Linked to original sources

arXiv2020

Optimal Distributed Control for Leader-Follower Networks: A Scalable Design

The focus of this paper is directed towards optimal control of multi-agent systems consisting of one leader and a number of followers in the presence of noise. The dynamics of every agent is assumed to be linear, and the performance index is a quadratic function of the states and actions of the leader and followers. The leader and followers are coupled in both dynamics and cost. The state of the leader and the average of the states of all followers (called mean-field) are common information and known to all agents; however, the local state of the followers are private information and unknown to other agents. It is shown that the optimal distributed control strategy is linear time-varying, and its computational complexity is independent of the number of followers. This strategy can be computed in a distributed manner, where the leader needs to solve one Riccati equation to determine its optimal strategy while each follower needs to solve two Riccati equations to obtain its optimal strategy. This result is subsequently extended to the case of the infinite horizon discounted and undiscounted cost functions, where the optimal distributed strategy is shown to be stationary. A numerical example with $100$ followers is provided to demonstrate the efficacy of the results.

math.OC↗

arXiv2020

Near-Optimal Control Strategy in Leader-Follower Networks: A Case Study for Linear Quadratic Mean-Field Teams

In this paper, a decentralized stochastic control system consisting of one leader and many homogeneous followers is studied. The leader and followers are coupled in both dynamics and cost, where the dynamics are linear and the cost function is quadratic in the states and actions of the leader and followers. The objective of the leader and followers is to reach consensus while minimizing their communication and energy costs. The leader knows its local state and each follower knows its local state and the state of the leader. The number of required links to implement this decentralized information structure is equal to the number of followers, which is the minimum number of links for a communication graph to be connected. In the special case of leaderless, no link is required among followers, i.e., the communication graph is not even connected. We propose a near-optimal control strategy that converges to the optimal solution as the number of followers increases. One of the salient features of the proposed solution is that it provides a design scheme, where the convergence rate \edit{as well as} the collective behavior of the followers can be designed by choosing appropriate cost functions. In addition, the computational complexity of the proposed solution does not depend on the number of followers. Furthermore, the proposed strategy can be computed in a distributed manner, where the leader solves one Riccati equation and each follower solves two Riccati equations to calculate their strategies. Two numerical examples are provided to demonstrate the effectiveness of the results in the control of multi-agent systems.

math.OC↗

arXiv2020

Deep Structured Teams with Linear Quadratic Model: Partial Equivariance and Gauge Transformation

Motivated by the recent developments in artificial intelligence, we introduce linear quadratic deep structured teams in this paper. Two notions of equivariant and partially equivariant systems are defined, and it is shown that such systems can be partitioned into a few sub-populations of decision makers, where every decision maker in each sub-population is coupled in both dynamics and cost function through a set of linear regressions of the states and actions of all decision makers. Two non-classical information structures are considered: deep-state sharing and partial deep-state sharing, where deep state refers to the linear regression of the states of the decision makers in each sub-population. For a risk-sensitive cost function with deep-state sharing structure, a closed-form low-complexity representation of the globally optimal strategy is obtained, whose computational complexity is independent of the number of decision makers in each sub-population. In addition, it is shown that the risk-sensitive solution converges to the risk-neutral one as the number of decision makers increases to infinity. Moreover, two sub-optimal sequential strategies under partial deep-state sharing information structure are proposed by introducing two Kalman-like filters, one based on the finite-population model and the other one based on the infinite-population model. It is proved that the prices of information associated with the above sub-optimal solutions converge to zero as the number of decision makers goes to infinity. Furthermore, a class of feed-forward deep neural networks with multiple layers of weighted sums and products is introduced wherein the optimal weights and biases are explicitly obtained. A supply-chain management example is presented to demonstrate the efficacy of the obtained results.

math.OC↗

arXiv2020

Deep Teams: Decentralized Decision Making with Finite and Infinite Number of Agents

Inspired by the concepts of deep learning in artificial intelligence and fairness in behavioural economics, we introduce deep teams in this paper. In such systems, agents are partitioned into a few sub-populations so that the dynamics and cost of agents in each sub-population is invariant to the indexing of agents. The goal of agents is to minimize a common cost function in such a manner that the agents in each sub-population are not discriminated or privileged by the way they are indexed. Two non-classical information structures are studied. In the first one, each agent observes its local state as well as the empirical distribution of the states of agents in each sub-population, called deep state, whereas in the second one, the deep states of a subset (possibly all) of sub-populations are not observed. Novel dynamic programs are developed to identify globally optimal and sub-optimal solutions for the first and second information structures, respectively. The computational complexity of finding the optimal solution in both space and time is polynomial (rather than exponential) with respect to the number of agents in each sub-population and is linear (rather than exponential) with respect to the control horizon. This complexity is further reduced in time by introducing a forward equation, that we call deep Chapman-Kolmogorov equation, described by multiple convolutional layers of Binomial probability distributions. Two different prices are defined for computation and communication, and it is shown that under mild assumptions they converge to zero as the quantization level and the number of agents tend to infinity. In addition, the main results are extended to the infinite-horizon discounted model and arbitrarily asymmetric cost function. Finally, a service-management example with 200 users is presented.

math.OC↗

arXiv2019

Deep Nash and Sequential Mean-Field Equilibria in Cooperative and Non-cooperative Games with Imperfect Information Structures

A class of nonzero-sum stochastic dynamic games with imperfect information structure is investigated. The game involves an arbitrary number of players, modeled as homogeneous Markov decision processes, aiming to find a sequential Nash equilibrium. The players are coupled in both dynamics and cost functions through the empirical distribution of states and actions of players. Two non-classical information structures are considered: deep state sharing and no-sharing, where deep state refers to the empirical distribution of the states of players. In the former, each player observes its local state as well as the deep state while in the latter each player observes only its local state. For both finite- and infinite-horizon cost functions, a sequential equilibrium, called deep Nash equilibrium, is identified, where the dynamics of deep state resembles a convolutional neural network. In addition, an approximate sequential equilibrium, called sequential mean-field equilibrium, under no-sharing information structure is proposed, whose performance converges to that of the deep Nash equilibrium despite the fact that the strategy is not necessarily continuous with respect to the deep state. The proposed strategies are robust to trembling-hand imperfection in both microscopic and macroscopic levels. Finally, the extension to multiple sub-populations and arbitrarily-coupled (asymmetric) cost functions are demonstrated.

math.OC↗

arXiv2018

Linear Quadratic Mean Field Teams: Optimal and Approximately Optimal Decentralized Solutions

We consider team optimal control of decentralized systems with linear dynamics, quadratic costs, and arbitrary disturbance that consist of multiple sub-populations with exchangeable agents (i.e., exchanging two agents within the same sub-population does not affect the dynamics or the cost). Such a system is equivalent to one where the dynamics and costs are coupled across agents through the mean-field (or empirical mean) of the states and actions (even when the primitive random variables are non-exchangeable). Two information structures are investigated. In the first, all agents observe their local state and the mean-field of all sub-populations, in the second, all agents observe their local state but the mean-field of only a subset of the sub-populations. Both information structures are non-classical and not partially nested. Nonetheless, it is shown that linear control strategies are optimal for the first and approximately optimal for the second, the approximation error is inversely proportional to the size of the sub-populations whose mean-fields are not observed. The corresponding gains are determined by the solution of K+1 decoupled standard Riccati equations, where K is the number of sub-populations. The dimensions of the Riccati equations do not depend on the size of the sub-populations, thus the solution complexity is independent of the number of agents. Generalizations to major-minor agents, tracking cost, weighted mean-field, and infinite horizon are provided. The results are illustrated using an example of demand response in smart grids.

eess.SY↗