arXiv subjects

Siavash Alemzadeh

Publications and source records attributed to Siavash Alemzadeh.

9 recordsLinked to original sources

arXiv2023

Data-Driven Structured Policy Iteration for Homogeneous Distributed Systems

Control of networked systems, comprised of interacting agents, is often achieved through modeling the underlying interactions. Constructing accurate models of such interactions--in the meantime--can become prohibitive in applications. Data-driven control methods avoid such complications by directly synthesizing a controller from the observed data. In this paper, we propose an algorithm referred to as Data-driven Structured Policy Iteration (D2SPI), for synthesizing an efficient feedback mechanism that respects the sparsity pattern induced by the underlying interaction network. In particular, our algorithm uses temporary "auxiliary" communication links in order to enable the required information exchange on a (smaller) sub-network during the "learning phase" -- links that will be removed subsequently for the final distributed feedback synthesis. We then proceed to show that the learned policy results in a stabilizing structured policy for the entire network. Our analysis is then followed by showing the stability and convergence of the proposed distributed policies throughout the learning phase, exploiting a construct referred to as the "Patterned monoid.'' The performance of D2SPI is then demonstrated using representative simulation scenarios.

eess.SY↗

arXiv2022

On Regularizability and its Application to Online Control of Unstable LTI Systems

Learning, say through direct policy updates, often requires assumptions such as knowing a priori that the initial policy (gain) is stabilizing, or persistently exciting (PE) input-output data, is available. In this paper, we examine online regulation of (possibly unstable) partially unknown linear systems with no prior access to an initial stabilizing controller nor PE input-output data; we instead leverage the knowledge of the input matrix for online regulation. First, we introduce and characterize the notion of "regularizability" for linear systems that gauges the extent by which a system can be regulated in finite-time in contrast to its asymptotic behavior (commonly characterized by stabilizability/controllability). Next, having access only to the input matrix, we propose the Data-Guided Regulation (DGR) synthesis procedure that -- as its name suggests -- regulates the underlying state while also generating informative data that can subsequently be used for data-driven stabilization or system identification. We further improve the computational performance of DGR via a rank-one update and demonstrate its utility in online regulation of the X-29 aircraft.

eess.SY↗

arXiv2020

Adaptive Traffic Control with Deep Reinforcement Learning: Towards State-of-the-art and Beyond

In this work, we study adaptive data-guided traffic planning and control using Reinforcement Learning (RL). We shift from the plain use of classic methods towards state-of-the-art in deep RL community. We embed several recent techniques in our algorithm that improve the original Deep Q-Networks (DQN) for discrete control and discuss the traffic-related interpretations that follow. We propose a novel DQN-based algorithm for Traffic Control (called TC-DQN+) as a tool for fast and more reliable traffic decision-making. We introduce a new form of reward function which is further discussed using illustrative examples with comparisons to traditional traffic control methods.

cs.LG↗

arXiv2020

Deep Learning-based Resource Allocation for Infrastructure Resilience

From an optimization point of view, resource allocation is one of the cornerstones of research for addressing limiting factors commonly arising in applications such as power outages and traffic jams. In this paper, we take a data-driven approach to estimate an optimal nodal restoration sequence for immediate recovery of the infrastructure networks after natural disasters such as earthquakes. We generate data from td-INDP, a high-fidelity simulator of optimal restoration strategies for interdependent networks, and employ deep neural networks to approximate those strategies. Despite the fact that the underlying problem is NP-complete, the restoration sequences obtained by our method are observed to be nearly optimal. In addition, by training multiple models---the so-called estimators---for a variety of resource availability levels, our proposed method balances a trade-off between resource utilization and restoration time. Decision-makers can use our trained models to allocate resources more efficiently after contingencies, and in turn, improve the community resilience. Besides their predictive power, such trained estimators unravel the effect of interdependencies among different nodal functionalities in the restoration strategies. We showcase our methodology by the real-world interdependent infrastructure of Shelby County, TN.

eess.SP↗

arXiv2019

Distributed Q-Learning for Dynamically Decoupled Systems

Control of large-scale networked systems often necessitates the availability of complex models for the interactions amongst the agents. However in many applications, building accurate models of agents or interactions amongst them might be infeasible or computationally prohibitive due to the curse of dimensionality or the complexity of these interactions. In the meantime, data-guided control methods can circumvent model complexity by directly synthesizing the controller from the observed data. In this paper, we propose a distributed Q-learning algorithm to design a feedback mechanism based on a given underlying graph structure parameterizing the agents' interaction network. We assume that the distributed nature of the system arises from the cost function of the corresponding control problem and show that for the specific case of identical dynamically decoupled systems, the learned controller converges to the optimal Linear Quadratic Regulator (LQR) controller for each subsystem. We provide a convergence analysis and verify the result with an example.

math.OC↗

arXiv2018

Linear Model Regression on Time-series Data: Non-asymptotic Error Bounds and Applications

Data-driven methods for modeling dynamic systems have received considerable attention as they provide a mechanism for control synthesis directly from the observed time-series data. In the absence of prior assumptions on how the time-series had been generated, regression on the system model has been particularly popular. In the linear case, the resulting least squares setup for model regression, not only provides a computationally viable method to fit a model to the data, but also provides useful insights into the modal properties of the underlying dynamics. Although probabilistic estimates for this model regression have been reported, deterministic error bounds have not been examined in the literature, particularly as they pertain to the properties of the underlying system. In this paper, we provide deterministic non-asymptotic error bounds for fitting a linear model to the observed time-series data, with a particular attention to the role of symmetry and eigenvalue multiplicity in the underlying system matrix.

math.OC↗

arXiv2018

Influence Models on Layered Uncertain Networks: A Guaranteed-Cost Design Perspective

Control and estimation on large-scale social networks often necessitate the availability of models for the interactions amongst the agents. However characterizing accurate models of social interactions pose new challenges due to inherent complexity and unpredictability. Moreover, model uncertainty becomes more pronounced for large-scale networks. For certain classes of social networks, the layering structure allows a compositional approach. In this paper, we present such an approach to determine performance guarantees on layered networks with inherent model uncertainties. A factorization method is used to determine robust stability and performance and this is accomplished by a layered cost-guaranteed design via a layered Riccati-type solver, mirroring the network structure. We provide an example of the proposed methodology in the context of opinion dynamics on large-scale social networks.

math.OC↗

arXiv2017

Controllability and data-driven identification of bipartite consensus on nonlinear signed networks

Nonlinear networked systems are of interest in several areas of research, such as multi-agent systems and social networks. In this paper, we examine the controllability of several classes of nonlinear networked dynamics on which the underlying graph admits negative weights. Such signed networks exhibit bipartite clustering when the underlying graph is structurally balanced. We show that structural balance is the key ingredient inducing uncontrollability when combined with a leader-node symmetry and a certain type of dynamical symmetry. We then examine the problem of extracting the bipartite structure of such graphs from data using Extended Dynamic Mode Decomposition to approximate the corresponding Koopman operator.

math.OC↗

arXiv2017

Controllability and Stabilizability Analysis of Signed Consensus Networks

Signed networks have been a topic of recent interest in the network control community as they allow studying antagonistic interactions in multi-agent systems. Although dynamical characteristics of signed networks have been well-studied, notions such as controllability and stabilizability for signed networks for protocols such as consensus are missing in the literature. Classically, graph automorphisms with respect to the input nodes have been used to characterize uncontrollability of consensus networks. In this paper, we show that in addition to the graph symmetry, the topological property of structural balance facilitates the derivation of analogous sufficient conditions for uncontrollability for signed networks. In particular, we provide an analysis which shows that a gauge transformation induced by structural balance allows symmetry arguments to hold for signed consensus networks. Lastly, we use fractional automorphisms to extend our observations to output controllability and stabilizability of signed networks.

math.OC↗