arXiv subjects

Aranya Chakrabortty

Publications and source records attributed to Aranya Chakrabortty.

At least 37 records · Page 2Linked to original sources

arXiv2020

Fast Online Reinforcement Learning Control using State-Space Dimensionality Reduction

In this paper, we propose a fast reinforcement learning (RL) control algorithm that enables online control of large-scale networked dynamic systems. RL is an effective way of designing model-free linear quadratic regulator (LQR) controllers for linear time-invariant (LTI) networks with unknown state-space models. However, when the network size is large, conventional RL can result in unacceptably long learning times. The proposed approach is to construct a compressed state vector by projecting the measured state through a projective matrix. This matrix is constructed from online measurements of the states in a way that it captures the dominant controllable subspace of the open-loop network model. Next, a RL-controller is learned using the reduced-dimensional state instead of the original state such that the resultant cost is close to the optimal LQR cost. Numerical benefits as well as the cyber-physical implementation benefits of the approach are verified using illustrative examples including an example of wide-area control of the IEEE 68-bus benchmark power system.

eess.SY↗

arXiv2020

A Stackelberg Security Investment Game for Voltage Stability of Power Systems

We formulate a Stackelberg game between an attacker and a defender of a power system. The attacker attempts to alter the load setpoints of the power system covertly and intelligently, so that the voltage stability margin of the grid is reduced, driving the entire system towards a voltage collapse. The defender, or the system operator, aims to compensate for this reduction by retuning the reactive power injection to the grid by switching on control devices, such as a bank of shunt capacitors. A modified Backward Induction method is proposed to find a cost-based Stackelberg equilibrium (CBSE) of the game, which saves the players' costs while providing the optimal allocation of both players' investment resources under budget and covertness constraints. We analyze the proposed game extensively for the IEEE 9-bus power system model and present an example of its performance for the IEEE 39-bus power system model. It is demonstrated that the defender is able to maintain system stability unless its security budget is much lower than the attacker's budget.

eess.SY↗

arXiv2020

Hierarchical Control of Multi-Agent Systems using Online Reinforcement Learning

We propose a new reinforcement learning based approach to designing hierarchical linear quadratic regulator (LQR) controllers for heterogeneous linear multi-agent systems with unknown state-space models and separated control objectives. The separation arises from grouping the agents into multiple non-overlapping groups, and defining the control goal as two distinct objectives. The first objective aims to minimize a group-wise block-decentralized LQR function that models group-level mission. The second objective, on the other hand, tries to minimize an LQR function between the average states (centroids) of the groups. Exploiting this separation, we redefine the weighting matrices of the LQR functions in a way that they allow us to decouple their respective algebraic Riccati equations. Thereafter, we develop a reinforcement learning strategy that uses online measurements of the agent states and the average states to learn the respective controllers based on the approximate Riccati equations. Since the first controller is block-decentralized and, therefore, can be learned in parallel, while the second controller is reduced-dimensional due to averaging, the overall design enjoys a significantly reduced learning time compared to centralized reinforcement learning.

eess.SY↗

arXiv2020

Reduced-Dimensional Reinforcement Learning Control using Singular Perturbation Approximations

We present a set of model-free, reduced-dimensional reinforcement learning (RL) based optimal control designs for linear time-invariant singularly perturbed (SP) systems. We first present a state-feedback and output-feedback based RL control design for a generic SP system with unknown state and input matrices. We take advantage of the underlying time-scale separation property of the plant to learn a linear quadratic regulator (LQR) for only its slow dynamics, thereby saving a significant amount of learning time compared to the conventional full-dimensional RL controller. We analyze the sub-optimality of the design using SP approximation theorems and provide sufficient conditions for closed-loop stability. Thereafter, we extend both designs to clustered multi-agent consensus networks, where the SP property reflects through clustering. We develop both centralized and cluster-wise block-decentralized RL controllers for such networks, in reduced dimensions. We demonstrate the details of the implementation of these controllers using simulations of relevant numerical examples and compare them with conventional RL designs to show the computational benefits of our approach.

eess.SY↗

arXiv2020

Co-Design of Delays and Sparse Controllers for Bandwidth-Constrained Cyber-Physical Systems

We address the problem of sparsity-promoting optimal control of cyber-physical systems with feedback delays. The delays are categorized into two classes - namely, intra-layer delay, and inter-layer delay between the cyber and the physical layers. Our objective is to minimize the H2-norm of the closed-loop system by designing an optimal combination of these two delays along with a sparse state-feedback controller, while respecting a given bandwidth constraint. We propose a two-loop optimization algorithm for this. The inner loop, based on alternating directions method of multipliers (ADMM), handles the conflicting directions of decreasing H2-norm and increasing sparsity of the controller. The outer loop comprises of semidefinite program (SDP)-based relaxations of non-convex inequalities necessary for stable co-design of the delays with the controller. We illustrate this algorithm using simulations that highlight various aspects of how delays and sparsity impact the stability and $\mc{H}_2$ performance of a LTI system.

math.OC↗

arXiv2019

Game-Theoretic Mixed $H_2/H_{\infty}$ Control with Sparsity Constraint for Multi-agent Networked Control Systems

Multi-agent networked control systems (NCSs) are often subject to model uncertainty and are limited by large communication cost, associated with feedback of data between the system nodes. To provide robustness against model uncertainty and to reduce the communication cost, this paper investigates the mixed $H_2/H_{\infty}$ control problem for NCS under the sparsity constraint. First, proximal alternating linearized minimization (PALM) is employed to solve the centralized social optimization where the agents have the same optimization objective. Next, we investigate a sparsity-constrained noncooperative game, which accommodates different control-performance criteria of different agents, and propose a best-response dynamics algorithm based on PALM that converges to an approximate Generalized Nash Equilibrium (GNE) of this game. A special case of this game, where the agents have the same $H_2$ objective, produces a partially-distributed social optimization solution. We validate the proposed algorithms using a network with unstable node dynamics and demonstrate the superiority of the proposed PALM-based method to a previously investigated sparsity-constrained mixed $H_2/H_{\infty}$ controller.

eess.SY↗

arXiv2019

Sparsity-Promoting Optimal Control of Cyber-Physical Systems over Shared Communication Networks

Recent years have seen several new directions in the design of sparse control of cyber-physical systems (CPSs) driven by the objective of reducing communication cost. One common assumption made in these designs is that the communication happens over a dedicated network. For many practical applications, however, communication must occur over shared networks, leading to two critical design challenges, namely - time-delays in the feedback and fair sharing of bandwidth among users. In this paper, we present a set of sparse H2 control designs under these two design constraints. An important aspect of our design is that the delay itself can be a function of sparsity, which leads to an interesting pattern of trade-offs in the H2 performance. We present three distinct algorithms. The first algorithm preconditions the assignable bandwidth to the network and produces an initial guess for a stabilizing controller. This is followed by our second algorithm, which sparsifies this controller while simultaneously adapting the feedback delay and optimizing the H2 performance using alternating directions method of multipliers (ADMM). The third algorithm extends this approach to a multiple user scenario where optimal number of communication links, whose total sum is fixed, is distributed fairly among users by minimizing the variance of their H2 performances. The problem is cast as a difference-of-convex (DC) program with mixed-integer linear program (MILP) constraints. We provide theorems to prove convergence of these algorithms, followed by validation through numerical simulations.

eess.SY↗

arXiv2019

A New Cyber-Secure Countermeasure for LTI systems under DoS attacks

This paper presents a new counter-measure to mitigate denial-of-service cyber-attacks in linear time-invariant (LTI) systems. We first design a sparse linear quadratic regulator (LQR) optimal controller for a given LTI plant and evaluate the priority of the feedback communication links in terms of the loss of closed-loop performance when the corresponding block of the feedback gain matrix is removed. An attacker may know about this priority ordering, and thereby attack the links with the highest priority. To prevent this, we present a message rerouting strategy by which the states that are scheduled to be transmitted through the high priority links can be rerouted through lower priority ones in case the former get attacked. Since the attacked link is not available for service, and the states of the low priority links can no longer be accommodated either, we run a structured $\mathcal{H}_2$ control algorithm to determine the post-attack optimal feedback gains. We illustrate various aspects of the proposed algorithms by simulations.

eess.SY↗

arXiv2019

Modeling and Quantifying the Impact of Wind Power Penetration on Power System Coherency

This paper presents a mathematical analysis of how wind generation impacts the coherency property of power systems. Coherency arises from time-scale separation in the dynamics of synchronous generators, where generator states inside a coherent area synchronize over a fast time-scale due to stronger coupling, while the areas themselves synchronize over a slower time-scale due to weaker coupling. This time-scale separation is reflected in the form of a spectral separation in the weighted Laplacian matrix describing the swing dynamics of the generators. However, when wind farms with doubly-fed induction generators (DFIG) are integrated in the system then this Laplacian matrix changes based on both the level of wind penetration and the location of the wind farms. The modified Laplacian changes the effective slow eigenspace of the generators. Depending on penetration level, this change may result in changing the identities of the coherent areas. We develop a theoretical framework to quantify this modification, and validate our results with numerical simulations of the IEEE 68-bus system with one and multiple wind farms. We compare our model based results on clustering with results using measurement-based principal component analysis to substantiate our derivations.

eess.SY↗

arXiv2018

Sparse Wide-Area Control of Power Systems using Data-driven Reinforcement Learning

In this paper we present an online wide-area oscillation damping control (WAC) design for uncertain models of power systems using ideas from reinforcement learning. We assume that the exact small-signal model of the power system at the onset of a contingency is not known to the operator and use the nominal model and online measurements of the generator states and control inputs to rapidly converge to a state-feedback controller that minimizes a given quadratic energy cost. However, unlike conventional linear quadratic regulators (LQR), we intend our controller to be sparse, so its implementation reduces the communication costs. We, therefore, employ the gradient support pursuit (GraSP) optimization algorithm to impose sparsity constraints on the control gain matrix during learning. The sparse controller is thereafter implemented using distributed communication. Using the IEEE 39-bus power system model with 1149 unknown parameters, it is demonstrated that the proposed learning method provides reliable LQR performance while the controller matched to the nominal model becomes unstable for severely uncertain systems.

eess.SY↗

arXiv2018

A Cyber-Security Investment Game for Networked Control Systems

We formulate a resource-planning game between an attacker and a defender of a network control system. We consider the network to be operating in closed-loop with a linear quadratic regulator (LQR). We construct a general-sum, two-player, mixed strategy game, where the attacker attempts to destroy communication equipment of some nodes, and thereby render the LQR feedback gain matrix to be sparse, leading to degradation of closed-loop performance. The defender, on the other hand, aims to prevent this loss. Both players trade their control performance objectives for the cost of their actions. A Mixed Strategy Nash Equilibrium (MSNE) of the game represents the allocation of the players' respective resources for attacking or protecting the network nodes. We analyze the dependence of a MSNE on the relative budgets of the players as well as on the important network nodes that must be preserved to achieve a desirable LQR performance. MSNE is computed using nonlinear programming. Results are validated using the New England power system model, and it is shown that reliable defense is feasible unless the cost of attack is very low or much smaller than the cost of protection per generator.

eess.SY↗

arXiv2018

Dynamic Modeling, Stability, and Control of Power Systems with Distributed Energy Resources

This article presents a suite of new control designs for next-generation electric smart grids. The future grid will consist of thousands of non-conventional renewable generation sources such as wind, solar, and energy storage. These new components are collectively referred to as distributed energy resources (DER). The article presents a comprehensive list of dynamic models for DERs, and shows their coupling with the conventional generators and loads. It then presents several innovative control designs that can be used for facilitating large-scale DER integration. Ideas from decentralized retrofit control and distributed sparsity-promoting optimal control are used for developing these designs, followed by illustrations on an IEEE power system test model.

eess.SY↗

arXiv2018

Coordinated Control of Energy Storage in Networked Microgrids under Unpredicted Load Demands

In this paper a nonlinear control design for power balancing in networked microgrids using energy storage devices is presented. Each microgrid is considered to be interfaced to the distribution feeder though a solid-state transformer (SST). The internal duty cycle based controllers of each SST ensures stable regulation of power commands during normal operation. But problem arises when a sudden change in load or generation occurs in any microgrid in a completely unpredicted way in between the time instants at which the SSTs receive their power setpoints. In such a case, the energy storage unit in that microgrid must produce or absorb the deficit power. The challenge lies in designing a suitable regulator for this purpose owing to the nonlinearity of the battery model and its coupling with the nonlinear SST dynamics. We design an input-output linearization based controller, and show that it guarantees closed-loop stability via a cascade connection with the SST model. The design is also extended to the case when multiple SSTs must coordinate their individual storage controllers to assist a given SST whose storage capacity is insufficient to serve the unpredicted load. The design is verified using the IEEE 34-bus distribution system with nine SST-driven microgrids.

eess.SY↗

arXiv2018

Control Inversion: A Clustering-Based Method for Distributed Wide-Area Control of Power Systems

Wide-area control (WAC) has been shown to be an effective tool for damping low-frequency oscillations in power systems. In the current state of art, WAC is challenged by two main factors - namely, scalability of design and complexity of implementation. In this paper we present a control design called control inversion that bypasses both of these challenges using the idea of clustering. The basic philosophy behind this method is to project the original power system model into a lower-dimensional state-space through clustering and aggregation of generator states, and then designing an LQR controller for the lower-dimensional model. This controller is finally projected back to the original coordinates for wide-area implementation. The main problem is, therefore, posed as finding the projection which best matches the closed-loop performance of the WAC controller with that of a reference LQR controller for damping low-frequency oscillations. We verify the effectiveness of the proposed design using the NPCC 48-machine power system model.

eess.SY↗

arXiv2018

Synchronization and Aggregation of Nonlinear Power Systems with Consideration of Bus Network Structures

We study nonlinear power systems consisting of generators, generator buses, and non-generator buses. First, looking at a generator and its bus' variables jointly, we introduce a synchronization concept for a pair of such joint generators and buses. We show that this concept is related to graph symmetry. Next, we extend, in two ways, the synchronization from a pair to a partition of all generators in the networks and show that they are related to either graph symmetry or equitable partitions. Finally, we show how an exact reduced model can be obtained by aggregating the generators and associated buses in the network when the original system is synchronized with respect to a partition, provided that the initial condition respects the partition. Additionally, the aggregation-based reduced model is again a power system.

eess.SY↗

arXiv2017

Optimal Control of Large-Scale Networks using Clustering Based Projections

In this paper we present a set of projection-based designs for constructing simplified linear quadratic regulator (LQR) controllers for large-scale network systems. When such systems have tens of thousands of states, the design of conventional LQR controllers becomes numerically challenging, and their implementation requires a large number of communication links. Our proposed algorithms bypass these difficulties by clustering the system states using structural properties of its closed-loop transfer matrix. The assignment of clusters is defined through a structured projection matrix P, which leads to a significantly lower-dimensional LQR design. The reduced-order controller is finally projected back to the original coordinates via an inverse projection. The problem is, therefore, posed as a model matching problem of finding the optimal set of clusters or P that minimizes the H2-norm of the error between the transfer matrix of the full-order network with the full-order LQR and that with the projected LQR. We derive a tractable relaxation for this model matching problem, and design a P that solves the relaxation. The design is shown to be implementable by a convenient, hierarchical two-layer control architecture, requiring far less number of communication links than full-order LQR.

eess.SY↗

arXiv2017

Hierarchical H2 Control of Large-Scale Network Dynamic Systems

Standard H2 optimal control of networked dynamic systems tend to become unscalable with network size. Structural constraints can be imposed on the design to counteract this problem albeit at the risk of making the solution non-convex. In this paper, we present a special class of structural constraints such that the H2 design satisfies a quadratic invariance condition, and therefore can be reformulated as a convex problem. This special class consists of structured and weighted projections of the input and output spaces. The choice of these projections can be optimized to match the closed-loop performance of the reformulated controller with that of the standard H2 controller. The advantage is that unlike the latter, the reformulated controller results in a hierarchical implementation which requires significantly lesser number of communication links, while also admitting model and controller reduction that helps the design to scale computationally. We illustrate our design with simulations of a 500-node network.

eess.SY↗

arXiv2017

Digital Grid: Transforming the Electric Power Grid into an Innovation Engine for the United States

The electric power grid is one of the largest and most complex infrastructures ever built by mankind. Modern civilization depends on it for industry production, human mobility, and comfortable living. However, many critical technologies such as the 60 Hz transformers were developed at the beginning of the 20th century and have changed very little since then.1 The traditional unidirectional power from the generation to the customer through the transmission-distribution grid has also changed nominally, but it no longer meets the need of the 21st century market energy customers. On one hand, 128m US residential customers pay $15B/per month for their utility bill, yet they have no option to select their energy supplier. In a world of where many traditional industries are transformed by digital Internet technology (Amazon, Ebay, Uber, Airbnb), the traditional electric energy market is lagging significantly behind. A move towards a true digital grid is needed. Such a digital grid requires a tight integration of the physical layer (energy and power) with digital and cyber information to allow an open and real time market akin to the world of e-commerce. Another major factor that is pushing for this radical transformation are the rapidly changing patterns in energy resources ownership and load flow. Driven by the decreasing cost in distributed solar, energy storage, electric vehicle, on site generation and microgrids, the high penetration of Distributed Energy Resource (DER) is shifting challenges substantially towards the edge of grid from the control point of view. The envisioned Digital Grid must facilitate the open competition and open innovation needed to accelerate of the adoption of new DER technologies while satisfying challenges in grid stability, data explosion and cyber security.

cs.CY↗