arXiv subjects

Baosen Zhang

Publications and source records attributed to Baosen Zhang.

At least 55 records · Page 3Linked to original sources

arXiv2023

A Frequency Domain Approach to Predict Power System Transients

The dynamics of power grids are governed by a large number of nonlinear differential and algebraic equations (DAEs). To safely operate the system, operators need to check that the states described by these DAEs stay within prescribed limits after various potential faults. However, current numerical solvers of DAEs are often too slow for real-time system operations. In addition, detailed system parameters are often not exactly known. Machine learning approaches have been proposed to reduce computational efforts, but existing methods generally suffer from overfitting and failures to predict unstable behaviors. This paper proposes a novel framework to predict power system transients by learning in the frequency domain. The intuition is that although the system behavior is complex in the time domain, there are relatively few dominant modes in the frequency domain. Therefore, we learn to predict by constructing neural networks with Fourier transform and filtering layers. System topology and fault information are encoded by taking a multi-dimensional Fourier transform, allowing us to leverage the fact that the trajectories are sparse both in time and spatial frequencies. We show that the proposed approach does not need detailed system parameters, greatly speeds up prediction computations and is highly accurate for different fault types.

eess.SY↗

arXiv2022

Efficient Reinforcement Learning Through Trajectory Generation

A key barrier to using reinforcement learning (RL) in many real-world applications is the requirement of a large number of system interactions to learn a good control policy. Off-policy and Offline RL methods have been proposed to reduce the number of interactions with the physical environment by learning control policies from historical data. However, their performances suffer from the lack of exploration and the distributional shifts in trajectories once controllers are updated. Moreover, most RL methods require that all states are directly observed, which is difficult to be attained in many settings. To overcome these challenges, we propose a trajectory generation algorithm, which adaptively generates new trajectories as if the system is being operated and explored under the updated control policies. Motivated by the fundamental lemma for linear systems, assuming sufficient excitation, we generate trajectories from linear combinations of historical trajectories. For linear feedback control, we prove that the algorithm generates trajectories with the exact distribution as if they are sampled from the real system using the updated control policy. In particular, the algorithm extends to systems where the states are not directly observed. Experiments show that the proposed method significantly reduces the number of sampled data needed for RL algorithms.

cs.LG↗

arXiv2022

State-of-Charge Aware EV Charging

Recent proliferation in electric vehicles (EVs) are posing profound impacts over the operation of electrical grids. In particular, due to the physical constraints on charging stations' capacity and uncertainty in charging demand, it becomes an emerging challenge to design high performance scheduling algorithms to better serve charging sessions. In this paper, we design a predictive charging controller by actively incorporating each EV's state-of-charge (SOC) information, which has strong effects on the utilization of dispatchable power during peak hours. Simulation results on both synthetic and real-world EV session and charging demand data demonstrate the proposed algorithm's benefits on maximizing charging throughput and achieving higher rate of feasible charging sessions while satisfying battery and station physical constraints at the same time.

eess.SY↗

arXiv2022

Optimal Inverter-Based Resources Placement in Low-Inertia Power Systems

An increase in the integration of Inverter Based Resources (IBRs) to the electric grid, will lead to a corresponding decrease in the amount of connected synchronous generators, resulting in a decline in the available rotational inertia system-wide. This can lead to pronounced frequency deviations when there are disturbances and faults in the grid. This decline in available rotational inertia can be compensated for by fast acting IBRs participating in frequency response services, by rapidly injecting into or removing power from the grid. Currently, there are still relative small number of sizable IBRs in the grid. Therefore, the placement of the IBRs in the system, as well as the inverter configuration type and controller, will have a material impact on the frequency response of the grid. In this work, we present an optimal placement algorithm that maximizes the benefits of utilizing IBRs in providing frequency response services. That is, we minimize the overall system frequency deviation while using a minimal amount of electric power injection from the IBRs. The proposed algorithm uses the resistance distance to place the IBRs at nodes that are central to the rest of the nodes in the network thus minimizing the distance of power flow. The proposed greedy algorithm achieves a near optimal performance and relies on the supermodularity of the resistance distances. We validate the performance of the placement algorithm on three IEEE test systems of varying sizes, by comparing its performance to an exhaustive search algorithm. We further evaluate the performance of the placed IBRs on an IEEE test system, to determine their impact on frequency stability. The IBRs are configured in a grid-forming mode and equipped with a model predictive control (MPC)-based inverter power control.

eess.SY↗

arXiv2022

Safe and Efficient Model Predictive Control Using Neural Networks: An Interior Point Approach

Model predictive control (MPC) provides a useful means for controlling systems with constraints, but suffers from the computational burden of repeatedly solving an optimization problem in real time. Offline (explicit) solutions for MPC attempt to alleviate real time computational challenges using either multiparametric programming or machine learning. The multiparametric approaches are typically applied to linear or quadratic MPC problems, while learning-based approaches can be more flexible and are less memory-intensive. Existing learning-based approaches offer significant speedups, but the challenge becomes ensuring constraint satisfaction while maintaining good performance. In this paper, we provide a neural network parameterization of MPC policies that explicitly encodes the constraints of the problem. By exploring the interior of the MPC feasible set in an unsupervised learning paradigm, the neural network finds better policies faster than projection-based methods and exhibits substantially shorter solve times. We use the proposed policy to solve a robust MPC problem, and demonstrate the performance and computational gains on a standard test system.

eess.SY↗

arXiv2022

Cyber-Physical Attack Leveraging Subsynchronous Resonance

This paper discusses how a cyber attack could take advantage of torsional resonances in the shaft of turbo-generators to inflict severe physical damage to a power system. If attackers were able to take over the control of a battery energy storage device, they could modulate the injection of this device at a frequency that matches one of the sub-synchronous resonance frequencies of a generator. Small changes in injection might be sufficient to excite one of these mechanical resonances, resulting in metal fatigue and ultimately a catastrophic failure in the shaft of the generator. Using a state-space model of the electromechanical system, the paper develops transfer functions linking the magnitude of the malicious injections to the magnitude of oscillations in the speed and angle of the various masses connected to the shaft. Numerical results from a two-area power system demonstrate the existence of vulnerable frequencies and show that damaging mechanical oscillations can be triggered without causing easily detectable signals at the generator terminals.

eess.SY↗

arXiv2022

Equilibrium-Independent Stability Analysis for Distribution Systems with Lossy Transmission Lines

Power distribution systems are becoming much more active with increased penetration of distributed energy resources. Because of the intermittent nature of these resources, the stability of distribution systems under large disturbances and time-varying conditions is becoming a key issue in practical operations. Because the transmission lines in distribution systems are lossy, standard approaches in power system stability analysis do not readily apply and the understanding of transient stability remains open even for simplified models. This paper proposes a novel equilibrium-independent transient stability analysis of distribution systems with lossy lines. We certify network-level stability by breaking the network into subsystems, and by looking at the equilibrium-independent passivity of each subsystem, the network stability is certified through a diagonal stability property of the interconnection matrix. This allows the analysis scale to large networked systems with time-varying equilibria. The proposed method gracefully extrapolates between lossless and lossy systems, and provides a simple yet effective approach to optimize control efforts with guaranteed stability regions. Case studies verify that the proposed method is much less conservative than existing approaches and also scales to large systems.

eess.SY↗

arXiv2022

Stable Reinforcement Learning for Optimal Frequency Control: A Distributed Averaging-Based Integral Approach

Frequency control plays a pivotal role in reliable power system operations. It is conventionally performed in a hierarchical way that first rapidly stabilizes the frequency deviations and then slowly recovers the nominal frequency. However, as the generation mix shifts from synchronous generators to renewable resources, power systems experience larger and faster frequency fluctuations due to the loss of inertia, which adversely impacts the frequency stability. This has motivated active research in algorithms that jointly address frequency degradation and economic efficiency in a fast timescale, among which the distributed averaging-based integral (DAI) control is a notable one that sets controllable power injections directly proportional to the integrals of frequency deviation and economic inefficiency signals. Nevertheless, DAI do not typically consider the transient performance of the system following power disturbances and has been restricted to quadratic operational cost functions. This manuscript aims to leverage nonlinear optimal controllers to simultaneously achieve optimal transient frequency control and find the most economic power dispatch for frequency restoration. To this end, we integrate reinforcement learning (RL) to the classic DAI, which results in RL-DAI. Specifically, we use RL to learn a neural network-based control policy mapping from the integral variables of DAI to the controllable power injections which provides optimal transient frequency control, while DAI inherently ensures the frequency restoration and optimal economic dispatch. Compared to existing methods, we provide provable guarantees on the stability of the learned controllers and extend allowable cost functions to a much larger class. Simulations on the 39-bus New England system illustrate our results.

eess.SY↗

arXiv2022

Computationally Efficient Safe Reinforcement Learning for Power Systems

We propose a computationally efficient approach to safe reinforcement learning (RL) for frequency regulation in power systems with high levels of variable renewable energy resources. The approach draws on set-theoretic control techniques to craft a neural network-based control policy that is guaranteed to satisfy safety-critical state constraints, without needing to solve a model predictive control or projection problem in real time. By exploiting the properties of robust controlled-invariant polytopes, we construct a novel, closed-form "safety-filter" that enables end-to-end safe learning using any policy gradient-based RL algorithm. We then apply the safety filter in conjunction with the deep deterministic policy gradient (DDPG) algorithm to regulate frequency in a modified 9-bus power system, and show that the learned policy is more cost-effective than robust linear feedback control techniques while maintaining the same safety guarantee. We also show that the proposed paradigm outperforms DDPG augmented with constraint violation penalties.

eess.SY↗

arXiv2022

Data-Driven Modeling of Aggregate Flexibility under Uncertain and Non-Convex Load Models

Bundling a large number of distributed energy resources through a load aggregator has been advocated as an effective means to integrate such resources into whole-sale energy markets. To ease market clearing, system operators allow aggregators to submit bidding models of simple prespecified polytopic shapes. Aggregators need to carefully design and commit to a polytope that best captures their energy flexibility along a day-ahead scheduling horizon. This work puts forth a model-informed data-based optimal flexibility design for aggregators, which deals with the time-coupled, uncertain, and non-convex models of individual loads. The proposed solution first generates efficiently a labeled dataset of (non)-disaggregatable schedules. The feasible set of the aggregator is then approximated by an ellipsoid upon training a convex quadratic classifier using the labeled dataset. The ellipsoid is subsequently inner approximated by a polytope. Using Farkas lemma, the obtained polytope is finally inner approximated by the polytopic shape dictated by the market. Numerical tests show the effectiveness of the proposed flexibility design framework for designing the feasible sets of small- and large-sized aggregators coordinating solar photovoltaics, thermostatically-controlled loads, batteries, and electric vehicles. The tests further demonstrate that it is crucial for the aggregator to consider time-coupling and uncertainties in optimal flexibility design.

eess.SY↗

arXiv2021

Reinforcement Learning for Optimal Primary Frequency Control: A Lyapunov Approach

As more inverter-connected renewable resources are integrated into the grid, frequency stability may degrade because of the reduction in mechanical inertia and damping. A common approach to mitigate this degradation in performance is to use the power electronic interfaces of the renewable resources for primary frequency control. Since inverter-connected resources can realize almost arbitrary responses to frequency changes, they are not limited to reproducing the linear droop behaviors. To fully leverage their capabilities, reinforcement learning (RL) has emerged as a popular method to design nonlinear controllers to optimize a host of objective functions. Because both inverter-connected resources and synchronous generators would be a significant part of the grid in the near and intermediate future, the learned controller of the former should be stabilizing with respect to the nonlinear dynamics of the latter. To overcome this challenge, we explicitly engineer the structure of neural network-based controllers such that they guarantee system stability by construction, through the use of a Lyapunov function. A recurrent neural network architecture is used to efficiently train the controllers. The resulting controllers only use local information and outperform optimal linear droop as well as other state-of-the-art learning approaches.

eess.SY↗

arXiv2021

Learning to Solve the AC Optimal Power Flow via a Lagrangian Approach

Using deep neural networks to predict the solutions of AC optimal power flow (ACOPF) problems has been an active direction of research. However, because the ACOPF is nonconvex, it is difficult to construct a good data set that contains mostly globally optimal solutions. To overcome the challenge that the training data may contain suboptimal solutions, we propose a Lagrangian-based approach. First, we use a neural network to learn the dual variables of the ACOPF problem. Then we use a second neural network to predict solutions of the partial Lagrangian from the predicted dual variables. Since the partial Lagrangian has a much better optimization landscape, we use the predicted solutions from the neural network as a warm start for the ACOPF problem. Using standard and modified IEEE 22-bus, 39-bus, and 118-bus networks, we show that our approach is able to obtain the globally optimal cost even when the training data is mostly comprised of suboptimal solutions.

eess.SY↗

arXiv2021

Decentralized Safe Reinforcement Learning for Voltage Control

Inverter-based distributed energy resources provide the possibility for fast time-scale voltage control by quickly adjusting their reactive power. The power-electronic interfaces allow these resources to realize almost arbitrary control law, but designing these decentralized controllers is nontrivial. Reinforcement learning (RL) approaches are becoming increasingly popular to search for policy parameterized by neural networks. It is difficult, however, to enforce that the learned controllers are safe, in the sense that they may introduce instabilities into the system. This paper proposes a safe learning approach for voltage control. We prove that the system is guaranteed to be exponentially stable if each controller satisfies certain Lipschitz constraints. The set of Lipschitz bound is optimized to enlarge the search space for neural network controllers. We explicitly engineer the structure of neural network controllers such that they satisfy the Lipschitz constraints by design. A decentralized RL framework is constructed to train local neural network controller at each bus in a model-free setting.

eess.SY↗

arXiv2021

An Iterative Approach to Improving Solution Quality for AC Optimal Power Flow Problems

The existence of multiple solutions to AC optimal power flow (ACOPF) problems has been noted for decades. Existing solvers are generally successful in finding local solutions, which satisfy first and second order optimality conditions, but may not be globally optimal. In this paper, we propose a simple iterative approach to improve the quality of solutions to ACOPF problems. First, we call an existing solver for the ACOPF problem. From the solution and the associated dual variables, we form a partial Lagrangian. Then we optimize this partial Lagrangian and use its solution as a warm start to call the solver again for the ACOPF problem. By repeating this process, we can iteratively improve the solution quality, moving from local solutions to global ones. We show the effectiveness of our algorithm on standard IEEE networks. The simulation results show that our algorithm can escape from local solutions to achieve global optimums within a few iterations.

eess.SY↗

arXiv2021

Lyapunov-Regularized Reinforcement Learning for Power System Transient Stability

Transient stability of power systems is becoming increasingly important because of the growing integration of renewable resources. These resources lead to a reduction in mechanical inertia but also provide increased flexibility in frequency responses. Namely, their power electronic interfaces can implement almost arbitrary control laws. To design these controllers, reinforcement learning (RL) has emerged as a powerful method in searching for optimal non-linear control policy parameterized by neural networks. A key challenge is to enforce that a learned controller must be stabilizing. This paper proposes a Lyapunov regularized RL approach for optimal frequency control for transient stability in lossy networks. Because the lack of an analytical Lyapunov function, we learn a Lyapunov function parameterized by a neural network. The losses are specially designed with respect to the physical power system. The learned neural Lyapunov function is then utilized as a regularization to train the neural network controller by penalizing actions that violate the Lyapunov conditions. Case study shows that introducing the Lyapunov regularization enables the controller to be stabilizing and achieve smaller losses.

eess.SY↗

arXiv2021

An Iterative Approach to Finding Global Solutions of AC Optimal Power Flow Problems

The existence of multiple solutions to AC optimal power flow (ACOPF) problems has been noted for decades. Existing solvers are generally successful in finding local solutions, which are stationary points but may not be globally optimal. In this paper, we propose a simple iterative approach to find globally optimal solutions to ACOPF problems. First, we call an existing solver for the ACOPF problem. From the solution and the associated dual variables, we form a partial Lagrangian. Then we optimize this partial Lagrangian and use its solution as a warm start to call the solver again for the ACOPF problem. By repeating this process, we can iteratively improve the solution quality, moving from local solutions to global ones. We show the effectiveness our algorithm on standard IEEE networks. The simulation results show that our algorithm can escape from local solutions to achieve global optimums within a few iterations.

eess.SY↗

arXiv2020

Consensus-Based Set-Theoretic Control in Power Systems

Set-theoretic control is a useful technique for dealing with the uncertainty introduced into power systems by renewable energy resources. Although set operations are computationally expensive in large systems, distributed approaches serve as a remedy. In this paper, we propose a novel consensus-based approach for set-theoretic frequency control in power systems. A robust controlled-invariant set (RCI) for the system is generated by composing RCIs for each bus in the network. The process of generating these sets uses a consensus-based approach in order to facilitate discovery of mutually compatible subsystem RCIs. Each bus seeks to maximize the size of its own RCI while treating the effects of coupling as an unknown-but-bounded disturbance. The consensus routine, which demonstrates linear convergence, is embedded into a backwards reachability analysis of initial safe sets. Results for a 9-bus test case show that simple model predictive controllers associated with the resulting RCIs maintain safe operation when the system is subjected to worst case (adversarial) fluctuations in net demand, where conventional controllers are shown to fail.

eess.SY↗

arXiv2020

Energy Storage Arbitrage in Real-Time Markets via Reinforcement Learning

In this paper, we derive a temporal arbitrage policy for storage via reinforcement learning. Real-time price arbitrage is an important source of revenue for storage units, but designing good strategies have proven to be difficult because of the highly uncertain nature of the prices. Instead of current model predictive or dynamic programming approaches, we use reinforcement learning to design an optimal arbitrage policy. This policy is learned through repeated charge and discharge actions performed by the storage unit through updating a value matrix. We design a reward function that does not only reflect the instant profit of charge/discharge decisions but also incorporate the history information. Simulation results demonstrate that our designed reward function leads to significant performance improvement compared with existing algorithms.

eess.SY↗