arXiv subjects

Wenqi Cui

Publications and source records attributed to Wenqi Cui.

24 records · Page 2Linked to original sources

arXiv2022

Stable Reinforcement Learning for Optimal Frequency Control: A Distributed Averaging-Based Integral Approach

Frequency control plays a pivotal role in reliable power system operations. It is conventionally performed in a hierarchical way that first rapidly stabilizes the frequency deviations and then slowly recovers the nominal frequency. However, as the generation mix shifts from synchronous generators to renewable resources, power systems experience larger and faster frequency fluctuations due to the loss of inertia, which adversely impacts the frequency stability. This has motivated active research in algorithms that jointly address frequency degradation and economic efficiency in a fast timescale, among which the distributed averaging-based integral (DAI) control is a notable one that sets controllable power injections directly proportional to the integrals of frequency deviation and economic inefficiency signals. Nevertheless, DAI do not typically consider the transient performance of the system following power disturbances and has been restricted to quadratic operational cost functions. This manuscript aims to leverage nonlinear optimal controllers to simultaneously achieve optimal transient frequency control and find the most economic power dispatch for frequency restoration. To this end, we integrate reinforcement learning (RL) to the classic DAI, which results in RL-DAI. Specifically, we use RL to learn a neural network-based control policy mapping from the integral variables of DAI to the controllable power injections which provides optimal transient frequency control, while DAI inherently ensures the frequency restoration and optimal economic dispatch. Compared to existing methods, we provide provable guarantees on the stability of the learned controllers and extend allowable cost functions to a much larger class. Simulations on the 39-bus New England system illustrate our results.

eess.SY↗

arXiv2021

Reinforcement Learning for Optimal Primary Frequency Control: A Lyapunov Approach

As more inverter-connected renewable resources are integrated into the grid, frequency stability may degrade because of the reduction in mechanical inertia and damping. A common approach to mitigate this degradation in performance is to use the power electronic interfaces of the renewable resources for primary frequency control. Since inverter-connected resources can realize almost arbitrary responses to frequency changes, they are not limited to reproducing the linear droop behaviors. To fully leverage their capabilities, reinforcement learning (RL) has emerged as a popular method to design nonlinear controllers to optimize a host of objective functions. Because both inverter-connected resources and synchronous generators would be a significant part of the grid in the near and intermediate future, the learned controller of the former should be stabilizing with respect to the nonlinear dynamics of the latter. To overcome this challenge, we explicitly engineer the structure of neural network-based controllers such that they guarantee system stability by construction, through the use of a Lyapunov function. A recurrent neural network architecture is used to efficiently train the controllers. The resulting controllers only use local information and outperform optimal linear droop as well as other state-of-the-art learning approaches.

eess.SY↗

arXiv2021

Decentralized Safe Reinforcement Learning for Voltage Control

Inverter-based distributed energy resources provide the possibility for fast time-scale voltage control by quickly adjusting their reactive power. The power-electronic interfaces allow these resources to realize almost arbitrary control law, but designing these decentralized controllers is nontrivial. Reinforcement learning (RL) approaches are becoming increasingly popular to search for policy parameterized by neural networks. It is difficult, however, to enforce that the learned controllers are safe, in the sense that they may introduce instabilities into the system. This paper proposes a safe learning approach for voltage control. We prove that the system is guaranteed to be exponentially stable if each controller satisfies certain Lipschitz constraints. The set of Lipschitz bound is optimized to enlarge the search space for neural network controllers. We explicitly engineer the structure of neural network controllers such that they satisfy the Lipschitz constraints by design. A decentralized RL framework is constructed to train local neural network controller at each bus in a model-free setting.

eess.SY↗

arXiv2021

Design of a Graphical User Interface for Few-Shot Machine Learning Classification of Electron Microscopy Data

The recent growth in data volumes produced by modern electron microscopes requires rapid, scalable, and flexible approaches to image segmentation and analysis. Few-shot machine learning, which can richly classify images from a handful of user-provided examples, is a promising route to high-throughput analysis. However, current command-line implementations of such approaches can be slow and unintuitive to use, lacking the real-time feedback necessary to perform effective classification. Here we report on the development of a Python-based graphical user interface that enables end users to easily conduct and visualize the output of few-shot learning models. This interface is lightweight and can be hosted locally or on the web, providing the opportunity to reproducibly conduct, share, and crowd-source few-shot analyses.

cond-mat.mtrl-sci↗

arXiv2021

Lyapunov-Regularized Reinforcement Learning for Power System Transient Stability

Transient stability of power systems is becoming increasingly important because of the growing integration of renewable resources. These resources lead to a reduction in mechanical inertia but also provide increased flexibility in frequency responses. Namely, their power electronic interfaces can implement almost arbitrary control laws. To design these controllers, reinforcement learning (RL) has emerged as a powerful method in searching for optimal non-linear control policy parameterized by neural networks. A key challenge is to enforce that a learned controller must be stabilizing. This paper proposes a Lyapunov regularized RL approach for optimal frequency control for transient stability in lossy networks. Because the lack of an analytical Lyapunov function, we learn a Lyapunov function parameterized by a neural network. The losses are specially designed with respect to the physical power system. The learned neural Lyapunov function is then utilized as a regularization to train the neural network controller by penalizing actions that violate the Lyapunov conditions. Case study shows that introducing the Lyapunov regularization enables the controller to be stabilizing and achieve smaller losses.

eess.SY↗

arXiv2019

Multi-state Operating Reserve Model of Aggregate Thermostatically-Controlled-Loads for Power System Short-Term Reliability Evaluation

Thermostatically-controlled-loads (TCLs) have been regarded as a good candidate for maintaining the power system reliability by providing operating reserve. The short-term reliability evaluation of power systems, which is essential for power system operators in decision making to secure the system real time balancing, calls for the accurate modelling of operating reserve provided by TCLs. However, the particular characteristics of TCLs make their dynamic response different from the traditional generating units, resulting in difficulties to accurately represent the reliability of operating reserve provided by TCLs with conventional reliability model. This paper proposes a novel multi-state reliability model of operating reserve provided by TCLs considering their dynamic response during the reserve deployment process. An analytical model for characterizing dynamics of operating reserve provided by TCLs is firstly developed based on the migration of TCLs' room temperature within the temperature hysteresis band. Then, considering the stochastic consumers' behaviour and ambient temperature, the probability distribution functions of reserve capacity provided by TCLs are obtained by cumulants. On this basis, the states of reserve capacity and the corresponding probabilities at each time instant are obtained for representing the reliability of operating reserve provided by TCLs in the LZ-transform approach. Case studies are conducted to validate the proposed technique.

math.OC↗