arXiv subjects

Ramij R. Hossain

Publications and source records attributed to Ramij R. Hossain.

11 recordsLinked to original sources

arXiv2026

Learning-Based Augmentation and Adaptation for Grid Sim-to-Real Model Discrepancy

Modern power systems can encounter increased discrepancy between the operators' simulation model and the actual true dynamics of the grid, driven by uncertainties caused by integration of new inverter-based resources (IBRs), large loads, unmodeled dynamics, parameter drifts, etc., to name a few. All of these impact the control room operations, where some critical oscillations may not be captured during the transient studies. To circumvent these issues, we propose a learning-augmented hybrid approach where the operator simulation model is supplemented with artificial intelligence (AI)-learned residual models using the phasor measurement unit (PMU)/ point-on-wave (PoW) based sensed trajectory data. The physics-based operator model provides interpretability and structural consistency, while the learned residual captures discrepancies caused by non-idealities. The learned model employs advanced neural architectures and consists of a backbone encoder and multi-head decoder layers for heterogeneous grid channels. Subsequently, we formulated a continual learning-motivated adaptation framework such that the baseline residual AI model can also be updated when the underlying real grid model changes in future conditions. Extensive numerical simulations are performed on the IEEE 68-bus benchmark model with a diverse set of disturbances, and different state-of-the-art predictive architectures involving recurrent learners, latent neural ODEs, and transformers are explored to demonstrate both residual learning and adaptation capabilities.

eess.SY↗

arXiv2025

Policy Gradient-Based EMT-in-the-Loop Learning to Mitigate Sub-Synchronous Control Interactions

This paper explores the development of learning-based tunable control gains using EMT-in-the-loop simulation framework (e.g., PSCAD interfaced with Python-based learning modules) to address critical sub-synchronous oscillations. Since sub-synchronous control interactions (SSCI) arise from the mis-tuning of control gains under specific grid configurations, effective mitigation strategies require adaptive re-tuning of these gains. Such adaptiveness can be achieved by employing a closed-loop, learning-based framework that considers the grid conditions responsible for such sub-synchronous oscillations. This paper addresses this need by adopting methodologies inspired by Markov decision process (MDP) based reinforcement learning (RL), with a particular emphasis on simpler deep policy gradient methods with additional SSCI-specific signal processing modules such as down-sampling, bandpass filtering, and oscillation energy dependent reward computations. Our experimentation in a real-world event setting demonstrates that the deep policy gradient based trained policy can adaptively compute gain settings in response to varying grid conditions and optimally suppress control interaction-induced oscillations.

eess.SY↗

arXiv2025

On the System Theoretic Offline Learning of Continuous-Time LQR with Exogenous Disturbances

We analyze offline designs of linear quadratic regulator (LQR) strategies with uncertain disturbances. First, we consider the scenario where the exogenous variable can be estimated in a controlled environment, and subsequently, consider a more practical and challenging scenario where it is unknown in a stochastic setting. Our approach builds on the fundamental learning-based framework of adaptive dynamic programming (ADP), combined with a Lyapunov-based analytical methodology to design the algorithms and derive sample-based approximations motivated from the Markov decision process (MDP)-based approaches. For the scenario involving non-measurable disturbances, we further establish stability and convergence guarantees for the learned control gains under sample-based approximations. The overall methodology emphasizes simplicity while providing rigorous guarantees. Finally, numerical experiments focus on the intricacies and validations for the design of offline continuous-time LQR with exogenous disturbances.

eess.SY↗

arXiv2025

Managing Risks from Large Digital Loads Using Coordinated Grid-Forming Storage Network

Anticipated rapid growth of large digital load, driven by artificial intelligence (AI) data centers, is poised to increase uncertainty and large fluctuations in consumption, threatening the stability, reliability, and security of the energy infrastructure. Conventional measures taken by grid planners and operators to ensure stable and reliable integration of new resources are either cost-prohibitive (e.g., transmission upgrades) or ill-equipped (e.g., generation control) to resolve the unique challenges brought on by AI Data Centers (e.g., extreme load transients). In this work, we explore the feasibility of coordinating and managing available flexibility in the grid, in terms of grid-forming storage units, to ensure stable and reliable integration of AI Data Centers without the need for costly grid upgrades. Recently developed bi-layered coordinated control strategies -- involving fast-acting, local, autonomous, control at the storage to maintain transient safety in voltage and frequency at the point-of-interconnection, and a slower, coordinated (consensus) control to restore normal operating condition in the grid -- are used in the case studies. A comparison is drawn between broadly two scenarios: a network of coordinated, smaller, distributed storage vs. larger storage installations collocated with large digital loads. IEEE 68-bus network is used for the case studies, with large digital load profiles drawn from the MIT Supercloud Dataset.

eess.SY↗

arXiv2025

Physics-Informed Learning of Proprietary Inverter Models for Grid Dynamic Studies

This letter develops a novel physics-informed neural ordinary differential equations-based framework to emulate the proprietary dynamics of the inverters -- essential for improved accuracy in grid dynamic simulations. In current industry practice, the original equipment manufacturers (OEMs) often do not disclose the exact internal controls and parameters of the inverters, posing significant challenges in performing accurate dynamic simulations and other relevant studies, such as gain tunings for stability analysis and controls. To address this, we propose a Physics-Informed Latent Neural ODE Model (PI-LNM) that integrates system physics with neural learning layers to capture the unmodeled behaviors of proprietary units. The proposed method is validated using a grid-forming inverter (GFM) case study, demonstrating improved dynamic simulation accuracy over approaches that rely solely on data-driven learning without physics-based guidance.

eess.SY↗

arXiv2023

Resilient Control of Networked Microgrids using Vertical Federated Reinforcement Learning: Designs and Real-Time Test-Bed Validations

Improving system-level resiliency of networked microgrids is an important aspect with increased population of inverter-based resources (IBRs). This paper (1) presents resilient control design in presence of adversarial cyber-events, and proposes a novel federated reinforcement learning (Fed-RL) approach to tackle (a) model complexities, unknown dynamical behaviors of IBR devices, (b) privacy issues regarding data sharing in multi-party-owned networked grids, and (2) transfers learned controls from simulation to hardware-in-the-loop test-bed, thereby bridging the gap between simulation and real world. With these multi-prong objectives, first, we formulate a reinforcement learning (RL) training setup generating episodic trajectories with adversaries (attack signal) injected at the primary controllers of the grid forming (GFM) inverters where RL agents (or controllers) are being trained to mitigate the injected attacks. For networked microgrids, the horizontal Fed-RL method involving distinct independent environments is not appropriate, leading us to develop vertical variant Federated Soft Actor-Critic (FedSAC) algorithm to grasp the interconnected dynamics of networked microgrid. Next, utilizing OpenAI Gym interface, we built a custom simulation set-up in GridLAB-D/HELICS co-simulation platform, named Resilient RL Co-simulation (ResRLCoSIM), to train the RL agents with IEEE 123-bus benchmark test systems comprising 3 interconnected microgrids. Finally, the learned policies in simulation world are transferred to the real-time hardware-in-the-loop test-bed set-up developed using high-fidelity Hypersim platform. Experiments show that the simulator-trained RL controllers produce convincing results with the real-time test-bed set-up, validating the minimization of sim-to-real gap.

eess.SY↗

arXiv2022

Enhancing Cyber Resilience of Networked Microgrids using Vertical Federated Reinforcement Learning

This paper presents a novel federated reinforcement learning (Fed-RL) methodology to enhance the cyber resiliency of networked microgrids. We formulate a resilient reinforcement learning (RL) training setup which (a) generates episodic trajectories injecting adversarial actions at primary control reference signals of the grid forming (GFM) inverters and (b) trains the RL agents (or controllers) to alleviate the impact of the injected adversaries. To circumvent data-sharing issues and concerns for proprietary privacy in multi-party-owned networked grids, we bring in the aspects of federated machine learning and propose a novel Fed-RL algorithm to train the RL agents. To this end, the conventional horizontal Fed-RL approaches using decoupled independent environments fail to capture the coupled dynamics in a networked microgrid, which leads us to propose a multi-agent vertically federated variation of actor-critic algorithms, namely federated soft actor-critic (FedSAC) algorithm. We created a customized simulation setup encapsulating microgrid dynamics in the GridLAB-D/HELICS co-simulation platform compatible with the OpenAI Gym interface for training RL agents. Finally, the proposed methodology is validated with numerical examples of modified IEEE 123-bus benchmark test systems consisting of three coupled microgrids.

cs.LG↗

arXiv2022

Efficient Learning of Voltage Control Strategies via Model-based Deep Reinforcement Learning

This article proposes a model-based deep reinforcement learning (DRL) method to design emergency control strategies for short-term voltage stability problems in power systems. Recent advances show promising results in model-free DRL-based methods for power systems, but model-free methods suffer from poor sample efficiency and training time, both critical for making state-of-the-art DRL algorithms practically applicable. DRL-agent learns an optimal policy via a trial-and-error method while interacting with the real-world environment. And it is desirable to minimize the direct interaction of the DRL agent with the real-world power grid due to its safety-critical nature. Additionally, state-of-the-art DRL-based policies are mostly trained using a physics-based grid simulator where dynamic simulation is computationally intensive, lowering the training efficiency. We propose a novel model-based-DRL framework where a deep neural network (DNN)-based dynamic surrogate model, instead of a real-world power-grid or physics-based simulation, is utilized with the policy learning framework, making the process faster and sample efficient. However, stabilizing model-based DRL is challenging because of the complex system dynamics of large-scale power systems. We solved these issues by incorporating imitation learning to have a warm start in policy learning, reward-shaping, and multi-step surrogate loss. Finally, we achieved 97.5% sample efficiency and 87.7% training efficiency for an application to the IEEE 300-bus test system.

eess.SY↗

arXiv2022

Data-Driven Linear Koopman Embedding for Networked Systems: Model-Predictive Grid Control

This paper presents a data-learned linear Koopman embedding of nonlinear networked dynamics and uses it to enable real-time model predictive emergency voltage control in a power network. The approach involves a novel data-driven ``basis-dictionary free" lifting of the system dynamics into a higher dimensional linear space over which an MPC (model predictive control) is exercised, making it both scalable and rapid for practical real-time implementation. A Koopman-inspired deep neural network (KDNN) encoder-decoder architecture for the linear embedding of the underlying dynamics under distributed controls is presented, in which the end-to-end components of the KDNN comprising of a triple of transforms is learned from the system trajectory data in one go: A Neural Network (NN)-based lifting to a higher dimension, a linear dynamics within that higher dimension, and an NN-based projection to the original space. This data-learned approach relieves the burden of the ad-hoc selection of the nonlinear basis functions (e.g., polynomial or radial) used in conventional approaches for lifting to higher dimensional linear space. We validate the efficacy and robustness of the approach via application to the standard IEEE 39-bus system.

eess.SY↗

arXiv2022

Distributed-MPC with Data-Driven Estimation of Bus Admittance Matrix in Voltage Control

This article presents a distributed model-predictive control (MPC) design for real-time voltage control in power systems, including an online method to estimate the bus admittance matrix $\mathbf{Y}$ to let it be time-varying and unknown a priori. The prevalent control designs are either (a) centralized, providing optimal solutions but less scalable and susceptible to single-point failures/attacks, or (b) decentralized or localized, having increased scalability and attack resilience but are suboptimal. The proposed distributed solution offers the attractive features of both methodologies, where neighboring nodes share state information to attain a globally optimal solution. In addition, the presented framework provides a data-driven estimation of Y to circumvent the challenging issue of acquiring accurate knowledge of the line impedance (required to form Y). We first introduce the centralized version of the predictive voltage control problem and then transfer it to a distributed version. The distributed version is solved via the alternating direction method of multipliers (ADMM), using only local measurements and communication leveraging the graph structure of the power system. The proposed framework is resilient to prediction uncertainty, modeling error, and communication link failure, and the in-built redundancy within the proposed framework supports anomaly detection in cyberattacks. We validate the proposed methodology for IEEE-30 bus, IEEE-57 bus transmission systems, and IEEE-123 bus distribution systems.

eess.SY↗

arXiv2021

Data-Driven Pole Placement in LMI Regions with Robustness Constraints

This paper proposes a robust learning methodology to place the closed-loop poles in desired convex regions in the complex plane. We considered the system state and input matrices to be unknown and can only use the measurements of the system trajectories. The closed-loop pole placement problem in the linear matrix inequality (LMI) regions is considered a classic robust control problem; however, that requires knowledge about the state and input matrices of the linear system. We bring in ideas from the behavioral system theory and persistency of excitation condition-based fundamental lemma to develop a data-driven counterpart that satisfies multiple closed-loop robustness specifications, such as $\mathcal{D}$-stability and mixed $H_2/H_{\infty}$ performance specifications. Our formulations lead to data-driven semi-definite programs (SDPs) that are coupled with sufficient theoretical guarantees. We validate the theoretical results with numerical simulations on a third-order dynamic system.

eess.SY↗