Searcharxiv⌕ Search

arXiv · 2611.17612

Battery Energy Storage Sizing in PV Ramp-Rate Smoothing using Constraint-Aware Reinforcement Learning-Based Controller

Abstract

This paper presents a constraint-aware reinforcement learning (CARL) controller for battery energy storage system (BESS) sizing and photovoltaic (PV) ramp-rate smoothing. Rapid PV power fluctuations caused by cloud transitions can introduce operational and stability challenges in power systems. To mitigate these impacts, grid codes impose ramp-rate limits (RRL) on PV power injected to the grid. BESSs can support such grid codes by absorbing or supplying power during rapid variations. However, the extent to which BESSs can reduce PV-induced variability depends on (i) the BESS power and energy capacities; and (ii) the adopted controller. Thus, BESS sizing and controller design are coupled. Conventional moving-average control can increase battery usage due to filter-induced lag, while ramp-rate control is reactive and does not account for the long-term impact of actions on battery state of charge and future ramp events. Optimization-based methods can reduce BESS requirements by anticipating future PV variations, but their performance depends on accurate PV forecasts. To address these limitations, the CARL controller learns a causal state-feedback policy from historical PV data without requiring forecasts during deployment. Ramp-rate, converter-power, and state-of-charge constraints are incorporated into a causal feasibility interval that guarantees feasible actions by definition. A full-horizon input-convex neural network (ICNN)-based optimization is developed as a noncausal benchmark for the minimum achievable BESS requirements. Evaluation using three real-world PV datasets shows that, at a 10\%/min ramp-rate limit, CARL reduces peak BESS power by 8.7--10.4\% and energy capacity by 47.5--62.6\% relative to conventional ramp-rate control while maintaining the grid-codes compliance.

Explore related subjects

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Arash Omidi, Tanmay Mishra, Mads R. Almassalkhi. 2026-11-13. Battery Energy Storage Sizing in PV Ramp-Rate Smoothing using Constraint-Aware Reinforcement Learning-Based Controller. https://searcharxiv.com/abs/2611.17612

Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite Searcharxiv by Indiciae: BibTeX · RIS.

KEEP EXPLORING

Related papers

Mean-Field Learning for Storage Aggregation

Distributed energy storage devices can be aggregated to provide operational flexibility for power systems. Their efficient integration into system-level optimization calls for an accurate and computationally tractable surrogate that jointly represents aggregate flexibility and operating costs. However, constructing such a surrogate is challenging due to device heterogeneity, nonconvex operating constraints, and high-dimensional power trajectories. To address these challenges, this paper develops a mean-field learning framework for storage aggregation. Using random-set expectations, we show that the joint representation of aggregate flexibility and operating costs converges almost surely to a deterministic convex mean-field limit. This limit provides a learning benchmark for storage aggregation, against which we bound the approximation error of any convex surrogate. These theoretical results motivate a convex virtual-battery surrogate with physically interpretable parameters. We learn these parameters through a bilevel optimization problem that matches the optimal values of the surrogate and the storage fleet when both respond to a set of price scenarios. We derive closed-form gradients to solve this problem efficiently. Case studies illustrate the theoretical results and demonstrate the framework's approximation accuracy, computational efficiency, and robustness to device heterogeneity.

eess.SY↗

HyperKKL: Enabling Non-Autonomous State Estimation through Dynamic Weight Conditioning

This paper proposes HyperKKL, a novel learning approach for designing Kazantzis-Kravaris/Luenberger (KKL) observers for non-autonomous nonlinear systems. While KKL observers offer a rigorous theoretical framework by immersing nonlinear dynamics into a stable linear latent space, its practical realization relies on solving Partial Differential Equations (PDE) that are analytically intractable. Current existing learning-based approximations of the KKL observer are mostly designed for autonomous systems, failing to generalize to driven dynamics without expensive retraining or online gradient updates. HyperKKL addresses this by employing a hypernetwork architecture that encodes the exogenous input signal to instantaneously generate the parameters of the KKL observer, effectively learning a family of immersion maps parameterized by the external drive. We rigorously evaluate this approach against a curriculum learning strategy that attempts to generalize from autonomous regimes via training heuristics alone. The novel approach is illustrated on four numerical simulations in benchmark examples including the Duffing, Van der Pol, Lorenz, and Rössler systems.

eess.SY↗

Dynamic Flexibility Requests in Local Flexibility Markets: Quantifying the DSO Willingness to Pay

Local Flexibility Markets (LFMs) require Distribution System Operators (DSOs) to determine both the quantity of flexibility to procure and the corresponding willingness to pay during market clearing. Existing approaches typically rely on centralized AC-OPF clearing algorithms with limited practical scalability or strictly localized, static flexibility requests driven primarily by congestion management, while the economic value of flexibility is largely neglected. This paper proposes a dynamic flexibility-request methodology in which the DSO's willingness to pay is embedded directly into the market-clearing objective by monetizing transformer and cable aging, network losses, and voltage congestion. To enable computationally efficient clearing, exact convex piecewise-linear epigraph reformulations of the IEEE C57.91 transformer aging model and an Arrhenius-based cable aging model are developed and formally proven to preserve exactness. The proposed framework is validated on a modified CIGRE MV benchmark and compared with a recent state-of-the-art flexibility-request methodology. The results demonstrate significantly higher market liquidity, more efficient flexibility procurement, and improved network operations while preserving the market's transparency and non-discrimination principles.

eess.SY↗