arXiv subjects

Thomas Morstyn

Publications and source records attributed to Thomas Morstyn.

At least 37 records · Page 2Linked to original sources

arXiv2024

Multi-Agent Reinforcement Learning for Energy Networks: Computational Challenges, Progress and Open Problems

The rapidly changing architecture and functionality of electrical networks and the increasing penetration of renewable and distributed energy resources have resulted in various technological and managerial challenges. These have rendered traditional centralized energy-market paradigms insufficient due to their inability to support the dynamic and evolving nature of the network. This survey explores how multi-agent reinforcement learning (MARL) can support the decentralization and decarbonization of energy networks and mitigate the associated challenges. This is achieved by specifying key computational challenges in managing energy networks, reviewing recent research progress on addressing them, and highlighting open challenges that may be addressed using MARL.

cs.AI↗

arXiv2024

An Incentive Regulation Approach for Balancing Stakeholder Interests in Transmission Investment

The merchant-regulatory mechanism represents a promising tool that combines the benefits of merchant investment and regulated investment, thereby providing efficient incentives for merchant Transmission Companies (Transcos) subject to regulatory compliance. However, one of the drawbacks of the H-R-G-V merchant-regulated mechanism is that it allows the Transco to capture the entire surplus increase resulting from investment, without any economic benefits for consumers and generators. To address this issue, we propose an incentive tuning parameter, which is incorporated into the calculation of the incentive fee for the Transco. Accordingly, the regulatory framework can effectively manage the Transco's profit and allow market participants to access economic benefits, thus ensuring a fair distribution of economic advantages among the stakeholders, while the impact on overall social welfare remains relatively modest.

math.OC↗

arXiv2024

Datasets of Great Britain Primary Substations Integrated with Household Heating Information

The growing demand for electrified heating, electrified transportation, and power-intensive data centres challenge distribution networks. If electrification projects are carried out without considering electrical distribution infrastructure, there could be unexpected blackouts and financial losses. Datasets containing real-world distribution network information are required to address this. On the other hand, social data, such as household heating composition, are closely coupled with people's lives. Studying the coupling between the energy system and society is important in promoting social welfare. To fill these gaps, this paper introduces two datasets. The first is the main dataset for the distribution networks in Great Britain (GB), collecting information on firm capacity, peak demands, locations, and parent transmission nodes (the Grid Supply Point, namely GSP) for all primary substations (PSs). PSs are a crucial part of the UK distribution network and are at the lowest voltage level (11 kV) with publicly available data for most UK Distribution Network Operators (DNOs). Substation firm capacity and peak demand facilitate an understanding of the remaining room of the existing network. The parent GSP information helps link the dataset of distribution networks to datasets of transmission networks. The second dataset extends the main network dataset, linking each PS to information about the number of households that use different types of central heating recorded in census data. The derivation of the second dataset is based on locations of PSs collected in the main dataset with appropriate assumptions. The derivation process may also be replicated to integrate other social datasets.

eess.SY↗

arXiv2024

Depreciation Cost is a Poor Proxy for Revenue Lost to Aging in Grid Storage Optimization

Dispatch of a grid energy storage system for arbitrage is typically formulated into a rolling-horizon optimization problem that includes a battery aging model within the cost function. Quantifying degradation as a depreciation cost in the objective can increase overall profits by extending lifetime. However, depreciation is just a proxy metric for battery aging; it is used because simulating the entire system life is challenging due to computational complexity and the absence of decades of future data. In cases where the depreciation cost does not match the loss of possible future revenue, different optimal usage profiles result and this reduces overall profit significantly compared to the best case (e.g., by 30-50%). Representing battery degradation perfectly within the rolling-horizon optimization does not resolve this - in addition, the economic cost of degradation throughout life should be carefully considered. For energy arbitrage, optimal economic dispatch requires a trade-off between overuse, leading to high return rate but short lifetime, vs. underuse, leading to a long but not profitable life. We reveal the intuition behind selecting representative costs for the objective function, and propose a simple moving average filter method to estimate degradation cost. Results show that this better captures peak revenue, assuming reliable price forecasts are available.

eess.SY↗

arXiv2023

Home Electricity Data Generator (HEDGE): An open-access tool for the generation of electric vehicle, residential demand, and PV generation profiles

In this paper, we present the Home Electricity Data Generator (HEDGE), an open-access tool for the random generation of realistic residential energy data. HEDGE generates realistic daily profiles of residential PV generation, household electric loads, and electric vehicle consumption and at-home availability, based on real-life UK datasets. The lack of usable data is a major hurdle for research on residential distributed energy resources characterisation and coordination, especially when using data-driven methods such as machine learning-based forecasting and reinforcement learning-based control. A key issue is that while large data banks are available, they are not in a usable format, and numerous subsequent days of data for a given single home are unavailable. We fill these gaps with the open-access HEDGE tool which generates data sequences of energy data for several days in a way that is consistent for single homes, both in terms of profile magnitude and behavioural clusters. From raw datasets, pre-processing steps are conducted, including filling in incomplete data sequences and clustering profiles into behaviour clusters. Generative adversarial networks (GANs) are then trained to generate realistic synthetic data representative of each behaviour groups consistent with real-life behavioural and physical patterns.

eess.SY↗

arXiv2023

Centralised rehearsal of decentralised cooperation: Multi-agent reinforcement learning for the scalable coordination of residential energy flexibility

This paper investigates how deep multi-agent reinforcement learning can enable the scalable and privacy-preserving coordination of residential energy flexibility. The coordination of distributed resources such as electric vehicles and heating will be critical to the successful integration of large shares of renewable energy in our electricity grid and, thus, to help mitigate climate change. The pre-learning of individual reinforcement learning policies can enable distributed control with no sharing of personal data required during execution. However, previous approaches for multi-agent reinforcement learning-based distributed energy resources coordination impose an ever greater training computational burden as the size of the system increases. We therefore adopt a deep multi-agent actor-critic method which uses a \emph{centralised but factored critic} to rehearse coordination ahead of execution. Results show that coordination is achieved at scale, with minimal information and communication infrastructure requirements, no interference with daily activities, and privacy protection. Significant savings are obtained for energy users, the distribution network and greenhouse gas emissions. Moreover, training times are nearly 40 times shorter than with a previous state-of-the-art reinforcement learning approach without the factored critic for 30 homes.

eess.SY↗

arXiv2023

An Insurance Paradigm for Improving Power System Resilience via Distributed Investment

Extreme events, exacerbated by climate change, pose significant risks to the energy system and its consumers. However there are natural limits to the degree of protection that can be delivered from a centralised market architecture. Distributed energy resources provide resilience to the energy system, but their value remains inadequately recognized by regulatory frameworks. We propose an insurance framework to align residual outage risk exposure with locational incentives for distributed investment. We demonstrate that leveraging this framework in large-scale electricity systems could improve consumer welfare outcomes in the face of growing risks from extreme events via investment in distributed energy.

econ.GN↗

arXiv2022

Scalable multi-agent reinforcement learning for distributed control of residential energy flexibility

This paper proposes a novel scalable type of multi-agent reinforcement learning-based coordination for distributed residential energy. Cooperating agents learn to control the flexibility offered by electric vehicles, space heating and flexible loads in a partially observable stochastic environment. In the standard independent Q-learning approach, the coordination performance of agents under partial observability drops at scale in stochastic environments. Here, the novel combination of learning from off-line convex optimisations on historical data and isolating marginal contributions to total rewards in reward signals increases stability and performance at scale. Using fixed-size Q-tables, prosumers are able to assess their marginal impact on total system objectives without sharing personal data either with each other or with a central coordinator. Case studies are used to assess the fitness of different combinations of exploration sources, reward definitions, and multi-agent learning frameworks. It is demonstrated that the proposed strategies create value at individual and system levels thanks to reductions in the costs of energy imports, losses, distribution network congestion, battery depreciation and greenhouse gas emissions.

eess.SY↗

arXiv2022

Coordination of resources at the edge of the electricity grid: systematic review and taxonomy

This paper proposes a novel taxonomy of coordination strategies for distributed energy resources at the edge of the electricity grid, based on a systematic analysis of key literature trends. The coordination of distributed energy resources such as decentralised generation and flexibility sources is critical for decarbonising electricity and achieving climate goals. The literature on the topic is growing exponentially; however, there is ambiguity in the terminology used to date. We seek to resolve this lack of clarity by synthesising the categories of coordination strategies in a novel exhaustive, mutually exclusive taxonomy based on agency, information and game type. The relevance of these concepts in the literature is illustrated through a systematic literature review of 84,741 publications using a structured topic search query. Then 93 selected coordination strategies are analysed in more detail and mapped onto this framework. Clarity on structural assumptions is key for selecting appropriate coordination strategies for differing contexts within energy systems. We argue that a plurality of complementary strategies is needed to coordinate energy systems' different components and achieve deep decarbonisation.

eess.SY↗

arXiv2022

Distributionally Robust Joint Chance-Constrained Optimization for Networked Microgrids Considering Contingencies and Renewable Uncertainty

In light of a reliable and resilient power system under extreme weather and natural disasters, networked microgrids integrating local renewable resources have been adopted extensively to supply demands when the main utility experiences blackouts. However, the stochastic nature of renewables and unpredictable contingencies are difficult to address with the deterministic energy management framework. The paper proposes a comprehensive distributionally robust joint chance-constrained (DR-JCC) framework that incorporates microgrid island, power flow, distributed batteries and voltage control constraints. All chance constraints are solved jointly and each one is assigned to an optimized violation rate. To highlight, the JCC problem with the optimized violation rates has been recognized to be NP-hard and challenging to be solved. This paper proposes a novel evolutionary algorithm that successfully tackles the problem and reduces the solution conservativeness (i.e. operation cost) by around 50% comparing with the baseline Bonferroni Approximation. Considering the imperfect solar power forecast, we construct three data-driven ambiguity sets to model uncertain forecast error distributions. The solution is thus robust for any distribution in sets with the shared moment and shape assumptions. The proposed method is validated by robustness tests based on those sets and firmly secures the solution robustness.

eess.SY↗

arXiv2022

Incorporating forecasting and peer-to-peer negotiation frameworks into a distributed model predictive control approach for meshed electric networks

Continuous integration of renewable energy sources into power networks is causing a paradigm shift in energy generation and distribution with regards to trading and control; the intermittent nature of renewable sources affects pricing of energy sold or purchased; the networks are subject to operational constraints, voltage limits at each node, rated capacities for the power electronic devices, current bounds for distribution lines. These economic and technical constraints coupled with intermittent renewable injection may pose a threat to system stability and performance. We propose a novel holistic approach to energy trading composed of a distributed predictive control framework to handle physical interactions, i,e., voltage constraints and power dispatch, together with a negotiation framework to determine pricing policies for energy transactions. We study the effect of forecasting generation and consumption on the overall network's performance and market behaviours. We provide a rigorous convergence analysis for both the negotiation framework and the distributed control. Lastly, we assess the impact of forecasting in the proposed system with the aid of testing scenarios.

eess.SY↗

arXiv2021

On the Design of an Insurance Mechanism for Reliability Differentiation in Electricity Markets

Securing an adequate supply of dispatchable resources is critical for keeping a power system reliable under high penetrations of variable generation. Traditional resource adequacy mechanisms are poorly suited to exploiting the growing flexibility and heterogeneity of load enabled by advancements in distributed resource and control technology. To address these challenges this paper develops a resource adequacy mechanism for the electricity sector utilising insurance risk management frameworks that is adapted to a future with variable generation and flexible demand. The proposed design introduces a central insurance scheme with prudential requirements that align diverse consumer reliability preferences with the financial objectives of an insurer-of-last-resort. We illustrate the benefits of the scheme in (i) differentiating load by usage to enable better management of the system during times of extreme scarcity, (ii) incentivising incremental investment in generation infrastructure that is aligned with consumer reliability preferences and (iii) improving overall reliability outcomes for consumers.

eess.SY↗

arXiv2021

Electricity prices and tariffs to keep everyone happy: a framework for fixed and nodal prices coexistence in distribution grids with optimal tariffs for investment cost recovery

Some consumers, particularly households, are unwilling to face volatile electricity prices, and they can perceive as unfair price differentiation in the same local area. For these reasons, nodal prices in distribution networks are rarely employed. However, the increasing availability of renewable resources and emerging price-elastic behaviours pave the way for the effective introduction of marginal nodal pricing schemes in distribution networks. The aim of the proposed framework is to show how traditional non-flexible consumers can coexist with flexible users in a local distribution area. Flexible users will pay nodal prices, whereas non-flexible consumers will be charged a fixed price derived from the underlying nodal prices. Moreover, the developed approach shows how a distribution system operator should manage the local grid by optimally determining the lines to be expanded, and the collected network tariff levied on grid users, while accounting for both congestion rent and investment costs. The proposed model is formulated as a non-linear integer bilevel program, which is then recast as an equivalent single optimization problem, by using integer algebra and complementarity relations. The power flows in the distribution area are modelled by resorting to a second-order cone relaxation, whose solution is exact for radial networks under mild assumptions. The final model results in a mixed-integer quadratically constrained program, which can be solved with off-the-shelf solvers. Numerical test cases based on both 5-bus and 33-bus networks are reported to show the effectiveness of the proposed method.

econ.GN↗

arXiv2020

Peer-to-Peer Energy Systems for Connected Communities: A Review of Recent Advances and Emerging Challenges

After a century of relative stability of the electricity industry, extensive deployment of distributed energy resources and recent advances in computation and communication technologies have changed the nature of how we consume, trade, and apply energy. The power system is facing a transition from its traditional hierarchical structure to a more deregulated model by introducing new energy distribution models such as peer-to-peer sharing for connected communities. The proven effectiveness of P2P sharing in benefiting both prosumers and the grid has been demonstrated in many studies and pilot projects. However, there is still no extensive implementation of such sharing models in today's electricity markets. This paper aims to shed some light on this gap through a comprehensive overview of recent advances in the P2P energy system and an insightful discussion of the challenges that need to be addressed in order to establish P2P sharing as a viable energy management option in today's electricity market. To this end, in this article, we provide some background on different aspects of P2P sharing. Then, we discuss advances in P2P sharing through a systematic domain-based classification. We also review different pilot projects on P2P sharing across the globe. Finally, we identify and discuss a number of challenges that need to be addressed for scaling up P2P sharing in the electricity market followed by concluding remarks at the end of the paper.

cs.DC↗

arXiv2020

Improving the Scalability of a Prosumer Cooperative Game with K-Means Clustering

Among the various market structures under peer-to-peer energy sharing, one model based on cooperative game theory provides clear incentives for prosumers to collaboratively schedule their energy resources. The computational complexity of this model, however, increases exponentially with the number of participants. To address this issue, this paper proposes the application of K-means clustering to the energy profiles following the grand coalition optimization. The cooperative model is run with the "clustered players" to compute their payoff allocations, which are then further distributed among the prosumers within each cluster. Case studies show that the proposed method can significantly improve the scalability of the cooperative scheme while maintaining a high level of financial incentives for the prosumers.

cs.CE↗

arXiv2019

A Coalition Formation Game Framework for Peer-to-Peer Energy Trading

This paper studies social cooperation backed peer-to-peer energy trading technique by which prosumers can decide how they can use their batteries opportunistically for participating in the peer-to-peer trading. The objective is to achieve a solution in which the ultimate beneficiaries are the prosumers, i.e., a prosumer-centric solution. To do so, a coalition formation game is designed, which enables a prosumer to compare its benefit of participating in the peer-to-peer trading with and without using its battery and thus, allows the prosumer to form suitable social coalition groups with other similar prosumers in the network for conducting peer-to-peer trading. The properties of the formed coalitions are studied, and it is shown that 1) the coalition structure that stems from the social cooperation between participating prosumers at each time slot is both stable and optimal, and 2) the outcomes of the proposed peer- to-peer trading scheme is prosumer-centric. Case studies are conducted based on real household energy usage and solar generation data to highlight how the proposed scheme can benefit prosumers through exhibiting prosumer-centric properties.

eess.SP↗

arXiv2019

Grid Influenced Peer-to-Peer Energy Trading

This paper proposes a peer-to-peer energy trading scheme that can help the centralized power system to reduce the total electricity demand of its customers at the peak hour. To do so, a cooperative Stackelberg game is formulated, in which the centralized power system acts as the leader that needs to decide on a price at the peak demand period to incentivize prosumers to not seeking any energy from it. The prosumers, on the other hand, act as followers and respond to the leader's decision by forming suitable coalitions with neighboring prosumers in order to participate in P2P energy trading to meet their energy demand. The properties of the proposed Stackelberg game are studied. It is shown that the game has a unique and stable Stackelberg equilibrium, as a result of the stability of prosumers' coalitions. At the equilibrium, the leader chooses its strategy using a derived closed-form expression, while the prosumers choose their equilibrium coalition structure. An algorithm is proposed that enables the centralized power system and the prosumers to reach the equilibrium solution. Numerical case studies demonstrate the beneficial properties of the proposed scheme.

eess.SY↗

arXiv2019

A Stochastic Model for Uncontrolled Charging of Electric Vehicles Using Cluster Analysis

This paper proposes a probabilistic model for uncontrolled charging of electric vehicles (EVs). EV charging will add significant load to power systems in the coming years and, due to the convenience of charging at home, this is likely to occur in residential distribution systems. Estimating the size and shape of the load will allow necessary reinforcements to be identified. Models predicting EV charging are usually based on data from travel surveys, or from small trials. Travel surveys are recorded by hand and typically describe conventional vehicles, but represent a much larger and more diverse sample of the population. The model here utilizes both sources: trial data to parameterize the model, and survey data as the model input. Clustering is used to identify modes of vehicle use, thus reducing vehicle use to a single parameter -- which can be incorporated into the model without adding significant computational burden. Two case studies are included: one investigating the aggregated charging of 50 vehicles, and one predicting the increase in after diversity maximum demand for different regions of the UK.

eess.SP↗