arXiv subjects

Jalal Kazempour

Publications and source records attributed to Jalal Kazempour.

54 records · Page 3Linked to original sources

arXiv2022

Moving from Linear to Conic Markets for Electricity

We propose a new forward electricity market framework that admits heterogeneous market participants with second-order cone strategy sets, who accurately express the nonlinearities in their costs and constraints through conic bids, and a network operator facing conic operational constraints. In contrast to the prevalent linear-programming-based electricity markets, we highlight how the inclusion of second-order cone constraints improves uncertainty-, asset- and network-awareness of the market, which is key to the successful transition towards an electricity system based on weather-dependent renewable energy sources. We analyze our general market-clearing proposal using conic duality theory to derive efficient spatially-differentiated prices for the multiple commodities, comprising of energy and flexibility services. Under the assumption of perfect competition, we prove the equivalence of the centrally-solved market-clearing optimization problem to a competitive spatial price equilibrium involving a set of rational and self-interested participants and a price setter. Finally, under common assumptions, we prove that moving towards conic markets does not incur the loss of desirable economic properties of markets, namely market efficiency, cost recovery and revenue adequacy. Our numerical studies focus on the specific use case of uncertainty-aware market design and demonstrate that the proposed conic market brings advantages over existing alternatives within the linear programming market framework.

econ.TH↗

arXiv2022

Embedding Dependencies Between Wind Farms in Distributionally Robust Optimal Power Flow

The increasing share of renewables in the electricity generation mix comes along with an increasing uncertainty in power supply. In the recent years, distributionally robust optimization has gained significant interest due to its ability to make informed decisions under uncertainty, which are robust to misrepresentations of the distributional information (e.g., from probabilistic forecasts). This is achieved by introducing an ambiguity set that describes the potential deviations from an empirical distribution of all uncertain parameters. However, this set typically overlooks the inherent dependencies of uncertainty, e.g., spatial dependencies of weather-dependent energy sources. This paper goes beyond the state-of-the-art models by embedding such dependencies within the definition of ambiguity set. In particular, we propose a new copula-based ambiguity set which is tailored to capture any type of dependencies. The resulting problem is reformulated as a conic program which is kept generic such that it can be applied to any decision-making problem under uncertainty in power systems. Given the Optimal Power Flow (OPF) problem as one of the main potential applications, we illustrate the performance of our proposed distributionally robust model applied to i) a DC-OPF problem for a meshed transmission system and ii) an AC-OPF problem using LinDistFlow approximation for a radial distribution system.

math.OC↗

arXiv2022

Ecosystem for Demand-side Flexibility Revisited: The Danish Solution

Denmark has recently set a legislation called Market Model 3.0 to make the ecosystem for demand-side flexibility more attractive to stakeholders involved. The main change is to relax the previous mandate that required each aggregator to be associated with a retailer and a balance responsible party. We explain the rationale behind such a change and its implications, particularly on the pre-qualification of demand-side portfolios providing ancillary services.

eess.SY↗

arXiv2022

A Market for Trading Forecasts: A Wagering Mechanism

In many areas of industry and society, e.g., energy, healthcare, logistics, agents collect vast amounts of data that they deem proprietary. These data owners extract predictive information of varying quality and relevance from data depending on quantity, inherent information content, and their own technical expertise. Aggregating these data and heterogeneous predictive skills, which are distributed in terms of ownership, can result in a higher collective value for a prediction task. In this paper, we envision a platform for improving predictions via implicit pooling of private information in return for possible remuneration. Specifically, we design a wagering-based forecast elicitation market platform, where a buyer intending to improve their forecasts posts a prediction task, and sellers respond to it with their forecast reports and wagers. This market delivers an aggregated forecast to the buyer (pre-event) and allocates a payoff to the sellers (post-event) for their contribution. We propose a payoff mechanism and prove that it satisfies several desirable economic properties, including those specific to electronic platforms. Furthermore, we discuss the properties of the forecast aggregation operator and scoring rules to emphasize their effect on the sellers' payoff. Finally, we provide numerical examples to illustrate the structure and properties of the proposed market platform.

econ.TH↗

arXiv2022

Multi-Stage Linear Decision Rules for Stochastic Control of Natural Gas Networks with Linepack

The disturbances from variable and uncertain renewable generation propagate from power systems to natural gas networks, causing gas network operators to adjust gas supply nominations to ensure operational security. To alleviate expensive supply adjustments, we develop control policies to leverage instead the flexibility of linepack -- the gas stored in pipelines -- to balance stochastic gas extractions. These policies are based on multi-stage linear decision rules optimized on a finite discrete horizon to guide controllable network components, such as compressors and valves, towards feasible operations. Our approach offers several control applications. First, it treats the linepack as a main source of flexibility to balance disturbances from power systems without substantial impacts on nominal gas supply. Second, these policies can be optimized to minimize the variability (due to intermittency of renewables) and variance (due to their uncertainty) of network state variables, such as pressures. Finally, it enables topology optimization to decouple network parts and prevent uncertainty propagation through the network. This is demonstrated using illustrative numerical experiments.

math.OC↗

arXiv2022

Market Integration of Excess Heat

Excess heat will be an important heat source in future carbon-neutral district heating systems. A barrier to excess heat integration is the lack of appropriate scheduling and pricing systems for these producers, which generally have small capacity and limited flexibility. In this work, we formulate and analyze two methods for scheduling and pricing excess heat producers: self-scheduling and market participation. In the former, a price signal is sent to excess heat producers, based on which they determine their optimal schedule. The latter approach allows excess heat producers to participate in a market clearing. In a realistic case study of the Copenhagen district heating system, we investigate market outcomes for the two excess heat integration paradigms under increasing excess heat penetration. An important conclusion is that in systems of high excess heat penetration, simple price signal methods will not suffice, and more sophisticated price signals or coordinated dispatch become a necessity.

cs.CE↗

arXiv2022

Modeling Gas Flow Directions as State Variables: Does it Provide More Flexibility to Power Systems?

As a common practice, the direction of natural gas flow in every pipeline is determined ex-ante for simplification purposes, and treated as a given parameter within the scheduling problem. However, in integrated gas and electric power networks with a large share of intermittent renewable power supply, it is no longer straightforward to optimally predetermine the gas flow directions. A wrong predetermination of gas flow directions may result in feasible but not necessarily optimal schedules. We propose a mixed-integer linear optimization model to determine the optimal gas flow directions while scheduling the system. This unlocks additional flexibility to power systems, provided that a tight coordination between power and gas systems exists. The increased flexibility, although it comes at the cost of increased computational complexity, is quantified by comparing the total operational cost of the entire system with bidirectional gas flows as opposed to unidirectional gas flows. We numerically show that modeling gas flow directions as state variables may bring added value not only in the meshed but also in the radial gas networks.

math.OC↗

arXiv2022

Regression markets and application to energy forecasting

Energy forecasting has attracted enormous attention over the last few decades, with novel proposals related to the use of heterogeneous data sources, probabilistic forecasting, online learn-ing, etc. A key aspect that emerged is that learning and forecasting may highly benefit from distributed data, though not only in the geographical sense. That is, various agents collect and own data that may be useful to others. In contrast to recent proposals that look into distributed and privacy-preserving learning (incentive-free), we explore here a framework called regression markets. There, agents aiming to improve their forecasts post a regression task, for which other agents may contribute by sharing their data for their features and get monetarily rewarded for it.The market design is for regression models that are linear in their parameters, and possibly sep-arable, with estimation performed based on either batch or online learning. Both in-sample and out-of-sample aspects are considered, with markets for fitting models in-sample, and then for improving genuine forecasts out-of-sample. Such regression markets rely on recent concepts within interpretability of machine learning approaches and cooperative game theory, with Shapley additive explanations. Besides introducing the market design and proving its desirable properties, application results are shown based on simulation studies (to highlight the salient features of the proposal) and with real-world case studies.

stat.AP↗

arXiv2022

Trading Data for Wind Power Forecasting: A Regression Market with Lasso Regularization

This paper proposes a regression market for wind agents to monetize data traded among themselves for wind power forecasting. Existing literature on data markets often treats data disclosure as a binary choice or modulates the data quality based on the mismatch between the offer and bid prices. As a result, the market disadvantages either the data sellers due to the overestimation of their willingness to disclose data, or the data buyers due to the lack of useful data being provided. Our proposed regression market determines the data payment based on the least absolute shrinkage and selection operator (lasso), which not only provides the data buyer with a means for selecting useful features, but also enables each data seller to individualize the threshold for data payment. Using both synthetic data and real-world wind data, the case studies demonstrate a reduction in the overall losses for wind agents who buy data, as well as additional financial benefits to those who sell data.

stat.AP↗

arXiv2021

Stochastic Control and Pricing for Natural Gas Networks

We propose stochastic control policies to cope with uncertain and variable gas extractions in natural gas networks. Given historical gas extraction data, these policies are optimized to produce the real-time control inputs for nodal gas injections and for pressure regulation rates by compressors and valves. We describe the random network state as a function of control inputs, which enables a chance-constrained optimization of these policies for arbitrary network topologies. This optimization ensures the real-time gas flow feasibility and a minimal variation in the network state up to specified feasibility and variance criteria. Furthermore, the chance-constrained optimization provides the foundation of a stochastic pricing scheme for natural gas networks, which improves on a deterministic market settlement by offering the compensations to network assets for their contribution to uncertainty and variance control. We analyze the economic properties, including efficiency, revenue adequacy and cost recovery, of the proposed pricing scheme and make them conditioned on the network design.

math.OC↗

arXiv2021

Monetizing Customer Load Data for an Energy Retailer: A Cooperative Game Approach

When energy customers schedule loads ahead of time, this information, if acquired by their energy retailer, can improve the retailer's load forecasts. Better forecasts lead to wholesale purchase decisions that are likely to result in lower energy imbalance costs, and thus higher profits for the retailer. Therefore, this paper monetizes the value of the customer schedulable load data by quantifying the retailer's profit gain from adjusting the wholesale purchase based on such data. Using a cooperative game theoretic approach, the retailer translates their increased profit in expectation into the value of cooperation, and redistributes a portion of it among the customers as monetary incentives for them to continue providing their load data. Through case studies, this paper demonstrates the significance of the additional profit for the retailer from using the proposed framework, and evaluates the long-term monetary benefits to the customers based on different payoff allocation methods.

math.OC↗

arXiv2021

Design of a Continuous Local Flexibility Market with Network Constraints

To the best of our knowledge, this paper proposes for the first time a design of a continuous local flexibility market that explicitly considers network constraints. Continuous markets are expected to be the most appropriate design option during the early stages of local flexibility markets, where insufficient liquidity can hinder market development. At the same time, increasingly loaded distribution systems require to explicitly consider network constraints in local flexibility market clearing in order to help resolve rather than aggravate local network problems, such as line congestion and voltage issues. This paper defines the essential design considerations, introduces the local flexibility market clearing algorithm, and -- aiming to establish a starting point for future research -- discusses design options and research challenges that emerge during this procedure which require further investigation.

math.OC↗

arXiv2020

Differentially Private Optimal Power Flow for Distribution Grids

Although distribution grid customers are obliged to share their consumption data with distribution system operators (DSOs), a possible leakage of this data is often disregarded in operational routines of DSOs. This paper introduces a privacy-preserving optimal power flow (OPF) mechanism for distribution grids that secures customer privacy from unauthorised access to OPF solutions, e.g., current and voltage measurements. The mechanism is based on the framework of differential privacy that allows to control the participation risks of individuals in a dataset by applying a carefully calibrated noise to the output of a computation. Unlike existing private mechanisms, this mechanism does not apply the noise to the optimization parameters or its result. Instead, it optimizes OPF variables as affine functions of the random noise, which weakens the correlation between the grid loads and OPF variables. To ensure feasibility of the randomized OPF solution, the mechanism makes use of chance constraints enforced on the grid limits. The mechanism is further extended to control the optimality loss induced by the random noise, as well as the variance of OPF variables. The paper shows that the differentially private OPF solution does not leak customer loads up to specified parameters.

math.OC↗

arXiv2020

Differentially Private Convex Optimization with Feasibility Guarantees

This paper develops a novel differentially private framework to solve convex optimization problems with sensitive optimization data and complex physical or operational constraints. Unlike standard noise-additive algorithms, that act primarily on the problem data, objective or solution, and disregard the problem constraints, this framework requires the optimization variables to be a function of the noise and exploits a chance-constrained problem reformulation with formal feasibility guarantees. The noise is calibrated to provide differential privacy for identity and linear queries on the optimization solution. For many applications, including resource allocation problems, the proposed framework provides a trade-off between the expected optimality loss and the variance of optimization results.

cs.CR↗

arXiv2020

Chance-Constrained Equilibrium in Electricity Markets With Asymmetric Forecasts

We develop a stochastic equilibrium model for an electricity market with asymmetric renewable energy forecasts. In our setting, market participants optimize their profits using public information about a conditional expectation of energy production but use private information about the forecast error distribution. This information is given in the form of samples and incorporated into profit-maximizing optimizations of market participants through chance constraints. We model information asymmetry by varying the sample size of participants' private information. We show that with more information available, the equilibrium gradually converges to the ideal solution provided by the perfect information scenario. Under information scarcity, however, we show that the market converges to the ideal equilibrium if participants are to infer the forecast error distribution from the statistical properties of the data at hand or share their private forecasts.

math.OC↗

arXiv2020

Differentially Private Distributed Optimal Power Flow

Distributed algorithms enable private Optimal Power Flow (OPF) computations by avoiding the need in sharing sensitive information localized in algorithms sub-problems. However, adversaries can still infer this information from the coordination signals exchanged across iterations. This paper seeks formal privacy guarantees for distributed OPF computations and provides differentially private algorithms for OPF computations based on the consensus Alternating Direction Method of Multipliers (ADMM). The proposed algorithms attain differential privacy by introducing static and dynamic random perturbations of OPF sub-problem solutions at each iteration. These perturbations are Laplacian and designed to prevent the inference of sensitive information, as well as to provide theoretical privacy guarantees for ADMM sub-problems. Using a standard IEEE 118-node test case, the paper explores the fundamental trade-offs among privacy, algorithmic convergence, and optimality losses.

math.OC↗

arXiv2019

Electricity Market Equilibrium under Information Asymmetry

We study a competitive electricity market equilibrium with two trading stages, day-ahead and real-time. The welfare of each market agent is exposed to uncertainty (here from renewable energy production), while agent information on the probability distribution of this uncertainty is not identical at the day-ahead stage. We show a high sensitivity of the equilibrium solution to the level of information asymmetry and demonstrate economic, operational, and computational value for the system stemming from potential information sharing.

math.OC↗

arXiv2018

A Consensus-ADMM Approach for Strategic Generation Investment in Electricity Markets

This paper addresses a multi-stage generation investment problem for a strategic (price-maker) power producer in electricity markets. This problem is exposed to different sources of uncertainty, including short-term operational (e.g., rivals' offering strategies) and long-term macro (e.g., demand growth) uncertainties. This problem is formulated as a stochastic bilevel optimization problem, which eventually recasts as a large-scale stochastic mixed-integer linear programming (MILP) problem with limited computational tractability. To cope with computational issues, we propose a consensus version of alternating direction method of multipliers (ADMM), which decomposes the original problem by both short- and long-term scenarios. Although the convergence of ADMM to the global solution cannot be generally guaranteed for MILP problems, we introduce two bounds on the optimal solution, allowing for the evaluation of the solution quality over iterations. Our numerical findings show that there is a trade-off between computational time and solution quality.

math.OC↗