arXiv subjects

Yury Dvorkin

Publications and source records attributed to Yury Dvorkin.

At least 19 recordsLinked to original sources

arXiv2026

Iterative Single-Loop Coordination of Transmission and Distribution Systems and DER Aggregators with Limited Information Sharing

The growing integration of DERs underscores the importance of unified coordination across TSO, DSO, and aggregator levels. This paper proposes a coordination method between TSO, DSOs and DER aggregators, based on a transparent, well-scalable, and decentralized single-loop iterative mechanism with limited information sharing. The approach enables iterative price-based coordination between adjacent levels, where the higher-level entities provide price signals and the lower-level ones respond with desired power schedules. The proposed method uses the proximal-gradient-based iterative co-optimization algorithm extended to a three-level case, which enjoys convergence characteristic of proximal algorithms. The performance of the proposed approach is compared with both a centralized formulation and a sequential reference formulation based on fixed transmission-level energy prices. The results demonstrate that the proposed coordination achieves outcomes very close to those obtained with the centralized formulation, while significantly improving performance relative to the sequential reference, particularly at higher levels of DER flexibility.

eess.SY↗

arXiv2026

Reachability Guarantees for Energy Arbitrage

Batteries performing energy arbitrage may overdischarge ahead of a high-demand or critical operating hour when they re-optimize in real time and depart from the day-ahead schedule. This paper introduces a framework for battery energy arbitrage under uncertain market prices that integrates chance-constrained terminal state-of-charge (SoC) requirements with online threshold policies. We first cast the multi-interval arbitrage problem as a stochastic dynamic program enhanced by a probabilistic end-of-horizon SoC constraint, ensuring with high confidence that the battery terminates within a prescribed energy band. We then apply a $k$-search algorithm to derive explicit charging (buying) and discharging (selling) thresholds with a provable worst-case competitive ratio, and compute the corresponding action probabilities over the decision horizon. To compute exact distributions under operational limits, we develop a probability redistribution pruning method and use it to quantify the likelihood of meeting the terminal SoC band. Leveraging the resulting SoC distribution, we estimate the minimum stopping time required to satisfy the SoC chance constraint. Computational experiments on historical real price data demonstrate that the proposed framework improves SoC estimation and supports chance-constraint satisfaction, while also showing that reachability is highly sensitive to a given initial SoC and operating start time.

eess.SY↗

arXiv2026

Mixed-Integer Nonlinear Differentiable Predictive Control for Underground Pumped Hydro Energy Storage Systems

This paper extends Mixed-Integer Differentiable Predictive Control (MI-DPC) to multi-modal discrete decisions and nonconvex polynomial dynamics arising in Underground Pumped Hydro Energy Storage Systems (UPHES). A neural policy mapping problem parameters to continuous setpoints and integer mode selections via a Gumbel-Softmax layer is trained in a self-supervised manner by differentiating the expectation of the finite horizon control objective through the nonlinear dynamics model. Three methodological contributions enable this extension: a parallel differentiable simulator that preserves gradient magnitude, a Transformer encoder that captures long-range temporal dependencies, and a Gumbel-Softmax temperature annealing schedule that regularizes the combinatorial search. We demonstrate the framework on day-ahead scheduling of a UPHES, a large-scale mixed-integer optimal control problem with nonlinear unit performance curves and volume-head coupling. MI-DPC achieves only 1.6% suboptimality relative to a piecewise mixed-integer quadratic programming baseline, while providing five orders of magnitude speedup in online scheduling time.

eess.SY↗

arXiv2026

Shift or curtail? How much data-center flexibility is worth depends on the host power grid

Data-center growth risks overbuilding power grid infrastructure and stranding capital. Flexible data-center operation can defer infrastructure investments, but its value depends on the flexibility mechanism and the host power grid characteristics. We classify data-center load as firm, flexible or interruptible, and embed them in capacity expansion applied to market-organized, fossil-heavy PJM and carbon-capped, centrally coordinated Korea. In PJM, the flexibility value is spatial: shifting workloads between zones reduces system cost by 6% in 2028 and 19% in 2038, avoiding 4.4 GW and 8.9 GW of gas and nuclear generation. In Korea, it is temporal: shifting load into midday solar hours makes 0.5 GW of additional solar worth building in 2028 and avoids 1.2 GW of gas and 0.3 GW of batteries in 2038. In both, realistic event-shape limits diminish the value of curtailment. The results show that flexibility procurement and its value are driven by grid characteristics and policy objectives.

physics.soc-ph↗

arXiv2026

Accelerating Underground Pumped Hydro Energy Storage Scheduling with Decision-Focused Learning

Underground pumped hydro energy storage (UPHES) systems play a critical role in grid-scale energy storage for renewable integration, yet optimal day-ahead scheduling remains computationally prohibitive due to nonlinear turbine performance characteristics and discrete operational modes. This paper presents a decision-focused learning (DFL) framework that addresses the computational-accuracy trade-off in UPHES day-ahead scheduling. The proposed methodology employs neural networks to predict penalty weights that guide recursive linearization, transforming the intractable MINLP into a sequence of convex quadratic programs trained end-to-end via differentiable optimization layers. Case studies across 19 representative Belgian electricity market scenarios demonstrate that the DFL framework effectively navigates the trade-off between solution quality and computation time. As a refiner, DFL adds approximately 1.2 s of post-processing to improve profit by 1.1% over piecewise MIQP baselines. As a real-time scheduler initialized with a fast global linear MIQP, the end-to-end time is 3.87 s (a 300-fold speedup) while maintaining profitability within 3.6% of the piecewise MIQP benchmark. An MIP-free deployment mode, which warm-starts from a historical schedule database and thus needs no mixed-integer solve, retains 92% of benchmark profit on held-out price scenarios.

eess.SY↗

arXiv2026

Robust Capacity Expansion under Wildfire Ignition Risk and High Renewable Penetration

In power systems, the risk of wildfire ignition has increased significantly in recent years. The impact and severity of these events on energy dispatch, as well as their societal ramifications, make wildfire prevention critical for power system planning and operation. A common intervention by system operators is to de-energize transmission lines to mitigate the risk of fire caused by equipment failures. With the growing integration of variable renewable generation, managing and preparing the system to de-energization under wildfire risk has become even more challenging. In this context, mitigation decisions such as installing battery energy storage systems and undergrounding transmission lines can reduce the risk and adverse effects associated with de-energization and renewable generation variability. This paper presents a robust optimization model to determine the optimal location of battery storage and undergrounding of transmission line investment, utilizing representative weeks and uncertainty sets to capture the temporal relationship of uncertain variables. Specifically, this paper addresses: (i) the worst-case realization of ignition risk leading to the de-energization of transmission lines, combined with the worst-case realization of renewable energy availability, and (ii) the optimal investment decisions for energy storage capacity and undergrounding of transmission lines that are exposed to ignition risk. The proposed model is formulated as a mixed-integer linear programming (MILP) problem, employing duality theory and binary decomposition to address nonlinearities, and is solved using a column-and-constraint generation algorithm. The proposed framework is evaluated on a model of the San Diego power system, demonstrating its practical effectiveness in improving the resilience to wildfire risk.

math.OC↗

arXiv2026

Learning Reachability of Energy Storage Arbitrage

Power systems face increasing weather-driven variability and, therefore, increasingly rely on flexible but energy-limited storage resources. Energy storage can buffer this variability, but its value depends on intertemporal decisions under uncertain prices. Without accounting for the future reliability value of stored energy, batteries may act myopically, discharging too early or failing to preserve reserves during critical hours. This paper introduces a stopping-time reward that, together with a state-of-charge (SoC) range target penalty, aligns arbitrage incentives with system reliability by rewarding storage that maintains sufficient SoC before critical hours. We formulate the problem as an online optimization with a chance-constrained terminal SoC and embed it in an end-to-end (E2E) learning framework, jointly training the price predictor and control policy. The proposed design enhances reachability of target SoC ranges, improves profit under volatile conditions, and reduces its standard deviation.

eess.SY↗

arXiv2026

Cross-Atlantic Research Agenda for Scalable Grid Architectures and Distributed Flexibility

Electric power systems are rapidly evolving into deeply digital, cyber-physical infrastructures in which large fleets of distributed energy resources must be coordinated as system-level flexibility across multiple spatial and temporal scales. Despite growing distributed energy resource deployment, existing grid and market architectures lack scalable, interoperable mechanisms to reliably translate device-level flexibility into grid-aware services, creating risks to reliability, affordability, and resilience at high penetration. We propose that scalable and reliable coordination of distributed energy resource-based flexibility in future power systems is fundamentally an architectural problem that can be addressed through laminar cyber-physical design using minimal, standardized interoperability interfaces that link device autonomy with system-level objectives. To assess this claim, we present and discuss a layered cyber-physical systems architecture and explicate its implementation through standards-based interfaces, Flexibility Functions, hierarchical control, and case studies spanning U.S. and Danish regulatory, market, and operational contexts. Empirical evidence from New York's Grid of the Future proceedings, Danish Smart Energy Operating System pilots, and operational aggregator deployments demonstrates that such architecture enables predictable, grid-aware flexibility while preserving device autonomy, interoperability, reliability, and quality of service. These results support a cross-Atlantic research agenda centered on joint testbeds, harmonized interoperability mechanisms, and coordinated policy experiments to accelerate the deployment of resilient, scalable, and flexible clean energy systems.

eess.SY↗

arXiv2026

Grid-Supporting Equipment Supply Chains Constrain the Feasible Pace of Power System Expansion

Power system expansion depends on the equipment required to connect, convert, regulate, and condition electricity, yet grid-supporting equipment (GSE) is rarely modeled as an explicit constraint. We develop a framework integrating dynamic stock-flow modeling, bill-of-materials accounting, multi-regional supply-use analysis, and expansion optimization to quantify GSE deployment requirements and upstream material dependence. Because manufacturing data are often fragmented or proprietary, we use critical material requirements as a physically grounded proxy for GSE supply constraints. In a U.S. case study, GSE shortages reach 269.6--274.1 GVA (28.5%--28.6%) by 2030 under high-growth conditions. Copper becomes fully binding, with steel and nickel forming additional constraints. Trade disruption intensifies shortages, while grid-enhancing technologies provide limited relief. These results show that grid expansion depends on the timely manufacturability, replacement, and material support of GSE, motivating planning frameworks that explicitly incorporate deliverability, supply chain exposure, and resilience strategies.

eess.SY↗

arXiv2026

L2O-CCG: Adversarial Learning with Set Generalization for Adaptive Robust Optimization

The adversarial subproblem in two-stage adaptive robust optimization (ARO), which identifies the worst-case uncertainty realization, is a major computational bottleneck. This difficulty is exacerbated when the recourse value function is non-concave and the uncertainty set shifts across applications. Existing approaches typically exploit specific structural assumptions on the value function or the uncertainty set geometry to reformulate this subproblem, but degrade when these assumptions are violated or the geometry changes at deployment. To address this challenge, we propose L2O-CCG, a bi-level framework that enables the integration of structure-aware adversarial solvers within the constraint-and-column generation (CCG) algorithm. As one instantiation, we develop a generalizable adversarial learning method, which replaces solver-based adversarial search with a learned proximal gradient optimizer that can generalize across uncertainty set geometries without retraining. Here, an inner-level neural network approximates the recourse value function from offline data, while an outer-level pre-trained mapping generates iteration-dependent step sizes for a proximal gradient scheme. We also establish out-of-distribution convergence bounds under uncertainty set parameter shifts, showing how the trajectory deviation of the learned optimizer is bounded by the uncertainty set shift. We illustrate performance of the L2O-CCG method on a building HVAC management task.

eess.SY↗

arXiv2025

Generation Expansion Planning with Upstream Supply Chain Constraints on Materials, Manufacturing, and Deployment

Rising electricity demand underscores the need for secure and reliable generation expansion planning that accounts for upstream supply chain constraints. Traditional models often overlook limitations in materials, manufacturing capacity, lead times for deployment, and field availability, which can delay availability of planned resources and thus to threaten system reliability. This paper introduces a multi-stage supply chain-constrained generation expansion planning (SC-GEP) model that optimizes long-term investments while capturing material availability, production limits, spatial and temporal constraints, and material reuse from retired assets. A decomposition algorithm efficiently solves the resulting MILP. A Maryland case study shows that supply chain constraints shift technology choices, amplify deployment delays caused by lead times, and prompt earlier investment in shorter lead-time, low-material-intensity options. In the low-demand scenario, supply chain constraints raise investment costs by $1.2 billion. Under high demand, persistent generation and reserve shortfalls emerge, underscoring the need to integrate upstream constraints into long-term planning.

eess.SY↗

arXiv2025

Decision-Focused Learning for Neural Network-Constrained HVAC Scheduling

Heating, Ventilation, and Air Conditioning (HVAC) is a major electricity end-use with a substantial potential for providing grid services, such as demand response. Harnessing this flexibility requires accurate modeling of the thermal dynamics of buildings, a difficult task because nonlinear heat transfer and recurring daily cycles make historical data highly correlated and insufficient to generalize to new weather, occupancy, and control scenarios. This paper presents an HVAC management system formulated as a Mixed Integer Quadratic Program (MIQP), where Neural Network (NN) models of thermal dynamics are embedded as exact mixed-integer linear constraints. Unlike traditional training approaches that minimize prediction errors, we employ Decision-Focused Learning (DFL) to learn the NN parameters with the objective of directly improving the HVAC cost performance. However, the discrete nature of MIQP hinders DFL, as it leads to undefined and discontinuous gradients, thus impeding standard gradient-based training. We leverage Stochastic Smoothing (SS) to enable efficient gradient computation without the need to differentiate the MIQP. Experiments on a realistic five-zone building using a high-fidelity simulator demonstrate that the proposed SS-DFL approach outperforms conventional identify-then-optimize (i.e., the thermal dynamics model is identified on historical data then used in optimization) and relaxed DFL methods in both cost savings and grid service performance, highlighting its potential for scalable, grid-aware building control.

eess.SY↗

arXiv2025

Unlocking Transmission Flexibility under Uncertainty: Getting Dynamic Line Ratings into Electricity Markets

Static transmission line ratings may lead to underutilization of line capacity due to overly conservative assumptions. Grid-enhancing technologies (GETs) such as dynamic line ratings (DLRs), which adjust line capacity based on real-time conditions, are a techno-economically viable alternative to increase the utilization of existing power lines. Nonetheless, their adoption has been slow, partly due to the absence of operational tools that effectively account for simultaneous impacts on dispatch and pricing. In this paper, we represent transmission capacity with DLRs as a stock-like resource with time-variant interdependency, which is modeled via an approximation of line temperature evolution process, decoupling the impacts of ambient weather conditions and power flow on transmission line temperature and thus capacity. We integrate DLRs into a multi-period DC optimal power flow problem, with chance constrains addressing correlated uncertainty in DLRs and renewable generation. This yields non-convex problems that we transform into a tractable convex form by linearization. We derive locational marginal energy and ancillary services prices consistent with a competitive equilibrium. Numerical experiments on the 11-zone and 1814-node NYISO systems demonstrate its performance, including impacts on dispatch, pricing, and marginal carbon emissions.

eess.SY↗

arXiv2025

Decision-Focused Learning for Complex System Identification: HVAC Management System Application

As opposed to conventional training methods tailored to minimize a given statistical metric or task-agnostic loss (e.g., mean squared error), Decision-Focused Learning (DFL) trains machine learning models for optimal performance in downstream decision-making tools. We argue that DFL can be leveraged to learn the parameters of system dynamics, expressed as constraint of the convex optimization control policy, while the system control signal is being optimized, thus creating an end-to-end learning framework. This is particularly relevant for systems in which behavior changes once the control policy is applied, hence rendering historical data less applicable. The proposed approach can perform system identification - i.e., determine appropriate parameters for the system analytical model - and control simultaneously to ensure that the model's accuracy is focused on areas most relevant to control. Furthermore, because black-box systems are non-differentiable, we design a loss function that requires solely to measure the system response. We propose pre-training on historical data and constraint relaxation to stabilize the DFL and deal with potential infeasibilities in learning. We demonstrate the usefulness of the method on a building Heating, Ventilation, and Air Conditioning day-ahead management system for a realistic 15-zone building located in Denver, US. The results show that the conventional RC building model, with the parameters obtained from historical data using supervised learning, underestimates HVAC electrical power consumption. For our case study, the ex-post cost is on average six times higher than the expected one. Meanwhile, the same RC model with parameters obtained via DFL underestimates the ex-post cost only by 3%.

eess.SY↗

arXiv2025

Electricity Market-Clearing With Extreme Events

Extreme events jeopardize power network operations, causing beyond-design failures and massive supply interruptions. Existing market designs fail to internalize and systematically assess the risk of extreme and rare events. Efficiently maintaining the reliability of renewable-dominant power systems during extreme weather events requires co-optimizing system resources, while differentiating between large/rare and small/frequent deviations from forecast conditions. To address this gap in both research and practice, we propose managing the uncertainties associated with extreme weather events through an additional reserve service, termed extreme reserve. The procurement of extreme reserve is co-optimized with energy and regular reserve using a large deviation theory chance-constrained (LDT-CC) model, where LDT offers a mathematical framework to quantify the increased uncertainty during extreme events. To mitigate the high additional costs associated with reserve scheduling under the LDT-CC model, we also propose an LDT model based on weighted chance constraints (LDT-WCC). This model prepares the power system for extreme events at a lower cost, making it a less conservative alternative to the LDT-CC model. The proposed market design leads to a competitive equilibrium while ensuring cost recovery. Numerical experiments on an illustrative system and a modified 8-zone ISO New England system highlight the advantages of the proposed market design.

eess.SY↗

arXiv2024

Prescribing Decision Conservativeness in Two-Stage Power Markets: A Distributionally Robust End-to-End Approach

This paper presents an end-to-end framework for calibrating wind power forecast models to minimize operational costs in two-stage power markets, where the first stage involves a distributionally robust optimal power flow (DR-OPF) model. Unlike traditional methods that adjust forecast parameters and uncertainty quantification (UQ) separately, this framework jointly optimizes both the forecast model parameters and the decision conservativeness, which determines the size of the ambiguity set in the DR-OPF model. The framework aligns UQ with actual uncertainty realizations by directly optimizing downstream operational costs, a process referred to as cost-oriented calibration. The calibration is achieved using a gradient descent approach. To enable efficient differentiation, the DR-OPF problem is reformulated into a convex form, and the Envelope Theorem is leveraged to simplify gradient derivation in the two-stage setting. Additionally, the framework supports distributed implementation, enhancing data privacy and reducing computational overhead. By proactively calibrating forecast parameters and prescribing optimal decision conservativeness, the framework significantly enhances cost efficiency and reliability in power system operations. Numerical experiments on an IEEE 5-bus system demonstrate the effectiveness and efficiency of the proposed approach.

eess.SY↗

arXiv2024

A New Definition of Demand Response in the Distributed Energy Resource Era

Demand response is a concept that has been around since the very first electric power systems. However, we have seen an explosion of research on demand response and demand-side technologies in the past 30 years, coinciding with the shift towards liberalized/deregulated electricity markets and efforts to decarbonize the power sector. Now we are also seeing a shift towards more distributed/decentralized electric systems; we have entered the era of "distributed energy resources," which require new grid management, operational, and control strategies. Given this paradigm shift, we argue that the concept of demand response needs to be revisited, and more carefully/consistently defined to enable us to better utilize this massive resource for economic, technical, environmental, and societal aims. In this paper, we survey existing demand response definitions, highlight their shortcomings, propose a new definition, and describe how this new definition enables us to more effectively harness the value of demand response in modern power systems. We conclude with a demand response research agenda informed by a discussion of demand response barriers and enablers.

eess.SY↗

arXiv2024

Risk-Aware Security-Constrained Unit Commitment: Taming the Curse of Real-Time Volatility and Consumer Exposure

We propose an enhancement to wholesale electricity markets whereby the exposure of consumers to increasingly large and volatile consumer payments arising as a byproduct of volatile real-time net loads -- i.e., loads minus renewable outputs -- and prices, both compared to day-ahead cleared values. We incorporate a robust estimate of such excess payments into the day-ahead computation and specifically seek to account for volatility in real-time net loads and renewable generation. Our model features a data-driven uncertainty set based on principal component analysis, which accommodates both load and wind production volatility and captures locational correlation of uncertain data. To solve the model more efficiently, we develop a decomposition algorithm that can handle nonconvex subproblems. Our extensive experiments on a realistic NYISO data set show that the risk-aware model protects the consumers from potential high costs caused by adverse circumstances.

math.OC↗