arXiv subjects

Wangkun Xu

Publications and source records attributed to Wangkun Xu.

16 recordsLinked to original sources

arXiv2026

Structured Differentiable Optimization for Efficient Decision-focused Learning in Power Systems

Decision-focused learning (DfL) trains forecasting models to align downstream decision consequences, such as power-system operating costs. However, its application to realistic power networks is limited by the need to repeatedly solve and differentiate large optimization problems during training. This paper presents DiffAPQP, a solver-flexible framework and open-source Python package for scalable DfL with affine-parametric quadratic programs. To accelerate the forward pass, DiffAPQP automatically canonicalizes quadratic power-system models written in CVXPY into a differentiation-ready representation and takes advantage of the repetitive solving structure through solver warm-start and solver-data update during training. For the backward pass acceleration, we establish the equivalence between differentiation through the full KKT system and a reduced system obtained by eliminating inactive inequality constraints. For training losses depending solely on the optimal value, we further derive an envelope-theorem-based gradient that avoids solving an adjoint KKT system, resulting in eligible backward time. To our knowledge, this work presents the first solver-based end-to-end DfL demonstration on the IEEE 118-bus system with a 24-hour coupled economic-dispatch and redispatch horizon. Under matched SCS and Clarabel backends on a Linux machine, DiffAPQP achieves $2.27\times$--$3.58\times$ closed-loop and $3.62\times$--$4.38\times$ counterfactual end-to-end DfL training speedups over CvxpyLayers. The best solver configurations increase these speedups to $3.91\times$ (from 38.65 to 9.55 min/epoch) and $6.39\times$ (from 10.73 to 1.68 min/epoch), respectively. Additionally, DiffAPQP reduces peak memory usage by approximately $50\%$, while keeping similar operating costs as CvxpyLayers.

eess.SY↗

arXiv2026

Input Convex Neural Network as a Surrogate in Stability-Constrained Optimization for IBR-dominated Power Systems

Input convex neural networks (ICNNs) are increasingly used as surrogates for stability indices and embedded as constraints in power-system optimization. This letter clarifies two recurring formulation limitations that can negate ICNN convexity benefits: (i) applying generic Big-$M$ mixed-integer reformulations introduces auxiliary binaries that are unnecessary for enforcing ICNN sublevel constraints; and (ii) reversing the stability inequality transforms a convex sublevel set into a generally nonconvex superlevel set, invalidating global-convergence guarantees of cut-based methods. After clarifying the limitations, we provide (i) an exact LP-based epigraph reformulation for ReLU-ICNNs, (ii) an outer-approximation scheme with global guarantees under the sublevel convention, and (iii) a feasibility-preserving inner-approximation scheme for the superlevel convention, with simulations on IEEE 14- and 118-bus unit commitment instances.

math.OC↗

arXiv2026

Flow-based Polynomial Chaos Expansion for Uncertainty Quantification in Power System Dynamic Simulation

The large-scale integration of renewable energy sources introduces significant operational uncertainty into power systems. Although Polynomial Chaos Expansion (PCE) provides an efficient tool for uncertainty quantification (UQ) in power system dynamics, its accuracy depends critically on the faithful representation of input uncertainty, an assumption that is oftern violated in practice due to correlated, non-Gaussian, and otherwise complex data distributions. In contrast to purely data-driven surrogates that often overlook rigorous input distribution modelling, this paper introduces flow-based PCE, a unified framework that couples expressive input modelling with efficient uncertainty propagation. Specifically, normalising flows are employed to learn an invertible transport map from a simple base distribution to the empirical joint distribution of uncertain inputs, and this map is then integrated directly into the PCE construction. In addition, the Map Smoothness Index (MSI) is introduced as a new metric to quantify the quality of the learned map, and smoother transformations are shown to yield more accurate PCE surrogates. The proposed Flow-based PCE framework is validated on benchmark dynamic models, including the IEEE 14-bus system and the Great Britain transmission system, under a range of uncertainty scenarios.

eess.SY↗

arXiv2025

Learning-Augmented Power System Operations: A Unified Optimization View

With the increasing penetration of renewable energy and inverter-based resources, traditional physics-based power-system operation faces growing challenges in maintaining economic efficiency, security, and robustness. Machine learning (ML) has emerged as a powerful tool for modeling complex system dynamics and uncertainty. However, standalone ML pipelines, including model selection, training, and validation, are often designed separately from the downstream optimization problems they influence, which can lead to suboptimal system-level decisions. To address this gap, this paper proposes \emph{Learning-Augmented Power System Operations} (LAPSO), a unified optimization-centered framework that treats ML as an explicit component of power-system operational decision-making. First, LAPSO provides generalized mathematical template covering both decision-independent predictors that parameterize downstream optimization and decision-dependent learned surrogates that enter optimization as auxiliary constraints. Second, it designs ML pipelines using optimization-aware criteria, including solution-quality, computational tractability, constraint satisfaction, and economic performance. We instantiate LAPSO on both stability-constrained optimization (SCO) and objective-based forecasting (OBF), and show how the framework provides actionable guidance for selecting learned components. We further extend the framework to a hybrid forecast--operation--control chain and use it to organize heterogeneous uncertainty sources. Finally, we release an open-source Python package, \texttt{lapso}, for modularly augmenting existing power-system optimization models with ML components. Code and datasets are available at: https://github.com/xuwkk/lapso_exp.

eess.SY↗

arXiv2025

Qubit-Efficient Quantum Annealing for Stochastic Unit Commitment

Stochastic Unit Commitment (SUC) has been proposed to manage the uncertainties driven by renewable integration, but it leads to significant computational complexity. When accelerated by Benders Decomposition (BD), the master problem becomes binary integer programming, which is still NP-hard and computationally demanding for classical methods. Quantum Annealing (QA), known for efficiently solving Quadratic Unconstrained Binary Optimization (QUBO) problems, presents a potential solution. However, existing quantum algorithms rely on slack variables to handle linear binary inequality constraints, leading to increased qubit consumption and reduced computational efficiency. To solve the problem, this paper introduces the Powell-Hestenes-Rockafellar Augmented Lagrangian Multiplier (PHR-ALM) method to eliminate the need for slack variables, making qubit consumption independent of the increasing number of Benders cuts. To further reduce the qubit overhead, quantum ADMM is applied to break large-scale SUC into smaller blocks for sequential solutions, which does not scale with the number of generators. Finally, the simulation results on both 4-generator and the IEEE bus-118 systems demonstrate the feasibility and scalability of the proposed algorithm, indicating its superior qubit and runtime efficiency over classical and baseline quantum approaches on the D-Wave QPU platform.

quant-ph↗

arXiv2024

On the Incorporation of Stability Constraints into Sequential Operational Scheduling

With the increasing penetration of Inverter-Based Resources (IBRs), power system stability constraints must be incorporated into the operational framework, transforming it into stability-constrained optimization. Currently, there exist parallel research efforts on developing the stability constraints within DC power flow-based unit commitment (UC) and AC Optimal Power Flow (OPF). However, few studies discuss how including such constraints can interact with each other and eventually impact grid stability. In this context, this work simulates a realistic power system decision making framework and provides a thorough analysis on the necessity of incorporating frequency nadir and small signal stability constraints into these sequentially connected two operation stages. The simulation results demonstrate that including both stability constraints in the UC is essential to maintain power system stability, while the inclusion in AC OPF can further improve the stability index.

eess.SY↗

arXiv2024

Exact, Tractable Gauss-Newton Optimization in Deep Reversible Architectures Reveal Poor Generalization

Second-order optimization has been shown to accelerate the training of deep neural networks in many applications, often yielding faster progress per iteration on the training loss compared to first-order optimizers. However, the generalization properties of second-order methods are still being debated. Theoretical investigations have proved difficult to carry out outside the tractable settings of heavily simplified model classes -- thus, the relevance of existing theories to practical deep learning applications remains unclear. Similarly, empirical studies in large-scale models and real datasets are significantly confounded by the necessity to approximate second-order updates in practice. It is often unclear whether the observed generalization behaviour arises specifically from the second-order nature of the parameter updates, or instead reflects the specific structured (e.g.\ Kronecker) approximations used or any damping-based interpolation towards first-order updates. Here, we show for the first time that exact Gauss-Newton (GN) updates take on a tractable form in a class of deep reversible architectures that are sufficiently expressive to be meaningfully applied to common benchmark datasets. We exploit this novel setting to study the training and generalization properties of the GN optimizer. We find that exact GN generalizes poorly. In the mini-batch training setting, this manifests as rapidly saturating progress even on the \emph{training} loss, with parameter updates found to overfit each mini-batchatch without producing the features that would support generalization to other mini-batches. We show that our experiments run in the ``lazy'' regime, in which the neural tangent kernel (NTK) changes very little during the course of training. This behaviour is associated with having no significant changes in neural representations, explaining the lack of generalization.

cs.LG↗

arXiv2024

Efficient Sampling for Data-Driven Frequency Stability Constraint via Forward-Mode Automatic Differentiation

Encoding frequency stability constraints in the operation problem is challenging due to its complex dynamics. Recently, data-driven approaches have been proposed to learn the stability criteria offline with the trained model embedded as a constraint of online optimization. However, random sampling of stationary operation points is less efficient in generating balanced stable and unstable samples. Meanwhile, the performance of such a model is strongly dependent on the quality of the training dataset. Observing this research gap, we propose a gradient-based data generation method via forward-mode automatic differentiation. In this method, the original dynamic system is augmented with new states that represent the dynamic of sensitivities of the original states, which can be solved by invoking any ODE solver for a single time. To compensate for the contradiction between the gradient of various frequency stability criteria, gradient surgery is proposed by projecting the gradient on the normal plane of the other. In the end, we demonstrate the superior performance of the proposed sampling algorithm, compared with the unrolling differentiation and finite difference. All codes are available at https://github.com/xuwkk/frequency_sample_ad.

eess.SY↗

arXiv2024

Task-Aware Machine Unlearning and Its Application in Load Forecasting

Data privacy and security have become a non-negligible factor in load forecasting. Previous researches mainly focus on training stage enhancement. However, once the model is trained and deployed, it may need to `forget' (i.e., remove the impact of) part of training data if the these data are found to be malicious or as requested by the data owner. This paper introduces the concept of machine unlearning which is specifically designed to remove the influence of part of the dataset on an already trained forecaster. However, direct unlearning inevitably degrades the model generalization ability. To balance between unlearning completeness and model performance, a performance-aware algorithm is proposed by evaluating the sensitivity of local model parameter change using influence function and sample re-weighting. Furthermore, we observe that the statistical criterion such as mean squared error, cannot fully reflect the operation cost of the downstream tasks in power system. Therefore, a task-aware machine unlearning is proposed whose objective is a trilevel optimization with dispatch and redispatch problems considered. We theoretically prove the existence of the gradient of such an objective, which is key to re-weighting the remaining samples. We tested the unlearning algorithms on linear, CNN, and MLP-Mixer based load forecasters with a realistic load dataset. The simulation demonstrates the balance between unlearning completeness and operational cost. All codes can be found at https://github.com/xuwkk/task_aware_machine_unlearning.

cs.LG↗

arXiv2023

E2E-AT: A Unified Framework for Tackling Uncertainty in Task-aware End-to-end Learning

Successful machine learning involves a complete pipeline of data, model, and downstream applications. Instead of treating them separately, there has been a prominent increase of attention within the constrained optimization (CO) and machine learning (ML) communities towards combining prediction and optimization models. The so-called end-to-end (E2E) learning captures the task-based objective for which they will be used for decision making. Although a large variety of E2E algorithms have been presented, it has not been fully investigated how to systematically address uncertainties involved in such models. Most of the existing work considers the uncertainties of ML in the input space and improves robustness through adversarial training. We extend this idea to E2E learning and prove that there is a robustness certification procedure by solving augmented integer programming. Furthermore, we show that neglecting the uncertainty of COs during training causes a new trigger for generalization errors. To include all these components, we propose a unified framework that covers the uncertainties emerging in both the input feature space of the ML models and the COs. The framework is described as a robust optimization problem and is practically solved via end-to-end adversarial training (E2E-AT). Finally, the performance of E2E-AT is evaluated by a real-world end-to-end power system operation problem, including load forecasting and sequential scheduling tasks.

cs.LG↗

arXiv2023

Availability Adversarial Attack and Countermeasures for Deep Learning-based Load Forecasting

The forecast of electrical loads is essential for the planning and operation of the power system. Recently, advances in deep learning have enabled more accurate forecasts. However, deep neural networks are prone to adversarial attacks. Although most of the literature focuses on integrity-based attacks, this paper proposes availability-based adversarial attacks, which can be more easily implemented by attackers. For each forecast instance, the availability attack position is optimally solved by mixed-integer reformulation of the artificial neural network. To tackle this attack, an adversarial training algorithm is proposed. In simulation, a realistic load forecasting dataset is considered and the attack performance is compared to the integrity-based attack. Meanwhile, the adversarial training algorithm is shown to significantly improve robustness against availability attacks. All codes are available at https://github.com/xuwkk/AAA_Load_Forecast.

cs.LG↗

arXiv2022

Blending Data and Physics Against False Data Injection Attack: An Event-Triggered Moving Target Defence Approach

Fast and accurate detection of cyberattacks is a key element for a cyber-resilient power system. Recently, data-driven detectors and physics-based Moving Target Defences (MTD) have been proposed to detect false data injection (FDI) attacks on state estimation. However, the uncontrollable false positive rate of the data-driven detector and the extra cost of frequent MTD usage limit their wide applications. Few works have explored the overlap between these two areas. To fill this gap, this paper proposes blending data-driven and physics-based approaches to enhance the detection performance. To start, a physics-informed data-driven attack detection and identification algorithm is proposed. Then, an MTD protocol is triggered by the positive alarm from the data-driven detector. The MTD is formulated as a bilevel optimisation to robustly guarantee its effectiveness against the worst-case attack around the identified attack vector. Meanwhile, MTD hiddenness is also improved so that the defence cannot be detected by the attacker. To guarantee feasibility and convergence, the convex two-stage reformulation is derived through duality and linear matrix inequality. The simulation results verify that blending data and physics can achieve extremely high detection rate while simultaneously reducing the false positive rate of the data-driven detector and the extra cost of MTD. All codes are available at https://github.com/xuwkk/DDET-MTD.

eess.SY↗

arXiv2022

Robust Moving Target Defence Against False Data Injection Attacks in Power Grids

Recently, moving target defence (MTD) has been proposed to thwart false data injection (FDI) attacks in power system state estimation by proactively triggering the distributed flexible AC transmission system (D-FACTS) devices. One of the key challenges for MTD in power grid is to design its real-time implementation with performance guarantees against unknown attacks. Converting from the noiseless assumptions in the literature, this paper investigates the MTD design problem in a noisy environment and proposes, for the first time, the concept of robust MTD to guarantee the worst-case detection rate against all unknown attacks. We theoretically prove that, for any given MTD strategy, the minimal principal angle between the Jacobian subspaces corresponds to the worst-case performance against all potential attacks. Based on this finding, robust MTD algorithms are formulated for the systems with both complete and incomplete configurations. Extensive simulations using standard IEEE benchmark systems demonstrate the improved average and worst-case performances of the proposed robust MTD against state-of-the-art algorithms. All codes are available at https://github.com/xuwkk/Robust_MTD.

eess.SY↗

arXiv2022

Cyber-Physical Risk Assessment for False Data Injection Attacks Considering Moving Target Defences

In this paper, we examine the factors that influence the success of false data injection (FDI) attacks in the context of both cyber and physical styles of reinforcement. Many works consider the FDI attack in the context of the ability to change a measurement in a static system only. However, successful attacks will require first intrusion into a system followed by construction of an attack vector that can bypass bad data detection (BDD). In this way, we develop a full service framework for FDI risk assessment. The framework considers both the costs of system intrusion via a weighted graph assessment in combination with a physical, line overload-based vulnerability assessment. We present our simulations on a IEEE 14-bus system with an overlain RTU network to model the true risk of intrusion. The cyber model considers multiple methods of entry for the FDI attack including meter intrusion, RTU intrusion and combined style attacks. Post-intrusion our physical reinforcement model analyses the required level of topology divergence to protect against a branch overload from an optimised attack vector.

eess.SY↗

arXiv2022

A Deep Learning based Detection Method for Combined Integrity-Availability Cyber Attacks in Power System

As one of the largest and most complex systems on earth, power grid (PG) operation and control have stepped forward as a compound analysis on both physical and cyber layers which makes it vulnerable to assaults from economic and security considerations. A new type of attack, namely as combined data Integrity-Availability attack, has been recently proposed, where the attackers can simultaneously manipulate and blind some measurements on SCADA system to mislead the control operation and keep stealthy. Compared with traditional FDIAs, this combined attack can further complicate and vitiate the model-based detection mechanism. To detect such attack, this paper proposes a novel random denoising LSTM-AE (LSTMRDAE) framework, where the spatial-temporal correlations of measurements can be explicitly captured and the unavailable data is countered by the random dropout layer. The proposed algorithm is evaluated and the performance is verified on a standard IEEE 118-bus system under various unseen attack attempts.

eess.SY↗

arXiv2022

Multi-Agent Reinforcement Learning for Active Voltage Control on Power Distribution Networks

This paper presents a problem in power networks that creates an exciting and yet challenging real-world scenario for application of multi-agent reinforcement learning (MARL). The emerging trend of decarbonisation is placing excessive stress on power distribution networks. Active voltage control is seen as a promising solution to relieve power congestion and improve voltage quality without extra hardware investment, taking advantage of the controllable apparatuses in the network, such as roof-top photovoltaics (PVs) and static var compensators (SVCs). These controllable apparatuses appear in a vast number and are distributed in a wide geographic area, making MARL a natural candidate. This paper formulates the active voltage control problem in the framework of Dec-POMDP and establishes an open-source environment. It aims to bridge the gap between the power community and the MARL community and be a drive force towards real-world applications of MARL algorithms. Finally, we analyse the special characteristics of the active voltage control problems that cause challenges (e.g. interpretability) for state-of-the-art MARL approaches, and summarise the potential directions.

cs.LG↗