Searcharxiv⌕ Search

arXiv subjects

Qiaozhu Zhai

Publications and source records attributed to Qiaozhu Zhai.

15 recordsLinked to original sources

NeuroRisk: Physics-Informed Neural Optimization for Risk-Aware Traffic Engineering

In production Wide-Area Networks (WANs), correlated failures dominate availability losses, forcing operators to reserve large safety margins that leave substantial capacity underutilized. Achieving high utilization under strict availability targets therefore requires risk-aware Traffic Engineering (TE) over dozens to hundreds of probabilistic failure scenarios-yet solving this problem at operational timescales remains elusive. We demonstrate that existing risk-aware formulations can be unified under an embedded Sort-and-Select structure, exposing a fundamental trade-off between expressiveness and tractability: classical optimizers either restrict scenario selection for efficiency or incur prohibitive decomposition costs. While deep learning appears promising, prior Deep TE methods mainly target maximum link utilization and rely on scaling-based feasibility, which fundamentally breaks under explicit capacity constraints and scenario-dependent risk. We present NeuroRisk, a physics-informed deep unrolled optimizer that exploits the structure of Sort-and-Select. NeuroRisk enforces feasibility via gated edge-local reservations and represents scenario sets through permutation-invariant, gradient-aligned cues. Evaluations on production-style WANs show that NeuroRisk achieves small optimality gaps relative to the solver with orders of magnitude speedup $(10^2- 10^5 \times)$ on risk objectives, while outperforming neural baselines on nominal throughput.

cs.NI↗

A Fast Solver-Free Algorithm for Traffic Engineering in Large-Scale Data Center Network

Rapid growth of data center networks (DCNs) poses significant challenges for large-scale traffic engineering (TE). Existing acceleration strategies, which rely on commercial solvers or deep learning, face scalability issues and struggle with degrading performance or long computational time. Unlike existing algorithms adopting parallel strategies, we propose Sequential Source-Destination Optimization (SSDO), a sequential solver-free algorithm for TE. SSDO decomposes the problem into subproblems, each focused on adjusting the split ratios for a specific source-destination (SD) demand while keeping others fixed. To enhance the efficiency of subproblem optimization, we design a Balanced Binary Search Method (BBSM), which identifies the most balanced split ratios among multiple solutions that minimize Maximum Link Utilization (MLU). SSDO dynamically updates the sequence of SDs based on real-time utilization, which accelerates convergence and enhances solution quality. We evaluate SSDO on Meta DCNs and two wide-area networks. In a Meta topology, SSDO achieves a 65\% and 60\% reduction in normalized MLU compared to TEAL and POP, two state-of-the-art TE acceleration methods, while delivering a $12\times$ speedup over POP. These results demonstrate the superior performance of SSDO in large-scale TE.

cs.NI↗

OPBO: Order-Preserving Bayesian Optimization

Bayesian optimization is an effective method for solving expensive black-box optimization problems. Most existing methods use Gaussian processes (GP) as the surrogate model for approximating the black-box objective function, it is well-known that it can fail in high-dimensional space (e.g., dimension over 500). We argue that the reliance of GP on precise numerical fitting is fundamentally ill-suited in high-dimensional space, where it leads to prohibitive computational complexity. In order to address this, we propose a simple order-preserving Bayesian optimization (OPBO) method, where the surrogate model preserves the order, instead of the value, of the black-box objective function. Then we can use a simple but effective OP neural network (NN) to replace GP as the surrogate model. Moreover, instead of searching for the best solution from the acquisition model, we select good-enough solutions in the ordinal set to reduce computational cost. The experimental results show that for high-dimensional (over 500) black-box optimization problems, the proposed OPBO significantly outperforms traditional BO methods based on regression NN and GP. The source code is available at https://github.com/pengwei222/OPBO.

cs.LG↗

ATRO: A Fast Algorithm for Topology Engineering of Reconfigurable Datacenter Networks

Reconfigurable data center networks (DCNs) enhance traditional architectures with optical circuit switches (OCSs), enabling dynamic reconfiguration of inter-pod links, i.e., the logical topology. Optimizing this topology is crucial for adapting to traffic dynamics but is challenging due to its combinatorial nature. The complexity increases further when demands can be distributed across multiple paths, requiring joint optimization of topology and routing. We propose Alternating Topology and Routing Optimization (ATRO), a unified framework that supports both one-hop topology optimization (where traffic is routed via direct paths) and multi-hop joint optimization (where routing is also optimized). Although these settings differ in constraints, both are combinatorially hard and challenge solver-based methods. ATRO addresses both cases efficiently: in the one-hop case, it guarantees the global optimum via an accelerated binary search; in the multi-hop case, it alternates between topology and routing updates, with routing steps optionally accelerated by existing traffic engineering (TE) methods. ATRO supports warm-starting and improves solution quality monotonically across iterations. ATRO remains competitive even when paired with solver-free TE methods, forming a fully solver-free optimization pipeline that still outperforms prior approaches in runtime and maximum link utilization across diverse workloads.

cs.NI↗

Robust Co-Optimization of Distribution Network Hardening and Mobile Resource Scheduling with Decision-Dependent Uncertainty

This paper studies the robust co-planning of proactive network hardening and mobile hydrogen energy resources (MHERs) scheduling, which is to enhance the resilience of power distribution network (PDN) against the disastrous events. A decision-dependent robust optimization model is formulated with min-max resilience constraint and discrete recourse structure, which helps achieve the load survivability target considering endogenous uncertainties. Different from the traditional model with a fixed uncertainty set, we adopt a dynamic representation that explicitly captures the endogenous uncertainties of network contingency as well as the available hydrogen storage levels of MHERs, which induces a decision-dependent uncertainty (DDU) set. Also, the multi-period adaptive routing and energy scheduling of MHERs are modeled as a mixed-integer recourse problem for further decreasing the resilience cost. Then, a nested parametric column-and-constraint generation (N-PC&CG) algorithm is customized and developed to solve this challenging formulation. By leveraging the structural property of the DDU set as well as the combination of discrete recourse decisions and the corresponding extreme points, we derive a strengthened solution scheme with nontrivial enhancement strategies to realize efficient and exact computation. Numerical results on 14-bus test system and 56-bus real-world distribution network demonstrate the resilience benefits and economical feasibility of the proposed method under different damage severity levels. Moreover, the enhanced N-PC&CG shows a superior solution capability to support prompt decisions for resilient planning with DDU models.

eess.SY↗

Proactive Robust Hardening of Resilient Power Distribution Network: Decision-Dependent Uncertainty Modeling and Fast Solution Strategy

To address the power system hardening problem, traditional approaches often adopt robust optimization (RO) that considers a fixed set of concerned contingencies, regardless of the fact that hardening some components actually renders relevant contingencies impractical. In this paper, we directly adopt a dynamic uncertainty set that explicitly incorporates the impact of hardening decisions on the worst-case contingencies, which leads to a decision-dependent uncertainty (DDU) set. Then, a DDU-based robust-stochastic optimization (DDU-RSO) model is proposed to support the hardening decisions on distribution lines and distributed generators (DGs). Also, the randomness of load variations and available storage levels is considered through stochastic programming (SP) in the innermost level problem. Various corrective measures (e.g., the joint scheduling of DGs and energy storage) are included, coupling with a finite support of stochastic scenarios, for resilience enhancement. To relieve the computation burden of this new hardening formulation, an enhanced customization of parametric column-and-constraint generation (P-C&CG) algorithm is developed. By leveraging the network structural information, the enhancement strategies based on resilience importance indices are designed to improve the convergence performance. Numerical results on 33-bus and 118-bus test distribution networks have demonstrated the effectiveness of DDU-RSO aided hardening scheme. Furthermore, in comparison to existing solution methods, the enhanced P-C&CG has achieved a superior performance by reducing the solution time by a few orders of magnitudes.

eess.SY↗

Stochastic-Robust Planning of Networked Hydrogen-Electrical Microgrids: A Study on Induced Refueling Demand

Hydrogen-electrical microgrids are increasingly assuming an important role on the pathway toward decarbonization of energy and transportation systems. This paper studies networked hydrogen-electrical microgrids planning (NHEMP), considering a critical but often-overlooked issue, i.e., the demand-inducing effect (DIE) associated with infrastructure development decisions. Specifically, higher refueling capacities will attract more refueling demand of hydrogen-powered vehicles (HVs). To capture such interactions between investment decisions and induced refueling demand, we introduce a decision-dependent uncertainty (DDU) set and build a trilevel stochastic-robust formulation. The upper-level determines optimal investment strategies for hydrogen-electrical microgrids, the lower-level optimizes the risk-aware operation schedules across a series of stochastic scenarios, and, for each scenario, the middle-level identifies the "worst" situation of refueling demand within an individual DDU set to ensure economic feasibility. Then, an adaptive and exact decomposition algorithm, based on Parametric Column-and-Constraint Generation (PC&CG), is customized and developed to address the computational challenge and to quantitatively analyze the impact of DIE. Case studies on an IEEE exemplary system validate the effectiveness of the proposed NHEMP model and the PC&CG algorithm. It is worth highlighting that DIE can make an important contribution to the economic benefits of NHEMP, yet its significance will gradually decrease when the main bottleneck transits to other system restrictions.

math.OC↗

Robust Approximate Dynamic Programming for Large-scale Unit Commitment with Energy Storages

The multistage robust unit commitment (UC) is of paramount importance for achieving reliable operations considering the uncertainty of renewable realizations. The typical affine decision rule method and the robust feasible region method may achieve uneconomic dispatches as the dispatch decisions just rely on the current-stage information. Through approximating the future cost-to-go functions, the dual dynamic programming based methods have been shown adaptive to the multistage robust optimization problems, while suffering from high computational complexity. Thus, we propose the robust approximate dynamic programming (RADP) method to promote the computational speed and the economic performance for large-scale robust UC problems. RADP initializes the candidate points for guaranteeing the feasibility of upper bounding the value functions, solves the linear McCormick relaxation based bilinear programming to obtain the worst cases, and combines the primal and dual updates for this hybrid binary and continuous decision-making problem to achieve fast convergence. We can verify that the RADP method enjoys a finite termination guarantee for the multistage robust optimization problems with achieving suboptimal solutions. Numerical tests on 118-bus and 2383-bus transmission systems have demonstrated that RADP can approach the suboptimal economic performance at significantly improved computational efficiency.

math.OC↗

A Fast Solution Method for Large-scale Unit Commitment Based on Lagrangian Relaxation and Dynamic Programming

The unit commitment problem (UC) is crucial for the operation and market mechanism of power systems. With the development of modern electricity, the scale of power systems is expanding, and solving the UC problem is also becoming more and more difficult. To this end, this paper proposes a new fast solution method based on Lagrangian relaxation and dynamic program-ming. Firstly, the UC solution is estimated to be an initial trial UC solution by a fast method based on Lagrangian relaxation. This initial trial UC solution fully considers the system-wide con-straints. Secondly, a dynamic programming module is introduced to adjust the trial UC solution to make it satisfy the unit-wise constraints. Thirdly, a method for constructing a feasible UC solution is proposed based on the adjusted trial UC solution. Specifically, a feasibility-testing model and an updating strategy for the trial UC solution are established in this part. Numerical tests are implemented on IEEE 24-bus, IEEE 118-bus, Polish 2383-bus, and French 6468-bus systems, which verify the effec-tiveness and efficiency of the proposed method.

math.OC↗

A General Formulation for Evaluating the Performance of Linear Power Flow Models

Linear power flow (LPF) models are essential in power system analysis. Various LPF models are proposed, but some crucial questions are still remained: what is the performance bound (e.g., the error bound) of LPF models, how to know a branch is applicable for LPF models or not, and what is the best LPF model. In this paper, these crucial questions are answered and a general formulation (GF) for evaluating the performance of LPF models is proposed. The GF actually figure out two core difficulties, the one is how to define the definition range of the LPF models, and the second is how to analytically obtain the best LPF model and evaluate the performance of a given LPF model. Besides, the key factors that affect the performance of LPF models are also analyzed through the proposed framework. The case studies compare the proposed LPF model with the DC power flow model, the physical-model-driven LPF model, and the data-driven LPF model, and the results show the effectiveness as well as the superiority of the proposed method.

eess.SY↗

Data Based Linearization: Least-Squares Based Approximation

Linearization of power flow is an important topic in power system analysis. The computational burden can be greatly reduced under the linear power flow model while the model error is the main concern. Therefore, various linear power flow models have been proposed in literature and dedicated to seek the optimal approximation. Most linear power flow models are based on some kind of transformation/simplification/Taylor expansion of AC power flow equations and fail to be accurate under cold-start mode. It is surprising that data-based linearization methods have not yet been fully investigated. In this paper, the performance of a data-based least-squares approximation method is investigated. The resulted cold-start sensitive factors are named as least-squares distribution factors (LSDF). Compared with the traditional power transfer distribution factors (PTDF), it is found that the LSDF can work very well for systems with large load variation, and the average error of LSDF is only about 1% of the average error of PTDF. Comprehensive numerical testing is performed and the results show that LSDF has attractive performance in all studied cases and has great application potential in occasions requiring only cold-start linear power flow models.

eess.SY↗

A Memory-Efficient Sketch Method for Estimating High Similarities in Streaming Sets

Estimating set similarity and detecting highly similar sets are fundamental problems in areas such as databases, machine learning, and information retrieval. MinHash is a well-known technique for approximating Jaccard similarity of sets and has been successfully used for many applications such as similarity search and large scale learning. Its two compressed versions, b-bit MinHash and Odd Sketch, can significantly reduce the memory usage of the original MinHash method, especially for estimating high similarities (i.e., similarities around 1). Although MinHash can be applied to static sets as well as streaming sets, of which elements are given in a streaming fashion and cardinality is unknown or even infinite, unfortunately, b-bit MinHash and Odd Sketch fail to deal with streaming data. To solve this problem, we design a memory efficient sketch method, MaxLogHash, to accurately estimate Jaccard similarities in streaming sets. Compared to MinHash, our method uses smaller sized registers (each register consists of less than 7 bits) to build a compact sketch for each set. We also provide a simple yet accurate estimator for inferring Jaccard similarity from MaxLogHash sketches. In addition, we derive formulas for bounding the estimation error and determine the smallest necessary memory usage (i.e., the number of registers used for a MaxLogHash sketch) for the desired accuracy. We conduct experiments on a variety of datasets, and experimental results show that our method MaxLogHash is about 5 times more memory efficient than MinHash with the same accuracy and computational cost for estimating high similarities.

cs.DS↗

A Variable Reduction Method for Large-Scale Security Constrained Unit Commitment

Efficient methods for large-scale security constrained unit commitment (SCUC) problems have long been an important research topic and a challenge especially in market clearing computation. For large-scale SCUC, the Lagrangian relaxation methods (LR) and the mixed integer programming methods (MIP) are most widely adopted. However, LR usually suffers from slow convergence; and the computational burden of MIP is heavy when the binary variable number is large. In this paper, a variable reduction method is proposed: First, the time-coupled constraints in the original SCUC problem are relaxed and many single-period SCUC problems (s-UC) are obtained. Second, LR is used to solve the s-UCs. Different from traditional LR with iterative subgradient method, it is found that the optimal multipliers and the approximate UC solutions of s-UCs can be obtained by solving linear programs. Third, a criterion for choosing and fixing the UC variables in the SCUC problem is established, hence the number of binary variables is reduced. Last, the SCUC with reduced binary variables is solved by MIP solver to obtain the final UC solution. The proposed method is tested on the IEEE 118-bus system and a 6484-bus system. The results show the method is very efficient and effective.

math.OC↗

Multi-Stage Robust Transmission Constrained Unit Commitment: A Decomposition Framework with Implicit Decision Rules

With the integration of large-scale renewable energy sources to power systems, many optimization methods have been applied to solve the stochastic/uncertain transmission-constrained unit commitment (TCUC) problem. Among all methods, two-stage and multi-stage robust optimization-based methods are the most widely adopted ones. In the two-stage methods, nonanticipativity of economic dispatch (ED) decisions are not considered. While in multi-stage methods, explicit decision rules (for example, affine decision rules) are usually adopted to guarantee nonanticipativity of ED decisions. With explicit decision rules, the computational burden can be heavy and the optimality of the solution is affected. In this paper, a multi-stage robust TCUC formulation with implicit decision rules is proposed, as well as a decomposition framework to solve it. The solutions are proved to be multi-stage robust and nonanticipativity of ED decisions is guaranteed. Meanwhile, a computationally efficient time-decoupled solution method for the feasibility check subproblems is also proposed such that the method is suitable for large-scale TCUC problems with uncertain loads/renewable injections. Numerical tests are conducted on the IEEE 118-bus system and Polish 2383-bus system. Performances of several state-of-the-art methods are compared.

math.OC↗

Robust Transmission Constrained Unit Commitment:A Column Merging Method

With rapid integration of power sources with uncertainty, robustness must be carefully considered in the transmission constrained unit commitment (TCUC) problem. The overall computational complexity of the robust TCUC methods is closely related to the vertex number of the uncertainty set. The vertex number is further associated with 1) the period number in the scheduling horizon as well as 2) the number of nodes with uncertain injections. In this paper, a column merging method (CMM) is proposed to reduce the computation burden by merging the uncertain nodes, while still guar-anteeing the robustness of the solution. By the CMM, the transmission constraints are modified, with the parameters obtained based on an analytical solution of a uniform approximation problem, so that the computational time is negligi-ble. The CMM is applied under a greedy-algorithm based framework, where the number of merged nodes and the ap-proximation error can be well balanced. The CMM is designed as a preprocessing tool to improve the solution efficiency for robust TCUC problems and is compatible with many solution methods (like two-stage and multi-stage robust optimi-zation methods). Numerical tests show the method is effective.

math.OC↗