Searcharxiv⌕ Search

arXiv subjects

Danny H. K. Tsang

Publications and source records attributed to Danny H. K. Tsang.

At least 19 recordsLinked to original sources

ML-OPF-Bench: Benchmarking Machine Learning for Optimal Power Flow

Machine Learning (ML) methods promise a fast solution process for Optimal Power Flow (OPF). While inconsistent test cases, implementations, and evaluation metrics across existing studies make it challenging to determine which algorithmic advances are most critical for real-world deployment. To this end, we propose ML-OPF-Bench, a unified benchmark for AC- and DC-OPF that evaluates representative ML algorithms under a consistent pipeline, stress-tests them across system sizes, distribution shifts, and resource budgets, and ranks them with a multi-objective framework. We find that prediction accuracy alone is not a reliable indicator of operational feasibility. Under heavily loaded, congested conditions, even the strongest in-distribution performers lose their advantage, while feasibility is maintained largely by post-processing that enforces the target constraints rather than by the underlying pure ML predictor. Data scaling shows that prediction accuracy and constraint violations follow different trajectories, whereas compute scaling shows that returns diminish and that larger models do not consistently perform better. These results expose critical trade-offs among ML methods' speed, accuracy, and feasibility, and offer practical guidance for future ML-OPF design. We open-source the benchmark as an extensible Python package for integrating new learning-based OPF algorithms and evaluating them under the same standard as the existing baselines.

eess.SY↗

Beyond Imitation: Filtering On-Policy Distillation by Reasoning Progress

On-policy distillation (OPD) has emerged as an effective framework for post-training language models by pairing student-generated trajectories with dense token-level supervision from a teacher. However, OPD implicitly assumes that teacher-derived rewards are an appropriate proxy for reasoning progress, and therefore treats all teacher feedback equally during policy optimization. While in practice, this assumption does not always hold. We observe that teacher-derived rewards often conflict with genuine reasoning progress, as reasoning steps with clear reasoning advancement may still receive lower distillation rewards, simply due to deviation from teacher's outputs. To address this mismatch, we propose Reasoning-Progress-Aware Reward Filtering for On-Policy Distillation (R2-OPD), which constructs two within-trajectory rankings of reasoning spans, one from teacher-derived rewards and the other from independently estimated progress reward. Distillation rewards are selectively suppressed whenever the two rankings disagree, reducing supervision that conflicts with reasoning progress while preserving effective teacher guidance. Our approach shows consistent improvement over standard OPD especially regarding reasoning performances.

cs.AI↗

Pareto-Aware Hierarchical Reinforcement Learning for Online Resource Allocation in RIS-assisted Large-Scale IoT Systems

With the rapid evolution of 5G and emerging 6G networks, reconfigurable intelligent surfaces (RIS) have become a critical technology for enhancing wireless communication scenarios. However, optimizing RIS-assisted multi-user systems typically introduces high-dimensional physical layer variables and non-convex Pareto-optimal rate sets, posing severe computational challenges for real-time applications. To address these limitations, this paper proposes a dimension-reduced, hierarchical reinforcement learning (RL) framework, termed Pareto-aware autoencoder-assisted RL (PAAERL), to optimize online resource allocation in RIS-assisted Internet of Things (IoT) networks. Our approach first substitutes high-dimensional continuous RIS beamforming variables with lower-dimensional weight vectors that strictly represent the Pareto-optimal frontier, theoretically avoiding geometric information loss across both convex and non-convex rate regions. To further mitigate the curse of dimensionality in dense networks, an autoencoder architecture is integrated to execute a secondary, data-driven compression phase, mapping the priority space into a highly condensed continuous latent action space. Extensive simulations conducted across practical communication scenarios, including multi-user mobile edge computing (MEC) networks, demonstrate that the proposed PAAERL framework drastically reduces offline training times, accelerates online policy convergence, and significantly decreases overall network costs compared to state-of-the-art benchmarks, underscoring its exceptional scalability and practical viability for next-generation intelligent IoT environments.

cs.NI↗

Physics-Informed Neural Optimization Based Antenna Coding Design for Pixel Antenna Systems

Pixel antennas enable highly radiation pattern reconfigurability to enhance wireless systems, but its antenna coding design, that is optimizing the states of switches embedded in pixel antennas, remains an NP-hard challenge. Conventional approaches for antenna coding design typically rely on heuristic search algorithms, which suffer from high computational complexity. To overcome this issue, we propose a novel efficient data-free optimization algorithm called physics-informed neural optimizer (PINO) for antenna coding design. By integrating a deep convolutional neural network prior and a Gumbel-Sigmoid continuous relaxation into a differentiable physics engine, the proposed algorithm transforms the binary optimization problem into a continuous differentiable problem, which enables the antenna coding optimization problem to be efficiently solved via gradient descent. Simulation results demonstrate that the proposed algorithm outperforms the heuristic search based algorithms, reducing computational time while achieving higher average channel gain.

eess.SP↗

Decomposition and Successive Decomposition Methods and Algorithms for Nonconvex Optimization

Existing results on decomposition methods and algorithms for nonconvex problems are minimal. Parallel decomposition algorithms do not exist for nonconvex problems with coupling nonlinear equality constraints. Besides, decomposition structures (i.e., coupling variables and constraints) are not fully exploited in designing decomposition methods and algorithms. In this paper, we consider nonconvex problems with decomposition structures that are more general than those handled in the existing literature. We propose primal and dual decomposition and successive primal and successive dual decomposition methods and algorithms for these nonconvex problems, which exploit decomposition structures, allow for parallel and distributed implementations, produce the original nonconvex problems' stationary points, and offer good opportunities to achieve superior tradeoff between convergence performance and computation time. Finally, we compare the proposed methods and algorithms, extend them to indirect and two-level decomposition methods and algorithms, and provide examples and numerical results to demonstrate their respective values. Notably, the proposed decomposition and successive decomposition methods and algorithms generalize the basic ones for convex problems, and the proposed successive decomposition methods and algorithms extend the existing ones for nonconvex problems, together enriching the decomposition theory.

math.OC↗

Learning Design-Score Manifold to Guide Diffusion Models for Offline Optimization

Optimizing complex systems, from discovering therapeutic drugs to designing high-performance materials, remains a fundamental challenge across science and engineering, as the underlying rules are often unknown and costly to evaluate. Offline optimization aims to optimize designs for target scores using pre-collected datasets without system interaction. However, conventional approaches may fail beyond training data, predicting inaccurate scores and generating inferior designs. This paper introduces ManGO, a diffusion-based framework that learns the design-score manifold, capturing the design-score interdependencies holistically. Unlike existing methods that treat design and score spaces in isolation, ManGO unifies forward prediction and backward generation, attaining generalization beyond training data. Key to this is its derivative-free guidance for conditional generation, coupled with adaptive inference-time scaling that dynamically optimizes denoising paths. Extensive evaluations demonstrate that ManGO outperforms 24 single- and 10 multi-objective optimization methods across diverse domains, including synthetic tasks, robot control, material design, DNA sequence, and real-world engineering optimization.

cs.LG↗

FREESH: Fair, Resource- and Energy-Efficient Scheduling for LLM Serving on Heterogeneous GPUs

The ever-increasing computation and energy demand for LLM and AI agents call for holistic and efficient optimization of LLM serving systems. In practice, heterogeneous GPU clusters can be deployed in a geographically distributed manner, while LLM load also observes diversity in terms of both query traffic and serving patterns. LLM queries running on advanced GPUs during a high-emission hour at one location can lead to significantly higher carbon footprints versus same queries running on mid-level GPUs at a low-emission time and location. By observing LLM serving requirements and leveraging spatiotemporal computation flexibility, we consider the joint routing and scheduling problem, and propose FREESH to cooperatively run a group of data centers while minimizing user-specified carbon or energy objectives. FREESH identifies the optimal configurations of balanced load serving by matching distinct GPU instance's power-throughput characteristics with predictable LLM query length and workloads. To ensure both latency and fairness requirements, FREESH identifies optimized parallelism and query routing schedules together with dynamic GPU frequency scaling for power saving, and Least-Laxity-First (LLF) serving strategy for query scheduling. During the 1-hour serving on production workloads, FREESH reduces energy by 28.6% and emissions by 45.45% together with improvements in SLO attainment and fairness.

cs.DC↗

AMP-based Joint Activity Detection and Channel Estimation for Massive Grant-Free Access in OFDM-based Wideband Systems

To realize orthogonal frequency division multiplexing (OFDM)-based grant-free access for wideband systems under frequency-selective fading, existing device activity detection and channel estimation methods need substantial accuracy improvement or computation time reduction. In this paper, we aim to resolve this issue. First, we present an exact time-domain signal model for OFDM-based grant-free access under frequency-selective fading. Then, we present a maximum a posteriori (MAP)-based device activity detection problem and two minimum mean square error (MMSE)-based channel estimation problems. The MAP-based device activity detection problem and one of the MMSE-based channel estimation problems are formulated for the first time. Next, we build a new factor graph that captures the exact statistics of time-domain channels and device activities. Based on it, we propose two approximate message passing (AMP)-based algorithms, AMP-A-EC and AMP-A-AC, to approximately solve the MAP-based device activity detection problem and two MMSE-based channel estimation problems. Both proposed algorithms alleviate the AMP's inherent convergence problem when the pilot length is smaller or comparable to the number of active devices. Then, we analyze AMP-A-EC's error probability of activity detection and mean square error (MSE) of channel estimation via state evolution and show that AMP-A-AC has the lower computational complexity (in dominant term). Finally, numerical results show the two proposed AMP-based algorithms' superior performance and respective preferable regions, revealing their significant values for OFDM-based grant-free access.

cs.IT↗

Cued-Agent: A Collaborative Multi-Agent System for Automatic Cued Speech Recognition

Cued Speech (CS) is a visual communication system that combines lip-reading with hand coding to facilitate communication for individuals with hearing impairments. Automatic CS Recognition (ACSR) aims to convert CS hand gestures and lip movements into text via AI-driven methods. Traditionally, the temporal asynchrony between hand and lip movements requires the design of complex modules to facilitate effective multimodal fusion. However, constrained by limited data availability, current methods demonstrate insufficient capacity for adequately training these fusion mechanisms, resulting in suboptimal performance. Recently, multi-agent systems have shown promising capabilities in handling complex tasks with limited data availability. To this end, we propose the first collaborative multi-agent system for ACSR, named Cued-Agent. It integrates four specialized sub-agents: a Multimodal Large Language Model-based Hand Recognition agent that employs keyframe screening and CS expert prompt strategies to decode hand movements, a pretrained Transformer-based Lip Recognition agent that extracts lip features from the input video, a Hand Prompt Decoding agent that dynamically integrates hand prompts with lip features during inference in a training-free manner, and a Self-Correction Phoneme-to-Word agent that enables post-process and end-to-end conversion from phoneme sequences to natural language sentences for the first time through semantic refinement. To support this study, we expand the existing Mandarin CS dataset by collecting data from eight hearing-impaired cuers, establishing a mixed dataset of fourteen subjects. Extensive experiments demonstrate that our Cued-Agent performs superbly in both normal and hearing-impaired scenarios compared with state-of-the-art methods. The implementation is available at https://github.com/DennisHgj/Cued-Agent.

cs.CV↗

Vertex-Guided Redundant Constraints Identification for Unit Commitment

Power systems Unit Commitment (UC) problem determines the generator commitment schedule and dispatch decisions to realize the reliable and economic operation of power networks. The growing penetration of stochastic renewables and demand behaviors makes it necessary to solve the UC problem timely. It is possible to derive lightweight, faster-to-solve UC models via constraint screening to eliminate redundant constraints. However, the screening process remains computationally cumbersome due to the need of solving numerous linear programming (LP) problems. To reduce the number of LPs to solve, we introduce a novel perspective on such classic LP-based screening. Our key insights lie in the principle that redundant constraints will be satisfied by all vertices of the screened feasible region. Using the UC decision variables' bounds tightened by solving much fewer LPs, we build an outer approximation for the UC feasible region as the screened region. A matrix operation is then designed and applied to the outer approximation's vertices to identify all redundant constraints on-the-fly. Adjustments for the outer approximation are further explored to improve screening efficiency by considering the load operating range and cutting planes derived from UC cost and discrete unit status prediction. Extensive simulations are performed on a set of testbeds up to 2,383 buses to substantiate the effectiveness of the proposed schemes. Compared to classic LP-based screening, our schemes can achieve up to 8.8x acceleration while finding the same redundant constraints.

eess.SY↗

Causal-Aware Intelligent QoE Optimization for VR Interaction with Adaptive Keyframe Extraction

The optimization of quality of experience (QoE) in multi-user virtual reality (VR) interactions demands a delicate balance between ultra-low latency, high-fidelity motion synchronization, and equitable resource allocation. While adaptive keyframe extraction mitigates transmission overhead, existing approaches often overlook the causal relationships among allocated bandwidth, CPU frequency, and user perception, limiting QoE gains. This paper proposes an intelligent framework to maximize QoE by integrating adaptive keyframe extraction with causal-aware reinforcement learning (RL). First, a novel QoE metric is formulated using the Weber-Fechner Law, combining perceptual sensitivity, attention-driven priorities, and motion reconstruction accuracy. The QoE optimization problem is then modeled as a mixed integer programming (MIP) task, jointly optimizing keyframe ratios, bandwidth, and computational resources under horizon-fairness constraints. We propose Partial State Causal Deep Deterministic Policy Gradient (PS-CDDPG), which integrates the Deep Deterministic Policy Gradient (DDPG) method with causal influence detection. By leveraging causal information regarding how QoE is influenced and determined by various actions, we explore actions guided by weights calculated from causal inference (CI), which in turn improves training efficiency. Experiments conducted with the CMU Motion Capture Database demonstrate that our framework significantly reduces interactive latency, enhances QoE, and maintains fairness, achieving superior performance compared to benchmark methods.

cs.LG↗

Real-Time Network Traffic Forecasting with Missing Data: A Generative Model Approach

Real-time network traffic forecasting is crucial for network management and early resource allocation. Existing network traffic forecasting approaches operate under the assumption that the network traffic data is fully observed. However, in practical scenarios, the collected data are often incomplete due to various human and natural factors. In this paper, we propose a generative model approach for real-time network traffic forecasting with missing data. Firstly, we model the network traffic forecasting task as a tensor completion problem. Secondly, we incorporate a pre-trained generative model to achieve the low-rank structure commonly associated with tensor completion. The generative model effectively captures the intrinsic low-rank structure of network traffic data during pre-training and enables the mapping from a compact latent representation to the tensor space. Thirdly, rather than directly optimizing the high-dimensional tensor, we optimize its latent representation, which simplifies the optimization process and enables real-time forecasting. We also establish a theoretical recovery guarantee that quantifies the error bound of the proposed approach. Experiments on real-world datasets demonstrate that our approach achieves accurate network traffic forecasting within 100 ms, with a mean absolute error (MAE) below 0.002, as validated on the Abilene dataset.

cs.NI↗

Energy-Latency Aware Intelligent Reflecting Surface Aided Multi-cell Mobile Edge Computing

The explosive development of the Internet of Things (IoT) has led to increased interest in mobile edge computing (MEC), which provides computational resources at network edges to accommodate computation-intensive and latency-sensitive applications. Intelligent reflecting surfaces (IRSs) have gained attention as a solution to overcome blockage problems during the offloading uplink transmission in MEC systems. This paper explores IRS-aided multi-cell networks that enable servers to serve neighboring cells and cooperate to handle resource exhaustion. We aim to minimize the joint energy and latency cost, by jointly optimizing computation tasks, edge computing resources, user beamforming, and IRS phase shifts. The problem is decomposed into two subproblems--the MEC subproblem and the IRS communication subproblem--using the block coordinate descent (BCD) technique. The MEC subproblem is reformulated as a nonconvex quadratic constrained problem (QCP), while the IRS communication subproblem is transformed into a weight-sum-rate problem with auxiliary variables. We propose an efficient algorithm to iteratively optimize MEC resources and IRS communication until convergence. Numerical results show that our algorithm outperforms benchmarks and that multi-cell MEC systems achieve additional performance gains when supported by IRS.

cs.NI↗

Chance-Constrained Economic Dispatch with Flexible Loads and RES

With the increasing penetration of intermittent renewable energy sources (RESs), it becomes increasingly challenging to maintain the supply-demand balance of power systems by solely relying on the generation side. To combat the volatility led by the uncertain RESs, demand-side management by leveraging the multi-dimensional flexibility (MDF) has been recognized as an economic and efficient approach. Thus, it is important to integrate MDF into existing power systems. In this paper, we propose an enhanced day-ahead energy market, where the MDFs of aggregate loads are traded to minimize the generation cost and mitigate the volatility of locational marginal prices (LMPs) in the transmission network. We first explicitly capture the negative impact of the uncertainty from RESs on the day-ahead market by a chance-constrained economic dispatch problem (CEDP). Then, we propose a bidding mechanism for the MDF of the aggregate loads and combine this mechanism into the CEDP for the day-ahead market. Through multiple case studies, we show that MDF from load aggregators can reduce the volatility of LMPs. In addition, we identify the values of the different flexibilities in the MDF bids, which provide useful insights into the design of more complex MDF markets.

eess.SY↗

Robust Economic Dispatch with Flexible Demand and Adjustable Uncertainty Set

With more renewable energy sources (RES) integrated into the power system, the intermittency of RES places a heavy burden on the system. The uncertainty of RES is traditionally handled by controllable generators to balance the real time wind power deviation. As the demand side management develops, the flexibility of aggregate loads can be leveraged to mitigate the negative impact of the wind power. In view of this, we study the problem of how to exploit the multi-dimensional flexibility of elastic loads to balance the trade-off between a low generation cost and a low system risk related to the wind curtailment and the power deficiency. These risks are captured by the conditional value-at-risk. Also, unlike most of the existing studies, the uncertainty set of the wind power output in our model is not fixed. By contrast, it is undetermined and co-optimized based on the available load flexibility. We transform the original optimization problem into a convex one using surrogate affine approximation such that it can be solved efficiently. In case studies, we apply our model on a six-bus transmission network and demonstrate that how flexible load aggregators can help to determine the optimal admissible region for the wind power deviations.

eess.SY↗

Is Locational Marginal Price All You Need for Locational Marginal Emission?

Growing concerns over climate change call for improved techniques for estimating and quantifying the greenhouse gas emissions associated with electricity generation and transmission. Among the emission metrics designated for power grids, locational marginal emission (LME) can provide system operators and electricity market participants with valuable information on the emissions associated with electricity usage at various locations in the power network. In this paper, by investigating the operating patterns and physical interpretations of marginal emissions and costs in the security-constrained economic dispatch (SCED) problem, we identify and draw the exact connection between locational marginal price (LMP) and LME. Such interpretation helps instantly derive LME given nodal demand vectors or LMP, and also reveals the interplay between network congestion and nodal emission pattern. Our proposed approach helps reduce the computation time of LME by an order of magnitude compared to analytical approaches, while it can also serve as a plug-and-play module accompanied by an off-the-shelf market clearing and LMP calculation process.

eess.SY↗

Long-Term Carbon-Efficient Planning for Geographically Shiftable Resources: A Monte Carlo Tree Search Approach

Global climate challenge is demanding urgent actions for decarbonization, while electric power systems take the major roles in clean energy transition. Due to the existence of spatially and temporally dispersed renewable energy resources and the uneven distribution of carbon emission intensity throughout the grid, it is worth investigating future load planning and demand management to offset those generations with higher carbon emission rates. Such techniques include inter-region utilization of geographically shiftable resources and stochastic renewable energy. For instance, data center is considered to be a major carbon emission producer in the future due to increasing information load, while it holds the capability of geographical load balancing. In this paper, we propose a novel planning and operation model minimizing the system-level carbon emissions via sitting and operating geographically shiftable resources. This model decides the optimal locations for shiftable resources expansion along with power dispatch schedule. To accommodate future system operation patterns and a wide range of operating conditions, we incorporate 20-year fine-grained load and renewables scenarios for grid simulations of realistic sizes (e.g., up to 1888 buses). To tackle the computational challenges coming from the combinatorial nature of such large-scale planning problem, we develop a customized Monte Carlo Tree Search (MCTS) method, which can find reasonable solutions satisfying solution time limits. Besides, MCTS enables flexible time window settings and offline solution adjustments. Extensive simulations validate that our planning model can reduce more than 10\% carbon emission across all setups. Compared to off-the-shelf optimization solvers such as Gurobi, our method achieves up to 8.1X acceleration while the solution gaps are less than 1.5\% in large-scale cases.

eess.SY↗

Efficient Unit Commitment Constraint Screening under Uncertainty

Day-ahead unit commitment (UC) is a fundamental task for power system operators, where generator statuses and power dispatch are determined based on the forecasted nodal net demands. The uncertainty inherent in renewables and load forecasting requires the use of techniques in optimization under uncertainty to find more resilient and reliable UC solutions. However, the solution procedure of such specialized optimization may differ from the deterministic UC. The original constraint screening approach can be unreliable and inefficient for them. Thus, in this work we design a novel screening approach under the forecasting uncertainty. Our approach accommodates such uncertainties in both chance-constrained and robust forms, and can greatly reduce the UC instance size by screening out non-binding constraints. To further improve the screening efficiency, we utilize the multi-parametric programming theory to convert the underlying optimization problem of the screening model to a piecewise affine function. A multi-area screening approach is further developed to handle the computational intractability issues for large-scale problems. We verify the proposed method's performance on a variety of UC setups and uncertainty situations. Experimental results show that our robust screening procedure can guarantee better feasibility, while the CC screening can produce more efficient reduced models. The average screening time for a single line flow constraint can be accelerated by 71.2X to 131.3X using our proposed method.

eess.SY↗