arXiv subjects

Honglin Wen

Publications and source records attributed to Honglin Wen.

At least 19 recordsLinked to original sources

arXiv2026

Decision-calibrated prediction sets for robust power system operations

Robust optimization offers a tractable approach to balance operating costs and reliability in power systems dominated by weather-dependent renewable uncertainty, but its performance depends critically on the uncertainty set. Standard data-driven approaches often calibrate uncertainty sets to attain predictive coverage, which can produce unnecessarily large sets and costly operating decisions. In contrast, we introduce decision-calibrated prediction sets and embed them as uncertainty sets in robust optimization problems; these are conditional multivariate prediction sets where calibration is defined in terms of the reliability of downstream decisions, rather than in terms of the coverage. First, we learn these conditional prediction sets as sub-level sets of norm-based score functions represented by partially input-convex neural networks, capturing contextual information and multivariate dependence while preserving convexity and tractability in downstream robust formulations. Second, inspired by conformal risk control, we calibrate a score-threshold parameter that sets the volume of the uncertainty set, thereby controlling the expected violations of downstream operational constraints. We apply our approach to 15-minute-ahead reserve scheduling with network-constrained deliverability, which we formulate as a robust DC optimal power flow problem with affine recourse. Numerical experiments show that decision-calibrated sets attain prescribed constraint-satisfaction targets within about three percentage points, whereas standard coverage-based calibration systematically exceeds these targets by more than eleven percentage points, leading to larger sets and higher operating costs.

math.OC↗

arXiv2026

Stabilizing distribution-free probabilistic forecasts

Multi-step-ahead forecasts are often updated as new observations become available, since shorter forecast horizons typically improve forecast quality. However, such improvements come at the cost of forecast instability, i.e., variability in forecasts for the same target period. This instability can trigger costly changes to plans formulated based on the forecasts and may erode trust in the forecasting system. In this work, we integrate forecast stability alongside forecast quality into the training of distribution-free probabilistic time-series forecasting models, allowing us to control this trade-off. We propose a method for generating stabilized forecasted conditional quantile functions using regression splines parameterized by a neural network. This approach enables joint optimization of quality and stability, as it allows us to directly penalize dissimilarities arising from forecast updates. Furthermore, it allows assigning varying importance to stabilizing different parts of the forecast distributions (e.g., central parts vs. tails) to focus on the parts most relevant for the intended downstream use (e.g., the upper tail for inventory management). We empirically evaluate the proposed method on two datasets with different statistical properties and show that it can effectively reduce forecast instability without a substantial loss in forecast quality, and that it can target stabilization effort toward specific parts of the forecast distributions.

cs.LG↗

arXiv2025

Decision-Focused Continual Learning for Seaport Power-Logistics Scheduling: Generalization across Varying Tasks

Power-logistics scheduling in modern seaports typically follows a predict-then-optimize pipeline. To enhance the decision quality of predictions, decision-focused learning has been proposed, which aligns the training of forecasting models with downstream decision outcomes. However, this end-to-end design inherently restricts the value of forecasting models to a specific task structure and therefore generalizes poorly to evolving tasks induced by varying vessel arrivals. We address this gap with a decision-focused continual learning framework that adapts online to a stream of scheduling tasks. Specifically, we introduce Fisher-information-based regularization to enhance cross-task generalization by preserving parameters critical to prior tasks. A differentiable convex surrogate is also developed to stabilize gradient backpropagation. The proposed approach enables learning a decision-aligned forecasting model across a varying task stream with sustainable long-term computational and memory requirements. Experiments calibrated to Jurong Port show improved decision performance and cross-task generalization over existing methods, together with reduced computational cost and a bounded memory footprint.

cs.LG↗

arXiv2025

Pooling Probabilistic Forecasts for Cooperative Wind Power Offering

Wind power producers can benefit from forming coalitions to participate cooperatively in electricity markets. To support such collaboration, various profit allocation rules rooted in cooperative game theory have been proposed. However, existing approaches overlook the lack of coherence among producers regarding forecast information, which may lead to ambiguity in offering and allocations. In this paper, we introduce a ``reconcile-then-optimize'' framework for cooperative market offerings. This framework first aligns the individual forecasts into a coherent joint forecast before determining market offers. With such forecasts, we formulate and solve a two-stage stochastic programming problem to derive both the aggregate offer and the corresponding scenario-based dual values for each trading hour. Based on these dual values, we construct a profit allocation rule that is budget-balanced and stable. Finally, we validate the proposed method through empirical case studies, demonstrating its practical effectiveness and theoretical soundness.

eess.SY↗

arXiv2025

Value-oriented forecast reconciliation for renewables in electricity markets

Forecast reconciliation is considered an effective method to achieve coherence (within a forecast hierarchy) and to improve forecast quality. However, the value of reconciled forecasts in downstream decision-making tasks has been mostly overlooked. In a multi-agent setup with heterogeneous loss functions, this oversight may lead to unfair outcomes, hence resulting in conflicts during the reconciliation process. To address this, we propose a value-oriented forecast reconciliation approach that focuses on the forecast value for all individual agents. Fairness is ensured through the use of a Nash bargaining framework. Specifically, we model this problem as a cooperative bargaining game, where each agent aims to optimize their own gain while contributing to the overall reconciliation process. We then present a primal-dual algorithm for parameter estimation based on empirical risk minimization. From an application perspective, we consider an aggregated wind energy trading problem, where profits are distributed using a weighted allocation rule. We demonstrate the effectiveness of our approach through several numerical experiments, showing that it consistently results in increased profits for all agents involved.

stat.ML↗

arXiv2024

Efficient Unit Commitment Constraint Screening under Uncertainty

Day-ahead unit commitment (UC) is a fundamental task for power system operators, where generator statuses and power dispatch are determined based on the forecasted nodal net demands. The uncertainty inherent in renewables and load forecasting requires the use of techniques in optimization under uncertainty to find more resilient and reliable UC solutions. However, the solution procedure of such specialized optimization may differ from the deterministic UC. The original constraint screening approach can be unreliable and inefficient for them. Thus, in this work we design a novel screening approach under the forecasting uncertainty. Our approach accommodates such uncertainties in both chance-constrained and robust forms, and can greatly reduce the UC instance size by screening out non-binding constraints. To further improve the screening efficiency, we utilize the multi-parametric programming theory to convert the underlying optimization problem of the screening model to a piecewise affine function. A multi-area screening approach is further developed to handle the computational intractability issues for large-scale problems. We verify the proposed method's performance on a variety of UC setups and uncertainty situations. Experimental results show that our robust screening procedure can guarantee better feasibility, while the CC screening can produce more efficient reduced models. The average screening time for a single line flow constraint can be accelerated by 71.2X to 131.3X using our proposed method.

eess.SY↗

arXiv2024

Improving Sequential Market Coordination via Value-oriented Renewable Energy Forecasting

Large penetration of renewable energy sources (RESs) brings huge uncertainty into the electricity markets. The current deterministic clearing approach in the day-ahead (DA) market, where RESs participate based on expected production, has been criticized for causing a lack of coordination between the DA and real-time (RT) markets, leading to high overall operating costs. Previous works indicate that improving day-ahead RES entering quantities can significantly mitigate the drawbacks of deterministic clearing. In this work, we propose using a trained forecasting model, referred to as value-oriented forecasting, to determine RES Improved Entering Quantities (RIEQ) more efficiently during the operational phase. Unlike traditional models that minimize statistical forecasting errors, our approach trains model parameters to minimize the expected overall operating costs across both DA and RT markets. We derive the exact form of the loss function used for training, which becomes piecewise linear when market clearing is modeled by linear programs. Additionally, we provide the analytical gradient of the loss function with respect to the forecast, enabling an efficient training strategy. Numerical studies demonstrate that our forecasts significantly reduce overall operating costs for deterministic market clearing compared to conventional forecasts based on expected RES production.

eess.SY↗

arXiv2024

Probabilistic wind power forecasting resilient to missing values: an adaptive quantile regression approach

Probabilistic wind power forecasting approaches have significantly advanced in recent decades. However, forecasters often assume data completeness and overlook the challenge of missing values resulting from sensor failures, network congestion, etc. Traditionally, this issue is addressed during the data preprocessing procedure using methods such as deletion and imputation. Nevertheless, these ad-hoc methods pose challenges to probabilistic wind power forecasting at both parameter estimation and operational forecasting stages. In this paper, we propose a resilient probabilistic forecasting approach that smoothly adapts to missingness patterns without requiring preprocessing or retraining. Specifically, we design an adaptive quantile regression model with parameters capable of adapting to missing patterns, comprising two modules. The first is a feature extraction module where weights are kept static and biases are designed as a function of missingness patterns. The second is a non-crossing quantile neural network module, ensuring monotonicity of quantiles, with higher quantiles derived by adding non-negative amounts to lower quantiles. The proposed approach is applicable to cases under all missingness mechanisms including missing-not-at-random cases. Case studies demonstrate that our proposed approach achieves state-of-the-art results in terms of the continuous ranked probability score, with acceptable computational cost.

stat.AP↗

arXiv2024

Marginalize, Rather than Impute: Probabilistic Wind Power Forecasting with Incomplete Data

Machine learning methods are widely and successfully used for probabilistic wind power forecasting, yet the pervasive issue of missing values (e.g., due to sensor faults or communication outages) has received limited attention. The prevailing practice is impute-then-predict, but conditioning on point imputations biases parameter estimates and fails to propagate uncertainty from missing features. Our approach treats missing features and forecast targets uniformly: we learn a joint generative model of features and targets from incomplete data and, at operational deployment, condition on the observed features and marginalize the unobserved ones to produce forecasts. This imputation-free procedure avoids error introduced by imputation and preserves uncertainty aroused from missing features. In experiments, it improves forecast quality in terms of continuous ranked probability score relative to impute-then-predict baselines while incurring substantially lower computational cost than common alternatives.

cs.LG↗

arXiv2023

Deriving Loss Function for Value-oriented Renewable Energy Forecasting

Renewable energy forecasting is the workhorse for efficient energy dispatch. However, forecasts with small mean squared errors (MSE) may not necessarily lead to low operation costs. Here, we propose a forecasting approach specifically tailored for operational purposes, by incorporating operational problems into the estimation of forecast models via designing a loss function. We formulate a bilevel program, where the operation problem is at the lower level, and the forecast model estimation is at the upper level. We establish the relationship between the lower-level optimal solutions and forecasts through multiparametric programming. By integrating it into the upper-level objective for minimizing expected operation cost, we convert the bilevel problem to a single-level one and derive the loss function for training the model. It is proved to be piecewise linear, for linear operation problem. Compared to the commonly used loss functions, e.g. MSE, our approach achieves lower operation costs.

eess.SY↗

arXiv2023

Fast Constraint Screening for Multi-Interval Unit Commitment

Power systems Unit Commitment (UC) problem determines the generator commitment schedule and dispatch decisions for power networks based on forecasted electricity demand. However, with the increasing penetration of renewables and stochastic demand behaviors, it becomes challenging to solve the large-scale, multi-interval UC problem in an efficient manner. The main objective of this paper is to propose a fast and reliable scheme to eliminate a set of redundant or inactive physical constraints in the high-dimensional, multi-interval, mixed-integer UC problem, while the reduced problem is equivalent to the original full problem in terms of commitment decisions. Our key insights lie on pre-screening the constraints based on the load distribution and considering the physical feasibility regions of multi-interval UC problem. For the multistep UC formulation, we overcome screening conservativeness by utilizing the multi-step ramping relationships, and can reliably screen out more constraints compared to current practice. Extensive simulations on both specific load samples and load regions validate the proposed technique can screen out more than 80% constraints while preserving the feasibility of multi-interval UC problem.

eess.SY↗

arXiv2023

Toward Value-oriented Renewable Energy Forecasting: An Iterative Learning Approach

Energy forecasting is an essential task in power system operations. Operators usually issue forecasts and leverage them to schedule energy dispatch ahead of time. However, forecast models are typically developed in a way that overlooks the operational value of the forecasts. To bridge the gap, we design a value-oriented point forecasting approach for sequential energy dispatch problems with renewable energy sources. At the training phase, we align the loss function with the overall operation cost function, thereby achieving reduced operation costs. The forecast model parameter estimation is formulated as a bilevel program. Under mild assumptions, we convert the upper-level objective into an equivalent form using the dual solutions obtained from the lower-level operation problems. Additionally, a novel iterative solution strategy is proposed for the newly formulated bilevel program. Under such an iterative scheme, we show that the upper-level objective is locally linear regarding the forecast model output, and can act as the loss function. Numerical experiments demonstrate that, compared to commonly used statistical quality-oriented point forecasting methods, forecasts obtained by the proposed approach result in lower operation costs. Meanwhile, the proposed approach is more computationally efficient than traditional two-stage stochastic programs.

eess.SY↗

arXiv2023

A Contextual Bandit Approach for Value-oriented Prediction Interval Forecasting

Prediction interval (PI) is an effective tool to quantify uncertainty and usually serves as an input to downstream robust optimization. Traditional approaches focus on improving the quality of PI in the view of statistical scores and assume the improvement in quality will lead to a higher value in the power systems operation. However, such an assumption cannot always hold in practice. In this paper, we propose a value-oriented PI forecasting approach, which aims at reducing operational costs in downstream operations. For that, it is required to issue PIs with the guidance of operational costs in robust optimization, which is addressed within the contextual bandit framework here. Concretely, the agent is used to select the optimal quantile proportion, while the environment reveals the costs in operations as rewards to the agent. As such, the agent can learn the policy of quantile proportion selection for minimizing the operational cost. The numerical study regarding a two-timescale operation of a virtual power plant verifies the superiority of the proposed approach in terms of operational value. And it is especially evident in the context of extensive penetration of wind power.

eess.SY↗

arXiv2022

Enabling Fast Unit Commitment Constraint Screening via Learning Cost Model

Unit commitment (UC) are essential tools to transmission system operators for finding the most economical and feasible generation schedules and dispatch signals. Constraint screening has been receiving attention as it holds the promise for reducing a number of inactive or redundant constraints in the UC problem, so that the solution process of large scale UC problem can be accelerated by considering the reduced optimization problem. Standard constraint screening approach relies on optimizing over load and generations to find binding line flow constraints, yet the screening is conservative with a large percentage of constraints still reserved for the UC problem. In this paper, we propose a novel machine learning (ML) model to predict the most economical costs given load inputs. Such ML model bridges the cost perspectives of UC decisions to the optimization-based constraint screening model, and can screen out higher proportion of operational constraints. We verify the proposed method's performance on both sample-aware and sample-agnostic setting, and illustrate the proposed scheme can further reduce the computation time on a variety of setup for UC problems.

math.OC↗

arXiv2022

Efficient Demand Response Location Targeting for Price Spike Mitigation by Exploiting Price-demand Relationship

Demand response (DR) leverages demand-side flexibility, offering a promising approach to enhance market conditions like mitigating wholesale price spikes. However, poorly chosen DR locations can inadvertently increase electricity prices. For that, we introduce a method to rigorously select DR locations and corresponding demand reductions. We formulate a bilevel program where the upper level determines the DR locations and demand reductions while ensuring the average nodal prices meet a predetermined target. The lower level tackles an economic dispatch (ED) problem and feeds the resulting nodal prices back to the upper level based on post-DR demands. This bilevel formulation presents challenges due to the lower-level non-convexity affecting the upper-level constraints on average nodal prices. To address this, we propose to replace the lower level with a piecewise linear function representing the price-demand relationship, solving iteratively for each linear segment. This results in a tractable mixed-integer linear program. An acceleration strategy is proposed to further reduce the computation time. Numerical studies demonstrate the ability of the proposed approach to reduce prices to a desired level. Besides, we empirically show that the proposed approach is robust against inaccurate system parameters and can reduce computation time by over 50%.

eess.SY↗

arXiv2022

Wind energy forecasting with missing values within a fully conditional specification framework

Wind power forecasting is essential to power system operation and electricity markets. As abundant data became available thanks to the deployment of measurement infrastructures and the democratization of meteorological modelling, extensive data-driven approaches have been developed within both point and probabilistic forecasting frameworks. These models usually assume that the dataset at hand is complete and overlook missing value issues that often occur in practice. In contrast to that common approach, we rigorously consider here the wind power forecasting problem in the presence of missing values, by jointly accommodating imputation and forecasting tasks. Our approach allows inferring the joint distribution of input features and target variables at the model estimation stage based on incomplete observations only. We place emphasis on a fully conditional specification method owing to its desirable properties, e.g., being assumption-free when it comes to these joint distributions. Then, at the operational forecasting stage, with available features at hand, one can issue forecasts by implicitly imputing all missing entries. The approach is applicable to both point and probabilistic forecasting, while yielding competitive forecast quality within both simulation and real-world case studies. It confirms that by using a powerful universal imputation method like fully conditional specification, the proposed approach is superior to the common approach, especially in the context of probabilistic forecasting.

stat.AP↗

arXiv2022

Continuous and Distribution-free Probabilistic Wind Power Forecasting: A Conditional Normalizing Flow Approach

We present a data-driven approach for probabilistic wind power forecasting based on conditional normalizing flow (CNF). In contrast with the existing, this approach is distribution-free (as for non-parametric and quantile-based approaches) and can directly yield continuous probability densities, hence avoiding quantile crossing. It relies on a base distribution and a set of bijective mappings. Both the shape parameters of the base distribution and the bijective mappings are approximated with neural networks. Spline-based conditional normalizing flow is considered owing to its non-affine characteristics. Over the training phase, the model sequentially maps input examples onto samples of base distribution, given the conditional contexts, where parameters are estimated through maximum likelihood. To issue probabilistic forecasts, one eventually maps samples of the base distribution into samples of a desired distribution. Case studies based on open datasets validate the effectiveness of the proposed model, and allows us to discuss its advantages and caveats with respect to the state of the art.

eess.SY↗

arXiv2022

Optimal Adaptive Prediction Intervals for Electricity Load Forecasting in Distribution Systems via Reinforcement Learning

Prediction intervals offer an effective tool for quantifying the uncertainty of loads in distribution systems. The traditional central PIs cannot adapt well to skewed distributions, and their offline training fashion is vulnerable to unforeseen changes in future load patterns. Therefore, we propose an optimal PI estimation approach, which is online and adaptive to different data distributions by adaptively determining symmetric or asymmetric probability proportion pairs for quantiles. It relies on the online learning ability of reinforcement learning to integrate the two online tasks, i.e., the adaptive selection of probability proportion pairs and quantile predictions, both of which are modeled by neural networks. As such, the quality of quantiles-formed PI can guide the selection process of optimal probability proportion pairs, which forms a closed loop to improve the quality of PIs. Furthermore, to improve the learning efficiency of quantile forecasts, a prioritized experience replay strategy is proposed for online quantile regression processes. Case studies on both load and net load demonstrate that the proposed method can better adapt to data distribution compared with online central PIs method. Compared with offline-trained methods, it obtains PIs with better quality and is more robust against concept drift.

stat.AP↗