arXiv subjects

Nicole Ludwig

Publications and source records attributed to Nicole Ludwig.

18 recordsLinked to original sources

arXiv2026

Decision-Aware Training for Sample-Based Generative Models

Sample-based generative models are increasingly used for probabilistic forecasting in high-stakes decision settings, yet their training objectives are blind to the decision maker's cost structure. These models are commonly trained with strictly proper scoring rules, such as the energy score, which allocate their training signal in proportion to data density, with no awareness of where forecast errors are most costly for downstream decisions. We therefore propose decision-aware training for sample-based generative models, augmenting the energy score objective with a differentiable decision loss that directly penalises the cost incurred by acting on the model's forecast. This combined loss is theoretically grounded, as the decision loss is itself a proper scoring rule. We validate our method on one synthetic and two real-world tasks, showing targeted improvements in cost-sensitive regions while retaining full probabilistic forecasts.

cs.LG↗

arXiv2026

Bounded Graph Clustering with Graph Neural Networks

In community detection, many methods require the user to specify the number of clusters in advance since an exhaustive search over all possible values is computationally infeasible. While some classical algorithms can infer this number directly from the data, this is typically not the case for graph neural networks (GNNs): even when a desired number of clusters is specified, standard GNN-based methods often fail to return the exact number due to the way they are designed. In this work, we address this limitation by introducing a flexible and principled way to control the number of communities discovered by GNNs. Rather than assuming the true number of clusters is known, we propose a framework that allows the user to specify a plausible range and enforce these bounds during training. However, if the user wants an exact number of clusters, it may also be specified and reliably returned.

cs.LG↗

arXiv2025

Forecast Skill Is Not Decision Skill: Evidence from Weather-Dependent Decision Tasks

Standard weather forecast evaluations focus on the forecaster's perspective and on a statistical assessment comparing forecasts and observations. In practice, however, forecasts are used to make decisions, so it seems natural to take the decision-maker's perspective and quantify the value of a forecast by its ability to improve decision-making. Decision calibration provides a novel framework for evaluating probabilistic forecast performance at the decision level rather than the forecast level. We evaluate decision calibration to compare a Machine Learning and a classical numerical weather prediction model on various weather-dependent decision tasks, though the framework is applicable to any set of forecast models. We find that model performance at the forecast level does not reliably translate to performance in downstream decision-making: some performance differences only become apparent at the decision level, and even among seemingly similar decision tasks, model rankings can change. Our results confirm that typical forecast evaluations are insufficient for selecting the optimal forecast model for a specific decision task.

cs.LG↗

arXiv2025

Fault Detection in Solar Thermal Systems using Probabilistic Reconstructions

Solar thermal systems (STS) present a promising avenue for low-carbon heat generation, with a well-running system providing heat at minimal cost and carbon emissions. However, STS can exhibit faults due to improper installation, maintenance, or operation, often resulting in a substantial reduction in efficiency or even damage to the system. As monitoring at the individual level is economically prohibitive for small-scale systems, automated monitoring and fault detection should be used to address such issues. Recent advances in data-driven anomaly detection, particularly in time series analysis, offer a cost-effective solution by leveraging existing sensors to identify abnormal system states. Here, we propose a probabilistic reconstruction-based framework for anomaly detection. We evaluate our method on the publicly available PaSTS dataset of operational domestic STS, which features real-world complexities and diverse fault types. Our experiments show that reconstruction-based methods can detect faults in domestic STS both qualitatively and quantitatively, while generalizing to previously unseen systems. We also demonstrate that our model outperforms both simple and more complex deep learning baselines. Additionally, we show that heteroscedastic uncertainty estimation is essential to fault detection performance. Finally, we discuss the engineering overhead required to unlock these improvements and make a case for simple deep learning models.

eess.SY↗

arXiv2025

Assessing the risk of future Dunkelflaute events for Germany using generative deep learning

The European electricity power grid is transitioning towards renewable energy sources, characterized by an increasing share of off- and onshore wind and solar power. However, the weather dependency of these energy sources poses a challenge to grid stability, with so-called Dunkelflaute events -- periods of low wind and solar power generation -- being of particular concern due to their potential to cause electricity supply shortages. In this study, we investigate the impact of these events on the German electricity production in the years and decades to come. For this purpose, we adapt a recently developed generative deep learning framework to downscale climate simulations from the CMIP6 ensemble. We first compare their statistics to the historical record taken from ERA5 data. Next, we use these downscaled simulations to assess plausible future occurrences of Dunkelflaute events in Germany under the optimistic low (SSP2-4.5) and high (SSP5-8.5) emission scenarios. Our analysis indicates that both the frequency and duration of Dunkelflaute events in Germany in the ensemble mean are projected to remain largely unchanged compared to the historical period. This suggests that, under the considered climate scenarios, the associated risk is expected to remain stable throughout the century.

cs.LG↗

arXiv2025

A Generative Framework for Probabilistic, Spatiotemporally Coherent Downscaling of Climate Simulation

Local climate information is crucial for impact assessment and decision-making, yet coarse global climate simulations cannot capture small-scale phenomena. Current statistical downscaling methods infer these phenomena as temporally decoupled spatial patches. However, to preserve physical properties, estimating spatio-temporally coherent high-resolution weather dynamics for multiple variables across long time horizons is crucial. We present a novel generative framework that uses a score-based diffusion model trained on high-resolution reanalysis data to capture the statistical properties of local weather dynamics. After training, we condition on coarse climate model data to generate weather patterns consistent with the aggregate information. As this predictive task is inherently uncertain, we leverage the probabilistic nature of diffusion models and sample multiple trajectories. We evaluate our approach with high-resolution reanalysis information before applying it to the climate model downscaling task. We then demonstrate that the model generates spatially and temporally coherent weather dynamics that align with global climate output.

cs.LG↗

arXiv2025

RainShift: A Benchmark for Precipitation Downscaling Across Geographies

Earth System Models (ESM) are our main tool for projecting the impacts of climate change. However, running these models at sufficient resolution for local-scale risk-assessments is not computationally feasible. Deep learning-based super-resolution models offer a promising solution to downscale ESM outputs to higher resolutions by learning from data. Yet, due to regional variations in climatic processes, these models typically require retraining for each geographical area-demanding high-resolution observational data, which is unevenly available across the globe. This highlights the need to assess how well these models generalize across geographic regions. To address this, we introduce RainShift, a dataset and benchmark for evaluating downscaling under geographic distribution shifts. We evaluate state-of-the-art downscaling approaches including GANs and diffusion models in generalizing across data gaps between the Global North and Global South. Our findings reveal substantial performance drops in out-of-distribution regions, depending on model and geographic area. While expanding the training domain generally improves generalization, it is insufficient to overcome shifts between geographically distinct regions. We show that addressing these shifts through, for example, data alignment can improve spatial generalization. Our work advances the global applicability of downscaling methods and represents a step toward reducing inequities in access to high-resolution climate information.

cs.CV↗

arXiv2024

Climate data selection for multi-decadal wind power forecasts

Reliable wind speed data is crucial for applications such as estimating local (future) wind power. Global Climate Models (GCMs) and Regional Climate Models (RCMs) provide forecasts over multi-decadal periods. However, their outputs vary substantially, and higher-resolution models come with increased computational demands. In this study, we analyze how the spatial resolution of different GCMs and RCMs affects the reliability of simulated wind speeds and wind power, using ERA5 data as a reference. We present a systematic procedure for model evaluation for wind resource assessment as a downstream task. Our results show that higher-resolution GCMs and RCMs do not necessarily preserve wind speeds more accurately. Instead, the choice of model, both for GCMs and RCMs, is more important than the resolution or GCM boundary conditions. The IPSL model preserves the wind speed distribution particularly well in Europe, producing the most accurate wind power forecasts relative to ERA5 data.

stat.AP↗

arXiv2024

Turbine location-aware multi-decadal wind power predictions for Germany using CMIP6

Climate change will impact wind and therefore wind power generation with largely unknown effect and magnitude. Climate models can provide insights and should be used for long-term power planning. In this work we use Gaussian processes to predict power output given wind speeds from a global climate model and compare the aggregated predictions to actual power generation. Analyzing past climate model data supports the use of CMIP6 climate model data for multi-decadal wind power predictions and highlights the importance of being location-aware. Our predictions up to 2050 reveal only minor changes in yearly wind power generation. We find that wind power projections of the two in-between climate scenarios SSP2-4.5 and SSP3-7.0 closely align with actual wind power generation between 2015 and 2023. Our analysis also reveals larger uncertainty associated with Germany's coastal areas in the North as compared to Germany's South, motivating wind power expansion in regions where future wind is likely more reliable. Overall, our results indicate that wind energy will likely remain a reliable energy source in the future.

physics.ao-ph↗

arXiv2024

Wind Power Assessment based on Super-Resolution and Downscaling -- A Comparison of Deep Learning Methods

The efficient placement of wind turbines relies on accurate local wind speed forecasts. Climate projections provide valuable insight into long-term wind speed conditions, yet their spatial data resolution is typically insufficient for precise wind power forecasts. Deep learning methods, particularly models developed for image super-resolution, offer a promising solution to bridge this scale gap by increasing the spatial resolution of climate models. In this paper, we compare the performance of various deep learning models on two distinct tasks: super-resolution, where we map artificially coarsened ERA5 data to its native resolution, and downscaling, where we map native ERA5 to high-resolution COSMO-REA6 data. We evaluate the models on their downstream application in forecasting long-term wind power, emphasizing the impact of spatial wind speed resolution on wind power estimates. Our findings highlight the importance of aligning models and evaluation metrics with their specific downstream applications. We show that a diffusion model outperforms other models for estimating the wind power potential by better preserving the wind speeds' distributional and physical properties.

stat.AP↗

arXiv2024

Inductive biases in deep learning models for weather prediction

Deep learning has gained immense popularity in the Earth sciences as it enables us to formulate purely data-driven models of complex Earth system processes. Deep learning-based weather prediction (DLWP) models have made significant progress in the last few years, achieving forecast skills comparable to established numerical weather prediction models with comparatively lesser computational costs. In order to train accurate, reliable, and tractable DLWP models with several millions of parameters, the model design needs to incorporate suitable inductive biases that encode structural assumptions about the data and the modelled processes. When chosen appropriately, these biases enable faster learning and better generalisation to unseen data. Although inductive biases play a crucial role in successful DLWP models, they are often not stated explicitly and their contribution to model performance remains unclear. Here, we review and analyse the inductive biases of state-of-the-art DLWP models with respect to five key design elements: data selection, learning objective, loss function, architecture, and optimisation method. We identify the most important inductive biases and highlight potential avenues towards more efficient and probabilistic DLWP models.

physics.ao-ph↗

arXiv2023

Mind the (spectral) gap: How the temporal resolution of wind data affects multi-decadal wind power forecasts

To forecast wind power generation in the scale of years to decades, outputs from climate models are often used. However, one major limitation of the data projected by these models is their coarse temporal resolution - usually not finer than three hours and sometimes as coarse as one month. Due to the non-linear relationship between wind speed and wind power, and the long forecast horizon considered, small changes in wind speed can result in big changes in projected wind power generation. Our study indicates that the distribution of observed 10min wind speed data is relatively well preserved using three- or six-hourly instantaneous values. In contrast, daily or monthly values, as well as any averages, including three-hourly averages, are almost never capable of preserving the distribution of the underlying higher resolution data. Assuming that climate models behave in a similar manner to observations, our results indicate that output at three-hourly or six-hourly temporal resolution is high enough for multi-decadal wind power generation forecasting. In contrast, wind speed projections of lower temporal resolution, or averages over any time range, should be handled with care.

stat.AP↗

arXiv2022

Analytical Uncertainty Propagation for Multi-Period Stochastic Optimal Power Flow

The increase in renewable energy sources (RESs), like wind or solar power, results in growinguncertainty also in transmission grids. This affects grid stability through fluctuating energy supplyand an increased probability of overloaded lines. One key strategy to cope with this uncertainty isthe use of distributed energy storage systems (ESSs). In order to securely operate power systemscontaining renewables and use storage, optimization models are needed that both handle uncertaintyand apply ESSs. This paper introduces a compact dynamic stochastic chance-constrained DC optimalpower flow (CC-OPF) model, that minimizes generation costs and includes distributed ESSs. AssumingGaussian uncertainty, we use affine policies to obtain a tractable, analytically exact reformulation asa second-order cone problem (SOCP). We test the new model on five different IEEE networks withvarying sizes of 5, 39, 57, 118 and 300 nodes and include complexity analysis. The results showthat the model is computationally efficient and robust with respect to constraint violation risk. Thedistributed energy storage system leads to more stable operation with flattened generation profiles.Storage absorbed RES uncertainty, and reduced generation cost.

eess.SY↗

arXiv2022

Evaluating Ensemble Post-Processing for Wind Power Forecasts

Capturing the uncertainty in probabilistic wind power forecasts is challenging, especially when uncertain input variables, such as the weather, play a role. Since ensemble weather predictions aim to capture the uncertainty in the weather system, they can be used to propagate this uncertainty through to subsequent wind power forecasting models. However, as weather ensemble systems are known to be biased and underdispersed, meteorologists post-process the ensembles. This post-processing can successfully correct the biases in the weather variables but has not been evaluated thoroughly in the context of subsequent forecasts, such as wind power generation forecasts. The present paper evaluates multiple strategies for applying ensemble post-processing to probabilistic wind power forecasts. We use Ensemble Model Output Statistics (EMOS) as the post-processing method and evaluate four possible strategies: only using the raw ensembles without post-processing, a one-step strategy where only the weather ensembles are post-processed, a one-step strategy where we only post-process the power ensembles, and a two-step strategy where we post-process both the weather and power ensembles. Results show that post-processing the final wind power ensemble improves forecast performance regarding both calibration and sharpness, whilst only post-processing the weather ensembles does not necessarily lead to increased forecast performance.

stat.AP↗

arXiv2022

Analytical Uncertainty Propagation for Multi-Period Stochastic Optimal Power Flow

The increase in renewable energy sources (RESs), like wind or solar power, results in growing uncertainty also in transmission grids. This affects grid stability through fluctuating energy supply and an increased probability of overloaded lines. One key strategy to cope with this uncertainty is the use of distributed energy storage systems (ESSs). In order to securely operate power systems containing renewables and use storage, optimization models are needed that both handle uncertainty and apply ESSs. This paper introduces a compact dynamic stochastic chance-constrained optimal power flow (CC-OPF) model, that minimizes generation costs and includes distributed ESSs. Assuming Gaussian uncertainty, we use affine policies to obtain a tractable, analytically exact reformulation as a second-order cone problem (SOCP). We test the new model on five different IEEE networks with varying sizes of 5, 39, 57, 118 and 300 nodes and include complexity analysis. The results show that the model is computationally efficient and robust with respect to constraint violation risk. The distributed energy storage system leads to more stable operation with flattened generation profiles. Storage absorbed RES uncertainty, and reduced generation cost.

eess.SY↗

arXiv2022

A Collection and Categorization of Open-Source Wind and Wind Power Datasets

Wind power and other forms of renewable energy sources play an ever more important role in the energy supply of today's power grids. Forecasting renewable energy sources has therefore become essential in balancing the power grid. While a lot of focus is placed on new forecasting methods, little attention is given on how to compare, reproduce and transfer the methods to other use cases and data. One reason for this lack of attention is the limited availability of open-source datasets, as many currently used datasets are non-disclosed and make reproducibility of research impossible. This unavailability of open-source datasets is especially prevalent in commercially interesting fields such as wind power forecasting. However, with this paper we want to enable researchers to compare their methods on publicly available datasets by providing the, to our knowledge, largest up-to-date overview of existing open-source wind power datasets, and a categorization into different groups of datasets that can be used for wind power forecasting. We show that there are publicly available datasets sufficient for wind power forecasting tasks and discuss the different data groups properties to enable researchers to choose appropriate open-source datasets and compare their methods on them.

cs.LG↗

arXiv2021

Smart Data Representations: Impact on the Accuracy of Deep Neural Networks

Deep Neural Networks are able to solve many complex tasks with less engineering effort and better performance. However, these networks often use data for training and evaluation without investigating its representation, i.e.~the form of the used data. In the present paper, we analyze the impact of data representations on the performance of Deep Neural Networks using energy time series forecasting. Based on an overview of exemplary data representations, we select four exemplary data representations and evaluate them using two different Deep Neural Network architectures and three forecasting horizons on real-world energy time series. The results show that, depending on the forecast horizon, the same data representations can have a positive or negative impact on the accuracy of Deep Neural Networks.

cs.LG↗

arXiv2021

pyWATTS: Python Workflow Automation Tool for Time Series

Time series data are fundamental for a variety of applications, ranging from financial markets to energy systems. Due to their importance, the number and complexity of tools and methods used for time series analysis is constantly increasing. However, due to unclear APIs and a lack of documentation, researchers struggle to integrate them into their research projects and replicate results. Additionally, in time series analysis there exist many repetitive tasks, which are often re-implemented for each project, unnecessarily costing time. To solve these problems we present \texttt{pyWATTS}, an open-source Python-based package that is a non-sequential workflow automation tool for the analysis of time series data. pyWATTS includes modules with clearly defined interfaces to enable seamless integration of new or existing methods, subpipelining to easily reproduce repetitive tasks, load and save functionality to simply replicate results, and native support for key Python machine learning libraries such as scikit-learn, PyTorch, and Keras.

cs.LG↗