arXiv subjects

Manuel Dahmen

Publications and source records attributed to Manuel Dahmen.

32 records · Page 2Linked to original sources

arXiv2023

Nonlinear Isometric Manifold Learning for Injective Normalizing Flows

To model manifold data using normalizing flows, we employ isometric autoencoders to design embeddings with explicit inverses that do not distort the probability distribution. Using isometries separates manifold learning and density estimation and enables training of both parts to high accuracy. Thus, model selection and tuning are simplified compared to existing injective normalizing flows. Applied to data sets on (approximately) flat manifolds, the combined approach generates high-quality data.

cs.LG↗

arXiv2023

Multivariate Probabilistic Forecasting of Intraday Electricity Prices using Normalizing Flows

Electricity is traded on various markets with different time horizons and regulations. Short-term intraday trading becomes increasingly important due to the higher penetration of renewables. In Germany, the intraday electricity price typically fluctuates around the day-ahead price of the European Power EXchange (EPEX) spot markets in a distinct hourly pattern. This work proposes a probabilistic modeling approach that models the intraday price difference to the day-ahead contracts. The model captures the emerging hourly pattern by considering the four 15 min intervals in each day-ahead price interval as a four-dimensional joint probability distribution. The resulting nontrivial, multivariate price difference distribution is learned using a normalizing flow, i.e., a deep generative model that combines conditional multivariate density estimation and probabilistic regression. Furthermore, this work discusses the influence of different external impact factors based on literature insights and impact analysis using explainable artificial intelligence (XAI). The normalizing flow is compared to an informed selection of historical data and probabilistic forecasts using a Gaussian copula and a Gaussian regression model. Among the different models, the normalizing flow identifies the trends with the highest accuracy and has the narrowest prediction intervals. Both the XAI analysis and the empirical experiments highlight that the immediate history of the price difference realization and the increments of the day-ahead price have the most substantial impact on the price difference.

cs.LG↗

arXiv2022

Demand Response for Flat Nonlinear MIMO Processes using Dynamic Ramping Constraints

Volatile electricity prices make demand response (DR) attractive for processes that can modulate their production rate. However, if nonlinear dynamic processes must be scheduled simultaneously with their local multi-energy system, the resulting scheduling optimization problems often cannot be solved in real time. For single-input single-output processes, the problem can be simplified without sacrificing feasibility by dynamic ramping constraints that define a derivative of the production rate as the ramping degree of freedom. In this work, we extend dynamic ramping constraints to flat multi-input multi-output processes by a coordinate transformation that gives the true nonlinear ramping limits. Approximating these ramping limits by piecewise affine functions gives a mixed-integer linear formulation that guarantees feasible operation. As a case study, dynamic ramping constraints are derived for a heated reactor-separator process that is subsequently scheduled simultaneously with its multi-energy system. The dynamic ramping formulation bridges the gap between rigorous process models and simplified process representations for real-time scheduling.

math.OC↗

arXiv2022

Graph Machine Learning for Design of High-Octane Fuels

Fuels with high-knock resistance enable modern spark-ignition engines to achieve high efficiency and thus low CO2 emissions. Identification of molecules with desired autoignition properties indicated by a high research octane number and a high octane sensitivity is therefore of great practical relevance and can be supported by computer-aided molecular design (CAMD). Recent developments in the field of graph machine learning (graph-ML) provide novel, promising tools for CAMD. We propose a modular graph-ML CAMD framework that integrates generative graph-ML models with graph neural networks and optimization, enabling the design of molecules with desired ignition properties in a continuous molecular space. In particular, we explore the potential of Bayesian optimization and genetic algorithms in combination with generative graph-ML models. The graph-ML CAMD framework successfully identifies well-established high-octane components. It also suggests new candidates, one of which we experimentally investigate and use to illustrate the need for further auto-ignition training data.

cs.LG↗

arXiv2022

Dynamic Ramping for Demand Response of Processes and Energy Systems based on Exact Linearization

The increasing share of volatile renewable electricity production motivates demand response. Substantial potential for demand response is offered by flexible processes and their local multi-energy supply systems. Simultaneous optimization of their schedules can exploit the demand response potential, but leads to numerically challenging problems for nonlinear dynamic processes. In this paper, we propose to capture process dynamics using dynamic ramping constraints. In contrast to traditional static ramping constraints, dynamic ramping constraints are a function of the process state and can capture high-order dynamics. We derive dynamic ramping constraints rigorously for the case of single-input single-output processes that are exactly input-state linearizable. The resulting scheduling problem can be efficiently solved as a mixed-integer linear program. In a case study, we study two flexible reactors and a multi-energy system. The proper representation of process dynamics by dynamic ramping allows for faster transitions compared to static ramping constraints and thus higher economic benefits of demand response. The proposed dynamic ramping approach is sufficiently fast for application in online optimization.

math.OC↗

arXiv2022

Physical Pooling Functions in Graph Neural Networks for Molecular Property Prediction

Graph neural networks (GNNs) are emerging in chemical engineering for the end-to-end learning of physicochemical properties based on molecular graphs. A key element of GNNs is the pooling function which combines atom feature vectors into molecular fingerprints. Most previous works use a standard pooling function to predict a variety of properties. However, unsuitable pooling functions can lead to unphysical GNNs that poorly generalize. We compare and select meaningful GNN pooling methods based on physical knowledge about the learned properties. The impact of physical pooling functions is demonstrated with molecular properties calculated from quantum mechanical computations. We also compare our results to the recent set2set pooling approach. We recommend using sum pooling for the prediction of properties that depend on molecular size and compare pooling functions for properties that are molecular size-independent. Overall, we show that the use of physical pooling functions significantly enhances generalization.

cs.LG↗

arXiv2022

Graph neural networks for the prediction of molecular structure-property relationships

Molecular property prediction is of crucial importance in many disciplines such as drug discovery, molecular biology, or material and process design. The frequently employed quantitative structure-property/activity relationships (QSPRs/QSARs) characterize molecules by descriptors which are then mapped to the properties of interest via a linear or nonlinear model. In contrast, graph neural networks, a novel machine learning method, directly work on the molecular graph, i.e., a graph representation where atoms correspond to nodes and bonds correspond to edges. GNNs allow to learn properties in an end-to-end fashion, thereby avoiding the need for informative descriptors as in QSPRs/QSARs. GNNs have been shown to achieve state-of-the-art prediction performance on various property predictions tasks and represent an active field of research. We describe the fundamentals of GNNs and demonstrate the application of GNNs via two examples for molecular property prediction.

q-bio.BM↗

arXiv2022

Normalizing Flow-based Day-Ahead Wind Power Scenario Generation for Profitable and Reliable Delivery Commitments by Wind Farm Operators

We present a specialized scenario generation method that utilizes forecast information to generate scenarios for day-ahead scheduling problems. In particular, we use normalizing flows to generate wind power scenarios by sampling from a conditional distribution that uses wind speed forecasts to tailor the scenarios to a specific day. We apply the generated scenarios in a stochastic day-ahead bidding problem of a wind electricity producer and analyze whether the scenarios yield profitable decisions. Compared to Gaussian copulas and Wasserstein-generative adversarial networks, the normalizing flow successfully narrows the range of scenarios around the daily trends while maintaining a diverse variety of possible realizations. In the stochastic day-ahead bidding problem, the conditional scenarios from all methods lead to significantly more stable profitable results compared to an unconditional selection of historical scenarios. The normalizing flow consistently obtains the highest profits, even for small sets scenarios.

math.OC↗

arXiv2022

Personalized Algorithm Generation: A Case Study in Learning ODE Integrators

We study the learning of numerical algorithms for scientific computing, which combines mathematically driven, handcrafted design of general algorithm structure with a data-driven adaptation to specific classes of tasks. This represents a departure from the classical approaches in numerical analysis, which typically do not feature such learning-based adaptations. As a case study, we develop a machine learning approach that automatically learns effective solvers for initial value problems in the form of ordinary differential equations (ODEs), based on the Runge-Kutta (RK) integrator architecture. We show that we can learn high-order integrators for targeted families of differential equations without the need for computing integrator coefficients by hand. Moreover, we demonstrate that in certain cases we can obtain superior performance to classical RK methods. This can be attributed to certain properties of the ODE families being identified and exploited by the approach. Overall, this work demonstrates an effective learning-based approach to the design of algorithms for the numerical solution of differential equations. This can be readily extended to other numerical tasks.

math.NA↗

arXiv2022

Graph Neural Networks for Temperature-Dependent Activity Coefficient Prediction of Solutes in Ionic Liquids

Ionic liquids (ILs) are important solvents for sustainable processes and predicting activity coefficients (ACs) of solutes in ILs is needed. Recently, matrix completion methods (MCMs), transformers, and graph neural networks (GNNs) have shown high accuracy in predicting ACs of binary mixtures, superior to well-established models, e.g., COSMO-RS and UNIFAC. GNNs are particularly promising here as they learn a molecular graph-to-property relationship without pretraining, typically required for transformers, and are, unlike MCMs, applicable to molecules not included in training. For ILs, however, GNN applications are currently missing. Herein, we present a GNN to predict temperature-dependent infinite dilution ACs of solutes in ILs. We train the GNN on a database including more than 40,000 AC values and compare it to a state-of-the-art MCM. The GNN and MCM achieve similar high prediction performance, with the GNN additionally enabling high-quality predictions for ACs of solutions that contain ILs and solutes not considered during training.

cs.LG↗

arXiv2022

Simultaneous mixed-integer dynamic scheduling of processes and their energy systems

Increasingly volatile electricity prices make simultaneous scheduling optimization desirable for production processes and their energy systems. Simultaneous scheduling needs to account for both process dynamics and binary on/off-decisions in the energy system leading to challenging mixed-integer dynamic optimization problems. We propose an efficient scheduling formulation consisting of three parts: a linear scale-bridging model for the closed-loop process output dynamics, a data-driven model for the process energy demand, and a mixed-integer linear model for the energy system. Process dynamics are discretized by collocation yielding a mixed-integer linear programming (MILP) formulation. We apply the scheduling method to three case studies: a multi-product reactor, a single-product reactor, and a single-product distillation column, demonstrating the applicability to multi-input multi-output processes. For the first two case studies, we can compare our approach to nonlinear optimization and capture 82 % and 95 % of the improvement. The MILP formulation achieves optimization runtimes sufficiently fast for real-time scheduling.

math.OC↗

arXiv2022

Principal Component Density Estimation for Scenario Generation Using Normalizing Flows

Neural networks-based learning of the distribution of non-dispatchable renewable electricity generation from sources such as photovoltaics (PV) and wind as well as load demands has recently gained attention. Normalizing flow density models are particularly well suited for this task due to the training through direct log-likelihood maximization. However, research from the field of image generation has shown that standard normalizing flows can only learn smeared-out versions of manifold distributions. Previous works on normalizing flow-based scenario generation do not address this issue, and the smeared-out distributions result in the sampling of noisy time series. In this paper, we exploit the isometry of the principal component analysis (PCA), which sets up the normalizing flow in a lower-dimensional space while maintaining the direct and computationally efficient likelihood maximization. We train the resulting principal component flow (PCF) on data of PV and wind power generation as well as load demand in Germany in the years 2013 to 2015. The results of this investigation show that the PCF preserves critical features of the original distributions, such as the probability density and frequency behavior of the time series. The application of the PCF is, however, not limited to renewable power generation but rather extends to any data set, time series, or otherwise, which can be efficiently reduced using PCA.

cs.LG↗

arXiv2021

Validation Methods for Energy Time Series Scenarios from Deep Generative Models

The design and operation of modern energy systems are heavily influenced by time-dependent and uncertain parameters, e.g., renewable electricity generation, load-demand, and electricity prices. These are typically represented by a set of discrete realizations known as scenarios. A popular scenario generation approach uses deep generative models (DGM) that allow scenario generation without prior assumptions about the data distribution. However, the validation of generated scenarios is difficult, and a comprehensive discussion about appropriate validation methods is currently lacking. To start this discussion, we provide a critical assessment of the currently used validation methods in the energy scenario generation literature. In particular, we assess validation methods based on probability density, auto-correlation, and power spectral density. Furthermore, we propose using the multifractal detrended fluctuation analysis (MFDFA) as an additional validation method for non-trivial features like peaks, bursts, and plateaus. As representative examples, we train generative adversarial networks (GANs), Wasserstein GANs (WGANs), and variational autoencoders (VAEs) on two renewable power generation time series (photovoltaic and wind from Germany in 2013 to 2015) and an intra-day electricity price time series form the European Energy Exchange in 2017 to 2019. We apply the four validation methods to both the historical and the generated data and discuss the interpretation of validation results as well as common mistakes, pitfalls, and limitations of the validation methods. Our assessment shows that no single method sufficiently characterizes a scenario but ideally validation should include multiple methods and be interpreted carefully in the context of scenarios over short time periods.

cs.LG↗

arXiv2021

COMANDO: A Next-Generation Open-Source Framework for Energy Systems Optimization

Existing open-source modeling frameworks dedicated to energy systems optimization typically utilize (mixed-integer) linear programming ((MI)LP) formulations, which lack modeling freedom for technical system design and operation. We present COMANDO, an open-source Python package for component-oriented modeling and optimization for nonlinear design and operation of integrated energy systems. COMANDO allows to assemble system models from component models including nonlinear, dynamic and discrete characteristics. Based on a single system model, different deterministic and stochastic problem formulations can be obtained by varying objective function and underlying data, and by applying automatic or manual reformulations. The flexible open-source implementation allows for the integration of customized routines required to solve challenging problems, e.g., initialization, problem decomposition, or sequential solution strategies. We demonstrate features of COMANDO via case studies, including automated linearization, dynamic optimization, stochastic programming, and the use of nonlinear artificial neural networks as surrogate models in a reduced-space formulation for deterministic global optimization.

math.OC↗