arXiv subjects

Wolfgang Nowak

Publications and source records attributed to Wolfgang Nowak.

16 recordsLinked to original sources

arXiv2026

Uncertainty-Aware Surrogate-based Amortized Bayesian Inference for Computationally Expensive Models

Bayesian inference typically relies on a large number of model evaluations to estimate posterior distributions. Established methods like Markov Chain Monte Carlo (MCMC) and Amortized Bayesian Inference (ABI) can become computationally challenging. While ABI enables fast inference after training, generating sufficient training data still requires thousands of model simulations, which is infeasible for expensive models. Surrogate models offer a solution by providing approximate simulations at a lower computational cost, allowing the generation of large data sets for training. However, the introduced approximation errors and uncertainties can lead to overconfident posterior estimates. To address this, we propose Uncertainty-Aware Surrogate-based Amortized Bayesian Inference (UA-SABI) -- a framework that combines surrogate modeling and ABI while explicitly quantifying and propagating surrogate uncertainties through the inference pipeline. Our experiments show that this approach enables reliable, fast, and repeated Bayesian inference for computationally expensive models, even under tight time constraints.

stat.ML↗

arXiv2025

CODE: A global approach to ODE dynamics learning

Ordinary differential equations (ODEs) are a conventional way to describe the observed dynamics of physical systems. Scientists typically hypothesize about dynamical behavior, propose a mathematical model, and compare its predictions to data. However, modern computing and algorithmic advances now enable purely data-driven learning of governing dynamics directly from observations. In data-driven settings, one learns the ODE's right-hand side (RHS). Dense measurements are often assumed, yet high temporal resolution is typically both cumbersome and expensive. Consequently, one usually has only sparsely sampled data. In this work we introduce ChaosODE (CODE), a Polynomial Chaos ODE Expansion in which we use an arbitrary Polynomial Chaos Expansion (aPCE) for the ODE's right-hand side, resulting in a global orthonormal polynomial representation of dynamics. We evaluate the performance of CODE in several experiments on the Lotka-Volterra system, across varying noise levels, initial conditions, and predictions far into the future, even on previously unseen initial conditions. CODE exhibits remarkable extrapolation capabilities even when evaluated under novel initial conditions and shows advantages compared to well-examined methods using neural networks (NeuralODE) or kernel approximators (KernelODE) as the RHS representer. We observe that the high flexibility of NeuralODE and KernelODE degrades extrapolation capabilities under scarce data and measurement noise. Finally, we provide practical guidelines for robust optimization of dynamics-learning problems and illustrate them in the accompanying code.

cs.LG↗

arXiv2025

HydroStartML: A combined machine learning and physics-based approach to reduce hydrological model spin-up time

Finding the initial depth-to-water table (DTWT) configuration of a catchment is a critical challenge when simulating the hydrological cycle with integrated models, significantly impacting simulation outcomes. Traditionally, this involves iterative spin-up computations, where the model runs under constant atmospheric settings until steady-state is achieved. These so-called model spin-ups are computationally expensive, often requiring many years of simulated time, particularly when the initial DTWT configuration is far from steady state. To accelerate the model spin-up process we developed HydroStartML, a machine learning emulator trained on steady-state DTWT configurations across the contiguous United States. HydroStartML predicts, based on available data like conductivity and surface slopes, a DTWT configuration of the respective watershed, which can be used as an initial DTWT. Our results show that initializing spin-up computations with HydroStartML predictions leads to faster convergence than with other initial configurations like spatially constant DTWTs. The emulator accurately predicts configurations close to steady state, even for terrain configurations not seen in training, and allows especially significant reductions in computational spin-up effort in regions with deep DTWTs. This work opens the door for hybrid approaches that blend machine learning and traditional simulation, enhancing predictive accuracy and efficiency in hydrology for improving water resource management and understanding complex environmental interactions.

physics.geo-ph↗

arXiv2024

Inferring Underwater Topography with FINN

Spatiotemporal partial differential equations (PDEs) find extensive application across various scientific and engineering fields. While numerous models have emerged from both physics and machine learning (ML) communities, there is a growing trend towards integrating these approaches to develop hybrid architectures known as physics-aware machine learning models. Among these, the finite volume neural network (FINN) has emerged as a recent addition. FINN has proven to be particularly efficient in uncovering latent structures in data. In this study, we explore the capabilities of FINN in tackling the shallow-water equations, which simulates wave dynamics in coastal regions. Specifically, we investigate FINN's efficacy to reconstruct underwater topography based on these particular wave equations. Our findings reveal that FINN exhibits a remarkable capacity to infer topography solely from wave dynamics, distinguishing itself from both conventional ML and physics-aware ML models. Our results underscore the potential of FINN in advancing our understanding of spatiotemporal phenomena and enhancing parametrization capabilities in related domains.

cs.LG↗

arXiv2024

Overdispersion in gate tomography: Experiments and continuous, two-scale random walk model on the Bloch sphere

Noisy intermediate-scale quantum computers (NISQ) are in their childhood, but showing high promise. One main concern for NISQ machines is their inherent noisiness, as the qubit states are subject to disturbances with each algorithmic operation applied. In this study, we conduct experiments on quantum noise. Based on our data, we show that existing noise models fail to properly capture the aggregation of noise effects over an algorithm's runtime. They are underdispersed, meaning that observable frequencies scatter much more between repeated experiments than what the standard assumptions of the binomial distribution allow for. We develop noise model for the readout probabilities as a function of the number of gate operations. The model is based on a continuous random walk on the (Bloch) sphere, where the angular diffusion coefficient characterizes the noisiness of gate operations. We superimpose a second random walk at the scale of multiple readouts to account for overdispersion. The interaction of these two random walks predicts theoretical, runtime-dependent bounds for probabilities. Overall, it is a three-parameter distributional model that fits the data better than the corresponding one-scale model (without overdispersion). We demonstrate the fit and the plausibility of the predicted bounds via Bayesian data-model analysis.

quant-ph↗

arXiv2024

Probabilistic Regular Tree Priors for Scientific Symbolic Reasoning

Symbolic Regression (SR) allows for the discovery of scientific equations from data. To limit the large search space of possible equations, prior knowledge has been expressed in terms of formal grammars that characterize subsets of arbitrary strings. However, there is a mismatch between context-free grammars required to express the set of syntactically correct equations, missing closure properties of the former, and a tree structure of the latter. Our contributions are to (i) compactly express experts' prior beliefs about which equations are more likely to be expected by probabilistic Regular Tree Expressions (pRTE), and (ii) adapt Bayesian inference to make such priors efficiently available for symbolic regression encoded as finite state machines. Our scientific case studies show its effectiveness in soil science to find sorption isotherms and for modeling hyper-elastic materials.

cs.LG↗

arXiv2023

The Deep Arbitrary Polynomial Chaos Neural Network or how Deep Artificial Neural Networks could benefit from Data-Driven Homogeneous Chaos Theory

Artificial Intelligence and Machine learning have been widely used in various fields of mathematical computing, physical modeling, computational science, communication science, and stochastic analysis. Approaches based on Deep Artificial Neural Networks (DANN) are very popular in our days. Depending on the learning task, the exact form of DANNs is determined via their multi-layer architecture, activation functions and the so-called loss function. However, for a majority of deep learning approaches based on DANNs, the kernel structure of neural signal processing remains the same, where the node response is encoded as a linear superposition of neural activity, while the non-linearity is triggered by the activation functions. In the current paper, we suggest to analyze the neural signal processing in DANNs from the point of view of homogeneous chaos theory as known from polynomial chaos expansion (PCE). From the PCE perspective, the (linear) response on each node of a DANN could be seen as a $1^{st}$ degree multi-variate polynomial of single neurons from the previous layer, i.e. linear weighted sum of monomials. From this point of view, the conventional DANN structure relies implicitly (but erroneously) on a Gaussian distribution of neural signals. Additionally, this view revels that by design DANNs do not necessarily fulfill any orthogonality or orthonormality condition for a majority of data-driven applications. Therefore, the prevailing handling of neural signals in DANNs could lead to redundant representation as any neural signal could contain some partial information from other neural signals. To tackle that challenge, we suggest to employ the data-driven generalization of PCE theory known as arbitrary polynomial chaos (aPC) to construct a corresponding multi-variate orthonormal representations on each node of a DANN to obtain Deep arbitrary polynomial chaos neural networks.

cs.NE↗

arXiv2023

A fully Bayesian sparse polynomial chaos expansion approach with joint priors on the coefficients and global selection of terms

Polynomial chaos expansion (PCE) is a versatile tool widely used in uncertainty quantification and machine learning, but its successful application depends strongly on the accuracy and reliability of the resulting PCE-based response surface. High accuracy typically requires high polynomial degrees, demanding many training points especially in high-dimensional problems through the curse of dimensionality. So-called sparse PCE concepts work with a much smaller selection of basis polynomials compared to conventional PCE approaches and can overcome the curse of dimensionality very efficiently, but have to pay specific attention to their strategies of choosing training points. Furthermore, the approximation error resembles an uncertainty that most existing PCE-based methods do not estimate. In this study, we develop and evaluate a fully Bayesian approach to establish the PCE representation via joint shrinkage priors and Markov chain Monte Carlo. The suggested Bayesian PCE model directly aims to solve the two challenges named above: achieving a sparse PCE representation and estimating uncertainty of the PCE itself. The embedded Bayesian regularizing via the joint shrinkage prior allows using higher polynomial degrees for given training points due to its ability to handle underdetermined situations, where the number of considered PCE coefficients could be much larger than the number of available training points. We also explore multiple variable selection methods to construct sparse PCE expansions based on the established Bayesian representations, while globally selecting the most meaningful orthonormal polynomials given the available training data. We demonstrate the advantages of our Bayesian PCE and the corresponding sparsity-inducing methods on several benchmarks.

stat.CO↗

arXiv2022

Composing Partial Differential Equations with Physics-Aware Neural Networks

We introduce a compositional physics-aware FInite volume Neural Network (FINN) for learning spatiotemporal advection-diffusion processes. FINN implements a new way of combining the learning abilities of artificial neural networks with physical and structural knowledge from numerical simulation by modeling the constituents of partial differential equations (PDEs) in a compositional manner. Results on both one- and two-dimensional PDEs (Burgers', diffusion-sorption, diffusion-reaction, Allen--Cahn) demonstrate FINN's superior modeling accuracy and excellent out-of-distribution generalization ability beyond initial and boundary conditions. With only one tenth of the number of parameters on average, FINN outperforms pure machine learning and other state-of-the-art physics-aware models in all cases -- often even by multiple orders of magnitude. Moreover, FINN outperforms a calibrated physical model when approximating sparse real-world data in a diffusion-sorption scenario, confirming its generalization abilities and showing explanatory potential by revealing the unknown retardation factor of the observed process.

cs.LG↗

arXiv2021

Investigating the Pilot Point Ensemble Kalman Filter for geostatistical inversion and data assimilation

Parameter estimation has a high importance in the geosciences. The ensemble Kalman filter (EnKF) allows parameter estimation for large, time-dependent systems. For large systems, the EnKF is applied using small ensembles, which may lead to spurious correlations and, ultimately, to filter divergence. We present a thorough evaluation of the pilot point ensemble Kalman filter (PP-EnKF), a variant of the ensemble Kalman filter for parameter estimation. In this evaluation, we explicitly state the update equations of the PP-EnKF, discuss the differences of this update equation compared to the update equations of similar EnKF methods, and perform an extensive performance comparison. The performance of the PP-EnKF is tested and compared to the performance of seven other EnKF methods in two model setups, a tracer setup and a well setup. In both setups, the PP-EnKF performs well, ranking better than the classical EnKF. For the tracer setup, the PP-EnKF ranks third out of eight methods. At the same time, the PP-EnKF yields estimates of the ensemble variance that are close to EnKF results from a very large-ensemble reference, suggesting that it is not affected by underestimation of the ensemble variance. In a comparison of the ensemble variances, the PP-EnKF ranks first and third out of eight methods. Additionally, for the well model and ensemble size 50, the PP-EnKF yields correlation structures significantly closer to a reference than the classical EnKF, an indication of the method's skill to suppress spurious correlations for small ensemble sizes.

stat.AP↗

arXiv2021

Diagnosis of model-structural errors with a sliding time-window Bayesian analysis

Deterministic hydrological models with uncertain, but inferred-to-be-time-invariant parameters typically show time-dependent model structural errors. Such errors can occur if a hydrological process is active in certain time periods in nature, but is not resolved by the model. Such missing processes could become visible during calibration as time-dependent best-fit values of model parameters. We propose a formal time-windowed Bayesian analysis to diagnose this type of model error, formalizing the question \In which period of the calibration time-series does the model statistically disqualify itself as quasi-true?" Using Bayesian model evidence (BME) as model performance metric, we determine how much the data in time windows of the calibration time-series support or refute the model. Then, we track BME over sliding time windows to obtain a dynamic, time-windowed BME (tBME) and search for sudden decreases that indicate an onset of model error. tBME also allows us to perform a formal, sliding likelihood-ratio test of the model against the data. Our proposed approach is designed to detect error occurrence on various temporal scales, which is especially useful in hydrological modelling. We illustrate this by applying our proposed method to soil moisture modeling. We test tBME as model error indicator on several synthetic and real-world test cases that we designed to vary in error sources and error time scales. Results prove the usefulness of the framework for detecting structural errors in dynamic models. Moreover, the time sequence of posterior parameter distributions helps to investigate the reasons for model error and provide guidance for model improvement.

stat.ME↗

arXiv2021

Finite Volume Neural Network: Modeling Subsurface Contaminant Transport

Data-driven modeling of spatiotemporal physical processes with general deep learning methods is a highly challenging task. It is further exacerbated by the limited availability of data, leading to poor generalizations in standard neural network models. To tackle this issue, we introduce a new approach called the Finite Volume Neural Network (FINN). The FINN method adopts the numerical structure of the well-known Finite Volume Method for handling partial differential equations, so that each quantity of interest follows its own adaptable conservation law, while it concurrently accommodates learnable parameters. As a result, FINN enables better handling of fluxes between control volumes and therefore proper treatment of different types of numerical boundary conditions. We demonstrate the effectiveness of our approach with a subsurface contaminant transport problem, which is governed by a non-linear diffusion-sorption process. FINN does not only generalize better to differing boundary conditions compared to other methods, it is also capable to explicitly extract and learn the constitutive relationships (expressed by the retardation factor). More importantly, FINN shows excellent generalization ability when applied to both synthetic datasets and real, sparse experimental data, thus underlining its relevance as a data-driven modeling tool.

cs.LG↗

arXiv2021

Sequential pCN-MCMC, an efficient MCMC method for Bayesian inversion of high-dimensional multi-Gaussian priors

In geostatistics, Gaussian random fields are often used to model heterogeneities of soil or subsurface parameters. To give spatial approximations of these random fields, they are discretized. Then, different techniques of geostatistical inversion are used to condition them on measurement data. Among these techniques, Markov chain Monte Carlo (MCMC) techniques stand out, because they yield asymptotically unbiased conditional realizations. However, standard Markov Chain Monte Carlo (MCMC) methods suffer the curse of dimensionality when refining the discretization. This means that their efficiency decreases rapidly with an increasing number of discretization cells. Several MCMC approaches have been developed such that the MCMC efficiency does not depend on the discretization of the random field. The pre-conditioned Crank Nicolson Markov Chain Monte Carlo (pCN-MCMC) and the sequential Gibbs (or block-Gibbs) sampling are two examples. In this paper, we will present a combination of the pCN-MCMC and the sequential Gibbs sampling. Our algorithm, the sequential pCN-MCMC, will depend on two tuning-parameters: the correlation parameter $β$ of the pCN approach and the block size $κ$ of the sequential Gibbs approach. The original pCN-MCMC and the Gibbs sampling algorithm are special cases of our method. We present an algorithm that automatically finds the best tuning-parameter combination ($κ$ and $β$) during the burn-in-phase of the algorithm, thus choosing the best possible hybrid between the two methods. In our test cases, we achieve a speedup factors of $1-5.5$ over pCN and of $1-6.5$ over Gibbs. Furthermore, we provide the MATLAB implementation of our method as open-source code.

stat.CO↗

arXiv2020

Surrogate-based Bayesian Comparison of Computationally Expensive Models: Application to Microbially Induced Calcite Precipitation

Geochemical processes in subsurface reservoirs affected by microbial activity change the material properties of porous media. This is a complex biogeochemical process in subsurface reservoirs that currently contains strong conceptual uncertainty. This means, several modeling approaches describing the biogeochemical process are plausible and modelers face the uncertainty of choosing the most appropriate one. Once observation data becomes available, a rigorous Bayesian model selection accompanied by a Bayesian model justifiability analysis could be employed to choose the most appropriate model, i.e. the one that describes the underlying physical processes best in the light of the available data. However, biogeochemical modeling is computationally very demanding because it conceptualizes different phases, biomass dynamics, geochemistry, precipitation and dissolution in porous media. Therefore, the Bayesian framework cannot be based directly on the full computational models as this would require too many expensive model evaluations. To circumvent this problem, we suggest performing both Bayesian model selection and justifiability analysis after constructing surrogates for the competing biogeochemical models. Here, we use the arbitrary polynomial chaos expansion. We account for the approximation error in the Bayesian analysis by introducing novel correction factors for the resulting model weights. Thereby, we extend the Bayesian justifiability analysis and assess model similarities for computationally expensive models. We demonstrate the method on a representative scenario for microbially induced calcite precipitation in a porous medium. Our extension of the justifiability analysis provides a suitable approach for the comparison of computationally demanding models and gives an insight on the necessary amount of data for a reliable model performance.

stat.AP↗

arXiv2019

Using Surrogate Models and Data Assimilation for Efficient Mobile Simulations

Numerical simulations on mobile devices are an important tool for engineers and decision makers in the field. However, providing simulation results on mobile devices is challenging due to the complexity of the simulation, requiring remote server resources and distributed mobile computation. The additional large size of multi-dimensional simulation results leads to the insufficient performance of existing approaches, especially when the bandwidth of wireless communication is scarce. In this article, we present an optimized novel approach utilizing surrogate models and data assimilation techniques to reduce the communication overhead. Evaluations show that our approach is up to $6.5$ times faster than streaming results from the server while still meeting required quality constraints.

cs.DC↗

arXiv2018

Comparison of data-driven uncertainty quantification methods for a carbon dioxide storage benchmark scenario

A variety of methods is available to quantify uncertainties arising with\-in the modeling of flow and transport in carbon dioxide storage, but there is a lack of thorough comparisons. Usually, raw data from such storage sites can hardly be described by theoretical statistical distributions since only very limited data is available. Hence, exact information on distribution shapes for all uncertain parameters is very rare in realistic applications. We discuss and compare four different methods tested for data-driven uncertainty quantification based on a benchmark scenario of carbon dioxide storage. In the benchmark, for which we provide data and code, carbon dioxide is injected into a saline aquifer modeled by the nonlinear capillarity-free fractional flow formulation for two incompressible fluid phases, namely carbon dioxide and brine. To cover different aspects of uncertainty quantification, we incorporate various sources of uncertainty such as uncertainty of boundary conditions, of conceptual model definitions and of material properties. We consider recent versions of the following non-intrusive and intrusive uncertainty quantification methods: arbitary polynomial chaos, spatially adaptive sparse grids, kernel-based greedy interpolation and hybrid stochastic Galerkin. The performance of each approach is demonstrated assessing expectation value and standard deviation of the carbon dioxide saturation against a reference statistic based on Monte Carlo sampling. We compare the convergence of all methods reporting on accuracy with respect to the number of model runs and resolution. Finally we offer suggestions about the methods' advantages and disadvantages that can guide the modeler for uncertainty quantification in carbon dioxide storage and beyond.

cs.CE↗