arXiv subjects

Robert Scheichl

Publications and source records attributed to Robert Scheichl.

At least 37 records · Page 2Linked to original sources

arXiv2023

Multilevel Dimension-Independent Likelihood-Informed MCMC for Large-Scale Inverse Problems

We present a non-trivial integration of dimension-independent likelihood-informed (DILI) MCMC (Cui, Law, Marzouk, 2016) and the multilevel MCMC (Dodwell et al., 2015) to explore the hierarchy of posterior distributions. This integration offers several advantages: First, DILI-MCMC employs an intrinsic likelihood-informed subspace (LIS) (Cui et al., 2014) -- which involves a number of forward and adjoint model simulations -- to design accelerated operator-weighted proposals. By exploiting the multilevel structure of the discretised parameters and discretised forward models, we design a Rayleigh-Ritz procedure to significantly reduce the computational effort in building the LIS and operating with DILI proposals. Second, the resulting DILI-MCMC can drastically improve the sampling efficiency of MCMC at each level, and hence reduce the integration error of the multilevel algorithm for fixed CPU time. Numerical results confirm the improved computational efficiency of the multilevel DILI approach.

stat.CO↗

arXiv2023

Improving Met Office Weather and Climate Forecasts with Bespoke Multigrid Solvers

At the heart of the Met Office climate and weather forecasting capabilities lies a sophisticated numerical model which solves the equations of large-scale atmospheric flow. Since this model uses semi-implicit time-stepping, it requires the repeated solution of a large sparse system of linear equations with hundreds of millions of unknowns. This is one of the computational bottlenecks of operational forecasts and efficient numerical algorithms are crucial to ensure optimal performance. We developed and implemented a bespoke multigrid solver to address this challenge. Our solver reduces the time for solving the linear system by a factor two, compared to the previously used BiCGStab method. This leads to significant improvements of overall model performance: global forecasts can be produced 10%-15% faster. Multigrid also avoids stagnating convergence of the iterative scheme in single precision. By allowing better utilisation of computational resources, our work has led to estimated annual cost savings of GBP 300k for the Met Office.

physics.comp-ph↗

arXiv2023

Deep importance sampling using tensor trains with application to a priori and a posteriori rare event estimation

We propose a deep importance sampling method that is suitable for estimating rare event probabilities in high-dimensional problems. We approximate the optimal importance distribution in a general importance sampling problem as the pushforward of a reference distribution under a composition of order-preserving transformations, in which each transformation is formed by a squared tensor-train decomposition. The squared tensor-train decomposition provides a scalable ansatz for building order-preserving high-dimensional transformations via density approximations. The use of composition of maps moving along a sequence of bridging densities alleviates the difficulty of directly approximating concentrated density functions. To compute expectations over unnormalized probability distributions, we design a ratio estimator that estimates the normalizing constant using a separate importance distribution, again constructed via a composition of transformations in tensor-train format. This offers better theoretical variance reduction compared with self-normalized importance sampling, and thus opens the door to efficient computation of rare event probabilities in Bayesian inference problems. Numerical experiments on problems constrained by differential equations show little to no increase in the computational complexity with the event probability going to zero, and allow to compute hitherto unattainable estimates of rare event probabilities for complex, high-dimensional posterior densities.

stat.ML↗

arXiv2023

Lowering the Entry Bar to HPC-Scale Uncertainty Quantification

Treating uncertainties in models is essential in many fields of science and engineering. Uncertainty quantification (UQ) on complex and computationally costly numerical models necessitates a combination of efficient model solvers, advanced UQ methods and HPC-scale resources. The resulting technical complexities as well as lack of separation of concerns between UQ and model experts is holding back many interesting UQ applications. The aim of this paper is to close the gap between advanced UQ methods and advanced models by removing the hurdle of complex software stack integration, which in turn will offer a straightforward way to scale even prototype-grade UQ applications to high-performance resources. We achieve this goal by introducing a parallel software architecture based on UM-Bridge, a universal interface for linking UQ and models. We present three realistic applications from different areas of science and engineering, scaling from single machines to large clusters on the Google Cloud Platform.

cs.DC↗

arXiv2023

Scalable multiscale-spectral GFEM with an application to composite aero-structures

In this paper, the first large-scale application of multiscale-spectral generalized finite element methods (MS-GFEM) to composite aero-structures is presented. The crucial novelty lies in the introduction of A-harmonicity in the local approximation spaces, which in contrast to [Babuska, Lipton, Multiscale Model. Simul. 9, 2011] is enforced more efficiently via a constraint in the local eigenproblems. This significant modification leads to excellent approximation properties, which turn out to be essential to capture accurately material strains and stresses with a low dimensional approximation space, hence maximising model order reduction. The implementation of the framework in the DUNE software package, as well as a detailed description of all components of the method are presented and exemplified on a composite laminated beam under compressive loading. The excellent parallel scalability of the method, as well as its superior performance compared to the related, previously introduced GenEO method are demonstrated on two realistic application cases, including a C-shaped wing spar with complex geometry. Further, by allowing low-cost approximate solves for closely related models or geometries this efficient, novel technology provides the basis for future applications in optimisation or uncertainty quantification on challenging problems in composite aero-structures.

math.NA↗

arXiv2023

Separating internal and externally-forced contributions to global temperature variability using a Bayesian stochastic energy balance framework

Earth's temperature variability can be partitioned into internal and externally-forced components. Yet, underlying mechanisms and their relative contributions remain insufficiently understood, especially on decadal to centennial timescales. Important reasons for this are difficulties in isolating internal and externally-forced variability. Here, we provide a physically-motivated emulation of global mean surface temperature (GMST) variability, which allows for the separation of internal and external variations. To this end, we introduce the ``ClimBayes'' software package, which infers climate parameters from a stochastic energy balance model (EBM) with a Bayesian approach. We apply our method to GMST data from temperature observations and 20 last millennium simulations from climate models of intermediate to high complexity. This yields the best estimates of the EBM's forced and forced + internal response, which we refer to as emulated variability. The timescale-dependent variance is obtained from spectral analysis. In particular, we contrast the emulated forced and forced + internal variance on interannual to centennial timescales with that of the GMST target. Our findings show that a stochastic EBM closely approximates the power spectrum and timescale-dependent variance of GMST as simulated by modern climate models. Small deviations at interannual timescales can be attributed to the simplified representation of internal variability and, in particular, the absence of (pseudo-)oscillatory modes in the stochastic EBM. Altogether, we demonstrate the potential of combining Bayesian inference with conceptual climate models to emulate statistics of climate variables across timescales.

physics.ao-ph↗

arXiv2023

A Bayesian Approach to Modelling Biological Pattern Formation with Limited Data

Pattern formation in biological tissues plays an important role in the development of living organisms. Since the classical work of Alan Turing, a pre-eminent way of modelling has been through reaction-diffusion mechanisms. More recently, alternative models have been proposed, that link dynamics of diffusing molecular signals with tissue mechanics. In order to distinguish among different models, they should be compared to experimental observations. However, in many experimental situations only the limiting, stationary regime of the pattern formation process is observable, without knowledge of the transient behaviour or the initial state. The unstable nature of the underlying dynamics in all alternative models seriously complicates model and parameter identification, since small changes in the initial condition lead to distinct stationary patterns. To overcome this problem the initial state of the model can be randomised. In the latter case, fixed values of the model parameters correspond to a family of patterns rather than a fixed stationary solution, and standard approaches to compare pattern data directly with model outputs, e.g., in the least squares sense, are not suitable. Instead, statistical characteristics of the patterns should be compared, which is difficult given the typically limited amount of available data in practical applications. To deal with this problem, we extend a recently developed statistical approach for parameter identification using pattern data, the so-called Correlation Integral Likelihood (CIL) method. We suggest modifications that allow increasing the accuracy of the identification process without resizing the data set. The proposed approach is tested using different classes of pattern formation models. For all considered equations, parallel GPU-based implementations of the numerical solvers with efficient time stepping schemes are provided.

math.AP↗

arXiv2022

Multilevel quasi-Monte Carlo for random elliptic eigenvalue problems I: Regularity and error analysis

Stochastic PDE eigenvalue problems are useful models for quantifying the uncertainty in several applications from the physical sciences and engineering, e.g., structural vibration analysis, the criticality of a nuclear reactor or photonic crystal structures. In this paper we present a multilevel quasi-Monte Carlo (MLQMC) method for approximating the expectation of the minimal eigenvalue of an elliptic eigenvalue problem with coefficients that are given as a series expansion of countably-many stochastic parameters. The MLQMC algorithm is based on a hierarchy of discretisations of the spatial domain and truncations of the dimension of the stochastic parameter domain. To approximate the expectations, randomly shifted lattice rules are employed. This paper is primarily dedicated to giving a rigorous analysis of the error of this algorithm. A key step in the error analysis requires bounds on the mixed derivatives of the eigenfunction with respect to both the stochastic and spatial variables simultaneously. Under stronger smoothness assumptions on the parametric dependence, our analysis also extends to multilevel higher-order quasi-Monte Carlo rules. An accompanying paper [Gilbert and Scheichl, 2022], focusses on practical extensions of the MLQMC algorithm to improve efficiency, and presents numerical results.

math.NA↗

arXiv2022

Multilevel quasi-Monte Carlo for random elliptic eigenvalue problems II: Efficient algorithms and numerical results

Stochastic PDE eigenvalue problems often arise in the field of uncertainty quantification, whereby one seeks to quantify the uncertainty in an eigenvalue, or its eigenfunction. In this paper we present an efficient multilevel quasi-Monte Carlo (MLQMC) algorithm for computing the expectation of the smallest eigenvalue of an elliptic eigenvalue problem with stochastic coefficients. Each sample evaluation requires the solution of a PDE eigenvalue problem, and so tackling this problem in practice is notoriously computationally difficult. We speed up the approximation of this expectation in four ways: we use a multilevel variance reduction scheme to spread the work over a hierarchy of FE meshes and truncation dimensions; we use QMC methods to efficiently compute the expectations on each level; we exploit the smoothness in parameter space and reuse the eigenvector from a nearby QMC point to reduce the number of iterations of the eigensolver; and we utilise a two-grid discretisation scheme to obtain the eigenvalue on the fine mesh with a single linear solve. The full error analysis of a basic MLQMC algorithm is given in the companion paper [Gilbert and Scheichl, 2022], and so in this paper we focus on how to further improve the efficiency and provide theoretical justification for using nearby QMC points and two-grid methods. Numerical results are presented that show the efficiency of our algorithm, and also show that the four strategies we employ are complementary.

math.NA↗

arXiv2022

Wavenumber explicit convergence of a multiscale generalized finite element method for heterogeneous Helmholtz problems

In this paper, a generalized finite element method (GFEM) with optimal local approximation spaces for solving high-frequency heterogeneous Helmholtz problems is systematically studied. The local spaces are built from selected eigenvectors of carefully designed local eigenvalue problems defined on generalized harmonic spaces. At both continuous and discrete levels, $(i)$ wavenumber explicit and nearly exponential decay rates for local and global approximation errors are obtained without any assumption on the size of subdomains; $(ii)$ a quasi-optimal convergence of the method is established by assuming that the size of subdomains is $O(1/k)$ ($k$ is the wavenumber). A novel resonance effect between the wavenumber and the dimension of local spaces on the decay of error with respect to the oversampling size is implied by the analysis. Furthermore, for fixed dimensions of local spaces, the discrete local errors are proved to converge as $h\rightarrow 0$ ($h$ denoting the mesh size) towards the continuous local errors. The method at the continuous level extends the plane wave partition of unity method [I. Babuska and J. M. Melenk, Int.\;J.\;Numer.\;Methods Eng., 40 (1997), pp.~727--758] to the heterogeneous-coefficients case, and at the discrete level, it delivers an efficient non-iterative domain decomposition method for solving discrete Helmholtz problems resulting from standard FE discretizations. Numerical results are provided to confirm the theoretical analysis and to validate the proposed method.

math.NA↗

arXiv2022

Multilevel Delayed Acceptance MCMC

We develop a novel Markov chain Monte Carlo (MCMC) method that exploits a hierarchy of models of increasing complexity to efficiently generate samples from an unnormalized target distribution. Broadly, the method rewrites the Multilevel MCMC approach of Dodwell et al. (2015) in terms of the Delayed Acceptance (DA) MCMC of Christen & Fox (2005). In particular, DA is extended to use a hierarchy of models of arbitrary depth, and allow subchains of arbitrary length. We show that the algorithm satisfies detailed balance, hence is ergodic for the target distribution. Furthermore, multilevel variance reduction is derived that exploits the multiple levels and subchains, and an adaptive multilevel correction to coarse-level biases is developed. Three numerical examples of Bayesian inverse problems are presented that demonstrate the advantages of these novel methods. The software and examples are available in PyMC3.

stat.ME↗

arXiv2022

Multilevel simulation of hard-sphere mixtures

We present a multilevel Monte Carlo simulation method for analysing multi-scale physical systems via a hierarchy of coarse-grained representations, to obtain numerically-exact results, at the most detailed level. We apply the method to a mixture of size-asymmetric hard spheres, in the grand canonical ensemble. A three-level version of the method is compared with a previously-studied two-level version. The extra level interpolates between the full mixture and a coarse-grained description where only the large particles are present -- this is achieved by restricting the small particles to regions close to the large ones. The three-level method improves the performance of the estimator, at fixed computational cost. We analyse the asymptotic variance of the estimator, and discuss the mechanisms for the improved performance.

cond-mat.stat-mech↗

arXiv2022

Overlapping Schwarz methods with GenEO coarse spaces for indefinite and non-self-adjoint problems

GenEO (`Generalised Eigenvalue problems on the Overlap') is a method for computing an operator-dependent spectral coarse space to be combined with local solves on subdomains to form a robust parallel domain decomposition preconditioner for elliptic PDEs. It has previously been proved, in the self-adjoint and positive-definite case, that this method, when used as a preconditioner for conjugate gradients, yields iteration numbers which are completely independent of the heterogeneity of the coefficient field of the partial differential operator. We extend this theory to the case of convection-diffusion-reaction problems, which may be non-self-adjoint and indefinite, and whose discretisations are solved with preconditioned GMRES. The GenEO coarse space is defined here using a generalised eigenvalue problem based on a self-adjoint and positive-definite subproblem. We prove estimates on GMRES iteration counts which are independent of the variation of the coefficient of the diffusion term in the operator and depend only very mildly on variations of the other coefficients. These are proved under the assumption that the subdomain diameter is sufficiently small and the eigenvalue tolerance for building the coarse space is sufficiently large. While the iteration number estimates do grow as the non-self-adjointness and indefiniteness of the operator increases, practical tests indicate the deterioration is much milder. Thus we obtain an iterative solver which is efficient in parallel and very effective for a wide range of convection--diffusion--reaction problems.

math.NA↗

arXiv2021

Novel design and analysis of generalized FE methods based on locally optimal spectral approximations

In this paper, the generalized finite element method (GFEM) for solving second order elliptic equations with rough coefficients is studied. New optimal local approximation spaces for GFEMs based on local eigenvalue problems involving a partition of unity are presented. These new spaces have advantages over those proposed in [I. Babuska and R. Lipton, Multiscale Model.\;\,Simul., 9 (2011), pp.~373--406]. First, in addition to a nearly exponential decay rate of the local approximation errors with respect to the dimensions of the local spaces, the rate of convergence with respect to the size of the oversampling region is also established. Second, the theoretical results hold for problems with mixed boundary conditions defined on general Lipschitz domains. Finally, an efficient and easy-to-implement technique for generating the discrete $A$-harmonic spaces is proposed which relies on solving an eigenvalue problem associated with the Dirichlet-to-Neumann operator, leading to a substantial reduction in computational cost. Numerical experiments are presented to support the theoretical analysis and to confirm the effectiveness of the new method.

math.NA↗

arXiv2021

Error estimates for fully discrete generalized FEMs with locally optimal spectral approximations

This paper is concerned with error estimates of the fully discrete generalized finite element method (GFEM) with optimal local approximation spaces for solving elliptic problems with heterogeneous coefficients. The local approximation spaces are constructed using eigenvectors of local eigenvalue problems solved by the finite element method on some sufficiently fine mesh with mesh size $h$. The error bound of the discrete GFEM approximation is proved to converge as $h\rightarrow 0$ towards that of the continuous GFEM approximation, which was shown to decay nearly exponentially in previous works. Moreover, even for fixed mesh size $h$, a nearly exponential rate of convergence of the local approximation errors with respect to the dimension of the local spaces is established. An efficient and accurate method for solving the discrete eigenvalue problems is proposed by incorporating the discrete $A$-harmonic constraint directly into the eigensolver. Numerical experiments are carried out to confirm the theoretical results and to demonstrate the effectiveness of the method.

math.NA↗

arXiv2021

High Performance Uncertainty Quantification with Parallelized Multilevel Markov Chain Monte Carlo

Numerical models of complex real-world phenomena often necessitate High Performance Computing (HPC). Uncertainties increase problem dimensionality further and pose even greater challenges. We present a parallelization strategy for multilevel Markov chain Monte Carlo, a state-of-the-art, algorithmically scalable Uncertainty Quantification (UQ) algorithm for Bayesian inverse problems, and a new software framework allowing for large-scale parallelism across forward model evaluations and the UQ algorithms themselves. The main scalability challenge presents itself in the form of strong data dependencies introduced by the MLMCMC method, prohibiting trivial parallelization. Our software is released as part of the modular and open-source MIT UQ Library (MUQ), and can easily be coupled with arbitrary user codes. We demonstrate it using the DUNE and the ExaHyPE Engine. The latter provides a realistic, large-scale tsunami model in which identify the source of a tsunami from buoy-elevation data.

cs.MS↗

arXiv2021

GenEO coarse spaces for heterogeneous indefinite elliptic problems

Motivated by recent work on coarse spaces for Helmholtz problems, we provide in this paper a comparative study on the use of spectral coarse spaces of GenEO type for heterogeneous indefinite elliptic problems within an additive overlapping Schwarz method. In particular, we focus here on two different but related formulations of local generalised eigenvalue problems and compare their performance numerically. Even though their behaviour seems to be very similar for several well-known heterogeneous test cases that are mildly indefinite, only one of the coarse spaces has so far been analysed theoretically, while the other one leads to a significantly more robust domain decomposition method when the indefiniteness is increased. We present a summary of upcoming results developing such a theory and describe how the numerical experiments illustrate it.

math.NA↗

arXiv2021

Critical point for de-mixing of binary hard spheres

We use a two-level simulation method to analyse the critical point associated with demixing of binary hard sphere mixtures. The method exploits an accurate coarse-grained model with two-body and three-body effective interactions. Using this model within the two-level methodology allows computation of properties of the full (fine-grained) mixture. The critical point is located by computing the probability distribution for the number of large particles in the grand canonical ensemble, and matching to the universal form for the $3d$ Ising universality class. The results have a strong and unexpected dependence on the size ratio between large and small particles, which is related to three-body effective interactions, and the geometry of the underlying hard sphere packings.

cond-mat.stat-mech↗