arXiv subjects

Markus Bachmayr

Publications and source records attributed to Markus Bachmayr.

At least 37 records · Page 2Linked to original sources

arXiv2022

Particle Number Conservation and Block Structures in Matrix Product States

The eigenvectors of the particle number operator in second quantization are characterized by the block sparsity of their matrix product state representations. This is shown to generalize to other classes of operators. Imposing block sparsity yields a scheme for conserving the particle number that is commonly used in applications in physics. Operations on such block structures, their rank truncation, and implications for numerical algorithms are discussed. Explicit and rank-reduced matrix product operator representations of one- and two-particle operators are constructed that operate only on the non-zero blocks of matrix product states.

math.NA↗

arXiv2021

Approximation by tree tensor networks in high dimensions: Sobolev and compositional functions

This paper is concerned with convergence estimates for fully discrete tree tensor network approximations of high-dimensional functions from several model classes. For functions having standard or mixed Sobolev regularity, new estimates generalizing and refining known results are obtained, based on notions of linear widths of multivariate functions. In the main results of this paper, such techniques are applied to classes of functions with compositional structure, which are known to be particularly suitable for approximation by deep neural networks. As shown here, such functions can also be approximated by tree tensor networks without a curse of dimensionality -- however, subject to certain conditions, in particular on the depth of the underlying tree. In addition, a constructive encoding of compositional functions in tree tensor networks is given.

math.NA↗

arXiv2021

Atomic Cluster Expansion: Completeness, Efficiency and Stability

The Atomic Cluster Expansion (Drautz, Phys. Rev. B 99, 2019) provides a framework to systematically derive polynomial basis functions for approximating isometry and permutation invariant functions, particularly with an eye to modelling properties of atomistic systems. Our presentation extends the derivation by proposing a precomputation algorithm that yields immediate guarantees that a complete basis is obtained. We provide a fast recursive algorithm for efficient evaluation and illustrate its performance in numerical tests. Finally, we discuss generalisations and open challenges, particularly from a numerical stability perspective, around basis optimisation and parameter estimation, paving the way towards a comprehensive analysis of the convergence to a high-fidelity reference model.

math.NA↗

arXiv2020

Existence of dynamical low-rank approximations to parabolic problems

The existence and uniqueness of weak solutions to dynamical low-rank evolution problems for parabolic partial differential equations in two spatial dimensions is shown, covering also non-diagonal diffusion in the elliptic part. The proof is based on a variational time-stepping scheme on the low-rank manifold. Moreover, this scheme is shown to be closely related to practical methods for computing such low-rank evolutions.

math.NA↗

arXiv2020

Unified Analysis of Periodization-Based Sampling Methods for Matérn Covariances

The periodization of a stationary Gaussian random field on a sufficiently large torus comprising the spatial domain of interest is the basis of various efficient computational methods, such as the classical circulant embedding technique using the fast Fourier transform for generating samples on uniform grids. For the family of Matérn covariances with smoothness index $ν$ and correlation length $λ$, we analyse the nonsmooth periodization (corresponding to classical circulant embedding) and an alternative procedure using a smooth truncation of the covariance function. We solve two open problems: the first concerning the $ν$-dependent asymptotic decay of eigenvalues of the resulting circulant in the nonsmooth case, the second concerning the required size in terms of $ν$, $λ$ of the torus when using a smooth periodization. In doing this we arrive at a complete characterisation of the performance of these two approaches. Both our theoretical estimates and the numerical tests provided here show substantial advantages of smooth truncation.

math.NA↗

arXiv2019

Stability of Low-Rank Tensor Representations and Structured Multilevel Preconditioning for Elliptic PDEs

Folding grid value vectors of size $2^L$ into $L$th order tensors of mode sizes $2\times \cdots\times 2$, combined with low-rank representation in the tensor train format, has been shown to lead to highly efficient approximations for various classes of functions. These include solutions of elliptic PDEs on nonsmooth domains or with oscillatory data. This tensor-structured approach is attractive because it leads to highly compressed, adaptive approximations based on simple discretizations. Standard choices of the underlying basis, such as piecewise multilinear finite elements on uniform tensor product grids, entail the well-known matrix ill-conditioning of discrete operators. We demonstrate that for low-rank representations, the use of tensor structure itself additionally introduces representation ill-conditioning, a new effect specific to computations in tensor networks. We analyze the tensor structure of a BPX preconditioner for second-order linear elliptic operators and construct an explicit tensor-structured representation of the preconditioner, with ranks independent of the number $L$ of discretization levels. The straightforward application of the preconditioner yields discrete operators whose matrix conditioning is uniform with respect to the discretization parameter, but in decompositions that suffer from representation ill-conditioning. By additionally eliminating certain redundancies in the representations of the preconditioned discrete operators, we obtain reduced-rank decompositions that are free of both matrix and representation ill-conditioning. For an iterative solver based on soft thresholding of low-rank tensors, we obtain convergence and complexity estimates and demonstrate its reliability and efficiency for discretizations with up to $2^{50}$ nodes in each dimension.

math.NA↗

arXiv2019

Adaptive Low-Rank Approximations for Operator Equations: Accuracy Control and Computational Complexity

The challenge of mastering computational tasks of enormous size tends to frequently override questioning the quality of the numerical outcome in terms of accuracy. By this we do not mean the accuracy within the discrete setting, which itself may also be far from evident for ill-conditioned problems or when iterative solvers are involved. By accuracy-controlled computation we mean the deviation of the numerical approximation from the exact solution of an underlying continuous problem in a relevant metric, which has been the initiating interest in the first place. Can the accuracy of a numerical result be rigorously certified - a question that is particularly important in the context of uncertainty quantification, when many possible sources of uncertainties interact. This is the guiding question throughout this article, which reviews recent developments of low-rank approximation methods for problems in high spatial dimensions. In particular, we highlight the role of adaptivity when dealing with such strongly nonlinear methods that integrate in a natural way issues of discrete and continuous accuracy.

math.NA↗

arXiv2018

Identifiability of Diffusion Coefficients for Source Terms of Non-Uniform Sign

The problem of recovering a diffusion coefficient $a$ in a second-order elliptic partial differential equation from a corresponding solution $u$ for a given right-hand side $f$ is considered, with particular focus on the case where $f$ is allowed to take both positive and negative values. Identifiability of $a$ from $u$ is shown under mild smoothness requirements on $a$, $f$, and on the spatial domain $D$, assuming that either the gradient of $u$ is nonzero almost everywhere, or that $f$ as a distribution does not vanish on any open subset of $D$. Further results of this type under essentially minimal regularity conditions are obtained for the case of $D$ being an interval, including detailed information on the continuity properties of the mapping from $u$ to $a$.

math.AP↗

arXiv2018

Sequential sampling for optimal weighted least squares approximations in hierarchical spaces

We consider the problem of approximating an unknown function $u\in L^2(D,ρ)$ from its evaluations at given sampling points $x^1,\dots,x^n\in D$, where $D\subset \mathbb{R}^d$ is a general domain and $ρ$ is a probability measure. The approximation is picked in a linear space $V_m$ where $m=\dim(V_m)$ and computed by a weighted least squares method. Recent results show the advantages of picking the sampling points at random according to a well-chosen probability measure $μ$ that depends both on $V_m$ and $ρ$. With such a random design, the weighted least squares approximation is proved to be stable with high probability, and having precision comparable to that of the exact $L^2(D,ρ)$-orthonormal projection onto $V_m$, in a near-linear sampling regime $n\sim{m\log m}$. The present paper is motivated by the adaptive approximation context, in which one typically generates a nested sequence of spaces $(V_m)_{m\geq1}$ with increasing dimension. Although the measure $μ=μ_m$ changes with $V_m$, it is possible to recycle the previously generated samples by interpreting $μ_m$ as a mixture between $μ_{m-1}$ and an update measure $σ_m$. Based on this observation, we discuss sequential sampling algorithms that maintain the stability and approximation properties uniformly over all spaces $V_m$. Our main result is that the total number of computed sample at step $m$ remains of the order $m\log{m}$ with high probability. Numerical experiments confirm this analysis.

math.NA↗

arXiv2017

Fully discrete approximation of parametric and stochastic elliptic PDEs

It has recently been demonstrated that locality of spatial supports in the parametrization of coefficients in elliptic PDEs can lead to improved convergence rates of sparse polynomial expansions of the corresponding parameter-dependent solutions. These results by themselves do not yield practically realizable approximations, since they do not cover the approximation of the arising expansion coefficients, which are functions of the spatial variable. In this work, we study the combined spatial and parametric approximability for elliptic PDEs with affine or lognormal parametrizations of the diffusion coefficients and corresponding Taylor, Jacobi, and Hermite expansions, to obtain fully discrete approximations. Our analysis yields convergence rates of the fully discrete approximation in terms of the total number of degrees of freedom. The main vehicle consists in $\ell^p$ summability results for the coefficient sequences measured in higher-order Hilbertian Sobolev norms. We also discuss similar results for non-Hilbertian Sobolev norms which arise naturally when using adaptive spatial discretizations.

math.NA↗

arXiv2017

Parametric PDEs: Sparse or Low-Rank Approximations?

We consider adaptive approximations of the parameter-to-solution map for elliptic operator equations depending on a large or infinite number of parameters, comparing approximation strategies of different degrees of nonlinearity: sparse polynomial expansions, general low-rank approximations separating spatial and parametric variables, and hierarchical tensor decompositions separating all variables. We describe corresponding adaptive algorithms based on a common generic template and show their near-optimality with respect to natural approximability assumptions for each type of approximation. A central ingredient in the resulting bounds for the total computational complexity are new operator compression results for the case of infinitely many parameters. We conclude with a comparison of the complexity estimates based on the actual approximability properties of classes of parametric model problems, which shows that the computational costs of optimized low-rank expansions can be significantly lower or higher than those of sparse polynomial expansions, depending on the particular type of parametric problem.

math.NA↗

arXiv2016

Sparse polynomial approximation of parametric elliptic PDEs. Part I: affine coefficients

We consider elliptic partial differential equations with diffusion coefficients that depend affinely on countably many parameters. We study the summability properties of polynomial expansions of the function mapping parameter values to solutions of the PDE, considering both Taylor and Legendre series. Our results considerably improve on previously known estimates of this type, in particular taking into account structural features of the affine parametrization of the coefficient. Moreover, the results carry over to more general Jacobi polynomial expansions. We demonstrate that the new bounds are sharp in certain model cases and we illustrate them by numerical experiments.

math.NA↗

arXiv2016

Representations of Gaussian random fields and approximation of elliptic PDEs with lognormal coefficients

Approximation of elliptic PDEs with random diffusion coefficients typically requires a representation of the diffusion field in terms of a sequence $y=(y_j)_{j\geq 1}$ of scalar random variables. One may then apply high-dimensional approximation methods to the solution map $y\mapsto u(y)$. Although Karhunen-Loève representations are commonly used, it was recently shown, in the relevant case of lognormal diffusion fields, that they do not generally yield optimal approximation rates. Motivated by these results, we construct wavelet-type representations of stationary Gaussian random fields defined on bounded domains. The size and localization properties of these wavelets are studied, and used to obtain polynomial approximation results for the related elliptic PDE which outperform those achievable when using Karhunen-Loève representations. Our construction is based on a periodic extension of the random field, and the expansion on the domain is then obtained by simple restriction. This makes the approach easily applicable even when the computational domain of the PDE has a complicated geometry. In particular, we apply this construction to the class of Gaussian processes defined by the family of Matérn covariances.

math.NA↗

arXiv2015

Sparse polynomial approximation of parametric elliptic PDEs. Part II: lognormal coefficients

Elliptic partial differential equations with diffusion coefficients of lognormal form, that is $a=exp(b)$, where $b$ is a Gaussian random field, are considered. We study the $\ell^p$ summability properties of the Hermite polynomial expansion of the solution in terms of the countably many scalar parameters appearing in a given representation of $b$. These summability results have direct consequences on the approximation rates of best $n$-term truncated Hermite expansions. Our results significantly improve on the state of the art estimates available for this problem. In particular, they take into account the support properties of the basis functions involved in the representation of $b$, in addition to the size of these functions. One interesting conclusion from our analysis is that in certain relevant cases, the Karhunen-Loève representation of $b$ may not be the best choice concerning the resulting sparsity and approximability of the Hermite expansion.

math.NA↗

arXiv2015

Kolmogorov widths and low-rank approximations of parametric elliptic PDEs

Kolmogorov $n$-widths and low-rank approximations are studied for families of elliptic diffusion PDEs parametrized by the diffusion coefficients. The decay of the $n$-widths can be controlled by that of the error achieved by best $n$-term approximations using polynomials in the parametric variable. However, we prove that in certain relevant instances where the diffusion coefficients are piecewise constant over a partition of the physical domain, the $n$-widths exhibit significantly faster decay. This, in turn, yields a theoretical justification of the fast convergence of reduced basis or POD methods when treating such parametric PDEs. Our results are confirmed by numerical experiments, which also reveal the influence of the partition geometry on the decay of the $n$-widths.

math.NA↗

arXiv2015

Iterative Methods Based on Soft Thresholding of Hierarchical Tensors

We construct a soft thresholding operation for rank reduction of hierarchical tensors and subsequently consider its use in iterative thresholding methods, in particular for the solution of discretized high-dimensional elliptic problems. The proposed method for the latter case automatically adjusts the thresholding parameters, based only on bounds on the spectrum of the operator, such that the arising tensor ranks of the resulting iterates remain quasi-optimal with respect to the algebraic or exponential-type decay of the hierarchical singular values of the true solution. In addition, we give a modified algorithm using inexactly evaluated residuals that retains these features. The practical efficiency of the scheme is demonstrated in numerical experiments.

math.NA↗

arXiv2014

Adaptive Low-Rank Methods for Problems on Sobolev Spaces with Error Control in $L_2$

Low-rank tensor methods for the approximate solution of second-order elliptic partial differential equations in high dimensions have recently attracted significant attention. A critical issue is to rigorously bound the error of such approximations, not with respect to a fixed finite dimensional discrete background problem, but with respect to the exact solution of the continuous problem. While the energy norm offers a natural error measure corresponding to the underlying operator considered as an isomorphism from the energy space onto its dual, this norm requires a careful treatment in its interplay with the tensor structure of the problem. In this paper we build on our previous work on energy norm-convergent subspace-based tensor schemes contriving, however, a modified formulation which now enforces convergence only in $L_2$. In order to still be able to exploit the mapping properties of elliptic operators, a crucial ingredient of our approach is the development and analysis of a suitable asymmetric preconditioning scheme. We provide estimates for the computational complexity of the resulting method in terms of the solution error and study the practical performance of the scheme in numerical experiments. In both regards, we find that controlling solution errors in this weaker norm leads to substantial simplifications and to a reduction of the actual numerical work required for a certain error tolerance.

math.NA↗

arXiv2014

Adaptive Low-Rank Methods: Problems on Sobolev Spaces

This paper is concerned with the development and analysis of an iterative solver for high-dimensional second-order elliptic problems based on subspace-based low-rank tensor formats. Both the subspaces giving rise to low-rank approximations and corresponding sparse approximations of lower-dimensional tensor components are determined adaptively. A principal obstruction to a simultaneous control of rank growth and accuracy turns out to be the fact that the underlying elliptic operator is an isomorphism only between spaces that are not endowed with cross norms. Therefore, as central part of this scheme, we devise a method for preconditioning low-rank tensor representations of operators. Under standard assumptions on the data, we establish convergence to the solution of the continuous problem with a guaranteed error reduction. Moreover, for the case that the solution exhibits a certain low-rank structure and representation sparsity, we derive bounds on the computational complexity, including in particular bounds on the tensor ranks that can arise during the iteration. We emphasize that such assumptions on the solution do not enter in the formulation of the scheme, which in fact is shown to detect them automatically. Our findings are illustrated by numerical experiments that demonstrate the practical efficiency of the method in high spatial dimensions.

math.NA↗