Bayesian Operator Learning: Posterior Existence and Convergence of Point Estimates for Gaussian Priors

Abstract

We develop a Bayesian framework for learning nonlinear operators between infinite-dimensional spaces. Given a map $g_0:\mathcal{X}\to\mathcal{Y}$ between separable Hilbert spaces, we study the recovery of $g_0$ from $n\in\mathbb{N}$ noisy input-output pairs $(\boldsymbol{X},\boldsymbol{Z})=(X_i,Z_i)_{i=1}^n$ with $Z_i= g_0 (X_i ) + E_i$. Here the $X_i\in\mathcal{X}$ are randomly drawn 'design' points in a compact subset of $\mathcal X$, and the $E_i$ are assumed to be i.i.d. draws from a Gaussian white noise process indexed by $\mathcal{Y}$. For any 'operator-valued' prior $\mathbb{P}_G$ supported on the space of continuous operators, we show existence of the posterior $\mathbb P_{G|(\boldsymbol{X},\boldsymbol{Z})}$ as a regular conditional distribution, and provide a characterization of its Radon-Nikodym derivative. For Gaussian priors, we establish algebraic (in the sample size $n$) convergence rates for the posterior mean towards the ground truth; this corresponds to a ridge regularized kernel estimator. Moreover, we show that the posterior mean is minimax optimal (up to logarithmic factors) over hyperrectangles when the smoothness of the prior matches that of the ground truth. To illustrate the applicability of our analysis, we derive explicit learning rates for the Darcy flow solution operator.