Consistency of the posterior distribution in generalised linear inverse problems

Abstract

For ill-posed inverse problems, a regularised solution can be interpreted as a mode of the posterior distribution in a Bayesian framework. This framework enriches the set the solutions, as other posterior estimates can be used as a solution to the inverse problem, such as the posterior mean that can be easier to compute in practice. In this paper we prove consistency of Bayesian solutions of an ill-posed linear inverse problem in the Ky Fan metric for a general class of likelihoods and prior distributions in a finite dimensional setting. This result can be applied to study infinite dimensional problems by letting the dimension of the unknown parameter grow to infinity which can be viewed as discretisation on a grid or spectral approximation of an infinite dimensional problem. Likelihood and the prior distribution are assumed to be in an exponential form that includes distributions from the exponential family, and to be differentiable. The observations can be dependent. No assumption of finite moments of observations, such as expected value or the variance, is necessary thus allowing for possibly non-regular likelihoods, and allowing for non-conjugate and improper priors. If the variance exists, it may be heteroscedastic, namely, it may depend on the unknown function. We observe quite a surprising phenomenon when applying our result to the spectral approximation framework where it is possible to achieve the parametric rate of convergence, i.e the problem becomes self-regularised. We also consider a particular case of the unknown parameter being on the boundary of the parameter set, and show that the rate of convergence in this case is faster than for an interior point parameter.