Scale estimation and rate-unbiasedness for Gaussian processes under smoothness misspecification

Abstract

Gaussian process regression is used throughout statistics and machine learning for prediction and uncertainty quantification. A Gaussian process is specified by its mean and covariance functions. Many covariance functions, including Mat\'erns, have a smoothness parameter that is notoriously difficult to specify correctly or estimate from the data. In practice, the smoothness parameter is often selected more or less arbitrarily. We introduce rate-unbiasedness, a relaxed notion of asymptotic optimality which requires that the expected ratio of the mean-square error presumed by a potentially misspecified model and the true, but unknown, mean-square error remain bounded away from zero and infinity as more data are obtained. A rate-unbiased model provides uncertainty quantification that is of correct order of magnitude. We then prove that scale estimation suffices for rate-unbiasedness in a variety of common settings. As estimation of the scale of a Gaussian process is routine and requires no optimisation, rate-unbiasedness can be achieved in many applications.