Laplace approximation for fast Bayesian inference in generalized additive models based on penalized regression splines

Abstract

Generalized additive models (GAMs) are a well-established statistical tool for modeling complex nonlinear relationships between covariates and a response assumed to have a conditional distribution in the exponential family. In this article, P-splines and the Laplace approximation are coupled for flexible and fast approximate Bayesian inference in GAMs. The proposed Laplace-P-spline model contributes to the development of a new methodology to explore the posterior penalty space by considering a deterministic grid-based strategy or a Markov chain sampler, depending on the number of smooth additive terms in the predictor. Our approach has the merit of relying on closed form analytical expressions for the gradient and Hessian of the approximate posterior penalty vector, which enables to construct accurate posterior pointwise and credible set estimators for latent field variables at a relatively low computational budget even for a large number of smooth additive components. Based upon simple Gaussian approximations of the conditional latent field posterior, the suggested methodology enjoys excellent statistical properties. The performance of the Laplace-P-spline model is confirmed through different simulation scenarios and the method is illustrated on two real datasets.