Stochastic Variational Inference for Structured Additive Distributional Regression

Abstract

Structured additive distributional regression extends generalized additive models by allowing all parameters of a response distribution to depend on structured additive predictors. Bayesian formulations regularize these models through prior distributions that enforce smoothness or shrinkage, but Markov chain Monte Carlo methods, the standard computational tool, scale poorly with sample size and require substantial runtime. We develop a scalable stochastic variational inference framework for approximate Bayesian inference in structured additive distributional regression. Our key contribution is a variational family for the regression coefficients that is amortized over covariates, responses, and smoothing parameters. This construction allows the variational location and precision to adapt locally to the data, which leads to substantially higher ELBO values in markedly less computational time compared to existing approaches. To balance accuracy and scalability, we consider a block-structured variant that constrains the precision matrix but preserves essential dependence across smooth components. Both approaches are compared with a state-of-the-art dense variational approximation and with the widely used Integrated Nested Laplace Approximation (INLA). We establish theoretical results on posterior propriety and local asymptotic Gaussianity that motivate the proposed Gaussian approximations, assess their performance in simulation studies involving logistic and gamma distributional regression, and demonstrate their scalability in an application to modeling injury counts in motor vehicle crashes in New York City with more than 200,000 observations. Across all settings, incorporating covariates, responses, and smoothing parameters into the construction of the variational distribution yields higher ELBO values in substantially less computational time compared with existing variational approaches.