Bayesian Multinomial Logistic Regression for Numerous Categories

Abstract

Bayesian multinomial logistic regression provides a principled, interpretable approach to multiclass classification, but posterior sampling becomes increasingly expensive as the model dimension grows. Prior work has studied scalability in the number of subjects and covariates; in contrast, this paper focuses on how computation changes as the number of outcome categories increases. To improve scalability in settings with numerous categories, we adapt a gamma-augmentation strategy to decouple category-specific coefficient updates, so that each category's coefficients can be updated conditional on a single auxiliary variable per subject, rather than on the full set of other categories' coefficients. Because the resulting coefficient conditionals are non-conjugate, we couple this augmentation with either adaptive Metropolis-Hastings or elliptical slice sampling. Through simulation and a real-data example, we compare effective sample size and effective sampling rate across several standard competitors. We find that the best-performing sampler depends on the dimension and imbalance regime, and that the proposed augmentation provides substantial speedups in scenarios with numerous categories.