Grouped Mixture of Regressions

Abstract

Finite Mixture of Regressions (FMR) models are among the most widely used approaches in dealing with the heterogeneity among the observations in regression problems. One of the limitations of current approaches is their inability to incorporate group structure in data when available. In some applications, it is desired to cluster groups of observations together rather than the individual ones. In this work, we extend the FMR framework to allow for group structure among observations, and call the resulting model the Grouped Mixture of Regressions (GMR). We derive a fast fitting algorithm based on the Expectation-Maximization (EM) idea. We also show how the group structure can improve prediction by sharing information among members of each group, as reflected in the posterior predictive density under GMR. %that they don't consider clustering the data when there is group structure. In other words, sometimes it is desired to force the algorithm to cluster groups/blocks of observations, instead of individual observations. %In this work, we propose a maximum likelihood approach to cluster groups of observations. We call this algorithm Group Mixture of Regressions (GMR). Expectation Maximization (EM) is employed to maximize the likelihood. Posterior prediction density for predicting new observations is also derived and presented. The performance of the approach is assessed using both synthetic data as well as a real-world example.