arXiv subjects

Bettina Grün

Publications and source records attributed to Bettina Grün.

14 recordsLinked to original sources

arXiv2026

CliPS -- How to identify cluster distributions in Bayesian mixture models

We propose the CliPS procedure when fitting Bayesian mixture models in the context of model-based clustering to identify the cluster distributions while simultaneously assessing the suitability of a cluster solution and validating the cluster structure. The procedure relies on the point process representation of a mixture model and is based on the assumption that a suitable cluster solution requires the clusters to be distinguishable with respect to a low-dimensional functional of the component-specific parameters of the mixture. CliPS maps the component-specific MCMC draws to the point process representation and identifies clusters there, exploiting that, while data distributions usually overlap, the posterior of these functionals are more and more separated for increasing sample size. We outline the procedure and illustrate its use on several model-based clustering examples.

stat.ME↗

arXiv2025

Seeded Poisson Factorization: leveraging domain knowledge to fit topic models

Topic models are widely used for discovering latent thematic structures in large text corpora, yet traditional unsupervised methods often struggle to align with pre-defined conceptual domains. This paper introduces seeded Poisson Factorization (SPF), a novel approach that extends the Poisson Factorization (PF) framework by incorporating domain knowledge through seed words. SPF enables a structured topic discovery by modifying the prior distribution of topic-specific term intensities, assigning higher initial rates to pre-defined seed words. The model is estimated using variational inference with stochastic gradient optimization, ensuring scalability to large datasets. We present in detail the results of applying SPF to an Amazon customer feedback dataset, leveraging pre-defined product categories as guiding structures. SPF achieves superior performance compared to alternative guided probabilistic topic models in terms of computational efficiency and classification performance. Robustness checks highlight SPF's ability to adaptively balance domain knowledge and data-driven topic discovery, even in case of imperfect seed word selection. Further applications of SPF to four additional benchmark datasets, where the corpus varies in size and the number of topics differs, demonstrate its general superior classification performance compared to the unseeded PF model.

stat.ME↗

arXiv2025

Evolving Voices Based on Temporal Poisson Factorisation

The world is evolving and so is the vocabulary used to discuss topics in speech. Analysing political speech data from more than 30 years requires the use of flexible topic models to uncover the latent topics and their change in prevalence over time as well as the change in the vocabulary of the topics. We propose the temporal Poisson factorisation (TPF) model as an extension to the Poisson factorisation model to model sparse count data matrices obtained based on the bag-of-words assumption from text documents with time stamps. We discuss and empirically compare different model specifications for the time-varying latent variables consisting either of a flexible auto-regressive structure of order one or a random walk. Estimation is based on variational inference where we consider a combination of coordinate ascent updates with automatic differentiation using batching of documents. Suitable variational families are proposed to ease inference. We compare results obtained using independent univariate variational distributions for the time-varying latent variables to those obtained with a multivariate variant. We discuss in detail the results of the TPF model when analysing speeches from 18 sessions in the U.S. Senate (1981-2016).

stat.ME↗

arXiv2025

Revisiting Group Differences in High-Dimensional Choices: Method and Application to Congressional Speech

Gentzkow, Shapiro and Taddy, Econometrica Vol 87, No 4, 2019 (henceforth GST) use a supervised text-based regression model to assess changes in partisanship in U.S. congressional speech over time. Their estimates imply that partisanship is far greater in recent years than in the past, and that it increased sharply in the early 1990s. The paper at hand provides a replication in the wide sense of GST by complementing their analysis in three ways. First, we propose an alternative unsupervised language model, which combines ideas of topic models and ideal point models, to analyze the change in partisanship over time. We apply this model to the Senate speech data used in GST ranging from 1981-2017. Using our model we replicate their results on the specific evolution of partisanship. Second, our model provides additional insights such as the data-driven estimation of evolvement of topical contents over time. Third, we identify key phrases of partisanship on topic level.

econ.GN↗

arXiv2024

A Structural Text-Based Scaling Model for Analyzing Political Discourse

Scaling political actors based on their individual characteristics and behavior helps profiling and grouping them as well as understanding changes in the political landscape. In this paper we introduce the Structural Text-Based Scaling (STBS) model to infer ideological positions of speakers for latent topics from text data. We expand the usual Poisson factorization specification for topic modeling of text data and use flexible shrinkage priors to induce sparsity and enhance interpretability. We also incorporate speaker-specific covariates to assess their association with ideological positions. Applying STBS to U.S. Senate speeches from Congress session 114, we identify immigration and gun violence as the most polarizing topics between the two major parties in Congress. Additionally, we find that, in discussions about abortion, the gender of the speaker significantly influences their position, with female speakers focusing more on women's health. We also see that a speaker's region of origin influences their ideological position more than their religious affiliation.

stat.ME↗

arXiv2024

Without Pain -- Clustering Categorical Data Using a Bayesian Mixture of Finite Mixtures of Latent Class Analysis Models

We propose a Bayesian approach for model-based clustering of multivariate categorical data where variables are allowed to be associated within clusters and the number of clusters is unknown. The approach uses a two-layer mixture of finite mixtures model where the cluster distributions are approximated using latent class analysis models. A careful specification of priors with suitable hyperparameter values is crucial to identify the two-layer structure and obtain a parsimonious cluster solution. We outline the Bayesian estimation based on Markov chain Monte Carlo sampling with the telescoping sampler and describe how to obtain an identified clustering model by resolving the label switching issue. Empirical demonstrations in a simulation study using artificial data as well as a data set on low back pain indicate the good clustering performance of the proposed approach, provided hyperparameters are selected which induce sufficient shrinkage.

stat.ME↗

arXiv2024

Bayesian Finite Mixture Models

Finite mixture models are a useful statistical model class for clustering and density approximation. In the Bayesian framework finite mixture models require the specification of suitable priors in addition to the data model. These priors allow to avoid spurious results and provide a principled way to define cluster shapes and a preference for specific cluster solutions. A generic model estimation scheme for finite mixtures with a fixed number of components is available using Markov chain Monte Carlo (MCMC) sampling with data augmentation. The posterior allows to assess uncertainty in a comprehensive way, but component-specific posterior inference requires resolving the label switching issue. In this paper we focus on the application of Bayesian finite mixture models for clustering. We start with discussing suitable specification, estimation and inference of the model if the number of components is assumed to be known. We then continue to explain suitable strategies for fitting Bayesian finite mixture models when the number of components is not known. In addition, all steps required to perform Bayesian finite mixture modeling are illustrated on a data example where a finite mixture model of multivariate Gaussian distributions is fitted. Suitable prior specification, estimation using MCMC and posterior inference are discussed for this example assuming the number of components to be known as well as unknown.

stat.ME↗

arXiv2022

Mixture of Experts Distributional Regression: Implementation Using Robust Estimation with Adaptive First-order Methods

In this work, we propose an efficient implementation of mixtures of experts distributional regression models which exploits robust estimation by using stochastic first-order optimization techniques with adaptive learning rate schedulers. We take advantage of the flexibility and scalability of neural network software and implement the proposed framework in mixdistreg, an R software package that allows for the definition of mixtures of many different families, estimation in high-dimensional and large sample size settings and robust optimization based on TensorFlow. Numerical experiments with simulated and real-world data applications show that optimization is as reliable as estimation via classical approaches in many different settings and that results may be obtained for complicated scenarios where classical approaches consistently fail.

stat.CO↗

arXiv2022

Spying on the prior of the number of data clusters and the partition distribution in Bayesian cluster analysis

Cluster analysis aims at partitioning data into groups or clusters. In applications, it is common to deal with problems where the number of clusters is unknown. Bayesian mixture models employed in such applications usually specify a flexible prior that takes into account the uncertainty with respect to the number of clusters. However, a major empirical challenge involving the use of these models is in the characterisation of the induced prior on the partitions. This work introduces an approach to compute descriptive statistics of the prior on the partitions for three selected Bayesian mixture models developed in the areas of Bayesian finite mixtures and Bayesian nonparametrics. The proposed methodology involves computationally efficient enumeration of the prior on the number of clusters in-sample (termed as ``data clusters'') and determining the first two prior moments of symmetric additive statistics characterising the partitions. The accompanying reference implementation is made available in the R package 'fipp'. Finally, we illustrate the proposed methodology through comparisons and also discuss the implications for prior elicitation in applications.

stat.ME↗

arXiv2021

Generalized mixtures of finite mixtures and telescoping sampling

Within a Bayesian framework, a comprehensive investigation of mixtures of finite mixtures (MFMs), i.e., finite mixtures with a prior on the number of components, is performed. This model class has applications in model-based clustering as well as for semi-parametric density estimation and requires suitable prior specifications and inference methods to exploit its full potential. We contribute by considering a generalized class of MFMs where the hyperparameter $γ_K$ of a symmetric Dirichlet prior on the weight distribution depends on the number of components. We show that this model class may be regarded as a Bayesian non-parametric mixture outside the class of Gibbs-type priors. We emphasize the distinction between the number of components $K$ of a mixture and the number of clusters $K_+$, i.e., the number of filled components given the data. In the MFM model, $K_+$ is a random variable and its prior depends on the prior on $K$ and on the hyperparameter $γ_K$. We employ a flexible prior distribution for the number of components $K$ and derive the corresponding prior on the number of clusters $K_+$ for generalized MFMs. For posterior inference, we propose the novel telescoping sampler which allows Bayesian inference for mixtures with arbitrary component distributions without resorting to reversible jump Markov chain Monte Carlo (MCMC) methods. The telescoping sampler explicitly samples the number of components, but otherwise requires only the usual MCMC steps of a finite mixture model. The ease of its application using different component distributions is demonstrated on several data sets.

stat.ME↗

arXiv2021

How many data clusters are in the Galaxy data set? Bayesian cluster analysis in action

In model-based clustering, the Galaxy data set is often used as a benchmark data set to study the performance of different modeling approaches. Aitkin (2001) compares maximum likelihood and Bayesian analyses of the Galaxy data set and expresses reservations about the Bayesian approach due to the fact that the prior assumptions imposed remain rather obscure while playing a major role in the results obtained and conclusions drawn. The aim of the paper is to address Aitkin's concerns about the Bayesian approach by shedding light on how the specified priors influence the number of estimated clusters. We perform a sensitivity analysis of different prior specifications for the mixtures of finite mixture model, i.e., the mixture model where a prior on the number of components is included. We use an extensive set of different prior specifications in a full factorial design and assess their impact on the estimated number of clusters for the Galaxy data set. Results highlight the interaction effects of the prior specifications and provide insights into which prior specifications are recommended to obtain a sparse clustering solution. A simulation study with artificial data provides further empirical evidence to support the recommendations. A clear understanding of the impact of the prior specifications removes restraints preventing the use of Bayesian methods due to the complexity of selecting suitable priors. Also, the regularizing properties of the priors may be intentionally exploited to obtain a suitable clustering solution meeting prior expectations and needs of the application.

stat.AP↗

arXiv2018

Model-based Clustering

Mixture models extend the toolbox of clustering methods available to the data analyst. They allow for an explicit definition of the cluster shapes and structure within a probabilistic framework and exploit estimation and inference techniques available for statistical models in general. In this chapter an introduction to cluster analysis is provided, model-based clustering is related to standard heuristic clustering methods and an overview on different ways to specify the cluster model is given. Post-processing methods to determine a suitable clustering, infer cluster distribution characteristics and validate the cluster solution are discussed. The versatility of the model-based clustering approach is illustrated by giving an overview on the different areas of applications.

stat.ME↗

arXiv2016

Model-based clustering based on sparse finite Gaussian mixtures

In the framework of Bayesian model-based clustering based on a finite mixture of Gaussian distributions, we present a joint approach to estimate the number of mixture components and identify cluster-relevant variables simultaneously as well as to obtain an identified model. Our approach consists in specifying sparse hierarchical priors on the mixture weights and component means. In a deliberately overfitting mixture model the sparse prior on the weights empties superfluous components during MCMC. A straightforward estimator for the true number of components is given by the most frequent number of non-empty components visited during MCMC sampling. Specifying a shrinkage prior, namely the normal gamma prior, on the component means leads to improved parameter estimates as well as identification of cluster-relevant variables. After estimating the mixture model using MCMC methods based on data augmentation and Gibbs sampling, an identified model is obtained by relabeling the MCMC output in the point process representation of the draws. This is performed using $K$-centroids cluster analysis based on the Mahalanobis distance. We evaluate our proposed strategy in a simulation setup with artificial data and by applying it to benchmark data sets.

stat.ME↗

arXiv2016

Identifying Mixtures of Mixtures Using Bayesian Estimation

The use of a finite mixture of normal distributions in model-based clustering allows to capture non-Gaussian data clusters. However, identifying the clusters from the normal components is challenging and in general either achieved by imposing constraints on the model or by using post-processing procedures. Within the Bayesian framework we propose a different approach based on sparse finite mixtures to achieve identifiability. We specify a hierarchical prior where the hyperparameters are carefully selected such that they are reflective of the cluster structure aimed at. In addition this prior allows to estimate the model using standard MCMC sampling methods. In combination with a post-processing approach which resolves the label switching issue and results in an identified model, our approach allows to simultaneously (1) determine the number of clusters, (2) flexibly approximate the cluster distributions in a semi-parametric way using finite mixtures of normals and (3) identify cluster-specific parameters and classify observations. The proposed approach is illustrated in two simulation studies and on benchmark data sets.

stat.ME↗