How Robust Is Homogeneity Bias in LLMs? Evidence Across Models, Decoding Settings, and Identity Signals

Abstract

Large language models (LLMs) reproduce homogeneity bias -- the tendency to portray marginalized groups as more internally similar than dominant groups -- but whether this bias generalizes across models, is stable under different inference settings, or depends on how group identity is signaled remains unstudied. We map homogeneity bias across seven open-weight instruction-tuned LLMs (7-20B parameters), a 5x5 temperature x top-p decoding grid, and two paradigms for signaling group identity (explicit labels vs. racially distinctive names). In six of seven models, Hispanic and Asian Americans are portrayed as significantly more homogeneous than White Americans at the default configuration, and the effect remains positive on average at every temperature and top-p tested; African American and gender bias instead vary in direction across models. A conservative cell-level re-analysis confirms Hispanic and Asian homogeneity as robust while weaker African American and gender signals largely do not survive, establishing group-specific robustness. We also apply the same grid to a names-based paradigm in which group identity is signaled via racially distinctive surnames rather than explicit labels. The names paradigm corroborates Hispanic and Asian homogeneity bias, but Black-coded surnames elicit robustly less homogeneous outputs than White-coded names in every model tested -- a reversal absent from the label paradigm -- showing that how group identity is operationalized shapes which biases surface and in which direction.