Equitable Intelligence in Practice: A Fairness Audit of Open Large Language Models for Library Reference Services
As libraries explore large language models (LLMs) as a scalable layer for reference services, a core fairness question follows: can LLM-based services support all patrons fairly, regardless of demographic identity? LLMs could broaden access to information assistance, but they may also reproduce societal biases embedded in their training data and undermine libraries' commitments to impartial service. In this chapter, we apply an evaluation protocol that combines diagnostic classification, which detects systematic differences between responses, with linguistic analysis, which helps interpret them. Across three widely used open models (Llama-3.1 8B, Gemma-2 9B, and Ministral 8B), we find no compelling evidence of systematic differentiation by race/ethnicity, and only minor evidence of sex-linked differentiation in one model and one setting. We discuss implications for equitable AI adoption in libraries and the importance of ongoing monitoring in aligning LLM-based services with social justice goals.