AI-based scoring systematically underestimates conceptual understanding of linguistically weak students' explanations in physics
Students' explanations of scientific phenomena provide important evidence of their conceptual understanding. However, because conceptual understanding can only be inferred through language rather than observed directly, distinguishing conceptual understanding from linguistic quality represents a fundamental challenge for assessment. This challenge can also be expected to arise when artificial intelligence (AI) is used to score students' text-based explanations. We investigated whether AI-based scoring approaches assess students' conceptual understanding independently of the linguistic quality of their explanations about a phenomenon taken from a physics context. Conceptual understanding scores assigned by nine machine learning-based scoring approaches and two large language model-based scoring approaches were compared with expert-assigned scores for 116 explanations produced by secondary-school students in Germany. Multinomial logistic regression analyses examined whether expert-rated linguistic quality was associated with under- or overestimation of conceptual understanding. Across all eleven AI-based scoring approaches, lower linguistic quality was associated with a greater likelihood of underestimation, although the strength and statistical significance of this association varied between approaches. AI-based scoring approaches may thus systematically underestimate the conceptual understanding expressed in linguistically weaker explanations. This pattern resembles language bias previously documented in STEM teachers' assessment practices. The findings highlight the importance of examining not only agreement with expert ratings but also whether AI-based scoring approaches inadvertently rely on construct-irrelevant information. Addressing language bias is therefore essential for the validity and fairness of AI-supported assessment in physics education and STEM education more broadly.