LLAMA LIMA: A Living Meta-Analysis on the Effects of Generative AI on Learning Mathematics

Abstract

The capabilities of generative AI in mathematics education are rapidly evolving, posing significant challenges for research to keep pace. Research syntheses remain scarce and risk being outdated by the time of publication. To address this issue, we present a Living Meta-Analysis (LIMA) on the effects of generative AI-based interventions for learning mathematics. Following PRISMA-LSR guidelines, we continuously update the literature base, apply a Bayesian multilevel meta-regression model to account for nested and cumulative data, and publish updated versions on a preprint server at regular intervals. This paper reports results from the fourth version, including 27 studies, 5 of which were newly included since the third version. Analyses indicate an overall positive effect (g = 0.49, CrI [0.24, 0.75]) and no publication bias. Moderator analyses indicate strong evidence that generative AI is substantially more beneficial when it is implemented together with teacher instruction rather than replacing teachers. For other moderators, evidence is still inconclusive.