What exam scores can and cannot prove about unauthorized AI assistance: Evidence from a highly public classroom episode

Abstract

In spring 2026, an economics professor at Brown University gave a take-home midterm and, after unusually high scores, made the final exam proctored. Among the 59 students who completed the course, average scores fell from 95.7 out of 100 to 48.8. The instructor attributed the drop to unauthorized use of generative AI on the midterm; others proposed test anxiety, a harder final, student withdrawals, and regression to the mean. The publicly released scores, one midterm-final pair per student, show two striking patterns: the correlation between a student's two scores is only 0.06, and individual changes range from a 4-point gain to a 100-point loss. Permutation tests find no statistically detectable association between students' midterm and final scores, yet would detect association of the strength the alternative explanations predict more than 90 percent of the time. Once model complexity is accounted for, a model in which a student's midterm carries no information about that student's final describes the data as well as any model that permits an association. Neither result rules out a weak association, but together they show that the data do not require any. In simulated classes where each student's two scores remain linked through that student's own proficiency, as the alternative explanations imply, the two patterns almost never appear together; they appear together regularly only when that link is nearly severed. More than one mechanism could have severed it: midterm answers that were not the students' own work would have done so, and so would a final testing substantially different material, with no assistance involved. The scores cannot distinguish these possibilities or identify which students, if any, used unauthorized assistance. Our study offers a framework for evaluating statistical evidence in future disputes over generative AI use in academic assessment.