Missing Data on Physics Exams: Demographic Patterns, Course-Level Predictions, and Implications for Equity

Abstract

In a previous quantitative retrospective study we showed that different demographic groups of students leave different numbers of problems blank on physics exams, leading to inequities in course outcomes. In that work we argued that there were good reasons to treat these blanks as missing data, rather than indicators of a lack of understanding. In this paper, we refine this analysis and show more detailed breakdowns uncollected test item responses by race/ethnicity and first generation college student status, coming to the same conclusion: test item responses are uncollected for students with different ethnic and racial backgrounds at different rates, and these patterns exist even for high-performing students. We also correct an error from our previous work, finding here that there is no significant gender difference in uncollected test item responses. Finally, we provide a more robust analysis of course level data illustrating that blanks are a variable controlled at the course level rather than the student level, providing more evidence for the use of a course deficit model (rather than a student deficit model) when examining equity disparities, and also suggesting that there are plausible means for instructors to minimize uncollected test item responses, and therefore eliminate the bias associated with this missing data. We provide some suggestions for faculty who want to have more equitable course outcomes.