The impact of incoming preparation and demographics on performance in Physics I: a multi-institution comparison

Abstract

We have studied the impact of incoming preparation and demographic variables on student performance on the final exam in physics 1, the standard introductory, calculus-based mechanics course This was done at three different institutions using multivariable regression analysis to determine the extent to which exam scores can be predicted by a variety of variables that are available to most faculty and departments. We have found that the results are surprisingly consistent across the institutions, with the only two variables that have predictive power being math SAT/ACT scores and concept inventory pre-scores. The importance of both variables is comparable and fairly similar across the institutions. They explain 20 - 30 percent of the variation in students' performance on the final exam. Most notably, the demographic variables (gender, under-represented minority, first generation to attend college) are not significant. In all cases, although there appear to be gaps in exam performance if one considers only the demographic variable, once these two proxies of incoming preparation are included in the model, there is no longer a demographic gap. There is only a preparation gap that applies equally across the entire student population. This work shows that to properly understand differences in student performance across a diverse population, and hence to design more effective instruction, it is important to do statistical analyses that take multiple variables into account. It also illustrates the importance of having measures that are sensitive to both subject specific and more general preparation. The results suggest that better matching of the course design and teaching to the incoming student preparation will likely be the most effective way to eliminate observed performance gaps across demographic groups while also improving the success of all students.