arXiv subjects

Ben Van Dusen

Publications and source records attributed to Ben Van Dusen.

At least 19 recordsLinked to original sources

arXiv2026

Mechanics Cognitive Diagnostic: Testing Fine-Grained Learning Objectives in Introductory Physics

Physics courses use research-based assessments (RBAs) such as the Force Concept Inventory (FCI), Force and Motion Conceptual Evaluation (FMCE), and Energy and Momentum Conceptual Survey (EMCS) to measure learning in introductory mechanics, but their fixed-length, pretest-posttest design makes them retrospective: posttest scores summarize completed instruction and arrive after a course ends. We are developing the Mechanics Cognitive Diagnostic (MCD), a cognitive diagnostic computerized adaptive test that reports students' mastery of fine-grained learning objectives (LOs) throughout instruction. Using evidence-centered design, we defined 14 LOs from introductory mechanics textbooks and AP Physics standards, mapped FCI, FMCE, and EMCS items onto them with a Q-matrix, and refined the mapping with the deterministic inputs, noisy "and" gate (DINA) model, using posttest responses from 24,394 students in 807 courses across 79 institutions through LASSO. The FCI and EMCS achieved good DINA model fit; the FMCE showed marginal fit. Classification accuracy for most LOs met or exceeded benchmarks for low-stakes formative assessment. RBA items, though not developed for LO-level diagnosis, support it reliably, giving the MCD a working 14-LO item bank built from RBAs that physics courses already use. As data accumulate, we can revise or retire weak LOs and items and add new items through online calibration without interrupting testing. We plan to expand the MCD to 35 LOs, two per week, to cover a typical introductory mechanics course.

physics.ed-ph↗

arXiv2026

Challenging Data Aggregation Practices: A MAIHDA Study of Asian Student Outcomes in Introductory Physics

Aggregation of Asian student data can reinforce the model minority myth by obscuring educational disparities among Asian student subgroups. This study investigated variation in conceptual physics knowledge across Asian racial and ethnic subgroups using data from the LASSO platform, analyzing responses from 16,810 students enrolled in 493 introductory calculus-based physics courses across 64 U.S. institutions. We applied Multilevel Analysis of Individual Heterogeneity and Discriminatory Accuracy to examine predicted pre- and posttest performance on the Force Concept Inventory and Force and Motion Conceptual Evaluation. The findings revealed performance differences among 19 Asian subgroups that the pan-Asian strata (the single aggregated Asian group) concealed. Subgroup predicted means spanned 15.8 percentage points on the pretest and 15.4 percentage points on the posttest. The lowest-performing subgroup's posttest mean was roughly equal to the highest-performing subgroup's pretest mean, indicating a performance gap of about a full semester of instruction. Mean absolute error between the pan-Asian strata and the 19-subgroup estimates was 3.9 percentage points at pretest and 4.0 percentage points at posttest, equivalent to approximately 4-5 weeks of learning in a 16-week course. These findings demonstrate that fine-grained identity data collection can support identifying disparities that common aggregation practices conceal.

physics.ed-ph↗

arXiv2025

Applying Cognitive Diagnostic Models to Mechanics Concept Inventories

In physics education research, instructors and researchers often use research-based assessments (RBAs) to assess students' skills and knowledge. In this paper, we support the development of a mechanics cognitive diagnostic to test and implement effective and equitable pedagogies for physics instruction. Adaptive assessments using cognitive diagnostic models provide significant advantages over fixed-length RBAs commonly used in physics education research. As part of a broader project to develop a cognitive diagnostic assessment for introductory mechanics within an evidence-centered design framework, we identified and tested student models of four skills that cross content areas in introductory physics: apply vectors, conceptual relationships, algebra, and visualizations. We developed the student models in three steps. First, we based the model on learning objectives from instructors. Second, we coded the items on RBAs using the student models. Lastly, we then tested and refined this coding using a common cognitive diagnostic model, the deterministic inputs, noisy 'and' gate (DINA) model. The data included 19,889 students who completed either the Force Concept Inventory, Force and Motion Conceptual Evaluation, or Energy and Momentum Conceptual Survey on the LASSO platform. The results indicated a good to adequate fit for the student models with high accuracies for classifying students with many of the skills. The items from these three RBAs do not cover all of the skills in enough detail, however, they will form a useful initial item bank for the development of the mechanics cognitive diagnostic.

physics.ed-ph↗

arXiv2024

Society's educational debts in biology, chemistry, and physics across race, gender, and class

The success of collaborative instruction in helping students achieve higher grades in introductory science, technology, engineering, and mathematics (STEM) courses has led many educators and researchers to assume these methods also address inequities. However, little evidence tests this assumption. Structural inequities in our society have led to the chronic underrepresentation of Black, Hispanic, women, and first-generation students in STEM disciplines. Broadening participation from underrepresented groups in biology, chemistry, and physics would reduce social inequalities while harnessing diversity's economic impact on innovation and workforce expansion. We leveraged data on content knowledge from 18,791 students in 305 introductory courses using collaborative instruction at 45 institutions. We modeled student outcomes across the intersections of gender, race, ethnicity, and first-generation college status within and across science disciplines. Using these models, we examine the educational debts society owes college science students prior to instruction and whether instruction mitigates, perpetuates, or exacerbates those debts. The size of these educational debts and the extent to which courses added to or repaid these debts varied across disciplines. Across all three disciplines, society owed Black and Hispanic women and first-generation Black men the largest educational debts. Collaborative instructional strategies were not sufficient to repay society's educational debts.

physics.ed-ph↗

arXiv2024

Comparing the efficacy of fixed effect and MAIHDA models in predicting outcomes for intersectional social strata

This investigation examines the efficacy of multilevel analysis of individual heterogeneity and discriminatory accuracy (MAIHDA) over fixed effect models when performing intersectional studies. The research questions are: 1) What are typical strata representation rates and outcomes on physics research-based assessments? 2) To what extent do MAIHDA models create more accurate predicted strata outcomes than fixed effects models? 3) To what extent do MAIHDA models allow the modeling of smaller strata sample sizes? We simulated 3,000 datasets based on real-world data from 5,955 students on the LASSO platform. We found that MAIHDA created more accurate and precise predictions than fixed effect models. We also found that using MAIHDA could allow researchers to disaggregate their data further, creating smaller group sample sizes while maintaining more accurate findings than fixed effect models. We recommend using MAIHDA over fixed effect models for intersectional investigations.

physics.ed-ph↗

arXiv2022

How to Teach a Teacher: Challenges and Opportunities in Physics Teacher Education in Germany and the USA

Preparing future physics teachers for the demanding nature of their profession is an important and complex endeavor. Teacher education systems must provide a structure for the coherent professional development of prospective teachers. Worldwide, physics teacher education is organized in different ways, but have to face similar challenges, like the relation between academic studies and practical preparation. To meet these challenges, it is worth taking look at different teacher education systems. In this chapter, we compare physics teacher education in two countries, representing two different educational traditions: Germany and the USA. Comparing different aspects of physics teacher education (standards, organization and institutionalization, content of teacher education, quality assurance), we describe both systems in their current state and why they are organized in the way they are. In doing so, we identify surprising commonalities but also different opportunities for both systems to learn from each other.

physics.ed-ph↗

arXiv2021

How statistical model development can obscure inequities in STEM student outcomes

Researchers often frame quantitative research as objective, but every step in data collection and analysis can bias findings in often unexamined ways. In this investigation, we examined how the process of selecting variables to include in regression models (model specification) can bias findings about inequities in science and math student outcomes. We identified the four most used methods for model specification in discipline-based education research about equity: a priori, statistical significance, variance explained, and information criterion. Using a quantitative critical perspective that blends statistical theory with critical theory, we reanalyzed the data from a prior publication using each of the four methods and compared the findings from each. We concluded that using information criterion produced models that best aligned with our quantitative critical perspective's emphasis on intersectionality and models with more accurate coefficients and uncertainties. Based on these findings, we recommend researchers use information criterion for specifying models about inequities in STEM student outcomes.

physics.ed-ph↗

arXiv2021

Tenets of QuantCrit

Quantitative Critical (QuantCrit) research is a relatively new field of study grounded in critical theory (Crenshaw, 1990; Ladson-Billings, 2006; 2013). The nascency of QuantCrit has led multiple scholars to propose various related tenets (e.g., Gillborn et al., 2018; Stage, 2007; Covarrubias et al., 2018; López et al., 2018). In this paper, we offer a synthesis of several tenets of QuantCrit we have applied to our research, their applications to education research in general, and citations for more information.

physics.ed-ph↗

arXiv2021

Investigating society's educational debts due to racism and sexism in student attitudes about physics using quantitative critical race theory

The American Physical Society calls on its members to improve the diversity of physics by supporting an inclusive culture that encourages women and Black, Indigenous, and people of color to become physicists. In the current educational system, it is unlikely for a student to become a physicist if they do not share the same attitudes about what it means to learn and do physics as those held by most professional physicists. Evidence shows college physics courses and degree programs do not support students in developing these attitudes. Rather physics education filters out students who do not enter college physics courses with these attitudes. To better understand the role of attitudes in the lack of diversity in physics, we investigated the intersecting relationships between racism and sexism in inequities in student attitudes about learning and doing physics using a critical quantitative framework. The analyses used hierarchical linear models to examine students attitudes as measured by the Colorado learning attitudes about science survey. The data came from the LASSO database and included 2170 students in 46 calculus-based mechanics courses and 2503 students in 49 algebra-based mechanics courses taught at 18 institutions. Like prior studies, we found that attitudes either did not change or slightly decreased for most groups. Results identified large differences across intersecting race and gender groups representing educational debts society owes these students. White students, particularly White men in calculus-based courses, tended to have more expert-like attitudes than any other group of students. Instruction that addresses society's educational debts can help move physics toward an inclusive culture supportive of diverse students and professionals.

physics.ed-ph↗

arXiv2020

Online Administration of Research-Based Assessments

The number and use of research-based assessments (RBAs) has grown significantly over the last several decades. Data from RBAs can be compared against national datasets to provide instructors with empirical evidence on the efficacy of their teaching practices. Many physics instructors, however, opt not to use RBAs due to barriers such as having to use class time to administer them. In this article we examine how these barriers can be mitigated through online administrations of RBAs, particularly through the use of free online RBA platforms that automate administering, scoring, and analyzing RBAs (e.g., the Learning About STEM Student Outcomes [LASSO], Colorado Learning Attitudes About Science Survey for Experimental Physics [E-CLASS], Physics Lab Inventory of Critical thinking [PLIC], and PhysPort DataExplorer platforms). We also explore the research into common concerns of administering RBAs online and conclude with a practical how-to guide for instructors.

physics.ed-ph↗

arXiv2020

Associations between learning assistants, passing introductory physics, and equity: A quantitative critical race theory investigation

Many STEM degrees require passing an introductory physics course. Physics courses often have high failure rates that may disproportionately harm students who are marginalized by racism, sexism, and classism. We examined the associations between Learning Assistant (LA) supported courses and equity in non-passing grades (i.e., d, drop, fail, or withdrawal; DFW) in introductory physics courses. The data used in the study came from 2312 students in 41 sections of introductory physics courses at a regional Hispanic serving institution. We developed hierarchical generalized linear models of student DFW rates that accounted for gender, race, first-generation status and LA-supported instruction. We used a quantitative critical race theory (QuantCrit) perspective focused on the role of hegemonic power structures in perpetuating inequitable student outcomes. Our QuantCrit perspective informed our research questions, methods, and interpretations of findings. The models associated LAs with overall decreases in DFW rates and larger decreases in DFW rates for students of color than their white peers. While the inequities in DFW rates were lower in LA-supported courses, they were still present.

physics.ed-ph↗

arXiv2019

Modernizing use of regression models in physics education research: a review of hierarchical linear modeling

Physics education researchers (PER) often analyze student data with single-level regression models (e.g., linear and logistic regression). However, education datasets can have hierarchical structures, such as students nested within courses, that single-level models fail to account for. The improper use of single-level models to analyze hierarchical datasets can lead to biased findings. Hierarchical models (a.k.a., multi-level models) account for this hierarchical nested structure in the data. In this publication, we outline the theoretical differences between how single-level and multi-level models handle hierarchical datasets. We then present analysis of a dataset from 112 introductory physics courses using both multiple linear regression and hierarchical linear modeling to illustrate the potential impact of using an inappropriate analytical method on PER findings and implications. Research can leverage multi-institutional datasets to improve the field's understanding of how to support student success in physics. There is no post hoc fix, however, if researchers use inappropriate single-level models to analyze multi-level datasets. To continue developing reliable and generalizable knowledge, PER should use hierarchical models when analyzing hierarchical datasets. The supplemental materials include a sample dataset, R code to model the building and analysis presented in the paper, and an HTML output from the R code.

physics.ed-ph↗

arXiv2019

Equity in College Physics Student Learning: a Critical Quantitative Intersectionality Investigation

We investigated the intersectional nature of race/racism and gender/sexism in broad scale inequities in physics student learning using a critical quantitative intersectionality. To provide transparency and create a nuanced picture of learning, we problematized the measurement of equity by using two competing operationalizations of equity: Equity of Individuality and Equality of Learning. These two models led to conflicting conclusions. The analyses used hierarchical linear models to examine student's conceptual learning as measured by gains in scores on research-based assessments administered as pretests and posttests. The data came from the Learning About STEM Student Outcomes' (LASSO) national database and included data from 13,857 students in 187 first-semester college physics courses. Findings showed differences in student gains across gender and race. Large gender differences existed for White and Hispanic students but not for Asian, Black, and Pacific Islander students. The models predicted larger gains for students in collaborative learning than in lecture-based courses. The Equity of Individuality operationalization indicated that collaborative instruction improved equity because all groups learned more with collaborative learning. The Equality of Learning operationalization indicated that collaborative instruction did not improve equity because differences between groups were unaffected. We discuss the implications of these mixed findings and identify areas for future research using critical quantitative perspectives in education research.

physics.ed-ph↗

arXiv2019

Missing data and bias in physics education research: A case for using multiple imputation

Physics education researchers (PER) commonly use complete-case analysis to address missing data. For complete-case analysis, researchers discard all data from any student who is missing any data. Despite its frequent use, no PER article we reviewed that used complete-case analysis provided evidence that the data met the assumption of missing completely at random (MCAR) necessary to ensure accurate results. Not meeting this assumption raises the possibility that prior studies have reported biased results with inflated gains that may obscure differences across courses. To test this possibility, we compared the accuracy of complete-case analysis and multiple imputation (MI) using simulated data. We simulated the data based on prior studies such that students who earned higher grades participated at higher rates, which made the data missing at random (MAR). PER studies seldom use MI, but MI uses all available data, has less stringent assumptions, and is more accurate and more statistically powerful than complete-case analysis. Results indicated that complete-case analysis introduced more bias than MI and this bias was large enough to obscure differences between student populations or between courses. We recommend that the PER community adopt the use of MI for handling missing data to improve the accuracy in research studies.

physics.ed-ph↗

arXiv2018

LASSO: A New Tool to Support Instructors and Researchers

We developed the Learning About STEM Student Outcomes (LASSO) online assessment platform to increase instructor use of research-based assessments (RBAs). LASSO does this by making it easy to collect and analyze high-quality evidence about student learning in their courses. Specifically, LASSO simplifies the process of administering, scoring, and analyzing RBAs and saves class time by automating the process online. Course results are anonymized and aggregated in the LASSO database to provide instructors normative feedback about their student outcomes.

physics.ed-ph↗

arXiv2018

Teacher-Driven Professional Development and the Pursuit of a Sophisticated Understanding of Inquiry

The need for highly qualified physics teachers in the U.S. is well established, and reform efforts are underway to develop novel and innovative teacher professional development experiences to improve the quality of K-12 physics education. Streamline to Mastery is an NSF-funded, learner-centered professional development program that seeks to capitalize on teachers' knowledge and experience to move physics teachers toward mastery in their fields. Teacher participants in this teacher-driven program choose their own goals and areas of growth. One of these areas has been the development and implementation of inquiry-oriented curriculum, as well as the adaptation of traditional lessons toward a greater inquiry orientation. Results indicate that teachers' conceptions of inquiry teaching and learning have become more expert-like as they have engaged in teacher participant-driven experiences in the pursuit of greater understanding and more effective classroom practice.

physics.ed-ph↗

arXiv2018

Student Outcomes Across Collaborative Learning Environments

The Learning Assistant (LA) model supports instructors in implementing research-based teaching practices in their own courses. In the LA model, undergraduate students are hired to help facilitate research-based collaborative-learning activities. Using the Learning About STEM Student Outcomes (LASSO) database, we examined student learning from 112 first-semester physics courses that used either lecture-based instruction, collaborative instruction without LAs, or LA supported instruction. We measured student learning using 5959 students' responses on the Force and Motion Conceptual Evaluation (FMCE) or Force Concept Inventory (FCI). Results from Hierarchical Linear Models (HLM) indicated that LA supported courses had higher posttest scores than collaborative courses without LAs and that LA supported courses that used LAs in laboratory and recitation had higher posttest scores than those that used LAs in lecture.

physics.ed-ph↗

arXiv2018

A comparison of Hake's g and Cohen's d for analyzing gains on concept inventories

Measuring student learning is a complicated but necessary task for understanding the effectiveness of instruction and issues of equity in college STEM courses. Our investigation focused on the implications on claims about student learning that result from choosing between one of two commonly used methods for analyzing shifts in concept inventories. The methods are: Hake's gain (g), which is the most common method used in physics education research and other discipline based education research fields, and Cohen's d, which is broadly used in education research and many other fields. Data for the analyses came from the Learning Assistant Supported Student Outcomes (LASSO) database and included test scores from 4,551 students on physics, chemistry, biology, and math concept inventories from 89 courses at 17 institutions from across the United States. We compared the two methods across all of the concept inventories. The results showed that the two methods led to different inferences about student learning and equity due to g being biased in favor of high pretest populations. Recommendations for the analysis and reporting of findings on student learning data are included.

physics.ed-ph↗