arXiv subjects

Yongnam Kim

Publications and source records attributed to Yongnam Kim.

2 recordsLinked to original sources

arXiv2019

Apples to Oranges: Causal Effects of Answer Changing in Multiple-Choice Exams

Whether examinees' answer changing behavior while taking multiple-choice exams is beneficial or harmful is a long-standing puzzle in the educational and psychological measurement literature. Formalizing the problem using the potential outcomes framework, this article shows that the traditional method of comparing the proportions of "wrong to right" and "right to wrong" answer changing patterns--a method that has recently been criticized by van der Linden, Jeon, and Ferrara (2011)--indeed correctly identify the sign of the average answer changing effect, but only for those examinees who actually changed their initial responses. This subgroup effect is referred to as the average treatment effect on the treated (ATT) and generally differs from the average treatment effect on the untreated (ATU), that is, those who did not change their initial responses. Analyzing two real data sets, including van der Linden et al.'s (2011) controversial data, this article finds that the ATT of answer changing is positive while the ATU of answer changing is negative, therefore, the debate on answer changing effects can be easily resolved. The article also shows that answer changing and answer reviewing are two distinct treatments and knowing answer changing effects is not informative for predicting answer reviewing effects.

stat.AP↗

arXiv2019

Partial Identification of Answer Reviewing Effects in Multiple-Choice Exams

Does reviewing previous answers during multiple-choice exams help examinees increase their final score? This article formalizes the question using a rigorous causal framework, the potential outcomes framework. Viewing examinees' reviewing status as a treatment and their final score as an outcome, the article first explains the challenges of identifying the causal effect of answer reviewing in regular exam-taking settings. In addition to the incapability of randomizing the treatment selection (reviewing status) and the lack of other information to make this selection process ignorable, the treatment variable itself is not fully known to researchers. Looking at examinees' answer sheet data, it is unclear whether an examinee who did not change his or her answer on a specific item reviewed it but retained the initial answer (treatment condition) or chose not to review it (control condition). Despite such challenges, however, the article develops partial identification strategies and shows that the sign of the answer reviewing effect can be reasonably inferred. By analyzing a statewide math assessment data set, the article finds that reviewing initial answers is generally beneficial for examinees.

stat.AP↗