Computer-based and paper-and-pencil tests: A study in calculus for STEM majors

Abstract

Computer-based testing is an expanding use of technology offering advantages to teachers and students. We studied Calculus II classes for STEM majors using different testing modes. Three sections with 324 students employed: Paper-and-pencil testing, computer-based testing, and both. Computer tests gave immediate feedback, allowed multiple submissions, and pooling. Paper-and-pencil tests required work and explanation allowing inspection of high cognitive demand tasks. Each test mode used the strength of its method. Students were given the same lecture by the same instructor on the same day and the same homework assignments and due dates. The design is quasi-experimental, but students were not aware of the testing mode at registration. Two basic questions examined were: (1) Do paper-and-pencil and computer-based tests measure knowledge and skill in STEM Calculus II in a consistent manner? (2) How does the knowledge and skill gained by students in a fully computer-based Calculus II class compare to students in a class requiring pencil-and-paper tests and hence some paper-and-pencil work. These results indicate that computer-based tests are as consistent with paper-and-pencil tests as computer-based tests are with themselves. Results are also consistent with classes using paper-and-pencil tests having slightly better outcomes than fully computer-based classes using only computer assessments.