arXiv subjects

Bruce M. McLaren

Publications and source records attributed to Bruce M. McLaren.

4 recordsLinked to original sources

arXiv2026

Understanding Gaming the System by Analyzing Self-Regulated Learning in Think-Aloud Protocols

In digital learning systems, gaming the system refers to occasions when students attempt to succeed in an educational task by systematically taking advantage of system features rather than engaging meaningfully with the content. Often viewed as a form of behavioral disengagement, gaming the system is negatively associated with short- and long-term learning outcomes. However, little research has explored this phenomenon beyond its behavioral representation, leaving questions such as whether students are cognitively disengaged or whether they engage in different self-regulated learning (SRL) strategies when gaming largely unanswered. This study employs a mixed-methods approach to examine students' cognitive engagement and SRL processes during gaming versus non-gaming periods, using utterance length and SRL codes inferred from think-aloud protocols collected while students interacted with an intelligent tutoring system for chemistry. We found that gaming does not simply reflect a lack of cognitive effort; during gaming, students often produced longer utterances, were more likely to engage in processing information and realizing errors, but less likely to engage in planning, and exhibited reactive rather than proactive self-regulatory strategies. These findings provide empirical evidence supporting the interpretation that gaming may represent a maladaptive form of SRL. With this understanding, future work can address gaming and its negative impacts by designing systems that target maladaptive self-regulation to promote better learning.

cs.CY↗

arXiv2025

Carelessness Detection using Performance Factor Analysis: A New Operationalization with Unexpectedly Different Relationship to Learning

Detection of carelessness in digital learning platforms has relied on the contextual slip model, which leverages conditional probability and Bayesian Knowledge Tracing (BKT) to identify careless errors, where students make mistakes despite having the knowledge. However, this model cannot effectively assess carelessness in questions tagged with multiple skills due to the use of conditional probability. This limitation narrows the scope within which the model can be applied. Thus, we propose a novel model, the Beyond Knowledge Feature Carelessness (BKFC) model. The model detects careless errors using performance factor analysis (PFA) and behavioral features distilled from log data, controlling for knowledge when detecting carelessness. We applied the BKFC to detect carelessness in data from middle school students playing a learning game on decimal numbers and operations. We conducted analyses comparing the careless errors detected using contextual slip to the BKFC model. Unexpectedly, careless errors identified by these two approaches did not align. We found students' post-test performance was (corresponding to past results) positively associated with the carelessness detected using the contextual slip model, while negatively associated with the carelessness detected using the BKFC model. These results highlight the complexity of carelessness and underline a broader challenge in operationalizing carelessness and careless errors.

cs.CY↗

arXiv2023

Evaluating ChatGPT's Decimal Skills and Feedback Generation in a Digital Learning Game

While open-ended self-explanations have been shown to promote robust learning in multiple studies, they pose significant challenges to automated grading and feedback in technology-enhanced learning, due to the unconstrained nature of the students' input. Our work investigates whether recent advances in Large Language Models, and in particular ChatGPT, can address this issue. Using decimal exercises and student data from a prior study of the learning game Decimal Point, with more than 5,000 open-ended self-explanation responses, we investigate ChatGPT's capability in (1) solving the in-game exercises, (2) determining the correctness of students' answers, and (3) providing meaningful feedback to incorrect answers. Our results showed that ChatGPT can respond well to conceptual questions, but struggled with decimal place values and number line problems. In addition, it was able to accurately assess the correctness of 75% of the students' answers and generated generally high-quality feedback, similar to human instructors. We conclude with a discussion of ChatGPT's strengths and weaknesses and suggest several venues for extending its use cases in digital teaching and learning.

cs.HC↗

arXiv2020

Extending Deep Knowledge Tracing: Inferring Interpretable Knowledge and Predicting Post-System Performance

Recent student knowledge modeling algorithms such as Deep Knowledge Tracing (DKT) and Dynamic Key-Value Memory Networks (DKVMN) have been shown to produce accurate predictions of problem correctness within the same learning system. However, these algorithms do not attempt to directly infer student knowledge. In this paper we present an extension to these algorithms to also infer knowledge. We apply this extension to DKT and DKVMN, resulting in knowledge estimates that correlate better with a posttest than knowledge estimates from Bayesian Knowledge Tracing (BKT), an algorithm designed to infer knowledge, and another classic algorithm, Performance Factors Analysis (PFA). We also apply our extension to correctness predictions from BKT and PFA, finding that knowledge estimates produced with it correlate better with the posttest than BKT and PFA's standard knowledge estimates. These findings are significant since the primary aim of education is to prepare students for later experiences outside of the immediate learning activity.

cs.CY↗