arXiv · 2611.18609
Many Paths to the Same Mistake: Generating Diverse Reasoning Paths Behind the Same Student Mistake
Abstract
Realistic evaluation of AI systems requires user simulators that capture not only cooperative behavior, but also the errors and variability that arise in real interactions. In educational technology, simulating step-by-step flawed student reasoning is essential, yet datasets mapping these pathways are scarce, and Large Language Models (LLMs) often auto-correct rather than sustain simulated mistakes. To address this, we present a generate-and-filter pipeline that produces diverse and realistic student error traces. Our approach utilizes forward consistency, a metric verifying if a flawed reasoning trace can be used to recover the student's actual incorrect answer, paired with an LLM-as-a-judge to ensure realism. Our evaluation suggests that while many LLMs struggle to maintain incorrect reasoning mid-trace, smaller supervised fine-tuned models successfully capture these divergent paths. This work provides a framework for high-fidelity error trace generation, advancing realistic, non-cooperative user simulation.
Explore related subjects
Keep this discovery
Explore connections, maps & timelines
Fareya Ikram, Mengxuan Hu, Parker Glenn, Zhiqi Huang, Alfy Samuel. 2026-09-03. Many Paths to the Same Mistake: Generating Diverse Reasoning Paths Behind the Same Student Mistake. https://searcharxiv.com/abs/2611.18609
Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite Searcharxiv by Indiciae: BibTeX · RIS.