arXiv subjects

Engin Bumbacher

Publications and source records attributed to Engin Bumbacher.

4 recordsLinked to original sources

arXiv2026

Reflective Dialogue or Prompt Refinement? Effects of Tutor Scaffolding on Students' Independent LLM Use for Programming

While Large Language Models (LLMs) can provide personalized support in learning, several studies have raised concerns regarding their use in education. Importantly, learning depends on how students engage with LLMs. This study examined how two types of LLM-based tutors shape students' prompting practices, learning, and subsequent LLM-use: a Socratic-Guidance (SG) tutor, which structures interaction through dialogic questioning, and a Prompt-Refinement (PR) tutor that guides the formulation of effective prompts. We conducted a two-phase study in a graduate-level mobile robotics course: 66 students used either the SG or PR tutor during a 6-week intervention, followed by 52 students using an unconstrained LLM during a 3-week course project. Results show that while the SG- and PR tutors led to similar task performance and prompting patterns during guided use, they differ in learning outcomes and later LLM-use. SG-students, relative to PR-student, achieved higher learning gains in later sessions, and were more likely to adopt understanding-driven prompting strategies, which are predictive of higher understanding, when using an unconstrained LLM. Although learners perceived the SG tutor as less efficient, the findings suggest that Socratic guidance supports the development of students' capacity to learn with LLMs over time, highlighting its importance for LLM tutor design.

cs.AI↗

arXiv2025

Structured Prompts, Better Outcomes? Exploring the Effects of a Structured Interface with ChatGPT in a Graduate Robotics Course

Prior research shows that how students engage with Large Language Models (LLMs) influences their problem-solving and understanding, reinforcing the need to support productive LLM-uses that promote learning. This study evaluates the impact of a structured GPT platform designed to promote 'good' prompting behavior with data from 58 students in a graduate-level robotics course. The students were assigned to either an intervention group using the structured platform or a control group using ChatGPT freely for two practice lab sessions, before a third session where all students could freely use ChatGPT. We analyzed student perception (pre-post surveys), prompting behavior (logs), performance (task scores), and learning (pre-post tests). Although we found no differences in performance or learning between groups, we identified prompting behaviors - such as having clear prompts focused on understanding code - that were linked with higher learning gains and were more prominent when students used the structured platform. However, such behaviors did not transfer once students were no longer constrained to use the structured platform. Qualitative survey data showed mixed perceptions: some students perceived the value of the structured platform, but most did not perceive its relevance and resisted changing their habits. These findings contribute to ongoing efforts to identify effective strategies for integrating LLMs into learning and question the effectiveness of bottom-up approaches that temporarily alter user interfaces to influence students' interaction. Future research could instead explore top-down strategies that address students' motivations and explicitly demonstrate how certain interaction patterns support learning.

cs.CY↗

arXiv2025

"I forgot the formula:" How students can use coherence to reconstruct a (partially) forgotten equation

Introductory physics instruction emphasizes fluency with routine problem-solving procedures. However, even when applying these procedures, students frequently encounter challenges. This paper investigates how students navigate such moments when answering qualitative E&M problems in interviews. Students frequently noted they had partially forgotten a key equation on a problem involving RC circuits. We present focal cases that show how coherence-seeking approaches were employed to overcome this problem-solving challenge. In attempts to reconstruct these equations, participants were guided by identifying and chaining qualitative dependencies and seeking coherence between qualitative and mathematical understanding of the physical system. These moments of forgetting and reconstructing equations are a useful site for studying broader physics learning goals. While prior work investigates the use of mathematical sensemaking by examining how students respond to explicit prompts, our cases illustrate how students can spontaneously use mathematical sensemaking strategies. We reflect on these cases to consider how such adaptive reasoning can be a target for instruction and assessment.

physics.ed-ph↗

arXiv2025

FADE-CTP: A Framework for the Analysis and Design of Educational Computational Thinking Problems

In recent years, the emphasis on computational thinking (CT) has intensified as an effect of accelerated digitalisation. While most researchers are concentrating on defining CT and developing tools for its instruction and assessment, we focus on the characteristics of computational thinking problems (CTPs) - activities requiring CT to be solved - and how they influence the skills students can develop. In this paper, we present a comprehensive framework for systematically profiling CTPs by identifying specific components and characteristics, while establishing a link between these attributes and a structured catalogue of CT competencies. The purposes of this framework are (i) facilitating the analysis of existing CTPs to identify which abilities can be developed or measured based on their inherent characteristics, and (ii) guiding the design of new CTPs targeted at specific skills by outlining the necessary characteristics required for CT activation. To illustrate the framework functionalities, we begin by analysing prototypical activities in the literature, a process that leads to the definition of a taxonomy of CTPs across various domains, and we conclude with a case study on the design of a different version of one of these activities, the Cross Array Task (CAT), set in different cognitive environments. This approach allows an understanding of how CTPs in different contexts display unique and recurring characteristics that promote the development of distinct skills. In conclusion, this framework can inform the development of assessment tools, improve teacher training, and facilitate the analysis and comparison of existing CT activities, contributing to a deeper understanding of competency activation and guiding curriculum design in CT education.

cs.HC↗