arXiv subjects

Fabian Schrodt

Publications and source records attributed to Fabian Schrodt.

2 recordsLinked to original sources

arXiv2026

AI-based Verbal and Visual Scaffolding in a Serious Game: Effects on Learning and Cognitive Load

Due to their interactive nature, serious games offer valuable opportunities for supporting learning in educational contexts. Recent advances in large language models (LLMs) have further opened the door to new forms of personalized scaffolding in education. In this study, we combine both worlds and study three types of AI-based scaffolding designs in a serious game: (i) no scaffolding, (ii) chat-based (verbal) scaffolding provided by an AI-based non-player character (NPC), and (iii) combined chat-(verbal) and action-based (visual) scaffolding in which the AI may both try to explain or demonstrate the next step towards a solution. The scaffolding conditions are embedded in Qookies, a serious game designed to introduce fundamental concepts of quantum technologies. A total of 152 school students, university students, and members of the general public were randomly assigned to one of the three conditions. The results show that all groups experience significant learning gains, confirming the overall effectiveness of the serious game itself. No significant differences in learning outcomes emerged between scaffolding conditions. However, intrinsic cognitive load was lower in the combined chat-and-action (verbal+visual) scaffolding condition compared to the chat (verbal)-only condition, suggesting that visual demonstrations may offer more accessible support. Interaction analyses further revealed that players engaged with the AI character primarily for level-related questions and action recommendations, while deeper interactions were relatively rare.

physics.ed-ph↗

arXiv2020

Binding and Perspective Taking as Inference in a Generative Neural Network Model

The ability to flexibly bind features into coherent wholes from different perspectives is a hallmark of cognition and intelligence. Importantly, the binding problem is not only relevant for vision but also for general intelligence, sensorimotor integration, event processing, and language. Various artificial neural network models have tackled this problem with dynamic neural fields and related approaches. Here we focus on a generative encoder-decoder architecture that adapts its perspective and binds features by means of retrospective inference. We first train a model to learn sufficiently accurate generative models of dynamic biological motion or other harmonic motion patterns, such as a pendulum. We then scramble the input to a certain extent, possibly vary the perspective onto it, and propagate the prediction error back onto a binding matrix, that is, hidden neural states that determine feature binding. Moreover, we propagate the error further back onto perspective taking neurons, which rotate and translate the input features onto a known frame of reference. Evaluations show that the resulting gradient-based inference process solves the perspective taking and binding problem for known biological motion patterns, essentially yielding a Gestalt perception mechanism. In addition, redundant feature properties and population encodings are shown to be highly useful. While we evaluate the algorithm on biological motion patterns, the principled approach should be applicable to binding and Gestalt perception problems in other domains.

cs.LG↗