arXiv subjects

Eugenio Tufino

Publications and source records attributed to Eugenio Tufino.

13 recordsLinked to original sources

arXiv2026

A multi-agent AI classroom based on dual-process reasoning hazards: a pilot with prospective physics teachers

Responding productively, in real time, to the reasoning students actually produce is among the most difficult and slowest to acquire of the skills physics teachers develop, and prospective teachers get few opportunities to practise it before entering a classroom. In this pilot study, we describe a simulated class of five AI students, each consistently enacting a distinct reasoning pattern drawn from the dual-process theory (DPT) "hazards" framework of physics education research, and report on its first use in a university course for prospective physics teachers. Fifteen graduate students worked through an instructional sequence on DPT and questioning strategies. They then diagnosed two parallel sets of written vignettes, in a crossover arrangement, before and after interacting in pairs with the simulated class on a static-friction problem. The data set is completed by individual written hypotheses and reflections, the session logs and a short feedback questionnaire. Diagnostic scores improved significantly from PRE to POST (n = 11 paired, Wilcoxon p = 0.014, r = 0.79; on a 0-18 scale). During the simulation itself, however, participants asked predominantly uniform guiding questions, and the DPT vocabulary they had been taught appeared in only 2 of 71 substantive teacher turns, while seven of thirteen POST sheets used it readily. We read this knowing-doing gap not as a failure but as a snapshot of where each participant stands on the developmental trajectory of responsiveness to student ideas, a trajectory the simulation makes visible with transcript-level granularity. We discuss design, findings and implications for physics teacher preparation.

physics.ed-ph↗

arXiv2026

Embodied inquiry with AI as facilitator: an exploratory case study

Generative AI is entering science education at a time when the embodied education community is asking what such systems cannot do. Rather than asking whether AI can understand the body, this exploratory case study asks where a language-based AI can stand within an inquiry activity without displacing embodied experience. University students in a Master's course in physics education investigated the statics of fluids following the ISLE approach (Investigative Science Learning Environment) in a two-phase design: students first built the buoyancy model with their own hands, without AI; a purpose-configured AI assistant then facilitated the application of the model to a new phenomenon. We discuss what a language-based facilitator cannot reach and the value of a design in which AI complements embodied inquiry rather than replacing it.

physics.ed-ph↗

arXiv2026

Performance and failure modes of AI chatbots on a novel concept inventory on relativity in classical mechanics

AI chatbots are increasingly used by students as study tools in physics, raising practical questions about their reliability on conceptual tasks. Existing evaluations of large language models (LLMs) on physics concept inventories rely almost exclusively on instruments that have been publicly available for years and likely appear in model training data, making it difficult to disentangle physics competence from familiarity with the test items themselves. We address this issue by evaluating three frontier LLMs (GPT-5.2, Gemini 3 Pro, Gemini 3 Flash) on the Classical Relativity Concept Inventory (CRCI), a recently developed and validated 21-item instrument on Galilean relativity that was not publicly available at the time of testing. Each item was administered 30 times per model, and all 1890 responses were qualitatively coded along three dimensions: visual interpretation, physics reasoning, and coordination. Mean accuracy was 97% for Gemini 3 Flash, 89% for Gemini 3 Pro, and 73% for GPT-5.2, compared to 62% for the student sample (N = 267). However, all three models fail completely on a small number of items. The qualitative analysis shows that these failures stem predominantly from misinterpretations of visual content rather than from deficits in physics knowledge, and that LLM errors differ structurally from those of students: when models err, they converge on a single distractor with high consistency, whereas student errors are more broadly distributed. These findings indicate that chatbot reliability on conceptual physics is item-dependent and unpredictable, with direct implications for how concept inventories are administered.

physics.ed-ph↗

arXiv2026

Demystifying the oracle: A "20 Questions" game to promote AI ethics and literacy

As Generative AI becomes a key component in physics education, a significant ethical challenge has emerged: the tendency of students to anthropomorphize Large Language Models (LLMs), treating them as authoritative "oracles" that retrieve fixed facts from an internal database. However, LLMs operate fundamentally as probabilistic engines. This paper describes the design and implementation of a didactic activity, a reduced version of the "20 Questions" game, aimed at making this stochastic nature directly observable. Unlike a human player who fixes a target object at the start of the game, students discover that the model generates answers based solely on local coherence with the interaction history. By utilizing functionalities such as re-sampling and history rewinding, students act as experimenters, observing how identical interaction histories can yield diverging narrative paths. We discuss how mapping these behaviors to familiar physics concepts provides the epistemic scaffolding necessary to promote informed skepticism, framing the verification of AI outputs not merely as a compliance rule, but as a technical necessity derived from the system's probabilistic nature.

physics.ed-ph↗

arXiv2025

Creating a customisable Socratic AI physics tutor

This paper explores role engineering as an effective paradigm for customizing Large Language Models (LLMs) into specialized AI tutors for physics education. We demonstrate this methodology by designing a Socratic physics problem-solving tutor using Google's Gemini Gems feature, defining its pedagogical behavior through a detailed 'script' that specifies its role and persona. We present two illustrative use cases: the first demonstrates the Gem's multimodal ability to analyze a student's hand-drawn force diagram and apply notational rules from a 'Knowledge' file; the second showcases its capacity to guide conceptual reasoning in electromagnetism using its pre-trained knowledge without using specific documents provided by the instructor. Our findings show that the 'role-engineered' Gem successfully facilitates a Socratic dialogue, in stark contrast to a standard Gemini model, which tends to immediately provide direct solutions. We conclude that role engineering is a pivotal and accessible method for educators to transform a general-purpose 'solution provider' into a reliable pedagogical tutor capable of engaging students in an active reflection process. This approach offers a powerful tool for both instructors and students, while also highlighting the importance of addressing the technology's inherent limitations, such as the potential for occasional inaccuracies.

physics.ed-ph↗

arXiv2025

NotebookLM as a Socratic physics tutor: Design and preliminary observations of a RAG-based tool

This study explores NotebookLM, a Google Gemini - powered AI platform that integrates Retrieval-Augmented Generation (RAG) as a Socratic tutor for physics education. In this implementation, NotebookLM was configured to support students in solving conceptually oriented physics problems through a guided, questioning-based dialogue. When deployed as a collaborative tutor, the system restricts student interaction to a chat-only interface, promoting controlled and guided engagement. By grounding its responses in teacher-provided source documents, the AI tutor helps mitigate one of the major shortcomings of standard Large Language Models - hallucinations - thereby ensuring more traceable and reliable answers. This work details the methodological design of the tutor, including the iterative development of a pedagogical "Training Manual", and presents preliminary qualitative observations from demonstrations with pre-service and in-service teachers. These observations highlight both the promising potential of the tool and key pedagogical challenges, such as managing user motivation. While limitations remain, this work offers a promising and replicable model for educators seeking to implement grounded AI tutors in their own teaching contexts.

physics.ed-ph↗

arXiv2025

Exploring Large Language Models (LLMs) through interactive Python activities

This paper presents an approach to introduce physics students to the basic concepts of Large Language Models (LLMs) using Python-based activities in Google Colab. The teaching strategy integrates active learning strategies and combines theoretical ideas with practical, physics-related examples. Students engage with key technical concepts, such as word embeddings, through hands-on exploration of the Word2Vec neural network and GPT-2 - an LLM that gained a lot of attention in 2019 for its ability to generate coherent and plausible text from simple prompts. The activities highlight how words acquire meaning and how LLMs predict subsequent tokens by simulating simplified scenarios related to physics. By focusing on Word2Vec and GPT-2, the exercises illustrate fundamental principles underlying modern LLMs, such as semantic representation and contextual prediction. Through interactive experimenting in Google Colab, students observe the relationship between model parameters (such as temperature) in GPT-2 and output behaviour, understand scaling laws relating data quantity to model performance, and gain practical insights into the predictive capabilities of LLMs. This approach allows students to begin to understand how these systems work by linking them to physics concepts - systems that will shape their academic studies, professional careers and roles in society.

physics.ed-ph↗

arXiv2024

Exploring active learning in physics with ISLE-based modules in high school

This study presents a case study of active learning within the Investigative Science Learning Environment (ISLE), using the iOLab digital devices. We designed a pilot lab format to enhance student engagement and understanding through direct experimentation, taking advantage of the multifunctional capabilities of the iOLab devices. This paper evaluates the pedagogical effectiveness of integrating ISLE with digital tools for data collection and analysis in physics experiments. The initial findings provide insights into the pedagogical benefits and logistical considerations of using such technologies in a laboratory setting. Although no direct comparison with traditional teaching methods has been made, the observed student engagement and feedback suggest a positive impact on learning outcomes, even within the constraints of the short duration of the interventions.

physics.ed-ph↗

arXiv2024

Development of a global landscape of undergraduate physics laboratory courses

Physics Education Research (PER) is a global endeavour, with a wealth of work performed at a variety of institutions worldwide. However, results from research into undergraduate physics laboratory courses is often difficult to compare due to the broad variations in courses. We have created a survey to help classify these courses to compare and contrast them, which will be useful in two key endeavours: comparisons between PER studies and providing useful data for individual instructors hoping to improve their courses, thus providing information relevant to both researchers and instructors. While we are still in the process of collecting sufficient data to create a full taxonomy of laboratory courses, we present here details of the survey creation itself, as well as a first look at the data collected, which includes a broad landscape of lab courses in 41 countries. We used both quantitative and qualitative methods in analyzing the data collected. Some of these results include similarities between courses, such as students often using pre-constructed apparatuses and that instructors hope for students to learn technical skills. We also find differences in courses, such as in the number and types of goals of the course, as well as the activities students participate in.

physics.ed-ph↗

arXiv2024

Using Jupyter Notebooks to foster computational skills and professional practice in an introductory physics lab course

In this paper, we detail the integration of Python data analysis into a first-year physics laboratory course, a task accomplished without significant alterations to the existing course structure. We introduced tailored laboratory computational learning goals and designed activities to address them. We emphasise the development and application of Jupyter Notebooks, tailored with exercises and physics application examples, to facilitate students' mastery of data analysis programming within the laboratory setting. These Notebooks serve as a crucial tool in guiding students through the core principles of data handling and analysis in Python, while working on simple experimental tasks. The results of the evaluation of this intervention offer insights into the advantages and challenges associated with early integration of computational skills in laboratory courses, providing valuable information for educators in the field of physics education. This study demonstrates a practical and effective way of embedding computational skills into the physics curriculum, and contributes to the ongoing efforts of the physics education research community.

physics.ed-ph↗

arXiv2023

Evaluation of the effectiveness of an introductory mechanics Lab with Arduino and smartphone

This article describes the reform of a physics undergraduate lab course by introducing new technologies such as Arduino microcontroller and smartphone, to help students experience a more authentic lab experience. This paper outlines the structure of the lab activities and presents findings from pre- and post-course questionnaires, including the E-CLASS survey, completed by participating students. Some interesting observations about teamwork and equity emerged from these questionnaires, particularly about female students. These results will guide us in designing more effective and equitable lab courses.

physics.ed-ph↗

arXiv2023

Integrating Python data analysis in an existing introductory laboratory course

In this article we describe how we successfully incorporated data analysis in Python in a first-year laboratory course without significantly altering the course structure and without overburdening students. We show how we created and used carefully designed Jupyter Notebooks with exercises and physics application examples that allow students to master data analysis programming in the laboratory course. We use these Notebooks to guide students through the fundamentals of data handling and analysis in Python while performing simple experiments. We present our teaching approach and the developed materials. We discuss the effectiveness of our intervention based on the results from pre- and post- course questionnaires and students' group work. The results presented give insights about advantages and challenges of introducing computation at the early stage of the curriculum in a laboratory course setting and are informative for other instructors and the physics education research community.

physics.ed-ph↗

arXiv2023

Two simple experiments using FFT with digital devices in the introductory physics laboratory

We propose two experiments suited for high school and/or undergraduate physics laboratory which are aimed to the discovery of the practical meaning and usefulness of the FFT analysis as a mathematical and graphical instrument, intended to discriminate different periodic signals in various situations, ranging from basic, demonstrative measurements to very complex, real-research experiments. In our proposal, quite inexpensive digital instruments can be used, yet with accurate out-of-the-box results.

physics.ed-ph↗