arXiv subjects

Davinia Hernández-Leo

Publications and source records attributed to Davinia Hernández-Leo.

4 recordsLinked to original sources

arXiv2026

Responsible Institutional Analytics: Interpreting Bias with AI Support

Institutional Analytics (IA) dashboards inform decision-making in higher education, yet data limitations, constraints in analytical techniques, and missing contextual information often affect their interpretation. To support more responsible interpretation of IA, we introduce FACTRIA, a framework that organizes potential biasing factors across four areas: the analytics pipeline, institutional context, course-level characteristics, and demographics. We used the FACTRIA framework as input to a generative-AI chatbot designed to prompt users to reflect on these factors while analyzing IA. A qualitative study with stakeholders, drawing on four authentic IA cases, and a transition network analysis showed that the chatbot prompted participants to recognize how overlooked factors influenced their initial interpretation. Findings indicated that combining a structured framework with AI-based guidance can enhance context-aware, responsible interpretation of institutional data.

cs.HC↗

arXiv2026

Access Timing as Scaffolding: A Reinforcement Learning Approach to GenAI in Education

In recent years, generative AI (GenAI) in educational settings has become ubiquitous in university students' daily lives, despite its potential to induce over-reliance, metacognitive disengagement, and diminished learning when used unrestrictedly. While most prior research has focused on how to pedagogically scaffold its usage, the question of when to allow off-the-shelf GenAI remains understudied and lacks pedagogically grounded empirical investigation. We treat access timing itself as a form of implicit scaffolding and operationalize it through a reinforcement learning (RL) agent that decides when students should access GenAI, with a reward function grounded in metacognitive theory, cognitive load theory, and productive failure. In a mixed-methods controlled lab study with N=105 higher education students, we compared the agent's effect on learning gains and metacognitive engagement to unrestricted and fully restricted use. Results show that strategically timed GenAI access under the reinforcement learning condition improved objective post-test performance and metacognitive accuracy compared with unrestricted access, without requiring explicit metacognitive prompts or structured scaffolding. Exploratory comparisons with the fully restricted condition further suggest that timed access may reduce task errors and time on task relative to complete withholding. Overall, timing of GenAI access therefore is a tractable, theoretically grounded, and scalable pedagogical strategy that improves over completely unrestricted and withheld access, compatible with off-the-shelf tools and potentially low adoption barrier. This opens up a new research area that explores how access timing can be facilitated by educators and implemented in human-AI learning system design.

cs.CY↗

arXiv2026

Experiential Versus Instructional Approaches for Eliciting Metacognitive Awareness in AI-Assisted Learning: A Short-Term Longitudinal Study

With generative AI (GenAI) entering classrooms the question to which teaching approach best supports metacognitive skill acquisition in AI-assisted learning becomes pressing. In this short-term longitudinal study we investigate two contrasting approaches: experiential learning encompassing hands-on approaches and instructional learning such as classical lectures. We conducted a quasi-experiment with 126 university students from a first-year engineering course which were distributed across the two conditions and completed a two hour session on learning with GenAI in the corresponding learning style. Metacognitive awareness which encompasses both knowledge of cognition (understanding effective AI-use strategies) and regulation of cognition (applying that knowledge in practice) was measured before and after the session. Additionally, students longitudinal metacognitive awareness was tracked over the trimester and assessed again five weeks after the initial intervention. Results reveal that experiential methods outperform instructional approaches in engagement and knowledge of cognition immediately after the intervention. By five weeks, the two groups converged on these measures, while the experiential group showed a delayed, continuous within-group increase in regulation of cognition that was not observed in the instructional group. This suggests that the benefits of experiential approaches extend beyond conventional educational settings to AI-assisted learning, and that knowledge and regulation may develop on different timescales under experiential learning.

cs.HC↗

arXiv2024

Leveraging User Experience and Learning Analytics for Enhanced Student Well-being

This study explores the design and preliminary evaluation of the "Well-being Journey" (WB Journey), a digital tool aimed at enhancing student well-being within educational environments through tailored recommendations for students. The study examines the WB Journey prototype's user experience and its effectiveness in meeting learning analytics goals related to student preferences. To achieve both goals, we employ a mixed-methods approach, combining quantitative data from the User Experience Questionnaire (UEQ) and the Student Expectations of Learning Analytics Questionnaire (SELAQ) with qualitative feedback from a student discussion. Conducted among 25 students from an engineering school in a Spanish University, the study's data collection involved a 120-minute workshop. The findings suggest opportunities for enhancing the prototype, highlighting the importance of aligning similar digital tools with student needs and preferences for a supportive learning environment, which can be achieved by leveraging tools such as UEQ and SELAQ.

cs.HC↗