arXiv subjects

Mutlu Cukurova

Publications and source records attributed to Mutlu Cukurova.

33 records · Page 2Linked to original sources

arXiv2025

A Novel Approach to Scalable and Automatic Topic-Controlled Question Generation in Education

The development of Automatic Question Generation (QG) models has the potential to significantly improve educational practices by reducing the teacher workload associated with creating educational content. This paper introduces a novel approach to educational question generation that controls the topical focus of questions. The proposed Topic-Controlled Question Generation (T-CQG) method enhances the relevance and effectiveness of the generated content for educational purposes. Our approach uses fine-tuning on a pre-trained T5-small model, employing specially created datasets tailored to educational needs. The research further explores the impacts of pre-training strategies, quantisation, and data augmentation on the model's performance. We specifically address the challenge of generating semantically aligned questions with paragraph-level contexts, thereby improving the topic specificity of the generated questions. In addition, we introduce and explore novel evaluation methods to assess the topical relatedness of the generated questions. Our results, validated through rigorous offline and human-backed evaluations, demonstrate that the proposed models effectively generate high-quality, topic-focused questions. These models have the potential to reduce teacher workload and support personalised tutoring systems by serving as bespoke question generators. With its relatively small number of parameters, the proposals not only advance the capabilities of question generation models for handling specific educational topics but also offer a scalable solution that reduces infrastructure costs. This scalability makes them feasible for widespread use in education without reliance on proprietary large language models like ChatGPT.

cs.CY↗

arXiv2024

Adult learners recall and recognition performance and affective feedback when learning from an AI-generated synthetic video

The widespread use of generative AI has led to multiple applications of AI-generated text and media to potentially enhance learning outcomes. However, there are a limited number of well-designed experimental studies investigating the impact of learning gains and affective feedback from AI-generated media compared to traditional media (e.g., text from documents and human recordings of video). The current study recruited 500 participants to investigate adult learners recall and recognition performances as well as their affective feedback on the AI-generated synthetic video, using a mixed-methods approach with a pre-and post-test design. Specifically, four learning conditions, AI-generated framing of human instructor-generated text, AI-generated synthetic videos with human instructor-generated text, human instructor-generated videos, and human instructor-generated text frame (baseline), were considered. The results indicated no statistically significant difference amongst conditions on recall and recognition performance. In addition, the participants affective feedback was not statistically significantly different between the two video conditions. However, adult learners preferred to learn from the video formats rather than text materials.

cs.HC↗

arXiv2024

Biometrics and Behavior Analysis for Detecting Distractions in e-Learning

In this article, we explore computer vision approaches to detect abnormal head pose during e-learning sessions and we introduce a study on the effects of mobile phone usage during these sessions. We utilize behavioral data collected from 120 learners monitored while participating in a MOOC learning sessions. Our study focuses on the influence of phone-usage events on behavior and physiological responses, specifically attention, heart rate, and meditation, before, during, and after phone usage. Additionally, we propose an approach for estimating head pose events using images taken by the webcam during the MOOC learning sessions to detect phone-usage events. Our hypothesis suggests that head posture undergoes significant changes when learners interact with a mobile phone, contrasting with the typical behavior seen when learners face a computer during e-learning sessions. We propose an approach designed to detect deviations in head posture from the average observed during a learner's session, operating as a semi-supervised method. This system flags events indicating alterations in head posture for subsequent human review and selection of mobile phone usage occurrences with a sensitivity over 90%.

cs.CV↗

arXiv2024

The Interplay of Learning, Analytics, and Artificial Intelligence in Education: A Vision for Hybrid Intelligence

This paper presents a multi-dimensional view of AI's role in learning and education, emphasizing the intricate interplay between AI, analytics, and the learning processes. Here, I challenge the prevalent narrow conceptualisation of AI as tools, as exemplified in generative AI tools, and argue for the importance of alternative conceptualisations of AI for achieving human-AI hybrid intelligence. I highlight the differences between human intelligence and artificial information processing, the importance of hybrid human-AI systems to extend human cognition, and posit that AI can also serve as an instrument for understanding human learning. Early learning sciences and AI in Education research (AIED), which saw AI as an analogy for human intelligence, have diverged from this perspective, prompting a need to rekindle this connection. The paper presents three unique conceptualisations of AI: the externalization of human cognition, the internalization of AI models to influence human mental models, and the extension of human cognition via tightly coupled human-AI hybrid intelligence systems. Examples from current research and practice are examined as instances of the three conceptualisations in education, highlighting the potential value and limitations of each conceptualisation for education, as well as the perils of overemphasis on externalising human cognition. The paper concludes with advocacy for a broader approach to AIED that goes beyond considerations on the design and development of AI, but also includes educating people about AI and innovating educational systems to remain relevant in an AI-ubiquitous world.

cs.CY↗

arXiv2024

What Explains Teachers' Trust of AI in Education across Six Countries?

With growing expectations to use AI-based educational technology (AI-EdTech) to improve students' learning outcomes and enrich teaching practice, teachers play a central role in the adoption of AI-EdTech in classrooms. Teachers' willingness to accept vulnerability by integrating technology into their everyday teaching practice, that is, their trust in AI-EdTech, will depend on how much they expect it to benefit them versus how many concerns it raises for them. In this study, we surveyed 508 K-12 teachers across six countries on four continents to understand which teacher characteristics shape teachers' trust in AI-EdTech, and its proposed antecedents, perceived benefits and concerns about AI-EdTech. We examined a comprehensive set of characteristics including demographic and professional characteristics (age, gender, subject, years of experience, etc.), cultural values (Hofstede's cultural dimensions), geographic locations (Brazil, Israel, Japan, Norway, Sweden, USA), and psychological factors (self-efficacy and understanding). Using multiple regression analysis, we found that teachers with higher AI-EdTech self-efficacy and AI understanding perceive more benefits, fewer concerns, and report more trust in AI-EdTech. We also found geographic and cultural differences in teachers' trust in AI-EdTech, but no demographic differences emerged based on their age, gender, or level of education. The findings provide a comprehensive, international account of factors associated with teachers' trust in AI-EdTech. Efforts to raise teachers' understanding of, and trust in AI-EdTech, while considering their cultural values are encouraged to support its adoption in K-12 education.

cs.CY↗

arXiv2024

Harnessing Transparent Learning Analytics for Individualized Support through Auto-detection of Engagement in Face-to-Face Collaborative Learning

Using learning analytics to investigate and support collaborative learning has been explored for many years. Recently, automated approaches with various artificial intelligence approaches have provided promising results for modelling and predicting student engagement and performance in collaborative learning tasks. However, due to the lack of transparency and interpretability caused by the use of "black box" approaches in learning analytics design and implementation, guidance for teaching and learning practice may become a challenge. On the one hand, the black box created by machine learning algorithms and models prevents users from obtaining educationally meaningful learning and teaching suggestions. On the other hand, focusing on group and cohort level analysis only can make it difficult to provide specific support for individual students working in collaborative groups. This paper proposes a transparent approach to automatically detect student's individual engagement in the process of collaboration. The results show that the proposed approach can reflect student's individual engagement and can be used as an indicator to distinguish students with different collaborative learning challenges (cognitive, behavioural and emotional) and learning outcomes. The potential of the proposed collaboration analytics approach for scaffolding collaborative learning practice in face-to-face contexts is discussed and future research suggestions are provided.

cs.CY↗

arXiv2024

Predicting challenge moments from students' discourse: A comparison of GPT-4 to two traditional natural language processing approaches

Effective collaboration requires groups to strategically regulate themselves to overcome challenges. Research has shown that groups may fail to regulate due to differences in members' perceptions of challenges which may benefit from external support. In this study, we investigated the potential of leveraging three distinct natural language processing models: an expert knowledge rule-based model, a supervised machine learning (ML) model and a Large Language model (LLM), in challenge detection and challenge dimension identification (cognitive, metacognitive, emotional and technical/other challenges) from student discourse, was investigated. The results show that the supervised ML and the LLM approaches performed considerably well in both tasks, in contrast to the rule-based approach, whose efficacy heavily relies on the engineered features by experts. The paper provides an extensive discussion of the three approaches' performance for automated detection and support of students' challenge moments in collaborative learning activities. It argues that, although LLMs provide many advantages, they are unlikely to be the panacea to issues of the detection and feedback provision of socially shared regulation of learning due to their lack of reliability, as well as issues of validity evaluation, privacy and confabulation. We conclude the paper with a discussion on additional considerations, including model transparency to explore feasible and meaningful analytical feedback for students and educators using LLMs.

cs.CL↗

arXiv2023

Prototyping the use of Large Language Models (LLMs) for adult learning content creation at scale

As Large Language Models (LLMs) and other forms of Generative AI permeate various aspects of our lives, their application for learning and education has provided opportunities and challenges. This paper presents an investigation into the use of LLMs in asynchronous course creation, particularly within the context of adult learning, training and upskilling. We developed a course prototype leveraging an LLM, implementing a robust human-in-the-loop process to ensure the accuracy and clarity of the generated content. Our research questions focus on the feasibility of LLMs to produce high-quality adult learning content with reduced human involvement. Initial findings indicate that taking this approach can indeed facilitate faster content creation without compromising on accuracy or clarity, marking a promising advancement in the field of Generative AI for education. Despite some limitations, the study underscores the potential of LLMs to transform the landscape of learning and education, necessitating further research and nuanced discussions about their strategic and ethical use in learning design.

cs.CY↗

arXiv2023

M2LADS: A System for Generating MultiModal Learning Analytics Dashboards in Open Education

In this article, we present a Web-based System called M2LADS, which supports the integration and visualization of multimodal data recorded in learning sessions in a MOOC in the form of Web-based Dashboards. Based on the edBB platform, the multimodal data gathered contains biometric and behavioral signals including electroencephalogram data to measure learners' cognitive attention, heart rate for affective measures, visual attention from the video recordings. Additionally, learners' static background data and their learning performance measures are tracked using LOGCE and MOOC tracking logs respectively, and both are included in the Web-based System. M2LADS provides opportunities to capture learners' holistic experience during their interactions with the MOOC, which can in turn be used to improve their learning outcomes through feedback visualizations and interventions, as well as to enhance learning analytics models and improve the open content of the MOOC.

cs.HC↗

arXiv2023

Generative AI for learning: Investigating the potential of synthetic learning videos

Recent advances in generative artificial intelligence (AI) have captured worldwide attention. Tools such as Dalle-2 and ChatGPT suggest that tasks previously thought to be beyond the capabilities of AI may now augment the productivity of creative media in various new ways, including through the generation of synthetic video. This research paper explores the utility of using AI-generated synthetic video to create viable educational content for online educational settings. To date, there is limited research investigating the real-world educational value of AI-generated synthetic media. To address this gap, we examined the impact of using AI-generated synthetic video in an online learning platform on both learners content acquisition and learning experience. We took a mixed-method approach, randomly assigning adult learners (n=83) into one of two micro-learning conditions, collecting pre- and post-learning assessments, and surveying participants on their learning experience. The control condition included a traditionally produced instructor video, while the experimental condition included a synthetic video with a realistic AI-generated character. The results show that learners in both conditions demonstrated significant improvement from pre- to post-learning (p<.001), with no significant differences in gains between the two conditions (p=.80). In addition, no differences were observed in how learners perceived the traditional and synthetic videos. These findings suggest that AI-generated synthetic learning videos have the potential to be a viable substitute for videos produced via traditional methods in online educational settings, making high quality educational content more accessible across the globe.

cs.CV↗

arXiv2023

Adoption of Artificial Intelligence in Schools: Unveiling Factors Influencing Teachers Engagement

Albeit existing evidence about the impact of AI-based adaptive learning platforms, their scaled adoption in schools is slow at best. In addition, AI tools adopted in schools may not always be the considered and studied products of the research community. Therefore, there have been increasing concerns about identifying factors influencing adoption, and studying the extent to which these factors can be used to predict teachers engagement with adaptive learning platforms. To address this, we developed a reliable instrument to measure more holistic factors influencing teachers adoption of adaptive learning platforms in schools. In addition, we present the results of its implementation with school teachers (n=792) sampled from a large country-level population and use this data to predict teachers real-world engagement with the adaptive learning platform in schools. Our results show that although teachers knowledge, confidence and product quality are all important factors, they are not necessarily the only, may not even be the most important factors influencing the teachers engagement with AI platforms in schools. Not generating any additional workload, in-creasing teacher ownership and trust, generating support mechanisms for help, and assuring that ethical issues are minimised, are also essential for the adoption of AI in schools and may predict teachers engagement with the platform better. We conclude the paper with a discussion on the value of factors identified to increase the real-world adoption and effectiveness of adaptive learning platforms by increasing the dimensions of variability in prediction models and decreasing the implementation variability in practice.

cs.CY↗

arXiv2022

Impact of combining human and analytics feedback on students' engagement with, and performance in, reflective writing tasks

Reflective writing is part of many higher education courses across the globe. It is often considered a challenging task for students as it requires self-regulated learning skills to appropriately plan, timely engage and deeply reflect on learning experiences. Despite an advance in writing analytics and the pervasiveness of human feedback aimed to support student reflections, little is known about how to integrate feedback from humans and analytics to improve students' learning engagement and performance in reflective writing tasks. This study proposes a personalised behavioural feedback intervention based on students' writing engagement analytics utilising time-series analysis of digital traces from a ubiquitous online word processing platform. In a semester-long experimental study involving 81 postgraduate students, its impact on learning engagement and performance was studied. The results showed that the intervention cohort engaged statistically significantly more in their reflective writing task after receiving the combined feedback compared to the control cohort which only received human feedback on their reflective writing content. Further analyses revealed that the intervention cohort reflected more regularly at the weekly level, the regularity of weekly reflection led to better performance grades, and the impact on students with low self-regulated learning skills was higher. This study emphasizes the powerful benefits of implementing combined feedback approaches in which the strengths of analytics and human feedback are synthesized to improve student engagement and performance. Further research should explore the long-term sustainability of the observed effects and their validity in other contexts.

cs.CY↗

arXiv2022

An Artificial Intelligence driven Learning Analytics Method to Examine the Collaborative Problem solving Process from a Complex Adaptive Systems Perspective

Collaborative problem solving (CPS) enables student groups to complete learning tasks, construct knowledge, and solve problems. Previous research has argued the importance to examine the complexity of CPS, including its multimodality, dynamics, and synergy from the complex adaptive systems perspective. However, there is limited empirical research examining the adaptive and temporal characteristics of CPS which might lead to an oversimplified representation of the real complexity of the CPS process. To further understand the nature of CPS in online interaction settings, this research collected multimodal process and performance data (i.e., verbal audios, computer screen recordings, concept map data) and proposed a three-layered analytical framework that integrated AI algorithms with learning analytics to analyze the regularity of groups collaboration patterns. The results detected three types of collaborative patterns in groups, namely the behaviour-oriented collaborative pattern (Type 1) associated with medium-level performance, the communication - behaviour - synergistic collaborative pattern (Type 2) associated with high-level performance, and the communication-oriented collaborative pattern (Type 3) associated with low-level performance. The research further highlighted the multimodal, dynamic, and synergistic characteristics of groups collaborative patterns to explain the emergence of an adaptive, self-organizing system during the CPS process.

cs.AI↗

arXiv2022

A Transparency Index Framework for AI in Education

Numerous AI ethics checklists and frameworks have been proposed focusing on different dimensions of ethical AI such as fairness, explainability, and safety. Yet, no such work has been done on developing transparent AI systems for real-world educational scenarios. This paper presents a Transparency Index framework that has been iteratively co-designed with different stakeholders of AI in education, including educators, ed-tech experts, and AI practitioners. We map the requirements of transparency for different categories of stakeholders of AI in education and demonstrate that transparency considerations are embedded in the entire AI development process from the data collection stage until the AI system is deployed in the real world and iteratively improved. We also demonstrate how transparency enables the implementation of other ethical AI dimensions in Education like interpretability, accountability, and safety. In conclusion, we discuss the directions for future research in this newly emerging field. The main contribution of this study is that it highlights the importance of transparency in developing AI-powered educational technologies and proposes an index framework for its conceptualization for AI in education.

cs.CY↗

arXiv2022

A systematic review of physical-digital play technology and developmentally relevant child behaviour

New interactive physical-digital play technologies are shaping the way children plan. These technologies refer to digital play technologies that engage children in analogue forms of behaviour, either alone or with others. Current interactive physical-digital play technologies include robots, digital agents, mixed or augmented reality devices, and smart-eye based gaming. Little is known, however, about the ways in which these technologies could promote or damage child development. This systematic review was aimed at understanding if and how these physical-digital play technologies promoted developmentally relevant behaviour in typically developing 0 to 12 year-olds. Psychology, Education, and Computer Science databases were searched producing 635 paper. A total of 31 papers met the inclusion criteria, of which 17 were of high enough quality to be included for synthesis. Results indicate that these new interactive play technologies could have a positive effect on children's developmentally relevant behaviour. The review indicated specific ways in which different behaviour were promoted. Providing information about own performance promoted self-monitoring. Slowing interactivity, play interdependency, and joint object accessibility promoted collaboration. Offering delimited choices promoted decision making. Problem solving and physical activity were promoted by requiring children to engage in them to keep playing. Four principles underpinned the ways in which physical digital play technologies afforded child behaviour. These included social expectations framing play situations, the directiveness of action regulations (inviting, guiding or forcing behaviours), the technical features of play technologies (digital play mechanics and physical characteristics), and the alignment between play goals, play technology and the play behaviours promoted.

cs.HC↗