arXiv subjects

Shoeb Joarder

Publications and source records attributed to Shoeb Joarder.

13 recordsLinked to original sources

arXiv2026

Investigating the Effects of Different Levels of User Control in an Interactive Educational Recommender System

Educational recommender systems (ERSs) are becoming increasingly important in enhancing educational outcomes and personalizing learning experiences by providing recommendations of personalized resources and activities to learners, tailored to their individual learning needs. While user control is widely assumed to improve user experience, the effects of different levels of control in ERSs remain underexplored. To address this gap, we designed and evaluated an interactive ERS within the MOOC platform CourseMapper, where learners could interact with the input (i.e., user profile), process (i.e., recommendation algorithm), and output (i.e., recommendations) of the system. We conducted a between-subjects user study (N=184) to examine how varying levels of user control in an ERS influenced users' perceptions of the recommendation goals of perceived control, transparency, trust, satisfaction, and perceived quality. Our results show that enabling users to build and refine their profile is sufficient to promote positive perceptions of the ERS, while additional control options mainly reinforce these impressions. Moreover, perceived control is the only goal significantly affected by providing different levels of user control in the ERS, with input control exerting the strongest influence. Furthermore, different levels of control affect transparency, trust, satisfaction, and perceived quality in distinct yet interconnected ways. Overall, the findings provide empirical evidence that user control positively shapes transparency, trust, satisfaction, and perceived quality, though to varying extents.

cs.HC↗

arXiv2026

Usability Evaluation and Improvement of a Tool for Self-Service Learning Analytics

Self-Service Learning Analytics (SSLA) tools aim to support educational stakeholders in creating learning analytics indicators without requiring technical expertise. While such tools promise user control and trans- parency, their effectiveness and adoption depend critically on usability aspects. This paper presents a compre- hensive usability evaluation and improvement of the Indicator Editor, a no-code, exploratory SSLA tool that enables non-technical users to implement custom learning analytics indicators through a structured workflow. Using an iterative evaluation approach, we conduct an exploratory qualitative user study, usability inspections of high-fidelity prototypes, and a workshop-based evaluation in an authentic educational setting with n = 46 students using standardized instruments, namely System Usability Scale (SUS), User Experience Question- naire (UEQ), and Net Promoter Score (NPS). Based on the evaluation findings, we derive concrete design implications that inform improvements in workflow guidance, feedback, and information presentation in the Indicator Editor. Furthermore, our evaluation provides practical insights for the design of usable SSLA tools.

cs.CY↗

arXiv2025

Human-Centered Development of Indicators for Self-Service Learning Analytics: A Transparency through Exploration Approach

The aim of learning analytics is to turn educational data into insights, decisions, and actions to improve learning and teaching. The reasoning of the provided insights, decisions, and actions is often not transparent to the end-user, and this can lead to trust and acceptance issues when interventions, feedback, and recommendations fail. In this paper, we shed light on achieving transparent learning analytics by following a transparency through exploration approach. To this end, we present the design, implementation, and evaluation details of the Indicator Editor, which aims to support self-service learning analytics (SSLA) by empowering end-users to take control of the indicator implementation process. We systematically designed and implemented the Indicator Editor through an iterative human-centered design (HCD) approach. Further, we conducted a qualitative user study (n=15) to investigate the impact of following an SSLA approach on the users' perception of and interaction with the Indicator Editor. Our study showed qualitative evidence that supporting user interaction and providing user control in the indicator implementation process can have positive effects on different crucial aspects of learning analytics, namely transparency, trust, satisfaction, and acceptance.

cs.CY↗

arXiv2025

An Optimized Pipeline for Automatic Educational Knowledge Graph Construction

The automatic construction of Educational Knowledge Graphs (EduKGs) is essential for domain knowledge modeling by extracting meaningful representations from learning materials. Despite growing interest, identifying a scalable and reliable approach for automatic EduKG generation remains a challenge. In an attempt to develop a unified and robust pipeline for automatic EduKG construction, in this study we propose a pipeline for automatic EduKG construction from PDF learning materials. The process begins with generating slide-level EduKGs from individual pages/slides, which are then merged to form a comprehensive EduKG representing the entire learning material. We evaluate the accuracy of the EduKG generated from the proposed pipeline in our MOOC platform, CourseMapper. The observed accuracy, while indicative of partial success, is relatively low particularly in the educational context, where the reliability of knowledge representations is critical for supporting meaningful learning. To address this, we introduce targeted optimizations across multiple pipeline components. The optimized pipeline achieves a 17.5% improvement in accuracy and a tenfold increase in processing efficiency. Our approach offers a holistic, scalable and end-to-end pipeline for automatic EduKG construction, adaptable to diverse educational contexts, and supports improved semantic representation of learning content.

cs.CY↗

arXiv2025

Top-Down vs. Bottom-Up Approaches for Automatic Educational Knowledge Graph Construction in CourseMapper

The automatic construction of Educational Knowledge Graphs (EduKGs) is crucial for modeling domain knowledge in digital learning environments, particularly in Massive Open Online Courses (MOOCs). However, identifying the most effective approach for constructing accurate EduKGs remains a challenge. This study compares Top-down and Bottom-up approaches for automatic EduKG construction, evaluating their effectiveness in capturing and structuring knowledge concepts from learning materials in our MOOC platform CourseMapper. Through a user study and expert validation using Simple Random Sampling (SRS), results indicate that the Bottom-up approach outperforms the Top-down approach in accurately identifying and mapping key knowledge concepts. To further enhance EduKG accuracy, we integrate a Human-in-the-Loop approach, allowing course moderators to review and refine the EduKG before publication. This structured comparison provides a scalable framework for improving knowledge representation in MOOCs, ultimately supporting more personalized and adaptive learning experiences.

cs.CY↗

arXiv2025

Leveraging Graph Retrieval-Augmented Generation to Support Learners' Understanding of Knowledge Concepts in MOOCs

Massive Open Online Courses (MOOCs) lack direct interaction between learners and instructors, making it challenging for learners to understand new knowledge concepts. Recently, learners have increasingly used Large Language Models (LLMs) to support them in acquiring new knowledge. However, LLMs are prone to hallucinations which limits their reliability. Retrieval-Augmented Generation (RAG) addresses this issue by retrieving relevant documents before generating a response. However, the application of RAG across different MOOCs is limited by unstructured learning material. Furthermore, current RAG systems do not actively guide learners toward their learning needs. To address these challenges, we propose a Graph RAG pipeline that leverages Educational Knowledge Graphs (EduKGs) and Personal Knowledge Graphs (PKGs) to guide learners to understand knowledge concepts in the MOOC platform CourseMapper. Specifically, we implement (1) a PKG-based Question Generation method to recommend personalized questions for learners in context, and (2) an EduKG-based Question Answering method that leverages the relationships between knowledge concepts in the EduKG to answer learner selected questions. To evaluate both methods, we conducted a study with 3 expert instructors on 3 different MOOCs in the MOOC platform CourseMapper. The results of the evaluation show the potential of Graph RAG to empower learners to understand new knowledge concepts in a personalized learning experience.

cs.AI↗

arXiv2025

The ISC Creator: Human-Centered Design of Learning Analytics Interactive Indicator Specification Cards

Emerging research on human-centered learning analytics (HCLA) has demonstrated the importance of involving diverse stakeholders in co-designing learning analytics (LA) systems. However, there is still a demand for effective and efficient methods to co-design LA dashboards and indicators. Indicator Specification Cards (ISCs) have been introduced recently to facilitate the systematic co-design of indicators by different LA stakeholders. In this paper, we strive to enhance the user experience and usefulness of the ISC-based indicator design process. Towards this end, we present the systematic design, implementation, and evaluation details of the ISC Creator, an interactive LA tool that allows low-cost and flexible design of LA indicators. Our findings demonstrate the importance of carefully considered interactivity and recommendations for orienting and supporting non-expert LA stakeholders to design custom LA indicators.

cs.CY↗

arXiv2025

Designing and Evaluating an Educational Recommender System with Different Levels of User Control

Educational recommender systems (ERSs) play a crucial role in personalizing learning experiences and enhancing educational outcomes by providing recommendations of personalized resources and activities to learners, tailored to their individual learning needs. However, their effectiveness is often diminished by insufficient user control and limited transparency. To address these challenges, in this paper, we present the systematic design and evaluation of an interactive ERS, in which we introduce different levels of user control. Concretely, we introduce user control around the input (i.e., user profile), process (i.e., recommendation algorithm), and output (i.e., recommendations) of the ERS. To evaluate our system, we conducted an online user study (N=30) to explore the impact of user control on users' perceptions of the ERS in terms of several important user-centric aspects. Moreover, we investigated the effects of user control on multiple recommendation goals, namely transparency, trust, and satisfaction, as well as the interactions between these goals. Our results demonstrate the positive impact of user control on user perceived benefits of the ERS. Moreover, our study shows that user control strongly correlates with transparency and moderately correlates with trust and satisfaction. In terms of interaction between these goals, our results reveal that transparency moderately correlates and trust strongly correlates with satisfaction. Whereas, transparency and trust stand out as less correlated with each other.

cs.IR↗

arXiv2023

Interactive Explanation with Varying Level of Details in an Explainable Scientific Literature Recommender System

Explainable recommender systems (RS) have traditionally followed a one-size-fits-all approach, delivering the same explanation level of detail to each user, without considering their individual needs and goals. Further, explanations in RS have so far been presented mostly in a static and non-interactive manner. To fill these research gaps, we aim in this paper to adopt a user-centered, interactive explanation model that provides explanations with different levels of detail and empowers users to interact with, control, and personalize the explanations based on their needs and preferences. We followed a user-centered approach to design interactive explanations with three levels of detail (basic, intermediate, and advanced) and implemented them in the transparent Recommendation and Interest Modeling Application (RIMA). We conducted a qualitative user study (N=14) to investigate the impact of providing interactive explanations with varying level of details on the users' perception of the explainable RS. Our study showed qualitative evidence that fostering interaction and giving users control in deciding which explanation they would like to see can meet the demands of users with different needs, preferences, and goals, and consequently can have positive effects on different crucial aspects in explainable recommendation, including transparency, trust, satisfaction, and user experience.

cs.IR↗

arXiv2023

Justification vs. Transparency: Why and How Visual Explanations in a Scientific Literature Recommender System

Significant attention has been paid to enhancing recommender systems (RS) with explanation facilities to help users make informed decisions and increase trust in and satisfaction with the RS. Justification and transparency represent two crucial goals in explainable recommendation. Different from transparency, which faithfully exposes the reasoning behind the recommendation mechanism, justification conveys a conceptual model that may differ from that of the underlying algorithm. An explanation is an answer to a question. In explainable recommendation, a user would want to ask questions (referred to as intelligibility types) to understand results given by the RS. In this paper, we identify relationships between Why and How explanation intelligibility types and the explanation goals of justification and transparency. We followed the Human-Centered Design (HCD) approach and leveraged the What-Why-How visualization framework to systematically design and implement Why and How visual explanations in the transparent Recommendation and Interest Modeling Application (RIMA). Furthermore, we conducted a qualitative user study (N=12) to investigate the potential effects of providing Why and How explanations together in an explainable RS on the users' perceptions regarding transparency, trust, and satisfaction. Our study showed qualitative evidence confirming that the choice of the explanation intelligibility types depends on the explanation goal and user type.

cs.IR↗

arXiv2023

Towards a Flexible User Interface for 'Quick and Dirty' Learning Analytics Indicator Design

Research on Human-Centered Learning Analytics (HCLA) has provided demonstrations of a successful co-design process for LA tools with different stakeholders. However, there is a need for 'quick and dirty' methods to allow the low-cost design of LA indicators. Recently, Indicator Specification Cards (ISC) have been proposed to help different learning analytics stakeholders co-design indicators in a systematic manner. In this paper, we aim at improving the user experience, flexibility, and reliability of the ISC-based indicator design process. To this end, we present the development details of an intuitive and theoretically-sound ISC user interface that allows the low-cost design of LA indicators. Further, we propose two approaches to support the flexible design of indicators, namely a task-driven approach and a data-driven approach.

cs.CY↗

arXiv2023

Is More Always Better? The Effects of Personal Characteristics and Level of Detail on the Perception of Explanations in a Recommender System

Despite the acknowledgment that the perception of explanations may vary considerably between end-users, explainable recommender systems (RS) have traditionally followed a one-size-fits-all model, whereby the same explanation level of detail is provided to each user, without taking into consideration individual user's context, i.e., goals and personal characteristics. To fill this research gap, we aim in this paper at a shift from a one-size-fits-all to a personalized approach to explainable recommendation by giving users agency in deciding which explanation they would like to see. We developed a transparent Recommendation and Interest Modeling Application (RIMA) that provides on-demand personalized explanations of the recommendations, with three levels of detail (basic, intermediate, advanced) to meet the demands of different types of end-users. We conducted a within-subject study (N=31) to investigate the relationship between user's personal characteristics and the explanation level of detail, and the effects of these two variables on the perception of the explainable RS with regard to different explanation goals. Our results show that the perception of explainable RS with different levels of detail is affected to different degrees by the explanation goal and user type. Consequently, we suggested some theoretical and design guidelines to support the systematic design of explanatory interfaces in RS tailored to the user's context.

cs.AI↗

arXiv2023

Designing Theory-Driven Analytics-Enhanced Self-Regulated Learning Applications

There is an increased interest in the application of learning analytics (LA) to promote self-regulated learning (SRL). A variety of LA dashboards and indicators were proposed to support different crucial SRL processes, such as planning, awareness, self-reflection, self-monitoring, and feedback. However, the design of these dashboards and indicators is often without reference to theories in learning science, human-computer interaction (HCI), and information visualization (InfoVis). Moreover, there is a lack of theoretically sound frameworks to guide the systematic design and development of LA dashboards and indicators to scaffold SRL. This chapter seeks to explore theoretical underpinnings of the design of LA-enhanced SRL applications, drawing from the fields of learning science, HCI, and InfoVis. We first present the Student-Centered Learning Analytics-enhanced Self-Regulated Learning (SCLA-SRL) methodology for building theory-driven LA-enhanced SRL applications for and with learners. We then put this methodology into practice by designing and developing LA indicators to support novice programmers' SRL in a higher education context.

cs.CY↗