arXiv subjects

Jiazi Hu

Publications and source records attributed to Jiazi Hu.

5 recordsLinked to original sources

arXiv2026

Beyond Benchmarking: Scenario-Based Evaluation of Large Language Models for Personalized Learning

While large language models (LLMs) are increasingly being adopted to support personalized learning, there remains limited understanding of how their pedagogical behaviors differ in authentic learning scenarios. Existing evaluation practices often emphasize benchmark scores and overall model rankings, but such approaches usually provide limited insight into how LLMs diagnose student understanding and generate personalized guidance. This study proposes a scenario-based evaluation framework for closely examining LLM behavior in personalized learning support. Using a post-class tutoring setting as an illustrative example, a dataset comprising a student's responses to a set of data structures questions is provided to multiple LLMs. Each model is required to identify the underlying knowledge concepts, infer the student's mastery profile, and generate personalized guidance for improvement. To support consistent, reproducible and scalable comparison, Gemini is employed as an external evaluator across multiple pedagogically relevant dimensions, including diagnostic accuracy, instructional clarity, actionability, misconception identification, and appropriateness to the student's level. The resulting pairwise preferences are then fitted using the Bradley-Terry model to derive comparative strength estimates, while qualitative analysis and semantic visualization are used to further examine differences in feedback structure, diagnostic depth, and recommendation specificity. The key findings show that different LLMs exhibit distinguishable pedagogical behaviors within the same learning scenario and demonstrates how scenario-based evaluation can provide educationally meaningful signals for understanding model behavior in AI-enhanced education.

cs.AI↗

arXiv2026

Consistent Evidence, Robust Recognition: Faithful Attribution Regularization under Geometric Transformations

Attribution methods are widely used to characterize the evidence underlying model predictions, yet their potential to improve model behavior remains underexplored. Attribution inconsistency under label-preserving geometric transformations may indicate transformation-sensitive evidence reliance, motivating attribution regularization. However, such supervision is valid only when attribution faithfully reflects the evidence driving predictions. Existing self-supervised methods typically align gradient-based maps such as Grad-CAM, whose limited faithfulness means that attribution consistency need not imply consistency of the underlying decision process, leaving transformation robustness unresolved. We propose an annotation-free attribution regularization framework based on submodular search over image regions. By measuring how candidate subsets affect model outputs, the search extracts compact, class-discriminative evidence as search-derived supervision. We further introduce a submodular ranking loss with path-consistency and termination-alignment terms that respectively align spatially corresponding candidate rankings along paired search trajectories and encourage the transformed trajectory to satisfy the stopping criterion at the target terminal step. The loss provides a differentiable surrogate for regularizing both final attributions and the otherwise discrete evidence-selection process. Experiments on ImageNet-100 show that our method substantially improves attribution stability, Insertion, and Deletion on ViT-B/16 with only a 0.28-point accuracy drop, with similar gains on ViT-L/16. On ImageNet-1K, it improves transformed-input accuracy on ResNet-50 and ConvNeXt-B while limiting the clean-accuracy drop to 0.30 points, demonstrating more consistent evidence reliance with minimal performance loss. Code will be released soon.

cs.CV↗

arXiv2026

Bridging MOOCs, Smart Teaching, and AI: A Decade of Evolution Toward a Unified Pedagogy

Over the past decade, higher education has undergone successive shifts driven by three major developments: Massive Open Online Courses (MOOCs), Smart Teaching technologies, and AI-enhanced learning. Each paradigm emerged to address specific limitations of traditional education: MOOCs enable ubiquitous access to learning resources; Smart Teaching supports real-time interaction with data-driven insights; and generative AI offers scalable personalization and on-demand content generation. However, these paradigms are often adopted in isolation, limiting their systemic pedagogical potential. This paper proposes a unified instructional framework that integrates these approaches under a coherent teaching-driven logic. The framework distinguishes three complementary dimensions of instructional design: structured exposure (MOOCs), adaptive allocation (Smart Teaching), and efficiency amplification (AI). To operationalize this integration, we formalize the framework as a layered knowledge transformation model and illustrate its behavior through a step-by-step learning example. The results demonstrate how each layer contributes to measurable and functionally distinct gains in knowledge mastery.

cs.CY↗

arXiv2025

An Exploration of Higher Education Course Evaluation by Large Language Models

Course evaluation plays a critical role in ensuring instructional quality and guiding curriculum development in higher education. However, traditional evaluation methods, such as student surveys, classroom observations, and expert reviews, are often constrained by subjectivity, high labor costs, and limited scalability. With recent advancements in large language models (LLMs), new opportunities have emerged for generating consistent, fine-grained, and scalable course evaluations. This study investigates the use of three representative LLMs for automated course evaluation at both the micro level (classroom discussion analysis) and the macro level (holistic course review). Using classroom interaction transcripts and a dataset of 100 courses from a major institution in China, we demonstrate that LLMs can extract key pedagogical features and generate structured evaluation results aligned with expert judgement. A fine-tuned version of Llama shows superior reliability, producing score distributions with greater differentiation and stronger correlation with human evaluators than its counterparts. The results highlight three major findings: (1) LLMs can reliably perform systematic and interpretable course evaluations at both the micro and macro levels; (2) fine-tuning and prompt engineering significantly enhance evaluation accuracy and consistency; and (3) LLM-generated feedback provides actionable insights for teaching improvement. These findings illustrate the promise of LLM-based evaluation as a practical tool for supporting quality assurance and educational decision-making in large-scale higher education settings.

cs.CL↗

arXiv2025

Generative AI as a Tool for Enhancing Reflective Learning in Students

Reflection is widely recognized as a cornerstone of student development, fostering critical thinking, self-regulation, and deep conceptual understanding. Traditionally, reflective skills have been cultivated through structured feedback, mentorship, and guided self-assessment. However, these approaches often face challenges such as limited scalability, difficulties in delivering individualized feedback, and a shortage of instructors proficient in facilitating meaningful reflection. This study pioneers the use of generative AI, specifically large language models (LLMs), as an innovative solution to these limitations. By leveraging the capacity of LLMs to deliver personalized, context-sensitive feedback at scale, this research investigates their potential to serve as effective facilitators of reflective exercises, sustaining deep engagement and promoting critical thinking. Through in-depth analyses of prompt engineering strategies and simulated multi-turn dialogues grounded in a project-based learning (PBL) context, the study demonstrates that, with pedagogically aligned prompts, LLMs can serve as accessible and adaptive tools for scalable reflective guidance. Furthermore, LLM-assisted evaluation is employed to objectively assess the performance of both tutors and students across multiple dimensions of reflective learning. The findings contribute to the evolving understanding of AI's role in reflective pedagogy and point to new opportunities for advancing AI-driven intelligent tutoring systems.

cs.HC↗