arXiv subjects

Xiaoyi Tian

Publications and source records attributed to Xiaoyi Tian.

At least 19 recordsLinked to original sources

arXiv2026

Will It Teach as Intended? How Teachers Configure Educational AI Chatbots

Teachers are increasingly using generative AI to support instruction, yet it remains unclear how pedagogical intentions are translated into chatbot configurations and reflected in chatbot behavior. We studied a teacher-facing chatbot authoring tool in professional development workshops with 27 middle school teachers, analyzing focus-group interviews alongside configuration and interaction logs. Teachers envisioned chatbots as instructional scaffolds that could provide differentiated support, extend access to assistance, and preserve student thinking within teacher-defined boundaries. Configuration analysis showed that Purpose primarily captured instructional goals and content focus, whereas Rules more often specified pedagogical behavior, guardrails, and learner-specific adaptations. Log-based evaluation showed stronger alignment for responsiveness (88.9%) and persona (81.5%) than for rules (70.4%) and purpose (59.3%). These findings show that configurable controls alone do not ensure pedagogical fidelity and highlight the need for authoring tools that help teachers express, test, and refine intended chatbot behavior.

cs.HC↗

arXiv2026

How Children Design and Reason about Trustworthy AI Chatbots

Children increasingly interact with AI chatbots, making trust calibration essential to AI literacy. Prior research has examined children's trust in AI mainly as users evaluating systems built by others, rather than as designers of their own chatbots. We developed a chatbot-building environment with adjustable trust-relevant traits (e.g., confidence, transparency, formality, assertiveness), rules, and persona. We conducted mixed-methods study with 115 learners (ages 8-18) who made 119 chatbots. We examined how children configured their chatbots, reasoned about trustworthiness, and how closely chatbot behavior aligned with their designs. Younger students (age 10-13) set significantly higher confidence than older students (age 14-18), and some deliberately built chatbots that gave wrong answers on purpose, yet still called them trustworthy, arguing that a chatbot does what it was built to do. Younger students equated trust with purpose-fulfillment, while older students linked it to transparent, calibrated design. Students also calibrated academic chatbots to be more transparent and formal than hobby chatbots. We identify seven design dimensions describing what children believe makes a chatbot trustworthy, and discuss implications for AI literacy tools.

cs.HC↗

arXiv2026

Confirming Correct, Missing the Rest: LLM Tutoring Agents Struggle Where Feedback Matters Most

Effective tutoring requires distinguishing optimal, valid but suboptimal, and incorrect student solutions, a distinction central to intelligent tutoring systems (ITS) but untested for LLM-based tutors. As LLMs are increasingly explored as conversational complements to ITS, evaluating their diagnostic precision is essential. We present a benchmark of seven LLM feedback agents in propositional logic using knowledge-graph-derived ground truth across 10,836 solution--feedback pairs and three feedback conditions. Models achieved near-ceiling performance on optimal steps but systematically over-rejected valid but suboptimal reasoning and over-validated incorrect solutions, precisely where adaptive tutoring matters most. These failures persisted across models regardless of solution context, suggesting architectural rather than informational limits. Moreover, accurate diagnosis did not reliably produce pedagogically actionable feedback, revealing a gap between diagnostic judgment and instructional effectiveness. Our findings suggest that LLMs are better suited for hybrid architectures where KG-grounded models handle diagnosis while LLMs support open-ended scaffolding and dialogue.

cs.AI↗

arXiv2026

Analyzing Middle School Students' Dialogue and Behaviors during Collaborative AI Chatbot Development Using Ordered Network Analysis

As Artificial Intelligence (AI) education has become a key component of K-12 curricula, activities such as designing and developing conversational agents are increasingly used as instructional practice. Prior work has primarily examined these activities by focusing on students' learning outcomes or the quality of final AI artifacts, offering limited insight into the collaborative processes through which learning unfolds during AI system development. Although the AIED community has a long history of studying collaborative learning in STEM and Computing education, the emergence of AI learning environments in which students build AI systems presents new opportunities to understand how collaboration unfolds in AI education contexts. Grounded in these foundational works, the current study examines collaborative interaction among middle school students engaged in the design and development of an AI chatbot. Using Ordered Network Analysis of students' dialogue and development actions, we characterize how collaboration is organized over time and how interaction patterns relate to chatbot quality and AI knowledge outcomes. Results reveal that higher-quality chatbots are associated with more integrated sequences linking explanation, testing, and refinement. Interaction patterns involving articulated reasoning and repeated testing and revision in response to chatbot output were also associated with stronger AI knowledge outcomes. These findings provide a process-oriented account of collaborative AI chatbot development and extend AIED research on collaborative learning processes to AI education contexts.

cs.HC↗

arXiv2026

Some applications of the matched projections of idempotents

For every idempotent $Q$ on a Hilbert space $H$, the matched projection $m(Q)$ is a well-established concept. This paper explores several applications of the matched projections. The first application addresses the distances from projections on $H$ to a given idempotent $Q$. Using $m(Q)$, a complete characterization of these distances is established, covering the minimum, maximum, and intermediate values. The second application focuses on the $C^*$-algebra $C^*\{Q\}$ generated by a single non-projection idempotent $Q$. A new $4\times 4$ block matrix representation of $Q$, induced by $m(Q)$, yields novel formulas for $Q$, leading to a full characterization of all elements in $C^*\{Q\}$ via explicit $4\times 4$ block matrices. Furthermore, for each $r>1$, a family of universal $r$-idempotents is introduced. These idempotents possess a universal property distinct from known properties of projection pairs. Some necessary and sufficient conditions are provided for such universal $r$-idempotents. The third application presents new characterizations of the numerical ranges. An operator version of the elliptical range theorem is established. Using a general non-projection idempotent $Q$ and its matched projection $m(Q)$, a non-quadratic operator is constructed, and its numerical range is described in detail. Additionally, another operator is introduced whose numerical range closure is not an elliptical disk, and the numerical range itself is neither closed nor open.

math.FA↗

arXiv2026

When Verification Hurts: Asymmetric Effects of Multi-Agent Feedback in Logic Proof Tutoring

Large language models (LLMs) are increasingly used for automated tutoring, but their reliability in structured symbolic domains remains unclear. We study step-level feedback for propositional logic proofs, which require precise symbolic reasoning aligned with a learner's current proof state. We introduce a knowledge-graph-grounded benchmark of 516 unique proof states with step-level annotations and difficulty metrics. Unlike prior tutoring evaluations that rely on model self-assessment or binary correctness, our framework enables fine-grained analysis of feedback quality against verified solution paths. We evaluate three role-specialized pipelines with varying solution access: Tutor (partial solution access), Teacher (full derivation access), and Judge (verification of Tutor feedback). Our results reveal a striking asymmetry: verification improves outcomes when upstream feedback is error-prone (<70% accuracy), but degrades performance by 4-6 percentage points through over-specification when feedback is already reliable (>85%). Critically, we identify a shared complexity ceiling; no model or pipeline reliably succeeds on proof states exceeding complexity 4-5. These findings challenge the assumption that adding verifiers or richer context universally improves tutoring, motivating adaptive, difficulty-aware architectures that route problems by estimated complexity and upstream reliability.

cs.AI↗

arXiv2026

Data-Driven Hints in Intelligent Tutoring Systems

This chapter explores the evolution of data-driven hint generation for intelligent tutoring systems (ITS). The Hint Factory and Interaction Networks have enabled the generation of next-step hints, waypoints, and strategic subgoals from historical student data. Data-driven techniques have also enabled systems to find the right time to provide hints. We explore further potential data-driven adaptations for problem solving based on behavioral problem solving data and the integration of Large Language Models (LLMs).

cs.AI↗

arXiv2026

Exploring Teacher-Chatbot Interaction and Affect in Block-Based Programming

AI-based chatbots have the potential to accelerate learning and teaching, but may also have counterproductive consequences without thoughtful design and scaffolding. To better understand teachers' perspectives on large language model (LLM)-based chatbots, we conducted a study with 11 teams of middle school teachers using chatbots for a science and computational thinking activity within a block-based programming environment. Based on a qualitative analysis of audio transcripts and chatbot interactions, we propose three profiles: explorer, frustrated, and mixed, that reflect diverse scaffolding needs. In their discussions, we found that teachers perceived chatbot benefits such as building prompting skills and self-confidence alongside risks including potential declines in learning and critical thinking. Key design recommendations include scaffolding the introduction to chatbots, facilitating teacher control of chatbot features, and suggesting when and how chatbots should be used. Our contribution informs the design of chatbots to support teachers and learners in middle school coding activities.

cs.CY↗

arXiv2026

Exploring the Design and Impact of Interactive Worked Examples for Learners with Varying Prior Knowledge

Tutoring systems improve learning through tailored interventions, such as worked examples, but often suffer from the aptitude-treatment interaction effect where low prior knowledge learners benefit more. We applied the ICAP learning theory to design two new types of worked examples, Buggy (students fix bugs), and Guided (students complete missing rules), requiring varying levels of cognitive engagement, and investigated their impact on learning in a controlled experiment with 155 undergraduate students in a logic problem solving tutor. Students in the Buggy and Guided examples groups performed significantly better on the posttest than those receiving passive worked examples. Buggy problems helped high prior knowledge learners whereas Guided problems helped low prior knowledge learners. Behavior analysis showed that Buggy produced more exploration-revision cycles, while Guided led to more help-seeking and fewer errors. This research contributes to the design of interventions in logic problem solving for varied levels of learner knowledge and a novel application of behavior analysis to compare learner interactions with the tutor.

cs.HC↗

arXiv2026

Adaptive Scaffolding for Cognitive Engagement in an Intelligent Tutoring System

The ICAP framework defines four cognitive engagement levels: Passive, Active, Constructive, and Interactive, where increased cognitive engagement can yield improved learning. However, personalizing learning activities that elicit the optimal level of cognitive engagement remains a key challenge in intelligent tutoring systems (ITS). In this work, we develop and evaluate a system that adaptively scaffolds cognitive engagement by dynamically selecting worked examples in two different ICAP modes: (active) Guided examples and (constructive) Buggy examples. We compare Bayesian Knowledge Tracing (BKT) and Deep Reinforcement Learning (DRL) as adaptive methods against a non-adaptive baseline method for selecting example type in a logic ITS. Our experiment with 113 students demonstrates that both adaptive policies significantly improved student performance on test problems. BKT yielded the largest improvement in posttest scores for low prior knowledge students, helping them catch up with their high prior knowledge peers, whereas DRL yielded significantly higher posttest scores among high prior knowledge students. This paper contributes new insights into the complex interactions of cognitive engagement and adaptivity and their results on learning outcomes.

cs.AI↗

arXiv2025

SnapClass: An AI-Enhanced Classroom Management System for Block-Based Programming

Block-Based Programming (BBP) platforms, such as Snap!, have become increasingly prominent in K-12 computer science education due to their ability to simplify programming concepts and foster computational thinking from an early age. While these platforms engage students through visual and gamified interfaces, teachers often face challenges in using them effectively and finding all the necessary features for classroom management. To address these challenges, we introduce SnapClass, a classroom management system integrated within the Snap! programming environment. SnapClass was iteratively developed drawing on established research about the pedagogical and logistical challenges teachers encounter in computing classrooms. Specifically, SnapClass allows educators to create and customize block-based coding assignments based on student skill levels, implement rubric-based auto-grading, and access student code history and recovery features. It also supports monitoring student engagement and idle time, and includes a help dashboard with a raise hand feature to assist students in real time. This paper describes the design and key features of SnapClass those are developed and those are under progress.

cs.CY↗

arXiv2025

The promise and limits of LLMs in constructing proofs and hints for logic problems in intelligent tutoring systems

Intelligent tutoring systems have demonstrated effectiveness in teaching formal propositional logic proofs, but their reliance on template-based explanations limits their ability to provide personalized student feedback. While large language models (LLMs) offer promising capabilities for dynamic feedback generation, they risk producing hallucinations or pedagogically unsound explanations. We evaluated the stepwise accuracy of LLMs in constructing multi-step symbolic logic proofs, comparing six prompting techniques across four state-of-the-art LLMs on 358 propositional logic problems. Results show that DeepSeek-V3 achieved superior performance up to 86.7% accuracy on stepwise proof construction and excelled particularly in simpler rules. We further used the best-performing LLM to generate explanatory hints for 1,050 unique student problem-solving states from a logic ITS and evaluated them on 4 criteria with both an LLM grader and human expert ratings on a 20% sample. Our analysis finds that LLM-generated hints were 75% accurate and rated highly by human evaluators on consistency and clarity, but did not perform as well explaining why the hint was provided or its larger context. Our results demonstrate that LLMs may be used to augment tutoring systems with logic tutoring hints, but require additional modifications to ensure accuracy and pedagogical appropriateness.

cs.AI↗

arXiv2025

The block matrix representations for the quasi-projection pairs on Hilbert $C^*$-modules

A quasi-projection pair consists of two operators $P$ and $Q$ acting on a Hilbert $C^*$-module $H$, where $P$ is a projection and $Q$ is an idempotent satisfying $Q^*=(2P-I)Q(2P-I)$, in which $Q^*$ denotes the adjoint operator of $Q$, and $I$ is the identity operator on $H$. Such a pair is said to be harmonious if both $P(I-Q)$ and $(I-P)Q$ admit polar decompositions. The primary goal of this paper is to present the block matrix representations for a harmonious quasi-projection pair $(P,Q)$ on a Hilbert $C^*$-module, and additionally to derive new block matrix representations for the matched projection, the range projection, and the null space projection of $Q$. Several applications of these newly obtained block matrix representations are also explored.

math.FA↗

arXiv2025

Characterizations of the semi-harmonious and harmonious quasi-projection pairs on Hilbert $C^*$-modules

For each adjointable idempotent $Q$ on a Hilbert $C^*$-module $H$, a specific projection $m(Q)$ called the matched projection of $Q$ was introduced recently due to the characterization of the minimum value among all the distances from projections to $Q$. Inspired by the relationship between $m(Q)$ and $Q$, another term called the quasi-projection pair $(P,Q)$ was also introduced recently, where $P$ is a projection on $H$ satisfying $Q^*=(2P-I)Q(2P-I)$, in which $Q^*$ is the adjoint operator of the idempotent $Q$ and $I$ is the identity operator on $H$. This paper aims to make systematical characterizations of the semi-harmonious and harmonious quasi-projection pairs on Hilbert $C^*$-modules, and meanwhile to provide examples illustrating the non-triviality of the associated characterizations.

math.OA↗

arXiv2025

The matched projections of idempotents on Hilbert $C^*$-modules

The aim of this paper is to give new characterizations of some fundamental issues about idempotents. In the general setting of adjointable operators on Hilbert $C^*$-modules, a new term of quasi-projection pair is introduced. For each idempotent $Q$, a projection $m(Q)$, called the matched projection of $Q$, is constructed. It is shown that $Q$ and $m(Q)$ as idempotents are homotopic, and $\big(m(Q),Q\big)$ is a quasi-projection pair. Some formulas for $m(Q)$ are derived. Based on these formulas, representations and norm estimations associated with $m(Q)$ are dealt with.

math.OA↗

arXiv2025

Combining Log Data and Collaborative Dialogue Features to Predict Project Quality in Middle School AI Education

Project-based learning plays a crucial role in computing education. However, its open-ended nature makes tracking project development and assessing success challenging. We investigate how dialogue and system interaction logs predict project quality during collaborative, project-based AI learning of 94 middle school students working in pairs. We used linguistic features from dialogue transcripts and behavioral features from system logs to predict three project quality outcomes: productivity (number of training phrases), content richness (word density), and lexical variation (word diversity) of chatbot training phrases. We compared the predictive accuracy of each modality and a fusion of the modalities. Results indicate log data better predicts productivity, while dialogue data is more effective for content richness. Both modalities modestly predict lexical variation. Multimodal fusion improved predictions for productivity and lexical variation of training phrases but not content richness. These findings suggest that the value of multimodal fusion depends on the specific learning outcome. The study contributes to multimodal learning analytics by demonstrating the nuanced interplay between behavioral and linguistic data in assessing student learning progress in open-ended AI learning environments.

cs.HC↗

arXiv2025

Investigating the Impact and Student Perceptions of Guided Parsons Problems for Learning Logic with Subgoals

Parsons problems (PPs) have shown promise in structured problem solving by providing scaffolding that decomposes the problem and requires learners to reconstruct the solution. However, some students face difficulties when first learning with PPs or solving more complex Parsons problems. This study introduces Guided Parsons problems (GPPs) designed to provide step-specific hints and improve learning outcomes in an intelligent logic tutor. In a controlled experiment with 76 participants, GPP students achieved significantly higher accuracy of rule application in both level-end tests and post-tests, with the strongest gains among students with lower prior knowledge. GPP students initially spent more time in training (1.52 vs. 0.81 hours) but required less time for post-tests, indicating improved problem solving efficiency. Our thematic analysis of GPP student self-explanations revealed task decomposition, better rule understanding, and reduced difficulty as key themes, while some students felt the structured nature of GPPs restricted their own way of reasoning. These findings reinforce that GPPs can effectively combine the benefits of worked examples and problem solving practice, but could be further improved by individual adaptation.

cs.HC↗

arXiv2024

Some aspects of semi-harmonious quasi-projection pairs

A term called the quasi-projection pair $(P,Q)$ was introduced recently by the authors, where $P$ is a projection and $Q$ is an idempotent on a Hilbert $C^*$-module $H$ satisfying $Q^*=(2P-I)Q(2P-I)$, in which $Q^*$ is the adjoint operator of the idempotent $Q$ and $I$ is the identity operator on $H$. Some fundamental issues on quasi-projection pairs, such as the block matrix representations for quasi-projection pairs and the $C^*$-morphisms associated with quasi-projection pairs, are worthwhile to be investigated. This paper aims to make some preparations. One object called the semi-harmonious quasi-projection pair is introduced in the general setting of the adjointable operators on Hilbert $C^*$-modules. Some related operator theories on the common similarity of operators and a norm equation associated with the Friedrichs angle are dealt with.

math.FA↗