How Children Design and Reason about Trustworthy AI Chatbots

Deniz Ozturk; Jiayu Li; Daksh Pratap Singh; Yasitha Rajapaksha; Fasika Melese; Bahare Riahi; Shiyan Jiang; Qiao Jin; Joey Huang; Veronica Cateté; Tiffany Barnes; Xiaoyi Tian

arXiv · 2609.25244

How Children Design and Reason about Trustworthy AI Chatbots

Deniz Ozturk·Jiayu Li·Daksh Pratap Singh·Yasitha Rajapaksha·Fasika Melese·Bahare Riahi·Shiyan Jiang·Qiao Jin·Joey Huang·Veronica Cateté·Tiffany Barnes·Xiaoyi Tian

View original source ↗Read source document ↗Export metadata ↓

Abstract

Children increasingly interact with AI chatbots, making trust calibration essential to AI literacy. Prior research has examined children's trust in AI mainly as users evaluating systems built by others, rather than as designers of their own chatbots. We developed a chatbot-building environment with adjustable trust-relevant traits (e.g., confidence, transparency, formality, assertiveness), rules, and persona. We conducted mixed-methods study with 115 learners (ages 8-18) who made 119 chatbots. We examined how children configured their chatbots, reasoned about trustworthiness, and how closely chatbot behavior aligned with their designs. Younger students (age 10-13) set significantly higher confidence than older students (age 14-18), and some deliberately built chatbots that gave wrong answers on purpose, yet still called them trustworthy, arguing that a chatbot does what it was built to do. Younger students equated trust with purpose-fulfillment, while older students linked it to transparent, calibrated design. Students also calibrated academic chatbots to be more transparent and formal than hobby chatbots. We identify seven design dimensions describing what children believe makes a chatbot trustworthy, and discuss implications for AI literacy tools.

Explore related subjects

cs.HC cs.AI

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Deniz Ozturk, Jiayu Li, Daksh Pratap Singh, Yasitha Rajapaksha, Fasika Melese, Bahare Riahi, Shiyan Jiang, Qiao Jin, Joey Huang, Veronica Cateté, Tiffany Barnes, Xiaoyi Tian. 2026-09-23. How Children Design and Reason about Trustworthy AI Chatbots. https://arxiv.org/abs/2609.25244

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Describe Me Something You Do Not Remember - Challenges and Risks of Exposure Design Using Generative Artificial Intelligence for Therapy of Complex Post-traumatic Stress Disorder

Post-traumatic stress disorder (PTSD) is associated with sudden, uncontrollable, and intense flashbacks of traumatic memories. Trauma exposure psychotherapy has proven effective in reducing the severity of trauma-related symptoms. It involves controlled recall of traumatic memories to train coping mechanisms for flashbacks and enable autobiographical integration of distressing experiences. In particular, exposure to visualizations of these memories supports successful recall. Although this approach is effective for various trauma types, it remains available for only a few. This is due to the lack of cost-efficient solutions for creating individualized exposure visualizations. This issue is particularly relevant for the treatment of Complex PTSD (CPTSD), where traumatic memories are highly individual and generic visualizations do not meet therapeutic needs. Generative Artificial Intelligence (GAI) offers a flexible and cost-effective alternative. GAI enables the creation of individualized exposure visualizations during therapy and, for the first time, allows patients to actively participate in the visualization process. While GAI opens new therapeutic perspectives and may improve access to trauma therapy, especially for CPTSD, it also introduces significant challenges and risks. The extreme uncertainty and lack of control that define both CPTSD and GAI raise concerns about feasibility and safety. To support safe and effective three-way communication, it is essential to understand the roles of patient, system, and therapist in exposure visualization and how each can contribute to safety. This paper outlines perspectives, challenges, and risks associated with the use of GAI in trauma therapy, with a focus on CPTSD.

cs.HC↗

arXiv2026

KnowTeX: Visualizing Mathematical Dependencies

Dependency graphs that show how definitions, theorems, and proofs relate to each other are valuable for understanding the structure of mathematical texts. Existing tools such as Lean Blueprint and plasTeXdepgraph generate such graphs within formal proof ecosystems, but they require familiarity with proof assistants or specific compilation pipelines. We present KnowTeX, a standalone Python tool that extracts dependency graphs directly from LaTeX sources without requiring any external framework. KnowTeX supports two complementary modes: a manual mode where authors annotate their source with lightweight commands compatible with Lean Blueprint, and an infer mode that automatically discovers dependencies through a layered system of deterministic and heuristic rules. The tool handles multi-file projects, detects cycles, applies transitive reduction, and exports graphs in DOT, TikZ, and PNG formats with an interactive preview. We evaluate KnowTeX on several mathematical texts and discuss how it complements recent tools such as LeanArchitect, which operates from the Lean side, while KnowTeX works entirely on the LaTeX side without requiring any formalization.

cs.HC↗

arXiv2026

Helping Customers in Distress: An LLM-powered Agent that Converses, Probes, and Routes

Banks receive millions of reports of fraud, scams, and disputed transactions every year, making it challenging to accurately direct customers to the appropriate specialist teams for assistance. The existing manual process driven by humans is slow and stressful for both customers and staff. To address this, we develop a customer-facing AI powered triaging agent that leverages large language models (LLMs) to conduct multi-turn conversations, ask relevant questions, and classify cases for accurate, policy-guided routing, making it embedded in the customer journey. To evaluate and continuously improve the agent, synthetic digital twins of real customers were simulated, generating realistic, labelled dialogues based on historical data to test a wide range of real-world scenarios. This work details the triage agent's modelling approach, integration with policy, safety guardrails and reasoning frameworks, the use of the synthetic agent for scalable evaluation, and findings on the AI system's accuracy, robustness, and compliance. Results show that the agent successfully improves triaging of historical cases, achieving a 30.6% increase in classification accuracy, with high satisfaction levels reported by our subject-matter experts, highlighting how targeted probing can lead to more effective triage in banking operations at scale.

cs.HC↗