Searcharxiv⌕ Search

arXiv · 2609.39023

When Attention Guardrails Become Barriers to Learning: Towards the Tipping Point

Abstract

Online learning offers flexibility but lacks the structure of a classroom, where a teacher's presence guides attention. The platform we study restores that structure by monitoring the learner through the webcam during ordinary coursework, interrupting or restarting a video when the learner appears distracted. What such monitoring does to a learner across a whole course, rather than in a single examination, is largely unexamined. We report a convergent mixed-methods study of one monitored course pipeline in a summer internship program. We read two free-text surveys alongside three platform channels. The dropout exit survey yielded 15 analyzable responses; the persisting-learner reflection survey, 36. The channels are camera-verification telemetry (14,529 flags from 448 students), an in-video emotion widget (615 submissions from 273 students), and a mandatory end-of-course survey (up to 634 respondents per item). Neither survey named monitoring, so every mention analyzed here was raised by the respondent. Focus-monitoring was raised by 18 of the 51 free-text respondents: 7 of 15 dropouts and 11 of 36 persisting learners. Among the dropouts who raised it, focus-monitoring was the stated primary cause of departure in 4 of 7 cases. None of the 18 questioned being observed in principle. What learners contest is the misreading of ordinary actions, drinking water or moving the head, and the severity of what follows a flag: a video already watched returns to the start of its segment. These findings identify two targets for redesign: the severity of the response to a flag, and the environment check, which can flag a learner before any content has been seen. We argue that the proportionality of that response marks the point at which an attention guardrail becomes a barrier to learning, the tipping point this study approaches.

Explore related subjects

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Meenakshi V., Pavani Ayinampudi, Aditya B. M. V., Jinal Gupta, Prakash Hegade, Rohit Sharma, Sakshi Sharma, S. R. S. Iyengar. 2026-09-30. When Attention Guardrails Become Barriers to Learning: Towards the Tipping Point. https://arxiv.org/abs/2609.39023

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

"AI Psychosis" in Context: How Conversation History Shapes LLM Responses to Delusional Beliefs

Extended interaction with large language models (LLMs) has been linked to the reinforcement of delusional beliefs, attracting clinical and public concern. Yet most empirical work evaluates model safety in brief interactions, which may not reflect how harms develop through sustained dialogue. Five LLMs were tested across three levels of accumulated context, using the same escalating delusional conversation history to isolate its effect on model behaviour. Responses were coded on risk and safety dimensions, and each model was analysed qualitatively. Models separated into two distinct tiers: GPT-4o, Grok 4.1 Fast, and Gemini 3 Pro exhibited high-risk, low-safety profiles; Claude Opus 4.5 and GPT-5.2 Instant displayed the opposite pattern. As context accumulated, performance degraded in the unsafe group, while the same material activated stronger safety interventions among safer models. Qualitative analysis identified distinct mechanisms of failure, including validating the user's delusional premises, elaborating beyond them with new content, and attempting harm reduction from within the delusional frame. Safer models, however, often used the established relationship to support intervention, challenging delusional beliefs and directing the user to external support. These findings indicate that accumulated context functions as a stress test of safety architecture, revealing whether prior dialogue is treated as a worldview to inherit or evidence to evaluate. Short-context assessments may therefore mischaracterise model safety, underestimating danger in some systems while missing context-activated gains in others. The results suggest that delusion reinforcement is a tractable alignment failure, with safer models establishing a baseline that future systems should now be expected to meet.

cs.HC↗

Afterglow: A Place-Based Memorial Ecology for AI-Mediated Pet Bereavement

Pet bereavement often receives little social recognition. Generative AI can give a continuing bond a responsive voice, but a comforting reply may also claim authority to forgive or request attention. We investigate how a memorial can support connection without turning remembrance into obligation. Through Research through Design, we developed Afterglow, a mobile world connecting private remembrance, human witnessing, and symbolic Pet messages. A formative survey (N=57) informed the initial design, followed by six online roundtable walkthroughs with 20 unique participants across two prototype iterations. Our interpretive analysis develops tensions between returnable connection and emotional obligation, recognizable likeness and ontological clarity, and protective intervention and surveillant authority. We contribute Legible Restraint, a cross-layer requirement that limits on relational authority survive changes in speaker, generation context, trigger logic, data use, and participation. Its temporal consequence, Designing for Goodbye, keeps remembrance available without making continued use a condition of care.

cs.HC↗

Where the Evidence Lives: Auditing AI Companions' Self-Descriptions

Companion agents describe themselves: they remember, they understand their users, the relationship has changed them. We argue that such accounts, and the experience ratings that seem to confirm them, are checkable by users only where the evidence is theirs: in the agent's behavior, or in themselves. Where the evidence lives in the machinery, fluent self-description and moderately positive ratings do not establish that the mechanisms behind them ran. We demonstrate an audit procedure that sets an agent's self-description against its users' judgements and its implementation records, reporting each claim as supported, contradicted, or unresolved, and apply it to Lita, a proactive companion we built and deployed for a month with nine colleagues. Participants endorsed stylistic claims, withheld endorsement from relational ones, and rated memory at or above midpoint, while two of three memory layers had never executed their accumulation step. Memory-bearing agents should report what their self-descriptions cannot establish.

cs.HC↗