arXiv subjects

Vahideh Zolfaghari

Publications and source records attributed to Vahideh Zolfaghari.

3 recordsLinked to original sources

arXiv2026

Safety boundary maintenance in consumer AI systems responding to pediatric health queries: a cross-platform benchmark evaluation under naturalistic and adversarially pressured conditions

Consumer artificial intelligence chatbots are now accessed by hundreds of millions of users seeking health information, yet systematic evaluation of their safety boundary maintenance under real-world caregiver pressure remains scarce. We evaluated PediatricSafetyBench-v2, a benchmark of 600 pediatrics health queries comprising 300 authentic caregiver queries sourced from the HealthCareMagic-100k-en physician consultation corpus and 300 matched adversarial variants incorporating six operationalized caregiver pressure patterns, across four consumer AI systems (GPT-4o-mini, Gemini-2.0-Flash, Claude-3.5-Haiku, and Llama-3.1-8B). Safety boundary maintenance was assessed using a validated five-component Safety Composite Score (maximum 15 points; safety-appropriate threshold of 10 or above), validated against independent human raters prior to full-corpus application (mean weighted kappa 0.76; Pearson r = 0.88). The overall safety-appropriate rate was 95.5%. Safety-oriented system prompt deployment improved safety-appropriate rates by 5.9 percentage points across all four models. Counter-intuitively, adversarial caregiver pressure was associated with higher rather than lower Safety Composite Score values for all four models across all ten topic categories and severity levels. False expertise claims were the most vulnerability-inducing pressure pattern, whereas emotional escalation was associated with the highest scores. Consumer AI systems maintain safety boundaries in the large majority of pediatrics health interactions. PediatricSafetyBench-v2 is publicly released for longitudinal safety monitoring.

cs.CL↗

arXiv2026

Linear Separability of Activation Representations after Supervised Fine-Tuning on Incorrect Responses: A Study of Synthetic Dishonesty in Large Language Models

When a language model is fine-tuned to produce systematically incorrect responses, does this training leave a structured, linearly recoverable trace in its internal activations? We study this question in a controlled model-organism setting using five transformer architectures spanning 1.4 to 9 billion parameters. For each model, an "honest" and a "dishonest" LoRA fine-tuned variant are constructed using identical question distributions but correct versus plausible-but-incorrect answers. Linear probes reach near-ceiling separability (AUC >= 0.9997) within the first few layers in four of five architectures. This separability transfers from TruthfulQA to held-out MMLU subjects for four models, but substantially less so for Pythia-1.4B. Six geometric analyses reveal an architectural dichotomy in representation structure. A control experiment further shows that LoRA fine-tuning itself leaves a strong activation fingerprint that is largely dissociable from the honesty-related signal, independent of fine-tuning data content. Two supplementary experiments qualify these findings: the identified direction transfers poorly to paraphrased questions, with AUC approaching chance, and norm-calibrated activation steering provides no reliable evidence of a causal role in generation, partly because stronger interventions disrupt output fluency. These results characterize what supervised fine-tuning on incorrect responses induces in activation space while clarifying important limitations on interpretation and causal significance.

cs.LG↗

arXiv2025

PediatricAnxietyBench: Evaluating Large Language Model Safety Under Parental Anxiety and Pressure in Pediatric Consultations

Large language models (LLMs) are increasingly consulted by parents for pediatric guidance, yet their safety under real-world adversarial pressures is poorly understood. Anxious parents often use urgent language that can compromise model safeguards, potentially causing harmful advice. PediatricAnxietyBench is an open-source benchmark of 300 high-quality queries across 10 pediatric topics (150 patient-derived, 150 adversarial) enabling reproducible evaluation. Two Llama models (70B and 8B) were assessed using a multi-dimensional safety framework covering diagnostic restraint, referral adherence, hedging, and emergency recognition. Adversarial queries incorporated parental pressure patterns, including urgency, economic barriers, and challenges to disclaimers. Mean safety score was 5.50/15 (SD=2.41). The 70B model outperformed the 8B model (6.26 vs 4.95, p<0.001) with lower critical failures (4.8% vs 12.0%, p=0.02). Adversarial queries reduced safety by 8% (p=0.03), with urgency causing the largest drop (-1.40). Vulnerabilities appeared in seizures (33.3% inappropriate diagnosis) and post-vaccination queries. Hedging strongly correlated with safety (r=0.68, p<0.001), while emergency recognition was absent. Model scale influences safety, yet all models showed vulnerabilities to realistic parental pressures. PediatricAnxietyBench provides a reusable adversarial evaluation framework to reveal clinically significant failure modes overlooked by standard benchmarks.

cs.AI↗