arXiv · 2611.18591
Where Language Models Fail Like Aphasic Patients and Where They Don't
Abstract
The computational architecture of human word production -- whether semantic-lexical retrieval and phonological encoding constitute dissociable processing stages -- has been debated for decades using hand-designed models with small vocabularies. We introduce computational architectural dissociation: using known architectural properties of vision-language models (VLMs) to derive falsifiable predictions about which patient error types should and should not align with model errors under perturbation. Because VLMs perform picture naming through learned semantic competition over full vocabularies but lack phonological encoding entirely, perturbation-induced errors should match patient patterns for semantic-stage categories but not phonological ones. Testing LLaVA 1.6 (Vicuna-13B) against 88 patients with aphasia across 175 Philadelphia Naming Test items (~37 million naming attempts, 50 perturbation seeds), we find that six of eight error categories show significant item-level alignment (Spearman $ρ$ = 0.32-0.50, all p < 0.001) while the two phonological categories -- formal errors and abstruse neologisms -- show no alignment ($ρ$ = 0.110 and 0.084, both n.s.), confirming the predicted dissociation. Spectral analysis of the output probabilities places each error type at a distinct position in a six-dimensional space, giving a mechanistic interpretability tool. The dissociation is layer-specific (peaking at L22, 55% depth), severity-independent, and stable across seeds (split-half r > 0.998). Weight perturbation of four additional 7B VLMs spanning the tested architectural combinations produced correct-rate alignment of comparable magnitude ($ρ$ = 0.21-0.27). Whether the per-category dissociation itself depends on scale or on attention design requires a comparison within a single inference implementation, which is reported separately.
Explore related subjects
Keep this discovery
Explore connections, maps & timelines
Yong Yang, Roger Newman-Norlund, Xiang Guan, Saeed Ahmadi, Regan Willis, Nadra Salman, Kalil Warren, Sophie Arheix-Parras, Srihari Nelakuditi, Rutvik H. Desai, Leonardo Bonilha, Christopher Rorden, Julius Fridriksson. 2026-08-24. Where Language Models Fail Like Aphasic Patients and Where They Don't. https://searcharxiv.com/abs/2611.18591
Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite Searcharxiv by Indiciae: BibTeX · RIS.