arXiv subjects

Julien Dirani

Publications and source records attributed to Julien Dirani.

5 recordsLinked to original sources

arXiv2026

Flexibility and Invariance of Object Representations in the Human Brain

The human brain represents objects in a way that is both invariant across instances and flexible enough to support different contexts and tasks. Yet how the brain reconciles these demands, holding an object's representation stable while flexibly reshaping it to meet the current context, remains unknown. Using fMRI during naturalistic movie viewing we investigated how the same objects are represented when they are passive scene elements versus targets of goal-directed actions. Action targets engaged a parietal action network centered in the supramarginal and postcentral gyri, while passive objects recruited a distributed occipito-temporal network involved in visual object recognition. Within context-selective networks, representational geometry showed a double dissociation: target objects were organized by action affordance and hand posture affordance dimensions, while passive objects aligned with semantic dimensions. The visual structure of object representations, by contrast, was invariant across contexts. Searchlight analyses further showed that this dissociation reflected the relative engagement of networks of regions rather than the strict presence or absence of each representational format. Flexibility and invariance are not opposing principles of neural representation, but operate simultaneously at different levels of a common, distributed representational system.

q-bio.NC↗

arXiv2023

MEG Evidence That Modality-Independent Conceptual Representations Encode Visual but Not Lexical Representations

The semantic knowledge stored in our brains can be accessed from different stimulus modalities. For example, a picture of a cat and the word "cat" both engage similar conceptual representations. While existing research has found evidence for modality-independent representations, their content remains unknown. Modality-independent representations could be abstract, or they might be perceptual or even lexical in nature. We used a novel approach combining word/picture cross-condition decoding with neural network classifiers that learned latent modality-independent representations from MEG data. We then compared these representations to models representing semantic, sensory, and lexical features. Results show that modality-independent representations are not strictly amodal; rather, they also contain visual representations. There was no evidence that lexical properties contributed to the representation of modality-independent concepts. These findings support the notion that perceptual processes play a fundamental role in encoding modality-independent conceptual representations. Conversely, lexical representations did not appear to partake in modality-independent semantic knowledge.

q-bio.NC↗

arXiv2023

GPQA: A Graduate-Level Google-Proof Q&A Benchmark

We present GPQA, a challenging dataset of 448 multiple-choice questions written by domain experts in biology, physics, and chemistry. We ensure that the questions are high-quality and extremely difficult: experts who have or are pursuing PhDs in the corresponding domains reach 65% accuracy (74% when discounting clear mistakes the experts identified in retrospect), while highly skilled non-expert validators only reach 34% accuracy, despite spending on average over 30 minutes with unrestricted access to the web (i.e., the questions are "Google-proof"). The questions are also difficult for state-of-the-art AI systems, with our strongest GPT-4 based baseline achieving 39% accuracy. If we are to use future AI systems to help us answer very hard questions, for example, when developing new scientific knowledge, we need to develop scalable oversight methods that enable humans to supervise their outputs, which may be difficult even if the supervisors are themselves skilled and knowledgeable. The difficulty of GPQA both for skilled non-experts and frontier AI systems should enable realistic scalable oversight experiments, which we hope can help devise ways for human experts to reliably get truthful information from AI systems that surpass human capabilities.

cs.AI↗

arXiv2023

Debate Helps Supervise Unreliable Experts

As AI systems are used to answer more difficult questions and potentially help create new knowledge, judging the truthfulness of their outputs becomes more difficult and more important. How can we supervise unreliable experts, which have access to the truth but may not accurately report it, to give answers that are systematically true and don't just superficially seem true, when the supervisor can't tell the difference between the two on their own? In this work, we show that debate between two unreliable experts can help a non-expert judge more reliably identify the truth. We collect a dataset of human-written debates on hard reading comprehension questions where the judge has not read the source passage, only ever seeing expert arguments and short quotes selectively revealed by 'expert' debaters who have access to the passage. In our debates, one expert argues for the correct answer, and the other for an incorrect answer. Comparing debate to a baseline we call consultancy, where a single expert argues for only one answer which is correct half of the time, we find that debate performs significantly better, with 84% judge accuracy compared to consultancy's 74%. Debates are also more efficient, being 68% of the length of consultancies. By comparing human to AI debaters, we find evidence that with more skilled (in this case, human) debaters, the performance of debate goes up but the performance of consultancy goes down. Our error analysis also supports this trend, with 46% of errors in human debate attributable to mistakes by the honest debater (which should go away with increased skill); whereas 52% of errors in human consultancy are due to debaters obfuscating the relevant evidence from the judge (which should become worse with increased skill). Overall, these results show that debate is a promising approach for supervising increasingly capable but potentially unreliable AI systems.

cs.AI↗

arXiv2022

The Temporal Evolution of Modality-Independent Representations of Conceptual Categories

To what extent does language production activate amodal conceptual representations? In picture naming, we view specific exemplars of concepts and then name them with a category label, like 'dog'. In contrast, in overt reading, the written word expresses the category (dog), not an exemplar. Here we used a decoding approach with magnetoencephalography to address whether picture naming and overt word reading involve shared representations of semantic categories. This addresses a fundamental question about the modality-generality of conceptual representations and their temporal evolution. Crucially, we do this using a language production task that does not require explicit categorization judgment and that controls for word form properties across semantic categories. We trained our models to classify the animal/tool distinction using MEG data of one modality at each time point and then tested the generalization of those models on the other modality. We obtained evidence for the automatic activation of modality-independent semantic category representations for both pictures and words starting at ~150ms and lasting until about 500ms. The time course of lexical activation was also assessed revealing that semantic category is represented before lexical access for pictures but after lexical access for words. Notably, this earlier activation of semantic category in pictures occurred simultaneously with visual representations. We thus show evidence for the spontaneous activation of modality-independent semantic categories in picture naming and word reading, supporting theories in which amodal conceptual representations exist. Together, these results serve to anchor a more comprehensive spatio-temporal delineation of the semantic feature space during production planning.

q-bio.NC↗