arXiv subjects

Alessandro T. Gifford

Publications and source records attributed to Alessandro T. Gifford.

5 recordsLinked to original sources

arXiv2026

A large dataset of human EEG responses to short naturalistic videos for studying dynamic visual event processing

Vision neuroscience has experienced a surge in the collection and use of large-scale datasets of brain responses to naturalistic images. However, static images lack the temporal dimension essential for understanding how vision is solved in the brain during dynamic real life settings. To facilitate the study of the neural correlates of dynamic visual event perception, we introduce the EEG Moments Dataset (EMD). EMD consists of 128-channel EEG responses and eye-tracking recordings of 6 human participants viewing 1,102 short naturalistic videos (3-second long; with audio track) while maintaining central fixation. We show that EMD's EEG responses well encode stimulus-related information, exhibit a temporal correspondence with the video stimuli, and have a rich representational content revealed by brain encoding models based on different feature spaces. Furthermore, complemented by the BOLD Moments Dataset (BMD) - an existing large-scale dataset of human functional magnetic resonance imaging (fMRI) responses for the same videos - EMD enables spatio-temporally resolved investigations of brain responses to dynamic visual events. We release EMD's EEG and eye-tracking data in both raw and preprocessed format, along with the 1,102 video stimuli, and rich stimulus metadata. Finally, we provide an interactive code tutorial to familiarize with EMD's preprocessed data, stimuli, and stimulus metadata.

q-bio.NC↗

arXiv2026

A 7T fMRI dataset of synthetic images for out-of-distribution modeling of vision

Now published in Nature Communications DOI: https://doi.org/10.1038/s41467-026-69345-9 Large-scale visual neural datasets such as the Natural Scenes Dataset (NSD) are boosting computational neuroscience research by enabling models of the brain with performances beyond what was possible just a decade ago. However, because the stimuli of these datasets typically live within a common naturalistic visual distribution, they do not allow for strict out-of-distribution (OOD) generalization tests which are crucial for the development of more robust models. Here, we address this limitation by releasing NSD-synthetic, a dataset consisting of 7T fMRI responses from the same eight NSD participants for 284 synthetic images. We show that NSD-synthetic's fMRI responses reliably encode stimulus-related information and are OOD with respect to NSD. Furthermore, we provide a proof of principle that OOD generalization tests on NSD-synthetic reveal differences between models of the brain that are not detected with the original NSD data; we demonstrate that the degree of OOD (quantified as the distance between a set of responses and the training data used for modeling) is predictive of the magnitude of model failures; and we show that less strict OOD generalization tests can can be usefully applied even within the domain of naturalistic stimuli. These results showcase how NSD-synthetic enables OOD generalization tests that facilitate the development of more robust models of visual processing and the formulation of more accurate theories of human vision.

q-bio.NC↗

arXiv2025

In silico discovery of representational relationships across visual cortex

Now published in Nature Human Behavior doi: https://doi.org/10.1038/s41562-025-02252-z Human vision is mediated by a complex interconnected network of cortical brain areas that jointly represent visual information. While these areas are increasingly understood in isolation, their representational relationships remain elusive. Here we developed relational neural control (RNC), and used it to investigate the representational relationships for univariate and multivariate fMRI responses of areas across visual cortex. Through RNC we generated and explored in silico fMRI responses for large amounts of images, discovering controlling images that align or disentangle responses across areas, thus indicating their shared or unique representational content. This revealed a typical network-level configuration of representational relationships in which shared or unique representational content varied based on cortical distance, categorical selectivity, and position within the visual hierarchy. Closing the empirical cycle, we validated the in silico discoveries on in vivo fMRI responses from independent subjects. Together, this reveals how visual areas jointly represent the world as an interconnected network.

q-bio.NC↗

arXiv2025

The Algonauts Project 2025 Challenge: How the Human Brain Makes Sense of Multimodal Movies

There is growing symbiosis between artificial and biological intelligence sciences: neural principles inspire new intelligent machines, which are in turn used to advance our theoretical understanding of the brain. To promote further collaboration between biological and artificial intelligence researchers, we introduce the 2025 edition of the Algonauts Project challenge: How the Human Brain Makes Sense of Multimodal Movies (https://algonautsproject.com/). In collaboration with the Courtois Project on Neuronal Modelling (CNeuroMod), this edition aims to bring forth a new generation of brain encoding models that are multimodal and that generalize well beyond their training distribution, by training them on the largest dataset of fMRI responses to movie watching available to date. Open to all, the 2025 challenge provides transparent, directly comparable results through a public leaderboard that is updated automatically after each submission to facilitate rapid model assessment and guide development. The challenge will end with a session at the 2025 Cognitive Computational Neuroscience (CCN) conference that will feature winning models. We welcome researchers interested in collaborating with the Algonauts Project by contributing ideas and datasets for future challenges.

q-bio.NC↗

arXiv2024

Limited but consistent gains in adversarial robustness by co-training object recognition models with human EEG

In contrast to human vision, artificial neural networks (ANNs) remain relatively susceptible to adversarial attacks. To address this vulnerability, efforts have been made to transfer inductive bias from human brains to ANNs, often by training the ANN representations to match their biological counterparts. Previous works relied on brain data acquired in rodents or primates using invasive techniques, from specific regions of the brain, under non-natural conditions (anesthetized animals), and with stimulus datasets lacking diversity and naturalness. In this work, we explored whether aligning model representations to human EEG responses to a rich set of real-world images increases robustness to ANNs. Specifically, we trained ResNet50-backbone models on a dual task of classification and EEG prediction; and evaluated their EEG prediction accuracy and robustness to adversarial attacks. We observed significant correlation between the networks' EEG prediction accuracy, often highest around 100 ms post stimulus onset, and their gains in adversarial robustness. Although effect size was limited, effects were consistent across different random initializations and robust for architectural variants. We further teased apart the data from individual EEG channels and observed strongest contribution from electrodes in the parieto-occipital regions. The demonstrated utility of human EEG for such tasks opens up avenues for future efforts that scale to larger datasets under diverse stimuli conditions with the promise of stronger effects.

cs.LG↗