arXiv subjects

Gabriel Mahuas

Publications and source records attributed to Gabriel Mahuas.

7 recordsLinked to original sources

arXiv2026

CoCoT-EEG: Contrastive-Pretrained Multiscale Convolutional Transformer for EEG Decoding

Self-supervised pretrained foundation models (FM) have shown early promise for non-invasive electroencephalogram (EEG) decoding applications. Many recent large-scale models converged on the approach of tokenizing raw EEG followed by masked reconstruction pretraining. However, this recipe has been shown to be suboptimal for data, like EEG, with high noise amplitude and information confined to limited dimensions such as narrow frequency bands. Building on this insight, we develop a novel contrastive-pretrained EEG model with multiscale temporal convolution input layers and Transformer encoder blocks (CoCoT). CoCoT matches or beats state-of-the-art reconstruction-pretrained EEG models on extensive benchmark decoding tasks with heterogeneous electrode configurations. Furthermore, CoCoT trained from scratch outperforms previous single-task decoding models and even rivals pretrained models, showcasing the architecture's flexibility and data efficiency. Through systematic ablations, including model architecture and pretraining objective, we demonstrate the viability of contrastive learning for building EEG FMs while suggesting key architectural design considerations, prompting further investigations in alternative large-scale pretraining strategies.

cs.LG↗

arXiv2026

The Variance Brain Foundation Models Forgot: Third-Order Statistics Predict Cognition Where Billion-Parameter Models Fail

Brain foundation models (BFMs) are self-supervised Transformers pretrained on fMRI data. We posit that these models should capture each subject's cognitive performance from their fMRI signal. Yet across three state-of-the-art BFMs and every readout we test, they predict cognition worse than a linear regression from the $\sim$80K parameters of the functional connectivity matrix (FC). The gap widens with scale: BrainLM's 650M model predicts cognition worse than its 111M. We attribute this to a \textbf{variance allocation problem}: BFM pretraining captures the variance components that dominate fMRI but not the higher-order structure that predicts cognition. Our per-cumulant analysis of the reconstructed signal shows that the second-order covariance is partially preserved, while the third-order co-skewness tensor is largely destroyed. To recover what BFMs lose, we design a linear pipeline that projects the fMRI signal into the subspace that best preserves its co-skewness and computes FC there. This \textbf{exceeds raw FC and every pretrained BFM} on every dataset and parcellation we test, outperforming prior state-of-the-art under controlled evaluation \textbf{with no pretraining and no GPU}. We \textbf{recover the raw-FC ceiling on BrainLM's forward pass} by finetuning with a loss targeted at this same subspace. This shows that the bottleneck is the pretraining objective, not the architecture or the model size.

q-bio.NC↗

arXiv2025

ReBaPL: Repulsive Bayesian Prompt Learning

Prompt learning has emerged as an effective technique for fine-tuning large-scale foundation models for downstream tasks. However, conventional prompt learning methods are prone to overfitting and can struggle with out-of-distribution generalization. To address these limitations, Bayesian prompt learning has been proposed, which frames prompt optimization as a Bayesian inference problem to enhance robustness. This paper introduces Repulsive Bayesian Prompt Learning (ReBaPL), a novel method for Bayesian prompt learning, designed to efficiently explore the complex and often multimodal posterior landscape of prompts. Our method integrates a cyclical step-size schedule with a stochastic gradient Hamiltonian Monte Carlo (SGHMC) algorithm, enabling alternating phases of exploration to discover new modes, and exploitation to refine existing modes. Furthermore, we introduce a repulsive force derived from a potential function over probability metrics (including Maximum Mean Discrepancy and Wasserstein distance) computed on the distributions of representations produced by different prompts. This representation-space repulsion diversifies exploration and prevents premature collapse to a single mode. Our approach allows for a more comprehensive characterization of the prompt posterior distribution, leading to improved generalization. In contrast to prior Bayesian prompt learning methods, our method provides a modular plug-and-play Bayesian extension of any existing prompt learning method based on maximum likelihood estimation. We demonstrate the efficacy of ReBaPL on several benchmark datasets, showing superior performance over state-of-the-art prompt learning methods.

cs.LG↗

arXiv2025

Diagrammatic expansion for the mutual-information rate in the realm of limited statistics

Neurons in sensory systems encode stimulus information into their stochastic spiking response. The mutual information has been extensively applied to these systems to quantify the neurons' capacity of transmitting such information. Yet, while for discrete stimuli, like flashed images or single tones, its computation is straightforward, for dynamical stimuli it is necessary to compute a (mutual) information rate (MIR), therefore integrating over the multiple temporal correlations which characterize sensory systems. Previous methods are based on extensive sampling of the neuronal response, require large amounts of data and are therefore prone to biases and inaccuracy. Here, we develop Moba-MIRA (moment-based mutual-information-rate approximation), a computational method to estimate the mutual information rate. To derive Moba-MIRA, we use Feynman diagrams to expand the mutual information to arbitrary order in the correlations around the corresponding value for the empirical spike count distributions of single bins. As a result, only the empirical estimation of the pairwise correlations between time bins and the single-bin entropies are required, without the need for the whole joint probability distributions. We tested Moba-MIRA on synthetic data generated with generalized linear models, and showed that it requires only a few tens of stimulus repetitions to provide an accurate estimate of the information rate. Finally, we applied it to ex-vivo electrophysiological recordings of rats retina, obtaining rates ranging between 5 to 20 bits per second, consistent with earlier estimates.

q-bio.NC↗

arXiv2024

Strong, but not weak, noise correlations are beneficial for population coding

Neural correlations play a critical role in sensory information coding. They are of two kinds: signal correlations, when neurons have overlapping sensitivities, and noise correlations from network effects and shared noise. In experiments from early sensory systems and cortex, many pairs of neurons typically show both types of correlations to be positive and large, especially between nearby neurons with similar stimulus sensitivity. However, theoretical arguments have suggested that stimulus and noise correlations should have opposite signs to improve coding, at odds with experimental observations. We analyze retinal recording in response to a large variety of stimuli, and show that, contrary to common belief, large noise correlations are beneficial for coding, even if aligned with signal correlations. To understand this result, we develop a theory of visual information coding by correlated neurons, which resolves that paradox. We show that noise correlations are always beneficial if they are strong enough, unless neurons are perfectly correlated by the stimulus. Finally, using neuronal recordings and modeling, we show that for high dimensional stimuli noise correlation benefits the encoding of fine-grained details of visual stimuli, at the expense of large-scale features, which are already well encoded.

q-bio.NC↗

arXiv2022

A small-correlation expansion to quantify information in noisy sensory systems

Neural networks encode information through their collective spiking activity in response to external stimuli. This population response is noisy and strongly correlated, with complex interplay between correlations induced by the stimulus, and correlations caused by shared noise. Understanding how these correlations affect information transmission has so far been limited to pairs or small groups of neurons, because the curse of dimensionality impedes the evaluation of mutual information in larger populations. Here we develop a small-correlation expansion to compute the stimulus information carried by a large population of neurons, yielding interpretable analytical expressions in terms of the neurons' firing rates and pairwise correlations. We validate the approximation on synthetic data and demonstrate its applicability to electrophysiological recordings in the vertebrate retina, allowing us to quantify the effects of noise correlations between neurons and of memory in single neurons.

q-bio.NC↗

arXiv2020

A new inference approach for training shallow and deep generalized linear models of noisy interacting neurons

Generalized linear models are one of the most efficient paradigms for predicting the correlated stochastic activity of neuronal networks in response to external stimuli, with applications in many brain areas. However, when dealing with complex stimuli, the inferred coupling parameters often do not generalize across different stimulus statistics, leading to degraded performance and blowup instabilities. Here, we develop a two-step inference strategy that allows us to train robust generalized linear models of interacting neurons, by explicitly separating the effects of correlations in the stimulus from network interactions in each training step. Applying this approach to the responses of retinal ganglion cells to complex visual stimuli, we show that, compared to classical methods, the models trained in this way exhibit improved performance, are more stable, yield robust interaction networks, and generalize well across complex visual statistics. The method can be extended to deep convolutional neural networks, leading to models with high predictive accuracy for both the neuron firing rates and their correlations.

cond-mat.dis-nn↗