SearcharxivSearch

arXiv subjects

William Snyder

Publications and source records attributed to William Snyder.

3 recordsLinked to original sources

Representation learning of human cortical folding to reveal long lasting neurodevelopmental signatures

The human brain folds in utero, primarily during late gestation. Shortly after birth, cortical folding patterns are established and remain stable thereafter, making them promising early neurodevelopmental markers. Yet it is unclear whether the representations given by current neuroimaging foundation models capture cortical folding variability. Here, we introduce Champollion, a self-supervised learning framework that learns interpretable local representations of cortical folding from structural MRI. Optimized on representative folding-related tasks, Champollion accurately captures known folding patterns across cortical regions and external datasets. In a comprehensive benchmark, it consistently outperforms neuroimaging and general-purpose foundation models. Furthermore, Champollion reveals richer genetic associations than conventional morphometric descriptors and identifies localized folding signatures associated with incomplete hippocampal inversion, prematurity, and maternal smoking. These results establish cortical folding as a rich and largely untapped source of neurodevelopmental information, and Champollion provides a unified framework for discovering, localizing and interpreting long lasting cortical folding signatures.

q-bio.QM

How Large Language Models Get Stuck: Early structure with persistent errors

Linguistic insights may help make Large Language Model (LLM) training more efficient. We trained Meta's OPT model on the 100M word BabyLM dataset, and evaluated it on the BLiMP benchmark, which consists of 67 classes, each defined by sentence pairs that differ in a targeted syntactic or semantic rule violation. We tested the model's preference for grammatical over ungrammatical sentences across training iterations and grammatical types. In nearly one-third of the BLiMP classes, OPT fails to consistently assign a higher likelihood to grammatical sentences, even after extensive training. When it fails, it often establishes a clear (erroneous) separation of the likelihoods at an early stage of processing and sustains this to the end of our training phase. We hypothesize that this mis-categorization is costly because it creates entrenched biases that must, eventually, be reversed in order for the model to perform well. We probe this phenomenon using a mixture of qualitative (based on linguistic theory and the theory of Deep Learning) and quantitative (based on numerical testing) assessments. Our qualitative assessments indicate that only some BLiMP tests are meaningful guides. We conclude by articulating a hypothesis, the Bigram Hypothesis, which claims that the learning process will exhibit erroneous entrenchment if bigram statistics bias the model toward wrong distinctions early in training, and we describe a method of testing the hypothesis on appropriately selected BLiMP classes.

cs.CL

Reduced Order Model Closures: A Brief Tutorial

In this paper, we present a brief tutorial on reduced order model (ROM) closures. First, we carefully motivate the need for ROM closure modeling in under-resolved simulations. Then, we construct step by step the ROM closure model by extending the classical Galerkin framework to the spaces of resolved and unresolved scales. Finally, we develop the data-driven variational multiscale ROM closure and then we test it in fluid flow simulations. Our tutorial on ROM closures is structured as a sequence of questions and answers, and is aimed at first year graduate students and advanced undergraduate students. Our goal is not to explain the "how," but the "why." That is, we carefully explain the principles used to develop ROM closures, without focusing on particular approaches. Furthermore, we try to keep the technical details to a minimum and describe the general ideas in broad terms while citing appropriate references for details.

math.NA