arXiv subjects

Xue-Xin Wei

Publications and source records attributed to Xue-Xin Wei.

17 recordsLinked to original sources

arXiv2026

In-context superposition: human-like working memory interference in large language models

Intelligent systems must maintain and manipulate task-relevant information online to adapt to dynamic environments. This capacity, known as working memory, is fundamental to human reasoning. Yet, human working memory is strikingly limited, maintaining only three to four items in a brain with billions of neurons. Surprisingly, large language models (LLMs), despite different substrates and direct access to prior context through attention, exhibit similar working memory limitations. Why should such different systems face analogous constraints? We propose that working memory limitations reflect a general trade-off of shared representations: representational compression and reuse support efficient learning and generalization, but also cause simultaneously active representations to interfere. We show a two-layer transformer trained on a working memory task can solve it perfectly, but diverse trained LLMs exhibit human-like limitations: performance declines with memory load, while retrieval is biased by recency and stimulus statistics. Mirroring humans, working memory performance in LLMs is also associated with broader model capability. Mechanistically, we show that LLMs encode multiple memories in entangled representations --- a condition we call \emph{in-context superposition} --- and progressively suppress competing content while aligning the target with the readout. Moreover, a causal intervention that suppresses interfering information improves performance. Together, these findings suggest that working memory capacity reflects the ability to select task-relevant information under interference, a computational challenge shared by biological and artificial systems.

cs.LG↗

arXiv2026

Large language models reorganize representational geometry during in-context learning

Large language models (LLMs) show remarkable flexibility in adapting to novel tasks without parameter updates, a capacity known as in-context learning (ICL). Prior work has sought to understand ICL by studying the circuits, algorithms, and representations that support it. Yet why some ICL tasks are easy to solve while others are difficult remains unresolved. In this paper, we ask whether LLMs can adapt their representations arbitrarily to solve a simple linear classification task. Specifically, we construct a family of binary classification tasks in which labels are defined by projecting LLMs' own representations onto different axes. Surprisingly, although all tasks are linearly separable by construction, their in-context learnability varies systematically across axes. We find that successful ICL is accompanied by a geometric reorganization of internal representations that increases task-relevant separability. Causal interventions that amplify neural activity along the axis defining the task are insufficient to improve behavioral performance or induce this representational reorganization. We also show that LLM behavior is best described by a prototype-like algorithm operating on representations that are themselves reorganized in context to adapt to the task. Together, these findings offer a geometric account of ICL in LLMs, showing that representations acquired through training constrain what can be exploited through in-context learning.

cs.CL↗

arXiv2026

Demystifying Adversarial Robustness in Diffusion Models: Compression, Randomness, and Geometry

Recent studies suggest that diffusion models significantly improve the empirical adversarial robustness of deep neural network models. While intuitive explanations have been proposed, the mechanisms underlying diffusion-based robustness remain largely unclear. This work aims to demystify how diffusion models improve adversarial robustness. We observe that diffusion models surprisingly increase the $\ell_p$ distance to clean samples, thus rejecting the hypothesis that purification denoises perturbed images closer to the clean ones. Next, we provide a unifying account of the robustness improvement in diffusion-based purification by decomposing it into two sources: (i) gradient masking induced by randomness; (ii) compression of the image space. First, we find that the purified images are heavily influenced by the internal randomness of diffusion models. This randomness leads to gradient masking that cannot be removed by the previously proposed remedy, i.e., expectation-over-transformation (EOT). The improvement in robustness due to randomness is determined by the cosine similarity of the optimal vs. empirical attack directions, as predicted by a hyperspherical cap model of the adversarial regions. Second, we find that, when fixing the randomness, diffusion models substantially compress the image space. Importantly, we discover a lawful relationship between the model's ability to compress the image space and the genuine adversarial robustness gain. Further theoretical analyses show that convergent score fields encoded in diffusion models explain these compression effects. Our findings reveal new insights into the mechanisms underlying diffusion-based purification, and offer guidance for developing more effective and principled adversarial purification systems.

cs.LG↗

arXiv2026

The Bayesian Origin of the Probability Weighting Function in Human Representation of Probabilities

Humans systematically misrepresent probability in a stereotyped inverse-S pattern. It has been documented for decades, but its origin remains unexplained. We propose a Bayesian encoding-decoding account in which probabilities are represented by noisy internal signals and decoded by Bayes-risk minimization. For bounded probability stimuli, we show that distortion decomposes into boundary regression, likelihood repulsion, and prior attraction, yielding a key prediction: the classic inverse-S-shaped weighting pattern implies a U-shaped allocation of encoding precision with greater sensitivity near 0 and 1. Across judgment of relative frequency, lottery pricing, and risky choice, this U-shape is recovered from data without imposing any functional form on the encoding, and our framework outperforms deterministic weighting functions, bounded log-odds models, uniform-encoding Bayesian accounts, and matched efficient-coding models on held-out data. In a new dot probability estimation experiment with bimodal stimulus statistics, the recovered prior tracks the new distribution while the recovered encoding remains U-shaped. Together, these results identify the inverse-S-shaped probability weighting function as the joint product of a stable U-shaped encoding and a flexible prior, integrated by optimal Bayesian decoding.

q-bio.NC↗

arXiv2026

Quantifying task-relevant representational similarity using decision variable correlation

Previous studies have compared neural activities in the visual cortex to representations in deep neural networks trained on image classification. Interestingly, while some suggest that their representations are highly similar, others argued the opposite. Here, we propose a new approach to characterize the similarity of the decision strategies of two observers (models or brains) using decision variable correlation (DVC). DVC quantifies the image-by-image correlation between the decoded decisions based on the internal neural representations in a classification task. Thus, it can capture task-relevant information rather than general representational alignment. We evaluate DVC using monkey V4/IT recordings and network models trained on image classification tasks. We find that model-model similarity is comparable to monkey-monkey similarity, whereas model-monkey similarity is consistently lower. Strikingly, DVC decreases with increasing network performance on ImageNet-1k. Adversarial training does not improve model-monkey similarity in task-relevant dimensions assessed using DVC, although it markedly increases the model-model similarity. Similarly, pre-training on larger datasets does not improve model-monkey similarity. These results suggest a divergence between the task-relevant representations in monkey V4/IT and those learned by models trained on image classification tasks.

cs.CV↗

arXiv2025

Diffusion models under low-noise regime

Recent work on diffusion models proposed that they operate in two regimes: memorization, in which models reproduce their training data, and generalization, in which they generate novel samples. While this has been tested in high-noise settings, the behavior of diffusion models as effective denoisers when the corruption level is small remains unclear. To address this gap, we systematically investigated the behavior of diffusion models under low-noise diffusion dynamics, with implications for model robustness and interpretability. Using (i) CelebA subsets of varying sample sizes and (ii) analytic Gaussian mixture benchmarks, we reveal that models trained on disjoint data diverge near the data manifold even when their high-noise outputs converge. We quantify how training set size, data geometry, and model objective choice shape denoising trajectories and affect score accuracy, providing insights into how these models actually learn representations of data distributions. This work starts to address gaps in our understanding of generative model reliability in practical applications where small perturbations are common.

cs.CV↗

arXiv2025

On Conformal Isometry of Grid Cells: Learning Distance-Preserving Position Embedding

This paper investigates the conformal isometry hypothesis as a potential explanation for the hexagonal periodic patterns in grid cell response maps. We posit that grid cell activities form a high-dimensional vector in neural space, encoding the agent's position in 2D physical space. As the agent moves, this vector rotates within a 2D manifold in the neural space, driven by a recurrent neural network. The conformal hypothesis proposes that this neural manifold is a conformal isometric embedding of 2D physical space, where local physical distance is preserved by the embedding up to a scaling factor (or unit of metric). Such distance-preserving position embedding is indispensable for path planning in navigation, especially planning local straight path segments. We conduct numerical experiments to show that this hypothesis leads to the hexagonal grid firing patterns by learning maximally distance-preserving position embedding, agnostic to the choice of the recurrent neural network. Furthermore, we present a theoretical explanation of why hexagon periodic patterns emerge by minimizing our loss function by showing that hexagon flat torus is maximally distance preserving.

q-bio.NC↗

arXiv2024

Emergence of Grid-like Representations by Training Recurrent Networks with Conformal Normalization

Grid cells in the entorhinal cortex of mammalian brains exhibit striking hexagon grid firing patterns in their response maps as the animal (e.g., a rat) navigates in a 2D open environment. In this paper, we study the emergence of the hexagon grid patterns of grid cells based on a general recurrent neural network (RNN) model that captures the navigation process. The responses of grid cells collectively form a high dimensional vector, representing the 2D self-position of the agent. As the agent moves, the vector is transformed by an RNN that takes the velocity of the agent as input. We propose a simple yet general conformal normalization of the input velocity of the RNN, so that the local displacement of the position vector in the high-dimensional neural space is proportional to the local displacement of the agent in the 2D physical space, regardless of the direction of the input velocity. We apply this mechanism to both a linear RNN and nonlinear RNNs. Theoretically, we provide an understanding that explains the connection between conformal normalization and the emergence of hexagon grid patterns. Empirically, we conduct extensive experiments to verify that conformal normalization is crucial for the emergence of hexagon grid patterns, across various types of RNNs. The learned patterns share similar profiles to biological grid cells, and the topological properties of the patterns also align with our theoretical understanding.

q-bio.NC↗

arXiv2022

Conformal Isometry of Lie Group Representation in Recurrent Network of Grid Cells

The activity of the grid cell population in the medial entorhinal cortex (MEC) of the mammalian brain forms a vector representation of the self-position of the animal. Recurrent neural networks have been proposed to explain the properties of the grid cells by updating the neural activity vector based on the velocity input of the animal. In doing so, the grid cell system effectively performs path integration. In this paper, we investigate the algebraic, geometric, and topological properties of grid cells using recurrent network models. Algebraically, we study the Lie group and Lie algebra of the recurrent transformation as a representation of self-motion. Geometrically, we study the conformal isometry of the Lie group representation where the local displacement of the activity vector in the neural space is proportional to the local displacement of the agent in the 2D physical space. Topologically, the compact abelian Lie group representation automatically leads to the torus topology commonly assumed and observed in neuroscience. We then focus on a simple non-linear recurrent model that underlies the continuous attractor neural networks of grid cells. Our numerical experiments show that conformal isometry leads to hexagon periodic patterns in the grid cell responses and our model is capable of accurate path integration. Code is available at \url{https://github.com/DehongXu/grid-cell-rnn}.

q-bio.NC↗

arXiv2021

On Path Integration of Grid Cells: Group Representation and Isotropic Scaling

Understanding how grid cells perform path integration calculations remains a fundamental problem. In this paper, we conduct theoretical analysis of a general representation model of path integration by grid cells, where the 2D self-position is encoded as a higher dimensional vector, and the 2D self-motion is represented by a general transformation of the vector. We identify two conditions on the transformation. One is a group representation condition that is necessary for path integration. The other is an isotropic scaling condition that ensures locally conformal embedding, so that the error in the vector representation translates conformally to the error in the 2D self-position. Then we investigate the simplest transformation, i.e., the linear transformation, uncover its explicit algebraic and geometric structure as matrix Lie group of rotation, and explore the connection between the isotropic scaling condition and a special class of hexagon grid patterns. Finally, with our optimization-based approach, we manage to learn hexagon grid patterns that share similar properties of the grid cells in the rodent brain. The learned model is capable of accurate long distance path integration. Code is available at https://github.com/ruiqigao/grid-cell-path.

q-bio.NC↗

arXiv2021

Neural tuning and representational geometry

A central goal of neuroscience is to understand the representations formed by brain activity patterns and their connection to behavior. The classical approach is to investigate how individual neurons encode the stimuli and how their tuning determines the fidelity of the neural representation. Tuning analyses often use the Fisher information to characterize the sensitivity of neural responses to small changes of the stimulus. In recent decades, measurements of large populations of neurons have motivated a complementary approach, which focuses on the information available to linear decoders. The decodable information is captured by the geometry of the representational patterns in the multivariate response space. Here we review neural tuning and representational geometry with the goal of clarifying the relationship between them. The tuning induces the geometry, but different sets of tuned neurons can induce the same geometry. The geometry determines the Fisher information, the mutual information, and the behavioral performance of an ideal observer in a range of psychophysical tasks. We argue that future studies can benefit from considering both tuning and geometry to understand neural codes and reveal the connections between stimulus, brain activity, and behavior.

q-bio.NC↗

arXiv2020

Learning identifiable and interpretable latent models of high-dimensional neural activity using pi-VAE

The ability to record activities from hundreds of neurons simultaneously in the brain has placed an increasing demand for developing appropriate statistical techniques to analyze such data. Recently, deep generative models have been proposed to fit neural population responses. While these methods are flexible and expressive, the downside is that they can be difficult to interpret and identify. To address this problem, we propose a method that integrates key ingredients from latent models and traditional neural encoding models. Our method, pi-VAE, is inspired by recent progress on identifiable variational auto-encoder, which we adapt to make appropriate for neuroscience applications. Specifically, we propose to construct latent variable models of neural activity while simultaneously modeling the relation between the latent and task variables (non-neural variables, e.g. sensory, motor, and other externally observable states). The incorporation of task variables results in models that are not only more constrained, but also show qualitative improvements in interpretability and identifiability. We validate pi-VAE using synthetic data, and apply it to analyze neurophysiological datasets from rat hippocampus and macaque motor cortex. We demonstrate that pi-VAE not only fits the data better, but also provides unexpected novel insights into the structure of the neural codes.

stat.ML↗

arXiv2020

A zero-inflated gamma model for deconvolved calcium imaging traces

Calcium imaging is a critical tool for measuring the activity of large neural populations. Much effort has been devoted to developing "pre-processing" tools for calcium video data, addressing the important issues of e.g., motion correction, denoising, compression, demixing, and deconvolution. However, statistical modeling of deconvolved calcium signals (i.e., the estimated activity extracted by a pre-processing pipeline) is just as critical for interpreting calcium measurements, and for incorporating these observations into downstream probabilistic encoding and decoding models. Surprisingly, these issues have to date received significantly less attention. In this work we examine the statistical properties of the deconvolved activity estimates, and compare probabilistic models for these random signals. In particular, we propose a zero-inflated gamma (ZIG) model, which characterizes the calcium responses as a mixture of a gamma distribution and a point mass that serves to model zero responses. We apply the resulting models to neural encoding and decoding problems. We find that the ZIG model outperforms simpler models (e.g., Poisson or Bernoulli models) in the context of both simulated and real neural data, and can therefore play a useful role in bridging calcium imaging analysis methods with tools for analyzing activity in large neural populations.

q-bio.NC↗

arXiv2020

Emergence of functional and structural properties of the head direction system by optimization of recurrent neural networks

Recent work suggests goal-driven training of neural networks can be used to model neural activity in the brain. While response properties of neurons in artificial neural networks bear similarities to those in the brain, the network architectures are often constrained to be different. Here we ask if a neural network can recover both neural representations and, if the architecture is unconstrained and optimized, the anatomical properties of neural circuits. We demonstrate this in a system where the connectivity and the functional organization have been characterized, namely, the head direction circuits of the rodent and fruit fly. We trained recurrent neural networks (RNNs) to estimate head direction through integration of angular velocity. We found that the two distinct classes of neurons observed in the head direction system, the Compass neurons and the Shifter neurons, emerged naturally in artificial neural networks as a result of training. Furthermore, connectivity analysis and in-silico neurophysiology revealed structural and mechanistic similarities between artificial networks and the head direction system. Overall, our results show that optimization of RNNs in a goal-driven task can recapitulate the structure and function of biological circuits, suggesting that artificial neural networks can be used to study the brain at the level of both neural activity and anatomical organization.

q-bio.NC↗

arXiv2018

Dynamic self-organized error-correction of grid cells by border cells

Grid cells in the entorhinal cortex are believed to establish their regular, spatially correlated firing patterns by path integration of the animal's motion. Mechanisms for path integration, e.g. in attractor network models, predict stochastic drift of grid responses, which is not observed experimentally. We demonstrate a biologically plausible mechanism of dynamic self-organization by which border cells, which fire at environmental boundaries, can correct such drift in grid cells. In our model, experience-dependent Hebbian plasticity during exploration allows border cells to learn connectivity to grid cells. Border cells in this learned network reset the phase of drifting grids. This error-correction mechanism is robust to environmental shape and complexity, including enclosures with interior barriers, and makes distinctive predictions for environmental deformation experiments. Our work demonstrates how diverse cell types in the entorhinal cortex could interact dynamically and adaptively to achieve robust path integration.

q-bio.NC↗

arXiv2018

Emergence of grid-like representations by training recurrent neural networks to perform spatial localization

Decades of research on the neural code underlying spatial navigation have revealed a diverse set of neural response properties. The Entorhinal Cortex (EC) of the mammalian brain contains a rich set of spatial correlates, including grid cells which encode space using tessellating patterns. However, the mechanisms and functional significance of these spatial representations remain largely mysterious. As a new way to understand these neural representations, we trained recurrent neural networks (RNNs) to perform navigation tasks in 2D arenas based on velocity inputs. Surprisingly, we find that grid-like spatial response patterns emerge in trained networks, along with units that exhibit other spatial correlates, including border cells and band-like cells. All these different functional types of neurons have been observed experimentally. The order of the emergence of grid-like and border cells is also consistent with observations from developmental studies. Together, our results suggest that grid cells, border cells and others as observed in EC may be a natural solution for representing space efficiently given the predominant recurrent connections in the neural circuits.

q-bio.NC↗

arXiv2013

The Sense of Place: Grid Cells in the Brain and the Transcendental Number e

Grid cells in the brain respond when an animal occupies a periodic lattice of "grid fields" during spatial navigation. The grid scale varies along the dorso-ventral axis of the entorhinal cortex. We propose that the grid system minimizes the number of neurons required to encode location with a given resolution. We derive several predictions that match recent experiments: (i) grid scales follow a geometric progression, (ii) the ratio between adjacent grid scales is the square root of e for idealized neurons, and robustly lies in the range 1.4-1.7 for realistic neurons, (iii) the scale ratio varies modestly within and between animals, (iv) the ratio between grid scale and individual grid field widths at that scale also lies in this range, (v) grid fields lie on a triangular lattice. The theory also predicts the optimal grids in one and three dimensions, and the total number of discrete scales.

q-bio.NC↗