arXiv subjects

Stephanie E Palmer

Publications and source records attributed to Stephanie E Palmer.

3 recordsLinked to original sources

arXiv2026

Representational drift changes the encoding of fast and slow-varying natural scene features differently

Representational drift refers to an unstable mapping between neural activity and input sensory or output behavioral variables. While much work has focused on the effect of representational drift on single, simple external variables, we investigate the differences in representational drift across spatiotemporal features in a moving visual stimulus. The neural responses across animals to the same movie reflect both common, encoded stimulus features and idiosyncratic individual variation. To extract the shared neural encoding of stimulus features only, we learn a latent space embedding using weakly supervised contrastive learning. This approach pulls neural activity together in the embedding space if they are responses to the same stimulus segment and push them apart if not. This approach enables us to probe how stimulus features fluctuating as fast as 33 ms (the movie frame rate) are encoded by variable neural codes across animals. It also allows us to investigate how representational drift changes the encoding in individuals across sessions. We observe that our learned embedding is near-optimal for decoding natural features (background scenery, local motion, complex spatio-temporal features, and time) and neural activity from novel animals. This suggests that our embedding retains the encoding of multiple features at higher temporal granularity compared to previous methods. To quantify representational drift, we apply the trained decoder (which achieves near-optimal performance in one session) to a subsequent session recorded 90 minutes later. We then use the decrease in decoding performance as a proxy for the magnitude of drift. We show that the drift changes the encoding of fast-varying local motion features at a rate 5-6 times higher than slower-varying scenery features. Drift also perturbs the local geometry in the embedding.

q-bio.NC↗

arXiv2025

Understanding temperature tuning in energy-based models

Generative models of complex systems often require post-hoc parameter adjustments to produce useful outputs. For example, energy-based models for protein design are sampled at an artificially low ''temperature'' to generate novel, functional sequences. This temperature tuning is a common yet poorly understood heuristic used across machine learning contexts to control the trade-off between generative fidelity and diversity. Here, we develop an interpretable, physically motivated framework to explain this phenomenon. We demonstrate that in systems with a large ''energy gap'' - separating a small fraction of meaningful states from a vast space of unrealistic states - learning from sparse data causes models to systematically overestimate high-energy state probabilities, a bias that lowering the sampling temperature corrects. More generally, we characterize how the optimal sampling temperature depends on the interplay between data size and the system's underlying energy landscape. Crucially, our results show that lowering the sampling temperature is not always desirable; we identify the conditions where \emph{raising} it results in better generative performance. Our framework thus casts post-hoc temperature tuning as a diagnostic tool that reveals properties of the true data distribution and the limits of the learned model.

q-bio.QM↗

arXiv2024

Towards understanding neural collapse in supervised contrastive learning with the information bottleneck method

Neural collapse describes the geometry of activation in the final layer of a deep neural network when it is trained beyond performance plateaus. Open questions include whether neural collapse leads to better generalization and, if so, why and how training beyond the plateau helps. We model neural collapse as an information bottleneck (IB) problem in order to investigate whether such a compact representation exists and discover its connection to generalization. We demonstrate that neural collapse leads to good generalization specifically when it approaches an optimal IB solution of the classification problem. Recent research has shown that two deep neural networks independently trained with the same contrastive loss objective are linearly identifiable, meaning that the resulting representations are equivalent up to a matrix transformation. We leverage linear identifiability to approximate an analytical solution of the IB problem. This approximation demonstrates that when class means exhibit $K$-simplex Equiangular Tight Frame (ETF) behavior (e.g., $K$=10 for CIFAR10 and $K$=100 for CIFAR100), they coincide with the critical phase transitions of the corresponding IB problem. The performance plateau occurs once the optimal solution for the IB problem includes all of these phase transitions. We also show that the resulting $K$-simplex ETF can be packed into a $K$-dimensional Gaussian distribution using supervised contrastive learning with a ResNet50 backbone. This geometry suggests that the $K$-simplex ETF learned by supervised contrastive learning approximates the optimal features for source coding. Hence, there is a direct correspondence between optimal IB solutions and generalization in contrastive learning.

cs.LG↗