arXiv subjects

Sarah Marzen

Publications and source records attributed to Sarah Marzen.

At least 19 recordsLinked to original sources

arXiv2026

The collective statistical mechanical personality of a group

We propose a mathematical framework for organizational psychology based on a Maximum Entropy model of a group's personalities. The Maximum Entropy model is then decimated to a single ``collective personality''. If the original personality scores are augmented by intelligence and emotional quotients, then a collective intelligence is also mathematically revealed. With simple matrix analyses of the collective personality, one can understand: that weak interpersonal coupling can strongly affect group character; that malleable rather than stubborn personalities control the group's collective personality; that one can mathematically solve for optimal top-down directives to achieve certain group personalities; and that groups can have a personality disorder even if the individuals composing the groups are neurotypical. We hope that this framework provides a useful starting point for future mathematical analyses in organizational psychology related to innate character rather than opinion dynamics or decision making, and note that the analysis can be applied to much more complex Maximum Entropy models than the one proposed here if empirical evidence suggests that the Gaussian model proposed here is overly simplistic.

q-bio.OT↗

arXiv2026

Predictions for and lack of maximal information transmission in the neuromuscular junction

A key question in theoretical biology is how effectively biological systems preserve information about their inputs while operating under physical and functional constraints. We examine that question at the neuromuscular junction (NMJ) by studying how neurotransmitter concentration is transformed into current at both cholinergic and glutamatergic NMJs. An information maximization analysis was used to derive a theoretical distribution over neurotransmitter concentrations based on biological understandings of dose-response relationships. These theoretical distributions were compared to an experimentally derived distribution obtained from a Drosophila NMJ. The theoretical and experimental distributions showed very little agreement, indicating that the Drosophila NMJ does not shape its distribution of synaptic vesicle release probabilities in order to maximize information transmission from nervous system to muscle. Predictions for cholinergic systems are provided.

q-bio.MN↗

arXiv2026

An Investigation of the Channel Capacity of Bacterial Chemotactic Sensors for Low Chemoattractant Concentrations

Bacterial chemotactic sensing converts noisy chemical signals into running and tumbling. We analyze the static sensing limits of mixed Tar/Tsr chemoreceptor clusters in individual \textit{Escherichia coli} cells using a heterogeneous Monod-Wyman-Changeux (MWC) model. Across a seven-dimensional parameter sweep, we compute three sensing-performance metrics -- channel capacity, dynamic range, and effective Hill coefficient -- in the limit that the cells are constantly in such low concentrations of chemoattractant that they need not adapt to new baseline chemoattractant concentration levels. What results are upper bounds on a more complicated trajectory mutual information rate, a quantitative understanding of the tight connection between channel capacity and dynamic range, and the finding that in this regime channel capacity is well described by a closed-form ceiling depending only on the receptor's baseline activity, which every wild-type and mutant strain in our sample achieves to within a few percent. In more realistic scenarios, adaptation plays a larger role and the exact temporal dynamics of chemoattractant concentrations seen by bacteria as they swim. This manuscript thus points to the importance of mapping out naturalistic chemoattractant concentration statistics in the wild as has been done for natural scene statistics.

q-bio.QM↗

arXiv2025

New analytic formulae for memory and prediction functions in reservoir computers with time delays

Time delays increase the effective dimensionality of reservoirs, thus suggesting that time delays in reservoirs can enhance their performance, particularly their memory and prediction abilities. We find new closed-form expressions for memory and prediction functions of linear time-delayed reservoirs in terms of the power spectrum of the input and the reservoir transfer function. We confirm this relationship numerically for some time-delayed reservoirs using simulations, including when the reservoir can be linearized but is actually nonlinear. Finally, we use these closed-form formulae to address the utility of multiple time delays in linear reservoirs in order to perform memory and prediction, finding similar results to previous work on nonlinear reservoirs. We hope these closed-form formulae can be used to understand memory and predictive capabilities in time-delayed reservoirs.

physics.comp-ph↗

arXiv2024

Comment on Deterministic Information Bottleneck

We make the case that although Deterministic Information Bottleneck may be a contribution to clustering, it should not be used to aid lossy compression without the addition of blocklength. We therefore suggest a new objective function that does so and leave its testing to future work.

q-bio.NC↗

arXiv2024

Cognitive biases can move opinion dynamics from consensus to signatures of transient chaos

Interest in how democracies form consensus has increased recently, with statistical physics and economics approaches both suggesting that there is convergence to a fixed point in belief networks, but with fluctuations in opinions when there are ``stubborn'' voters. We modify a model of opinion dynamics in which agents are fully Bayesian to account for two cognitive biases: confirmation bias and in-group bias. Confirmation bias occurs when the received information is considered to be more likely when it aligns with the receiver's beliefs. In-group bias occurs when the receiver further considers the information to be more likely when the receiver's beliefs and the sender's beliefs are aligned. We find that when there are no cognitive biases, a network of agents always converges to complete consensus. With confirmation bias alone, polarization can occur. With both biases present, consensus and polarization are possible, but when agents attempt to counteract confirmation bias, there can be signatures of transient chaos and ongoing opinion fluctuations. Based on this simple model, we conjecture that complex opinion fluctuations might be a generic feature of opinion dynamics when agents are Bayesian with biases.

physics.bio-ph↗

arXiv2024

Resource-rational reinforcement learning and sensorimotor causal states, and resource-rational maximiners

We propose a new computational-level objective function for theoretical biology and theoretical neuroscience that combines: reinforcement learning, the study of learning with feedback via rewards; rate-distortion theory, a branch of information theory that deals with compressing signals to retain relevant information; and computational mechanics, the study of minimal sufficient statistics of prediction also known as causal states. We highlight why this proposal is likely only an approximation, but is likely to be an interesting one, and propose a new algorithm for evaluating it to obtain the newly-coined ``reward-rate manifold''. The performance of real and artificial agents in partially observable environments can be newly benchmarked using these reward-rate manifolds. Finally, we describe experiments that can probe whether or not biological organisms are resource-rational reinforcement learners, using as an example maximin strategies, as bacteria have been shown to be approximate maximiners -- doing their best in the worst-case environment, regardless of what is actually happening.

q-bio.NC↗

arXiv2022

On the role of theory and modeling in neuroscience

In recent years, the field of neuroscience has gone through rapid experimental advances and a significant increase in the use of quantitative and computational methods. This growth has created a need for clearer analyses of the theory and modeling approaches used in the field. This issue is particularly complex in neuroscience because the field studies phenomena across a wide range of scales and often requires consideration of these phenomena at varying degrees of abstraction, from precise biophysical interactions to the computations they implement. We argue that a pragmatic perspective of science, in which descriptive, mechanistic, and normative approaches each play a distinct role in defining and bridging levels of abstraction will facilitate neuroscientific practice. This analysis leads to methodological suggestions, including selecting a level of abstraction that is appropriate for a given problem, identifying transfer functions to connect models and data, and the use of models themselves as a form of experiment.

q-bio.NC↗

arXiv2021

Learning about learning by many-body systems

Diverse many-body systems, from soap bubbles to suspensions to polymers, learn and remember patterns in the drives that push them far from equilibrium. This learning may be leveraged for computation, memory, and engineering. Until now, many-body learning has been detected with thermodynamic properties, such as work absorption and strain. We progress beyond these macroscopic properties first defined for equilibrium contexts: We quantify statistical mechanical learning using representation learning, a machine-learning model in which information squeezes through a bottleneck. By calculating properties of the bottleneck, we measure four facets of many-body systems' learning: classification ability, memory capacity, discrimination ability, and novelty detection. Numerical simulations of a classical spin glass illustrate our technique. This toolkit exposes self-organization that eludes detection by thermodynamic measures: Our toolkit more reliably and more precisely detects and quantifies learning by matter while providing a unifying framework for many-body learning.

cond-mat.stat-mech↗

arXiv2020

Time cells might be optimized for predictive capacity, not redundancy reduction or memory capacity

Recently, researchers have found time cells in the hippocampus that appear to contain information about the timing of past events. Some researchers have argued that time cells are taking a Laplace transform of their input in order to reconstruct the past stimulus. We argue that stimulus prediction, not stimulus reconstruction or redundancy reduction, is in better agreement with observed responses of time cells. In the process, we introduce new analyses of nonlinear, continuous-time reservoirs that model these time cells.

q-bio.NC↗

arXiv2020

First-principles prediction of the information processing capacity of a simple genetic circuit

Given the stochastic nature of gene expression, genetically identical cells exposed to the same environmental inputs will produce different outputs. This heterogeneity has been hypothesized to have consequences for how cells are able to survive in changing environments. Recent work has explored the use of information theory as a framework to understand the accuracy with which cells can ascertain the state of their surroundings. Yet the predictive power of these approaches is limited and has not been rigorously tested using precision measurements. To that end, we generate a minimal model for a simple genetic circuit in which all parameter values for the model come from independently published data sets. We then predict the information processing capacity of the genetic circuit for a suite of biophysical parameters such as protein copy number and protein-DNA affinity. We compare these parameter-free predictions with an experimental determination of protein expression distributions and the resulting information processing capacity of E. coli cells. We find that our minimal model captures the scaling of the cell-to-cell variability in the data and the inferred information processing capacity of our simple genetic circuit up to a systematic deviation.

q-bio.MN↗

arXiv2020

Quantifying many-body learning far from equilibrium with representation learning

Far-from-equilibrium many-body systems, from soap bubbles to suspensions to polymers, learn the drives that push them. This learning has been observed via thermodynamic properties, such as work absorption and strain. We move beyond these macroscopic properties that were first defined for equilibrium contexts: We quantify statistical mechanical learning with machine learning. Our toolkit relies on a structural parallel that we identify between far-from-equilibrium statistical mechanics and representation learning, which is undergone by neural networks that contain bottlenecks, including variational autoencoders. We train a variational autoencoder, via unsupervised learning, on configurations assumed by a many-body system during strong driving. We analyze the neural network's bottleneck to measure the many-body system's classification ability, memory capacity, discrimination ability, and novelty detection. Numerical simulations of a spin glass illustrate our technique. This toolkit exposes self-organization that eludes detection by thermodynamic measures, more reliably and more precisely identifying and quantifying learning by matter.

cond-mat.stat-mech↗

arXiv2017

Intrinsic computation of a Monod-Wyman-Changeux molecule

Causal states are minimal sufficient statistics of prediction of a stochastic process, their coding cost is called statistical complexity, and the implied causal structure yields a sense of the process' "intrinsic computation". We discuss how statistical complexity changes with slight variations on a biologically-motivated dynamical model, that of a Monod-Wyman-Changeux molecule. Perturbations to nonexistent transitions cause statistical complexity to jump from finite to infinite, while perturbations to existent transitions cause relatively slight variations in the statistical complexity. The same is not true for excess entropy, the mutual information between past and future. We discuss the implications of this for the relationship between intrinsic and useful computation of biological sensory systems.

cond-mat.stat-mech↗

arXiv2017

The difference between memory and prediction in linear recurrent networks

Recurrent networks are trained to memorize their input better, often in the hopes that such training will increase the ability of the network to predict. We show that networks designed to memorize input can be arbitrarily bad at prediction. We also find, for several types of inputs, that one-node networks optimized for prediction are nearly at upper bounds on predictive capacity given by Wiener filters, and are roughly equivalent in performance to randomly generated five-node networks. Our results suggest that maximizing memory capacity leads to very different networks than maximizing predictive capacity, and that optimizing recurrent weights can decrease reservoir size by half an order of magnitude.

cs.LG↗

arXiv2016

Weak universality in sensory tradeoffs

For many organisms, the number of sensory neurons is largely determined during development, before strong environmental cues are present. This is despite the fact that environments can fluctuate drastically both from generation to generation and within an organism's lifetime. How can organisms get by by hard-coding the number of sensory neurons? We approach this question using rate-distortion theory. A combination of simulation and theory suggests that when environments are large, the rate-distortion function---a proxy for material costs, timing delays, and energy requirements---depends only on coarse-grained environmental statistics that are expected to change on evolutionary, rather than ontogenetic, timescales.

q-bio.NC↗

arXiv2016

Memory and Information Processing in Recurrent Neural Networks

Recurrent neural networks (RNN) are simple dynamical systems whose computational power has been attributed to their short-term memory. Short-term memory of RNNs has been previously studied analytically only for the case of orthogonal networks, and only under annealed approximation, and uncorrelated input. Here for the first time, we present an exact solution to the memory capacity and the task-solving performance as a function of the structure of a given network instance, enabling direct determination of the function--structure relation in RNNs. We calculate the memory capacity for arbitrary networks with exponentially correlated input and further related it to the performance of the system on signal processing tasks in a supervised learning setup. We compute the expected error and the worst-case error bound as a function of the spectra of the network and the correlation structure of its inputs and outputs. Our results give an explanation for learning and generalization of task solving using short-term memory, which is crucial for building alternative computer architectures using physical phenomena based on the short-term memory principle.

cs.NE↗

arXiv2015

Signatures of Infinity: Nonergodicity and Resource Scaling in Prediction, Complexity, and Learning

We introduce a simple analysis of the structural complexity of infinite-memory processes built from random samples of stationary, ergodic finite-memory component processes. Such processes are familiar from the well known multi-arm Bandit problem. We contrast our analysis with computation-theoretic and statistical inference approaches to understanding their complexity. The result is an alternative view of the relationship between predictability, complexity, and learning that highlights the distinct ways in which informational and correlational divergences arise in complex ergodic and nonergodic processes. We draw out consequences for the resource divergences that delineate the structural hierarchy of ergodic processes and for processes that are themselves hierarchical.

cond-mat.stat-mech↗

arXiv2014

Understanding and Designing Complex Systems: Response to "A framework for optimal high-level descriptions in science and engineering---preliminary report"

We recount recent history behind building compact models of nonlinear, complex processes and identifying their relevant macroscopic patterns or "macrostates". We give a synopsis of computational mechanics, predictive rate-distortion theory, and the role of information measures in monitoring model complexity and predictive performance. Computational mechanics provides a method to extract the optimal minimal predictive model for a given process. Rate-distortion theory provides methods for systematically approximating such models. We end by commenting on future prospects for developing a general framework that automatically discovers optimal compact models. As a response to the manuscript cited in the title above, this brief commentary corrects potentially misleading claims about its state space compression method and places it in a broader historical setting.

cond-mat.stat-mech↗