arXiv subjects

William Bialek

Publications and source records attributed to William Bialek.

At least 73 records · Page 4Linked to original sources

arXiv2013

Complexity in genetic networks: topology vs. strength of interactions

Genetic regulatory networks are defined by their topology and by a multitude of continuously adjustable parameters. Here we present a class of simple models within which the relative importance of topology vs. interaction strengths becomes a well-posed problem. We find that complexity - the ability of the network to adopt multiple stable states - is dominated by the adjustable parameters. We comment on the implications for real networks and their evolution.

q-bio.MN↗

arXiv2013

Social interactions dominate speed control in driving natural flocks toward criticality

Flocks of birds exhibit a remarkable degree of coordination and collective response. It is not just that thousands of individuals fly, on average, in the same direction and at the same speed, but that even the fluctuations around the mean velocity are correlated over long distances. Quantitative measurements on flocks of starlings, in particular, show that these fluctuations are scale-free, with effective correlation lengths proportional to the linear size of the flock. Here we construct models for the joint distribution of velocities in the flock that reproduce the observed local correlations between individuals and their neighbors, as well as the variance of flight speeds across individuals, but otherwise have as little structure as possible. These minimally structured, or maximum entropy models provide quantitative, parameter-free predictions for the spread of correlations throughout the flock, and these are in excellent agreement with the data. These models are mathematically equivalent to statistical physics models for ordering in magnets, and the correct prediction of scale-free correlations arises because the parameters - completely determined by the data - are in the critical regime. In biological terms, criticality allows the flock to achieve maximal correlation across long distances with limited speed fluctuations.

physics.bio-ph↗

arXiv2013

Predictive information in a sensory population

Guiding behavior requires the brain to make predictions about future sensory inputs. Here we show that efficient predictive computation starts at the earliest stages of the visual system. We estimate how much information groups of retinal ganglion cells carry about the future state of their visual inputs, and show that every cell we can observe participates in a group of cells for which this predictive information is close to the physical limit set by the statistical structure of the inputs themselves. Groups of cells in the retina also carry information about the future state of their own activity, and we show that this information can be compressed further and encoded by downstream predictor neurons, which then exhibit interesting feature selectivity. Efficient representation of predictive information is a candidate principle that can be applied at each stage of neural computation.

q-bio.NC↗

arXiv2013

Statistical mechanics of the US Supreme Court

We build simple models for the distribution of voting patterns in a group, using the Supreme Court of the United States as an example. The least structured, or maximum entropy, model that is consistent with the observed pairwise correlations among justices' votes is equivalent to an Ising spin glass. While all correlations (perhaps surprisingly) are positive, the effective pairwise interactions in the spin glass model have both signs, recovering some of our intuition that justices on opposite sides of the ideological spectrum should have a negative influence on one another. Despite the competing interactions, a strong tendency toward unanimity emerges from the model, and this agrees quantitatively with the data. The model shows that voting patterns are organized in a relatively simple "energy landscape," correctly predicts the extent to which each justice is correlated with the majority, and gives us a measure of the influence that justices exert on one another. These results suggest that simple models, grounded in statistical physics, can capture essential features of collective decision making quantitatively, even in a complex political context.

physics.soc-ph↗

arXiv2013

Searching for collective behavior in a network of real neurons

Maximum entropy models are the least structured probability distributions that exactly reproduce a chosen set of statistics measured in an interacting network. Here we use this principle to construct probabilistic models which describe the correlated spiking activity of populations of up to 120 neurons in the salamander retina as it responds to natural movies. Already in groups as small as 10 neurons, interactions between spikes can no longer be regarded as small perturbations in an otherwise independent system; for 40 or more neurons pairwise interactions need to be supplemented by a global interaction that controls the distribution of synchrony in the population. Here we show that such "K-pairwise" models--being systematic extensions of the previously used pairwise Ising models--provide an excellent account of the data. We explore the properties of the neural vocabulary by: 1) estimating its entropy, which constrains the population's capacity to represent visual information; 2) classifying activity patterns into a small set of metastable collective modes; 3) showing that the neural codeword ensembles are extremely inhomogenous; 4) demonstrating that the state of individual neurons is highly predictable from the rest of the population, allowing the capacity for error correction.

q-bio.NC↗

arXiv2012

The simplest maximum entropy model for collective behavior in a neural network

Recent work emphasizes that the maximum entropy principle provides a bridge between statistical mechanics models for collective behavior in neural networks and experiments on networks of real neurons. Most of this work has focused on capturing the measured correlations among pairs of neurons. Here we suggest an alternative, constructing models that are consistent with the distribution of global network activity, i.e. the probability that K out of N cells in the network generate action potentials in the same small time bin. The inverse problem that we need to solve in constructing the model is analytically tractable, and provides a natural "thermodynamics" for the network in the limit of large N. We analyze the responses of neurons in a small patch of the retina to naturalistic stimuli, and find that the implied thermodynamics is very close to an unusual critical point, in which the entropy (in proper units) is exactly equal to the energy.

q-bio.NC↗

arXiv2012

Maximally informative "stimulus energies" in the analysis of neural responses to natural signals

The concept of feature selectivity in sensory signal processing can be formalized as dimensionality reduction: in a stimulus space of very high dimensions, neurons respond only to variations within some smaller, relevant subspace. But if neural responses exhibit invariances, then the relevant subspace typically cannot be reached by a Euclidean projection of the original stimulus. We argue that, in several cases, we can make progress by appealing to the simplest nonlinear construction, identifying the relevant variables as quadratic forms, or "stimulus energies." Natural examples include non-phase-locked cells in the auditory system, complex cells in visual cortex, and motion-sensitive neurons in the visual system. Generalizing the idea of maximally informative dimensions, we show that one can search for the kernels of the relevant quadratic forms by maximizing the mutual information between the stimulus energy and the arrival times of action potentials. Simple implementations of this idea successfully recover the underlying properties of model neurons even when the number of parameters in the kernel is comparable to the number of action potentials and stimuli are completely natural. We explore several generalizations that allow us to incorporate plausible structure into the kernel and thereby restrict the number of parameters. We hope that this approach will add significantly to the set of tools available for the analysis of neural responses to complex, naturalistic stimuli.

q-bio.NC↗

arXiv2011

Positional information, in bits

Cells in a developing embryo have no direct way of "measuring" their physical position. Through a variety of processes, however, the expression levels of multiple genes come to be correlated with position, and these expression levels thus form a code for "positional information." We show how to measure this information, in bits, using the gap genes in the Drosophila embryo as an example. Individual genes carry nearly two bits of information, twice as much as expected if the expression patterns consisted only of on/off domains separated by sharp boundaries. Taken together, four gap genes carry enough information to define a cell's location with an error bar of ~1% along the anterior-posterior axis of the embryo. This precision is nearly enough for each cell to have a unique identity, which is the maximum information the system can use, and is nearly constant along the length of the embryo. We argue that this constancy is a signature of optimality in the transmission of information from primary morphogen inputs to the output of the gap gene network.

q-bio.MN↗

arXiv2011

Optimizing information flow in small genetic networks. III. A self-interacting gene

Living cells must control the reading out or "expression" of information encoded in their genomes, and this regulation often is mediated by transcription factors--proteins that bind to DNA and either enhance or repress the expression of nearby genes. But the expression of transcription factor proteins is itself regulated, and many transcription factors regulate their own expression in addition to responding to other input signals. Here we analyze the simplest of such self-regulatory circuits, asking how parameters can be chosen to optimize information transmission from inputs to outputs in the steady state. Some nonzero level of self-regulation is almost always optimal, with self-activation dominant when transcription factor concentrations are low and self-repression dominant when concentrations are high. In steady state the optimal self-activation is never strong enough to induce bistability, although there is a limit in which the optimal parameters are very close to the critical point.

q-bio.MN↗

arXiv2011

Statistical mechanics for natural flocks of birds

Interactions among neighboring birds in a flock cause an alignment of their flight directions. We show that the minimally structured (maximum entropy) model consistent with these local correlations correctly predicts the propagation of order throughout entire flocks of starlings, with no free parameters. These models are mathematically equivalent to the Heisenberg model of magnetism, and define an "energy" for each configuration of flight directions in the flock. Comparing flocks of different densities, the range of interactions that contribute to the energy involves a fixed number of (topological) neighbors, rather than a fixed (metric) spatial range. Comparing flocks of different sizes, the model correctly accounts for the observed scale invariance of long ranged correlations among the fluctuations in flight direction.

physics.bio-ph↗

arXiv2010

When are correlations strong?

The inverse problem of statistical mechanics involves finding the minimal Hamiltonian that is consistent with some observed set of correlation functions. This problem has received renewed interest in the analysis of biological networks; in particular, several such networks have been described successfully by maximum entropy models consistent with pairwise correlations. These correlations are usually weak in an absolute sense (e.g., correlation coefficients ~ 0.1 or less), and this is sometimes taken as evidence against the existence of interesting collective behavior in the network. If correlations are weak, it should be possible to capture their effects in perturbation theory, so we develop an expansion for the entropy of Ising systems in powers of the correlations, carrying this out to fourth order. We then consider recent work on networks of neurons [Schneidman et al., Nature 440, 1007 (2006); Tkacik et al., arXiv:0912.5409 [q-bio.NC] (2009)], and show that even though all pairwise correlations are weak, the fact that these correlations are widespread means that their impact on the network as a whole is not captured in the leading orders of perturbation theory. More positively, this means that recent successes of maximum entropy approaches are not simply the result of correlations being weak.

q-bio.NC↗

arXiv2010

Searching for simplicity: Approaches to the analysis of neurons and behavior

What fascinates us about animal behavior is its richness and complexity, but understanding behavior and its neural basis requires a simpler description. Traditionally, simplification has been imposed by training animals to engage in a limited set of behaviors, by hand scoring behaviors into discrete classes, or by limiting the sensory experience of the organism. An alternative is to ask whether we can search through the dynamics of natural behaviors to find explicit evidence that these behaviors are simpler than they might have been. We review two mathematical approaches to simplification, dimensionality reduction and the maximum entropy method, and we draw on examples from different levels of biological organization, from the crawling behavior of C. elegans to the control of smooth pursuit eye movements in primates, and from the coding of natural scenes by networks of neurons in the retina to the rules of English spelling. In each case, we argue that the explicit search for simplicity uncovers new and unexpected features of the biological system, and that the evidence for simplification gives us a language with which to phrase new questions for the next generation of experiments. The fact that similar mathematical structures succeed in taming the complexity of very different biological systems hints that there is something more general to be discovered.

q-bio.NC↗

arXiv2010

Are biological systems poised at criticality?

Many of life's most fascinating phenomena emerge from interactions among many elements--many amino acids determine the structure of a single protein, many genes determine the fate of a cell, many neurons are involved in shaping our thoughts and memories. Physicists have long hoped that these collective behaviors could be described using the ideas and methods of statistical mechanics. In the past few years, new, larger scale experiments have made it possible to construct statistical mechanics models of biological systems directly from real data. We review the surprising successes of this "inverse" approach, using examples form families of proteins, networks of neurons, and flocks of birds. Remarkably, in all these cases the models that emerge from the data are poised at a very special point in their parameter space--a critical point. This suggests there may be some deeper theoretical principle behind the behavior of these diverse systems.

q-bio.QM↗

arXiv2009

Spin glass models for a network of real neurons

Ising models with pairwise interactions are the least structured, or maximum-entropy, probability distributions that exactly reproduce measured pairwise correlations between spins. Here we use this equivalence to construct Ising models that describe the correlated spiking activity of populations of 40 neurons in the salamander retina responding to natural movies. We show that pairwise interactions between neurons account for observed higher-order correlations, and that for groups of 10 or more neurons pairwise interactions can no longer be regarded as small perturbations in an independent system. We then construct network ensembles that generalize the network instances observed in the experiment, and study their thermodynamic behavior and coding capacity. Based on this construction, we can also create synthetic networks of 120 neurons, and find that with increasing size the networks operate closer to a critical point and start exhibiting collective behaviors reminiscent of spin glasses. We examine closely two such behaviors that could be relevant for neural code: tuning of the network to the critical point to maximize the ability to encode diverse stimuli, and using the metastable states of the Ising Hamiltonian as neural code words.

q-bio.NC↗

arXiv2009

Optimizing information flow in small genetic networks. II: Feed forward interactions

Central to the functioning of a living cell is its ability to control the readout or expression of information encoded in the genome. In many cases, a single transcription factor protein activates or represses the expression of many genes. As the concentration of the transcription factor varies, the target genes thus undergo correlated changes, and this redundancy limits the ability of the cell to transmit information about input signals. We explore how interactions among the target genes can reduce this redundancy and optimize information transmission. Our discussion builds on recent work [Tkacik et al, Phys Rev E 80, 031920 (2009)], and there are connections to much earlier work on the role of lateral inhibition in enhancing the efficiency of information transmission in neural circuits; for simplicity we consider here the case where the interactions have a feed forward structure, with no loops. Even with this limitation, the networks that optimize information transmission have a structure reminiscent of the networks found in real biological systems.

q-bio.MN↗

arXiv2009

The emergence of stereotyped behaviors in C. elegans

Animal behaviors are sometimes decomposable into discrete, stereotyped elements. In one model, such behaviors are triggered by specific commands; in the extreme case, the discreteness of behavior is traced to the discreteness of action potentials in the individual command neurons. We use the crawling behavior of the nematode C. elegans to explore the opposite extreme, in which discreteness and stereotypy emerges from the dynamics of the entire behavior. A simple stochastic model for the worm's continuously changing body shape during crawling has attractors corresponding to forward and backward motion; noise-driven transitions between these attractors correspond to abrupt reversals. We show that, with no free parameters, this model generates reversals at a rate within error bars of that observed experimentally, and the relatively stereotyped trajectories in the neighborhood of the reversal also are predicted correctly.

q-bio.NC↗

arXiv2009

Maximum entropy models for antibody diversity

Recognition of pathogens relies on families of proteins showing great diversity. Here we construct maximum entropy models of the sequence repertoire, building on recent experiments that provide a nearly exhaustive sampling of the IgM sequences in zebrafish. These models are based solely on pairwise correlations between residue positions, but correctly capture the higher order statistical properties of the repertoire. Exploiting the interpretation of these models as statistical physics problems, we make several predictions for the collective properties of the sequence ensemble: the distribution of sequences obeys Zipf's law, the repertoire decomposes into several clusters, and there is a massive restriction of diversity due to the correlations. These predictions are completely inconsistent with models in which amino acid substitutions are made independently at each site, and are in good agreement with the data. Our results suggest that antibody diversity is not limited by the sequences encoded in the genome, and may reflect rapid adaptation to antigenic challenges. This approach should be applicable to the study of the global properties of other protein families.

q-bio.GN↗

arXiv2009

From modes to movement in the behavior of C. elegans

Organisms move through the world by changing their shape, and here we explore the mapping from shape space to movements in the nematode C. elegans as it crawls on a planar agar surface. We characterize the statistics of the trajectories through the correlation functions of the orientation angular velocity, orientation angle and the mean-squared displacement, and we find that the loss of orientational memory has significant contributions from both abrupt, large amplitude turning events and the continuous dynamics between these events. Further, we demonstrate long-time persistence of orientational memory in the intervals between abrupt turns. Building on recent work demonstrating that C. elegans movements are restricted to a low-dimensional shape space, we construct a map from the dynamics in this shape space to the trajectory of the worm along the agar. We use this connection to illustrate that changes in the continuous dynamics reveal subtle differences in movement strategy that occur among mutants defective in two classes of dopamine receptors.

q-bio.NC↗