arXiv subjects

William Bialek

Publications and source records attributed to William Bialek.

At least 91 records · Page 5Linked to original sources

arXiv2009

Optimizing information flow in small genetic networks. I

In order to survive, reproduce and (in multicellular organisms) differentiate, cells must control the concentrations of the myriad different proteins that are encoded in the genome. The precision of this control is limited by the inevitable randomness of individual molecular events. Here we explore how cells can maximize their control power in the presence of these physical limits; formally, we solve the theoretical problem of maximizing the information transferred from inputs to outputs when the number of available molecules is held fixed. We start with the simplest version of the problem, in which a single transcription factor protein controls the readout of one or more genes by binding to DNA. We further simplify by assuming that this regulatory network operates in steady state, that the noise is small relative to the available dynamic range, and that the target genes do not interact. Even in this simple limit, we find a surprisingly rich set of optimal solutions. Importantly, for each locally optimal regulatory network, all parameters are determined once the physical constraints on the number of available molecules are specified. Although we are solving an over--simplified version of the problem facing real cells, we see parallels between the structure of these optimal solutions and the behavior of actual genetic regulatory networks. Subsequent papers will discuss more complete versions of the problem.

q-bio.MN↗

arXiv2008

Thermodynamics of natural images

The scale invariance of natural images suggests an analogy to the statistical mechanics of physical systems at a critical point. Here we examine the distribution of pixels in small image patches and show how to construct the corresponding thermodynamics. We find evidence for criticality in a diverging specific heat, which corresponds to large fluctuations in how "surprising" we find individual images, and in the quantitative form of the entropy vs. energy. The energy landscape derived from our thermodynamic framework identifies special image configurations that have intrinsic error correcting properties, and neurons which could detect these features have a strong resemblance to the cells found in primary visual cortex.

q-bio.NC↗

arXiv2007

Toward a statistical mechanics of four letter words

We consider words as a network of interacting letters, and approximate the probability distribution of states taken on by this network. Despite the intuition that the rules of English spelling are highly combinatorial (and arbitrary), we find that maximum entropy models consistent with pairwise correlations among letters provide a surprisingly good approximation to the full statistics of four letter words, capturing ~92% of the multi-information among letters and even "discovering" real words that were not represented in the data from which the pairwise correlations were estimated. The maximum entropy model defines an energy landscape on the space of possible words, and local minima in this landscape account for nearly two-thirds of words used in written English.

q-bio.NC↗

arXiv2007

Efficient representation as a design principle for neural coding and computation

Does the brain construct an efficient representation of the sensory world? We review progress on this question, focusing on a series of experiments in the last decade which use fly vision as a model system in which theory and experiment can confront each other. Although the idea of efficient representation has been productive, clearly it is incomplete since it doesn't tell us which bits of sensory information are most valuable to the organism. We suggest that an organism which maximizes the (biologically meaningful) adaptive value of its actions given fixed resources should have internal representations of the outside world that are optimal in a very specific information theoretic sense: they maximize the information about the future of sensory inputs at a fixed value of the information about their past. This principle contains as special cases computations which the brain seems to carry out, and it should be possible to test this optimization directly. We return to the fly visual system and report the results of preliminary experiments that are in encouraging agreement with theory.

q-bio.NC↗

arXiv2007

Information and fitness

The growth rate of organisms depends both on external conditions and on internal states, such as the expression levels of various genes. We show that to achieve a criterion mean growth rate over an ensemble of conditions, the internal variables must carry a minimum number of bits of information about those conditions. Evolutionary competition thus can select for cellular mechanisms that are more efficient in an abstract, information theoretic sense. Estimates based on recent experiments suggest that the minimum information required for reasonable growth rates is close to the maximum information that can be conveyed through biologically realistic regulatory mechanisms. These ideas are applicable most directly to unicellular organisms, but there are analogies to problems in higher organisms, and we suggest new experiments for both cases.

q-bio.PE↗

arXiv2007

Cell biology: Networks, regulation, pathways

This review was written for the Encyclopedia of Complexity and System Science (Springer-Verlag, Berlin, 2008), and is intended as a guide to the growing literature which approaches the phenomena of cell biology from a more theoretical point of view. We begin with the building blocks of cellular networks, and proceed toward the different classes of models being explored, finally discussing the "design principles" which have been suggested for these systems. Although largely a dispassionate review, we do draw attention to areas where there seems to be general consensus on ideas that have not been tested very thoroughly and, more optimistically, to areas where we feel promising ideas deserve to be more fully explored.

q-bio.MN↗

arXiv2007

Rediscovering the power of pairwise interactions

Two recent streams of work suggest that pairwise interactions may be sufficient to capture the complexity of biological systems ranging from protein structure to networks of neurons. In one approach, possible amino acid sequences in a family of proteins are generated by Monte Carlo annealing of a "Hamiltonian" that forces pairwise correlations among amino acid substitutions to be close to the observed correlations. In the other approach, the observed correlations among pairs of neurons are used to construct a maximum entropy model for the states of the network as a whole. We show that, in certain limits, these two approaches are mathematically equivalent, and we comment on open problems suggested by this framework

q-bio.QM↗

arXiv2007

Faster solutions of the inverse pairwise Ising problem

Recent work has shown that probabilistic models based on pairwise interactions-in the simplest case, the Ising model-provide surprisingly accurate descriptions of experiments on real biological networks ranging from neurons to genes. Finding these models requires us to solve an inverse problem: given experimentally measured expectation values, what are the parameters of the underlying Hamiltonian? This problem sits at the intersection of statistical physics and machine learning, and we suggest that more efficient solutions are possible by merging ideas from the two fields. We use a combination of recent coordinate descent algorithms with an adaptation of the histogram Monte Carlo method, and implement these techniques to take advantage of the sparseness found in data on real neurons. The resulting algorithm learns the parameters of an Ising model describing a network of forty neurons within a few minutes. This opens the possibility of analyzing much larger data sets now emerging, and thus testing hypotheses about the collective behaviors of these networks.

q-bio.QM↗

arXiv2007

Diffusion, dimensionality and noise in transcriptional regulation

The precision of biochemical signaling is limited by randomness in the diffusive arrival of molecules at their targets. For proteins binding to the specific sites on the DNA and regulating transcription, the ability of the proteins to diffuse in one dimension by sliding along the length of the DNA, in addition to their diffusion in bulk solution, would seem to generate a larger target for DNA binding, consequently reducing the noise in the occupancy of the regulatory site. Here we show that this effect is largely cancelled by the enhanced temporal correlations in one dimensional diffusion. With realistic parameters, sliding along DNA has surprisingly little effect on the physical limits to the precision of transcriptional regulation.

q-bio.MN↗

arXiv2007

Information capacity of genetic regulatory elements

Changes in a cell's external or internal conditions are usually reflected in the concentrations of the relevant transcription factors. These proteins in turn modulate the expression levels of the genes under their control and sometimes need to perform non-trivial computations that integrate several inputs and affect multiple genes. At the same time, the activities of the regulated genes would fluctuate even if the inputs were held fixed, as a consequence of the intrinsic noise in the system, and such noise must fundamentally limit the reliability of any genetic computation. Here we use information theory to formalize the notion of information transmission in simple genetic regulatory elements in the presence of physically realistic noise sources. The dependence of this "channel capacity" on noise parameters, cooperativity and cost of making signaling molecules is explored systematically. We find that, at least in principle, capacities higher than one bit should be achievable and that consequently genetic regulation is not limited the use of binary, or "on-off", components.

q-bio.MN↗

arXiv2007

Dimensionality and dynamics in the behavior of C. elegans

A major challenge in analyzing animal behavior is to discover some underlying simplicity in complex motor actions. Here we show that the space of shapes adopted by the nematode C. elegans is surprisingly low dimensional, with just four dimensions accounting for 95% of the shape variance, and we partially reconstruct "equations of motion" for the dynamics in this space. These dynamics have multiple attractors, and we find that the worm visits these in a rapid and almost completely deterministic response to weak thermal stimuli. Stimulus-dependent correlations among the different modes suggest that one can generate more reliable behaviors by synchronizing stimuli to the state of the worm in shape space. We confirm this prediction, effectively "steering" the worm in real time.

q-bio.OT↗

arXiv2007

Information flow and optimization in transcriptional control

In the simplest view of transcriptional regulation, the expression of a gene is turned on or off by changes in the concentration of a transcription factor (TF). We use recent data on noise levels in gene expression to show that it should be possible to transmit much more than just one regulatory bit. Realizing this optimal information capacity would require that the dynamic range of TF concentrations used by the cell, the input/output relation of the regulatory module, and the noise levels of binding and transcription satisfy certain matching relations. This parameter-free prediction is in good agreement with recent experiments on the Bicoid/Hunchback system in the early Drosophila embryo, and this system achieves ~90% of its theoretical maximum information transmission.

q-bio.MN↗

arXiv2007

Neural Decision Boundaries for Maximal Information Transmission

We consider here how to separate multidimensional signals into two categories, such that the binary decision transmits the maximum possible information transmitted about those signals. Our motivation comes from the nervous system, where neurons process multidimensional signals into a binary sequence of responses (spikes). In a small noise limit, we derive a general equation for the decision boundary that locally relates its curvature to the probability distribution of inputs. We show that for Gaussian inputs the optimal boundaries are planar, but for non-Gaussian inputs the curvature is nonzero. As an example, we consider exponentially distributed inputs, which are known to approximate a variety of signals from natural environment.

q-bio.NC↗

arXiv2007

The role of input noise in transcriptional regulation

Even under constant external conditions, the expression levels of genes fluctuate. Much emphasis has been placed on the components of this noise that are due to randomness in transcription and translation; here we analyze the role of noise associated with the inputs to transcriptional regulation, the random arrival and binding of transcription factors to their target sites along the genome. This noise sets a fundamental physical limit to the reliability of genetic control, and has clear signatures, but we show that these are easily obscured by experimental limitations and even by conventional methods for plotting the variance vs. mean expression level. We argue that simple, global models of noise dominated by transcription and translation are inconsistent with the embedding of gene expression in a network of regulatory interactions. Analysis of recent experiments on transcriptional control in the early Drosophila embryo shows that these results are quantitatively consistent with the predicted signatures of input noise, and we discuss the experiments needed to test the importance of input noise more generally.

q-bio.MN↗

arXiv2006

Neural coding of a natural stimulus ensemble: Uncovering information at sub-millisecond resolution

Our knowledge of the sensory world is encoded by neurons in sequences of discrete, identical pulses termed action potentials or spikes. There is persistent controversy about the extent to which the precise timing of these spikes is relevant to the function of the brain. We revisit this issue, using the motion-sensitive neurons of the fly visual system as a test case. New experimental methods allow us to deliver more nearly natural visual stimuli, comparable to those which flies encounter in free, acrobatic flight, and new mathematical methods allow us to draw more reliable conclusions about the information content of neural responses even when the set of possible responses is very large. We find that significant amounts of visual information are represented by details of the spike train at millisecond and sub-millisecond precision, even though the sensory input has a correlation time of ~60ms; different patterns of spike timing represent distinct motion trajectories, and the absolute timing of spikes points to particular features of these trajectories with high precision. Under these naturalistic conditions, the system continues to transmit more information at higher photon flux, even though individual photoreceptors are counting more than one million photons per second, and removes redundancy in the stimulus to generate a more efficient neural code.

q-bio.NC↗

arXiv2006

Ising models for networks of real neurons

Ising models with pairwise interactions are the least structured, or maximum-entropy, probability distributions that exactly reproduce measured pairwise correlations between spins. Here we use this equivalence to construct Ising models that describe the correlated spiking activity of populations of 40 neurons in the retina, and show that pairwise interactions account for observed higher-order correlations. By first finding a representative ensemble for observed networks we can create synthetic networks of 120 neurons, and find that with increasing size the networks operate closer to a critical point and start exhibiting collective behaviors reminiscent of spin glasses.

q-bio.NC↗

arXiv2006

Synergy from Silence in a Combinatorial Neural Code

The manner in which groups of neurons represent events in the external world is fundamental to neuroscience. Here, we analyze the population code of the retina during naturalistic stimulation and show that the information conveyed by specific multi-neuronal firing patterns can be very different from the sum of the pattern's parts. Synchronous spikes convey more information than either of the participating cells, but almost always less than their sum, making them redundant coding symbols. Surprisingly, patterns of spiking and silence are mostly synergistic - carrying information that is accessible only by observing the whole pattern of activity, rather than its components, and signifying unique features in the stimulus. These results demonstrate that the retina uses a combinatorial code and that the brain can benefit significantly from recognizing multi-neuronal firing patterns.

q-bio.NC↗

arXiv2005

Cooperativity, sensitivity and noise in biochemical signaling

Cooperative interactions among the binding of multiple signaling molecules is a common mechanism for enhancing the sensitivity of biological signaling systems. It is widely assumed that this increase in sensitivity of the mean response implies the ability to detect smaller signals. We show that, quite generally, there is a component of the noise in such systems that can be traced to the random arrival of the signaling molecules at their receptor sites, and this diffusive noise is not reduced by cooperativity. Cooperativity makes it easier for real systems to reach this physical limit, but cannot reduce the limit itself.

q-bio.MN↗