Searcharxiv⌕ Search

arXiv subjects

James P. Crutchfield

Publications and source records attributed to James P. Crutchfield.

158 records · Page 9Linked to original sources

Pattern Discovery and Computational Mechanics

Computational mechanics is a method for discovering, describing and quantifying patterns, using tools from statistical physics. It constructs optimal, minimal models of stochastic processes and their underlying causal structures. These models tell us about the intrinsic computation embedded within a process---how it stores and transforms information. Here we summarize the mathematics of computational mechanics, especially recent optimality and uniqueness results. We also expound the principles and motivations underlying computational mechanics, emphasizing its connections to the minimum description length principle, PAC theory, and other aspects of machine learning.

cs.LG↗

Comment on "Simple Measure of Complexity"

We critique the measure of complexity introduced by Shiner, Davison, and Landsberg in Ref. [1]. In particular, we point out that it is over-universal, in the sense that it has the same dependence on disorder for structurally distinct systems. We then give counterexamples to the claim that complexity is synonymous with being out of equilibrium: equilibrium systems can be structurally complex and nonequilibrium systems can be structurally simple. We also correct a misinterpretation of a result given by two of the present authors in Ref. [2]. [1] J. S. Shiner, M. Davison, and P. T. Landsberg, ``Simple Measure for Complexity'', Phys. Rev. E 59 (1999) 1459-1464. [2] J. P. Crutchfield and D. P. Feldman, ``Statistical Complexity of Simple One-Dimensional Spin Systems'', Phys. Rev. E 55:2 (1997) 1239R-1243R.

chao-dyn↗

Resource Sharing and Coevolution in Evolving Cellular Automata

Evolving one-dimensional cellular automata (CAs) with genetic algorithms has provided insight into how improved performance on a task requiring global coordination emerges when only local interactions are possible. Two approaches that can affect the search efficiency of the genetic algorithm are coevolution, in which a population of problems---in our case, initial configurations of the CA lattice---evolves along with the population of CAs; and resource sharing, in which a greater proportion of a limited fitness resource is assigned to those CAs which correctly solve problems that fewer other CAs in the population can solve. Here we present evidence that, in contrast to what has been suggested elsewhere, the improvements observed when both techniques are used together depend largely on resource sharing alone.

adap-org↗

Metastable Evolutionary Dynamics: Crossing Fitness Barriers or Escaping via Neutral Paths?

We analytically study the dynamics of evolving populations that exhibit metastability on the level of phenotype or fitness. In constant selective environments, such metastable behavior is caused by two qualitatively different mechanisms. One the one hand, populations may become pinned at a local fitness optimum, being separated from higher-fitness genotypes by a {\em fitness barrier} of low-fitness genotypes. On the other hand, the population may only be metastable on the level of phenotype or fitness while, at the same time, diffusing over {\em neutral networks} of selectively neutral genotypes. Metastability occurs in this case because the population is separated from higher-fitness genotypes by an {\em entropy barrier}: The population must explore large portions of these neutral networks before it discovers a rare connection to fitter phenotypes. We derive analytical expressions for the barrier crossing times in both the fitness barrier and entropy barrier regime. In contrast with ``landscape'' evolutionary models, we show that the waiting times to reach higher fitness depend strongly on the width of a fitness barrier and much less on its height. The analysis further shows that crossing entropy barriers is faster by orders of magnitude than fitness barrier crossing. Thus, when populations are trapped in a metastable phenotypic state, they are most likely to escape by crossing an entropy barrier, along a neutral path in genotype space. If no such escape route along a neutral path exists, a population is most likely to cross a fitness barrier where the barrier is {\em narrowest}, rather than where the barrier is shallowest.

adap-org↗

Neutral Evolution of Mutational Robustness

We introduce and analyze a general model of a population evolving over a network of selectively neutral genotypes. We show that the population's limit distribution on the neutral network is solely determined by the network topology and given by the principal eigenvector of the network's adjacency matrix. Moreover, the average number of neutral mutant neighbors per individual is given by the matrix spectral radius. This quantifies the extent to which populations evolve mutational robustness: the insensitivity of the phenotype to mutations. Since the average neutrality is independent of evolutionary parameters---such as, mutation rate, population size, and selective advantage---one can infer global statistics of neutral network topology using simple population data available from {\it in vitro} or {\it in vivo} evolution. Populations evolving on neutral networks of RNA secondary structures show excellent agreement with our theoretical predictions.

adap-org↗

The Evolutionary Unfolding of Complexity

We analyze the population dynamics of a broad class of fitness functions that exhibit epochal evolution---a dynamical behavior, commonly observed in both natural and artificial evolutionary processes, in which long periods of stasis in an evolving population are punctuated by sudden bursts of change. Our approach---statistical dynamics---combines methods from both statistical mechanics and dynamical systems theory in a way that offers an alternative to current ``landscape'' models of evolutionary optimization. We describe the population dynamics on the macroscopic level of fitness classes or phenotype subbasins, while averaging out the genotypic variation that is consistent with a macroscopic state. Metastability in epochal evolution occurs solely at the macroscopic level of the fitness distribution. While a balance between selection and mutation maintains a quasistationary distribution of fitness, individuals diffuse randomly through selectively neutral subbasins in genotype space. Sudden innovations occur when, through this diffusion, a genotypic portal is discovered that connects to a new subbasin of higher fitness genotypes. In this way, we identify innovations with the unfolding and stabilization of a new dimension in the macroscopic state space. The architectural view of subbasins and portals in genotype space clarifies how frozen accidents and the resulting phenotypic constraints guide the evolution to higher complexity.

adap-org↗

Optimizing Epochal Evolutionary Search: Population-Size Dependent Theory

Epochal dynamics, in which long periods of stasis in an evolving population are punctuated by a sudden burst of change, is a common behavior in both natural and artificial evolutionary processes. We analyze the population dynamics for a class of fitness functions that exhibit epochal behavior using a mathematical framework developed recently. In the latter the approximations employed led to a population-size independent theory that allowed us to determine optimal mutation rates. Here we extend this approach to include the destabilization of epochs due to finite-population fluctuations and show that this dynamical behavior often occurs around the optimal parameter settings for efficient search. The resulting, more accurate theory predicts the total number of fitness function evaluations to reach the global optimum as a function of mutation rate, population size, and the parameters specifying the fitness function. We further identify a generalized error threshold, smoothly bounding the two-dimensional regime of mutation rates and population sizes for which evolutionary search operates efficiently.

adap-org↗

Optimizing Epochal Evolutionary Search: Population-Size Independent Theory

Epochal dynamics, in which long periods of stasis in population fitness are punctuated by sudden innovations, is a common behavior in both natural and artificial evolutionary processes. We use a recent quantitative mathematical analysis of epochal evolution to estimate, as a function of population size and mutation rate, the average number of fitness function evaluations to reach the global optimum. This is then used to derive estimates of and bounds on evolutionary parameters that minimize search effort.

adap-org↗

The Evolutionary Design of Collective Computation in Cellular Automata

We investigate the ability of a genetic algorithm to design cellular automata that perform computations. The computational strategies of the resulting cellular automata can be understood using a framework in which ``particles'' embedded in space-time configurations carry information and interactions between particles effect information processing. This structural analysis can also be used to explain the evolutionary process by which the strategies were designed by the genetic algorithm. More generally, our goals are to understand how machine-learning processes can design complex decentralized systems with sophisticated collective computational abilities and to develop rigorous frameworks for understanding how the resulting dynamical systems perform computation.

adap-org↗

Thermodynamic Depth of Causal States: When Paddling around in Occam's Pool Shallowness Is a Virtue

Thermodynamic depth is an appealing but flawed structural complexity measure. It depends on a set of macroscopic states for a system, but neither its original introduction by Lloyd and Pagels nor any follow-up work has considered how to select these states. Depth, therefore, is at root arbitrary. Computational mechanics, an alternative approach to structural complexity, provides a definition for a system's minimal, necessary causal states and a procedure for finding them. We show that the rate of increase in thermodynamic depth, or {\it dive}, is the system's reverse-time Shannon entropy rate, and so depth only measures degrees of macroscopic randomness, not structure. To fix this we redefine the depth in terms of the causal state representation---$ε$-machines---and show that this representation gives the minimum dive consistent with accurate prediction. Thus, $ε$-machines are optimally shallow.

cond-mat.stat-mech↗

Statistical Measures of Complexity: Why?

We review several statistical complexity measures proposed over the last decade and a half as general indicators of structure or correlation. Recently, Lopez-Ruiz, Mancini, and Calbet [Phys. Lett. A 209 (1995) 321] introduced another measure of statistical complexity C_{LMC} that, like others, satisfies the ``boundary conditions'' of vanishing in the extreme ordered and disordered limits. We examine some properties of C_{LMC} and find that it is neither an intensive nor an extensive thermodynamic variable and that it vanishes exponentially in the thermodynamic limit for all one-dimensional finite-range spin systems. We propose a simple alteration of C_{LMC} that renders it extensive. However, this remedy results in a quantity that is a trivial function of the entropy density and hence of no use as a measure of structure or memory. We conclude by suggesting that a useful ``statistical complexity'' must not only obey the ordered-random boundary conditions of vanishing, it must also be defined in a setting that gives a clear interpretation to what structures are quantified.

cond-mat.stat-mech↗

Quantum Automata and Quantum Grammars

To study quantum computation, it might be helpful to generalize structures from language and automata theory to the quantum case. To that end, we propose quantum versions of finite-state and push-down automata, and regular and context-free grammars. We find analogs of several classical theorems, including pumping lemmas, closure properties, rational and algebraic generating functions, and Greibach normal form. We also show that there are quantum context-free languages that are not context-free.

quant-ph↗

Statistical Complexity of Simple 1D Spin Systems

We present exact results for two complementary measures of spatial structure generated by 1D spin systems with finite-range interactions. The first, excess entropy, measures the apparent spatial memory stored in configurations. The second, statistical complexity, measures the amount of memory needed to optimally predict the chain of spin values. These statistics capture distinct properties and are different from existing thermodynamic quantities.

cond-mat.stat-mech↗

Revisiting the Edge of Chaos: Evolving Cellular Automata to Perform Computations

We present results from an experiment similar to one performed by Packard (1988), in which a genetic algorithm is used to evolve cellular automata (CA) to perform a particular computational task. Packard examined the frequency of evolved CA rules as a function of Langton's lambda parameter (Langton, 1990), and interpreted the results of his experiment as giving evidence for the following two hypotheses: (1) CA rules able to perform complex computations are most likely to be found near ``critical'' lambda values, which have been claimed to correlate with a phase transition between ordered and chaotic behavioral regimes for CA; (2) When CA rules are evolved to perform a complex computation, evolution will tend to select rules with lambda values close to the critical values. Our experiment produced very different results, and we suggest that the interpretation of the original results is not correct. We also review and discuss issues related to lambda, dynamical-behavior classes, and computation in CA. The main constructive results of our study are identifying the emergence and competition of computational strategies and analyzing the central role of symmetries in an evolutionary system. In particular, we demonstrate how symmetry breaking can impede the evolution toward higher computational capability.

adap-org↗