arXiv subjects
Publications and source records attributed to Peter Gacs.
Andrei Toom, who died in September 2022, contributed some of the most fundamental results on probabilistic cellular automata. We want to acquaint the reader with these and will also try to give the reader a look at the environment in which they were born. Toom was an original and strong personality, and other aspects of his life (education, literature) will also deserve mention.
There are several proofs now for the stability of Toom's example of a two-dimensional stable cellular automaton and its application to fault-tolerant computation. Simon and Berman simplified and strengthened Toom's original proof: the present report is a simplified exposition of their proof.
A didactical survey of the foundations of Algorithmic Information Theory. These notes are short on motivation, history and background but introduce some of the main techniques and concepts of the field. The "manuscript" has been evolving over the years. Please, look at "Version history" below to see what has changed when.
Let v, w be infinite 0-1 sequences, and m a positive integer. We say that w is m-embeddable in v, if there exists an increasing sequence n_{i} of integers with n_{0}=0, such that 0< n_{i} - n_{i-1} < m, w(i) = v(n_i) for all i > 0. Let X and Y be independent coin-tossing sequences. We will show that there is an m with the property that Y is m-embeddable into X with positive probability. This answers a question that was open for a while. The proof generalizes somewhat the hierarchical method of an earlier paper of the author on dependent percolation.
We consider computations of a Turing machine under noise that causes consecutive violations of the machine's transition function. Given a constant upper bound B on the size of bursts of faults, we construct a Turing machine M(B) subject to faults that can simulate any fault-free machine under the condition that bursts are not closer to each other than V for an appropriate V = O(B^2).
The paper considers quantitative versions of different randomness notions: algorithmic test measures the amount of non-randomness (and is infinite for non-random sequences). We start with computable measures on Cantor space (and Martin-Lof randomness), then consider uniform randomness (test is a function of a sequence and a measure, not necessarily computable) and arbitrary constructive metric spaces. We also consider tests for classes of measures, in particular Bernoulli measures on Cantor space, and show how they are related to uniform tests and original Martin-Lof definition. We show that Hyppocratic (blind, oracle-free) randomness is equivalent to uniform randomness for measures in an effectively orthogonal effectively compact class. We also consider the notions of sparse set and on-line randomness and show how they can be expressed in our framework.
Two infinite walks on the same finite graph are called compatible if it is possible to introduce delays into them in such a way that they never collide. Years ago, Peter Winkler asked the question: for which graphs are two independent walks compatible with positive probability. Up to now, no such graphs were found. We show in this paper that large complete graphs have this property. The question is equivalent to a certain dependent percolation with a power-law behavior: the probability that the origin is blocked at distance n but not closer decreases only polynomially fast and not, as usual, exponentially.
While Kolmogorov complexity is the accepted absolute measure of information content in an individual finite object, a similarly absolute notion is needed for the information distance between two individual objects, for example, two pictures. We give several natural definitions of a universal information metric, based on length of shortest programs for either ordinary computations or reversible (dissipationless) computations. It turns out that these definitions are equivalent up to an additive logarithmic term. We show that the information distance is a universal cognitive similarity distance. We investigate the maximal correlation of the shortest programs involved, the maximal uncorrelation of programs (a generalization of the Slepian-Wolf theorem of classical information theory), and the density properties of the discrete metric spaces induced by the information distances. A related distance measures the amount of nonreversibility of a computation. Using the physical theory of reversible computation, we give an appropriate (universal, anti-symmetric, and transitive) measure of the thermodynamic work required to transform one object in another object by the most efficient process. Information distance between individual objects is needed in pattern recognition where one wants to express effective notions of "pattern similarity" or "cognitive similarity" between individual objects and in thermodynamics of computation where one wants to analyse the energy dissipation of a computation from a particular input to a particular output.
We extend the notion of randomness (in the version introduced by Schnorr) to computable Probability Spaces and compare it to a dynamical notion of randomness: typicality. Roughly, a point is typical for some dynamic, if it follows the statistical behavior of the system (Birkhoff's pointwise ergodic theorem). We prove that a point is Schnorr random if and only if it is typical for every mixing computable dynamics. To prove the result we develop some tools for the theory of computable probability spaces (for example, morphisms) that are expected to have other applications.
The angel-devil game is played on an infinite two-dimensional ``chessboard''. The squares of the board are all white at the beginning. The players called angel and devil take turns in their steps. When it is the devil's turn, he can turn a square black. The angel always stays on a white square, and when it is her turn she can fly at a distance of at most J steps (each of which can be horizontal, vertical or diagonal) to a new white square. Here J is a constant. The devil wins if the angel does not find any more white squares to land on. The result of the paper is that if J is sufficiently large then the angel has a strategy such that the devil will never capture her. This deceptively easy-sounding result has been a conjecture, surprisingly, for about thirty years. Several other independent solutions have appeared simultaneously, some of them prove that J=2 is sufficient (see the Wikipedia on the angel problem). Still, it is hoped that the hierarchical solution presented here may prove useful for some generalizations.
The algorithmic theory of randomness is well developed when the underlying space is the set of finite or infinite sequences and the underlying probability distribution is the uniform distribution or a computable distribution. These restrictions seem artificial. Some progress has been made to extend the theory to arbitrary Bernoulli distributions (by Martin-Loef), and to arbitrary distributions (by Levin). We recall the main ideas and problems of Levin's theory, and report further progress in the same framework. - We allow non-compact spaces (like the space of continuous functions, underlying the Brownian motion). - The uniform test (deficiency of randomness) d_P(x) (depending both on the outcome x and the measure P should be defined in a general and natural way. - We see which of the old results survive: existence of universal tests, conservation of randomness, expression of tests in terms of description complexity, existence of a universal measure, expression of mutual information as "deficiency of independence. - The negative of the new randomness test is shown to be a generalization of complexity in continuous spaces; we show that the addition theorem survives. The paper's main contribution is introducing an appropriate framework for studying these questions and related ones (like statistics for a general family of distributions).
For some m \ge 4, let us color each column of the integer lattice L = Z^2 independently and uniformly into one of m colors. We do the same for the rows, independently from the columns. A point of L will be called blocked if its row and column have the same color. We say that this random configuration percolates if there is a path in L starting at the origin, consisting of rightward and upward unit steps, and avoiding the blocked points. As a problem arising in distributed computing, it has been conjectured that for m \ge 4, the configuration percolates with positive probability. This has now been proved (in a later paper), for large m. Here, we prove that the probability that there is percolation to distance n but not to infinity is not exponentially small in n. This narrows the range of methods available for proving the conjecture.
Two infinite 0-1 sequences are called compatible when it is possible to cast out 0's from both in such a way that they become complementary to each other. Answering a question of Peter Winkler, we show that if the two 0-1-sequences are random i.i.d. and independent from each other, with probability p of 1's, then if p is sufficiently small they are compatible with positive probability. The question is equivalent to a certain dependent percolation with a power-law behavior: the probability that the origin is blocked at distance n but not closer decreases only polynomially fast and not, as usual, exponentially.
While Kolmogorov complexity is the accepted absolute measure of information content of an individual finite object, a similarly absolute notion is needed for the relation between an individual data sample and an individual model summarizing the information in the data, for example, a finite set (or probability distribution) where the data sample typically came from. The statistical theory based on such relations between individual objects can be called algorithmic statistics, in contrast to classical statistical theory that deals with relations between probabilistic ensembles. We develop the algorithmic theory of statistic, sufficient statistic, and minimal sufficient statistic. This theory is based on two-part codes consisting of the code for the statistic (the model summarizing the regularity, the meaningful information, in the data) and the model-to-data code. In contrast to the situation in probabilistic statistical theory, the algorithmic relation of (minimal) sufficiency is an absolute relation between the individual model and the individual data sample. We distinguish implicit and explicit descriptions of the models. We give characterizations of algorithmic (Kolmogorov) minimal sufficient statistic for all data samples for both description modes--in the explicit mode under some constraints. We also strengthen and elaborate earlier results on the ``Kolmogorov structure function'' and ``absolutely non-stochastic objects''--those rare objects for which the simplest models that summarize their relevant information (minimal sufficient statistics) are at least as complex as the objects themselves. We demonstrate a close relation between the probabilistic notions and the algorithmic ones.
We extend algorithmic information theory to quantum mechanics, taking a universal semicomputable density matrix (``universal probability'') as a starting point, and define complexity (an operator) as its negative logarithm. A number of properties of Kolmogorov complexity extend naturally to the new domain. Approximately, a quantum state is simple if it is within a small distance from a low-dimensional subspace of low Kolmogorov complexity. The von Neumann entropy of a computable density matrix is within an additive constant from the average complexity. Some of the theory of randomness translates to the new domain. We explore the relations of the new quantity to the quantum Kolmogorov complexity defined by Vitanyi (we show that the latter is sometimes as large as 2n - 2log n and the qubit complexity defined by Berthiaume, Dam and Laplante. The ``cloning'' properties of our complexity measure are similar to those of qubit complexity.
Toom's north-east-self voting cellular automaton rule R is known to suppress small minorities. A variant which we call R^+ is also known to turn an arbitrary initial configuration into a homogenous one (without changing the ones that were homogenous to start with). Here we show that R^+ always increases a certain property of sets called thickness. This result is intended as a step towards a proof of the fast convergence towards consensus under R^+. The latter is observable experimentally, even in the presence of some noise.
Consider a network of processors (sites) in which each site x has a finite set N(x) of neighbors. There is a transition function f that for each site x computes the next state ξ(x) from the states in N(x). But these transitions (updates) are applied in arbitrary order, one or many at a time. If the state of site x at time t is η(x,t) then let us define the sequence ζ(x,0), ζ(x,1), ... by taking the sequence η(x,0), η(x,1), ..., and deleting repetitions. The function f is said to have invariant histories if the sequence ζ(x,i), (while it lasts, in case it is finite) depends only on the initial configuration, not on the order of updates. This paper shows that though the invariant history property is typically undecidable, there is a useful simple sufficient condition, called commutativity: For any configuration, for any pair x,y of neighbors, if the updating would change both ξ(x) and ξ(y) then the result of updating first x and then y is the same as the result of doing this in the reverse order.
In a probabilistic cellular automaton in which all local transitions have positive probability, the problem of keeping a bit of information indefinitely is nontrivial, even in an infinite automaton. Still, there is a solution in 2 dimensions, and this solution can be used to construct a simple 3-dimensional discrete-time universal fault-tolerant cellular automaton. This technique does not help much to solve the following problems: remembering a bit of information in 1 dimension; computing in dimensions lower than 3; computing in any dimension with non-synchronized transitions. Our more complex technique organizes the cells in blocks that perform a reliable simulation of a second (generalized) cellular automaton. The cells of the latter automaton are also organized in blocks, simulating even more reliably a third automaton, etc. Since all this (a possibly infinite hierarchy) is organized in ``software'', it must be under repair all the time from damage caused by errors. A large part of the problem is essentially self-stabilization recovering from a mess of arbitrary size and content. The present paper constructs an asynchronous one-dimensional fault-tolerant cellular automaton, with the further feature of ``self-organization''. The latter means that the initial configuration does not have to encode an infinite hierarchy -- this will be built up over time. This is a corrected and strengthened version of the journal paper of 2001.