Searcharxiv⌕ Search

arXiv subjects

David H. Wolpert

Publications and source records attributed to David H. Wolpert.

At least 19 recordsLinked to original sources

The Logic of Emergence

``Emergence'' refers to the fact that different mathematical languages and theories seem to be required to describe scientific theories that operate at different spatial and temporal scales. Highly circumscribed, special cases of emergence have been investigated using many different formal tools, including coarse-graining (over space and / or time), renormalization, dimension reduction, broken symmetry, and various formulations of causality. Here we extend the mathematics of model theory to unify and extend these earlier investigations. As a preliminary application of our framework, we investigate canonical examples of emergence, including the emergence of genetics from chemistry, of psychology from neurobiology, and of different forms of collective behavior from the laws governing their constituent parts. By exploiting the power of model theory we also uncover many aspects of emergence that were previously obscured. In particular, we uncover some of the rich hierarchical structure that relates different scientific fields. For example, we show that scientific fields can be related via a collection of multiple partial orders, all defined over the same set. On the other hand, under an alternative relation among scientific fields, one which allows cycles, we show that it is possible that the set of all of scientific fields must collapse into a single scientific field. Going forward, these investigations point to the formal tools that might need to be developed in order to arrive at a deeper understanding of this emergent hierarchy of scientific reality.

math.LO↗

Distributional statistics of nonequilibrium

We establish a reference modeling framework for random probability distributions subject to constraints. Generalizing the hyperensemble approach of Crooks, we formulate Jaynesian priors of maximal uncertainty over the unit simplex of categorical distributions. This framework is entirely general, but we primarily investigate models with constraints on physically relevant quantities, namely, average energy and Shannon entropy. A constraint on the exact value of average energy uniquely determines a Boltzmann-Gibbs equilibrium, but also defines a hyperplanar subset of the probability simplex consisting of all distributions with that value of average energy. The associated hyperensemble has a uniform probability density across all such distributions, regardless of distance to equilibrium. Such reference models make predictions of generic nonequilibrium properties such as the mean and modal microstate distributions and the expected free energy of a random distribution, which we examine analytically and via exact sampling algorithms. Moreover, introducing constraints on Shannon entropy permits parametric access to tunably nonequilibrium regimes. The approach follows conventional Jaynesian modeling logic and is compatible with standard Bayesian inference, and reduces to Boltzmann-Gibbs statistics when Shannon entropy is maximized. We examine the physical predictions of this scheme, and demonstrate its wide compatibility with other approaches to nonequilibrium modeling. We illustrate the framework in three scenarios: (i) comparison to superstatistics quasi-equilibrium modeling, (ii) generation of random nonequilibrium steady states, and (iii) Bayesian inference under a prior hyperensemble. We highlight numerous prospects for theoretical extensions and empirical application.

cond-mat.stat-mech↗

Strengthened second law for periodic processes

We derive a lower bound on the entropy production of any physical system that evolves as a periodic process. Crucially, this lower bound is completely independent of the physical mechanism implementing the periodic dynamics; it follows purely from the definition of entropy production and the fact that whatever the physical mechanism is, it repeats in each period. In particular, the lower bound holds in both Markovian and non-Markovian systems, in systems that obey or violate (local) detailed balance, and even if the underlying physical mechanism uses currently unknown laws of physics. Moreover, the bound is strictly positive whenever the dynamics is not logically invertible. In this breadth of applicability, our bound is a strict strengthening of the second law. We illustrate our bound for spins coupled to two heat reservoirs and for finite automata.

cond-mat.stat-mech↗

Minimax entropy production for arbitrary processes

For any physical process, varying the initial distribution over its states will change the resulting expected entropy production (EP). An initial distribution that minimizes the EP of a particular process is called the prior of that process. Varying the microphysical details of the process will change its prior. We consider the following question: if an engineer wants to design a system to implement a given map, what prior should they design into the system to minimize the EP for the worst-case actual distribution? We refer to the resulting optimal worst-case value as the minimax EP, and investigate how it depends only on the dynamics of the process. We derive the closed-form solution to this minimax problem when the map is deterministic. We also show that for stochastic maps, a worst-case actual distribution can always be chosen to be a point mass on a single state, and that the associated minimax prior resembles a Boltzmann-Gibbs equation and can be found by a globally convergent numerical algorithm. We further show that the minimax EP is the maximum uncertainty about the input that remains after observing the output. Finally, we extend the analysis to the continuous-time regime, showing that the minimax instantaneous EP diverges, while the corresponding finite-time minimax EP has a universal short-time scaling set by the largest escape rate of the dynamics.

cond-mat.stat-mech↗

Finite relaxation protocols with minimal dissipation

Work extraction from nonequilibrium systems is a major challenge across biological, chemical, physical, and engineering systems. Idealized protocols generally require a quasistatic relaxation stage in which the Hamiltonian is gradually adjusted through a continuum of intermediaries. Here, we consider protocols restricted to a finite number $N$ of intermediary Hamiltonians, consisting of a sequence of quench-relax steps. We determine the sequence of quenches that minimizes the dissipated work, which can be expressed in terms of a recurrence involving the Lambert function. The optimal sequence converges to the Fisher-Rao geodesic, saturating known leading-order dissipation bounds at large $N$. We obtain lower bounds on work extraction from a nonequilibrium distribution as a function of its Fisher-Rao distance to equilibrium. We extend and apply the framework in two simple models: (i) an optical trap experiment, showing that the optimal intermediary distribution can be bimodal even for unimodal initial and final distributions, and (ii) an enzyme-catalyzed reaction, showing that that accounting for relaxation time in addition to dissipation can favor barrier-lowering.

cond-mat.stat-mech↗

Computational foundations of the human world

Human societies continuously transform scattered information into collective judgments and coordinated action, whether through markets discovering prices, governments allocating resources, communities enforcing norms, or science converging on reliable claims. Importantly, the computational difficulty of collective decision-making, particularly the time and communication required to reach solutions, imposes fundamental constraints on social organization. While theoretical computer science offers formal tools for analyzing such problems, for instance, by analyzing resource requirements, including time and memory, surprisingly, there is no domain of social science that focuses on the nature of computation in the human world. This perspective argues that we now have the opportunity to deploy these computational frameworks to study human social organization, opening research directions at the intersection of computer science and social science. We highlight core social phenomena that can be framed as computational, including (i) distributed consensus and coordinated action, (ii) societal restructuring with scale, (iii) hierarchical and modular structure, and (iv) externalized memory systems. We identify several concepts from theoretical computer science that may provide insight into these phenomena, especially emphasizing more recently developed approaches beyond the paradigm of Turing~Machines and worst-case computational complexity.

cs.SI↗

Implications of computer science theory for the simulation hypothesis

The simulation hypothesis has recently excited renewed interest in the physics and philosophy communities. However, the hypothesis specifically concerns {\textit{computers}} that simulate physical universes. So to formally investigate the hypothesis, we need to understand it in terms of computer science (CS) theory. In addition we need a formal way to couple CS theory with physics. Here I couple those fields by using the physical Church-Turing thesis. This allow me to exploit Kleene's second recursion, to prove that not only is it possible for {us} to be a simulation being run on a computer, but that we might be in a simulation being run a computer \emph{by us}. In such a ``self-simulation'', there would be two identical instances of us, both equally ``real''. I then use Rice's theorem to derive impossibility results concerning simulation and self-simulation; derive implications for (self-)simulation if we are being simulated in a program using fully homomorphic encryption; and briefly investigate the graphical structure of universes simulating other universes which contain computers running their own simulations. I end by describing some of the possible avenues for future research. While motivated in terms of the simulation hypothesis, the results in this paper are direct consequences of the Church-Turing thesis. So they apply far more broadly than the simulation hypothesis.

cs.LO↗

What does it mean for a system to compute?

Many real-world dynamic systems, both natural and artificial, are understood to be performing computations. For artificial dynamic systems, explicitly designed to perform computation - such as digital computers - by construction, we can identify which aspects of the dynamic system match the input and output of the computation that it performs, as well as the aspects of the dynamic system that match the intermediate logical variables of that computation. In contrast, in many naturally occurring dynamical systems that we understand to be computers, even though we neither designed nor constructed them - such as the human brain - it is not a priori clear how to identify the computation we presume to be encoded in the dynamic system. Regardless of their origin, dynamical systems capable of computation can, in principle, be mapped onto corresponding abstract computational machines that perform the same operations. In this paper, we begin by surveying a wide range of dynamic systems whose computational properties have been studied. We then introduce a very broadly applicable framework for identifying what computations(s) are emulated by a given dynamic system. After an introduction, we summarize key examples of dynamical systems whose computational properties have been studied. We then introduce a very broadly applicable framework that defines the computation performed by a given dynamical system in terms of maps between that system's evolution and the evolution of an abstract computational machine. We illustrate this framework with several examples from the literature, in particular discussing why some of those examples do not fully fall within the remit of our framework. We also briefly discuss several related issues, such as uncomputability in dynamical systems, and how to quantify the value of computation in naturally occurring computers.

physics.comp-ph↗

Uncertainty relations and fluctuation theorems for Bayes nets

Recent research has considered the stochastic thermodynamics of multiple interacting systems, representing the overall system as a Bayes net. I derive fluctuation theorems governing the entropy production (EP)of arbitrary sets of the systems in such a Bayes net. I also derive ``conditional'' fluctuation theorems, governing the distribution of EP in one set of systems conditioned on the EP of a different set of systems. I then derive thermodynamic uncertainty relations relating the EP of the overall system to the precisions of probability currents within the individual systems.

cond-mat.stat-mech↗

Stochastic thermodynamics of computation

One of the major resource requirements of computers - ranging from biological cells to human brains to high-performance (engineered) computers - is the energy used to run them. Those costs of performing a computation have long been a focus of research in physics, going back to the early work of Landauer. One of the most prominent aspects of computers is that they are inherently nonequilibrium systems. However, the early research was done when nonequilibrium statistical physics was in its infancy, which meant the work was formulated in terms of equilibrium statistical physics. Since then there have been major breakthroughs in nonequilibrium statistical physics, which are allowing us to investigate the myriad aspects of the relationship between statistical physics and computation, extending well beyond the issue of how much work is required to erase a bit. In this paper I review some of this recent work on the `stochastic thermodynamics of computation'. After reviewing the salient parts of information theory, computer science theory, and stochastic thermodynamics, I summarize what has been learned about the entropic costs of performing a broad range of computations, extending from bit erasure to loop-free circuits to logically reversible circuits to information ratchets to Turing machines. These results reveal new, challenging engineering problems for how to design computers to have minimal thermodynamic costs. They also allow us to start to combine computer science theory and stochastic thermodynamics at a foundational level, thereby expanding both.

cond-mat.stat-mech↗

Measuring amount of computation done by C.elegans using whole brain neural activity

Many dynamical systems found in biology, ranging from genetic circuits to the human brain to human social systems, are inherently computational. Although extensive research has explored their resulting functions and behaviors, the underlying computations often remain elusive. Even the fundamental task of quantifying the \textit{amount} of computation performed by a dynamical system remains under-investigated. In this study we address this challenge by introducing a novel framework to estimate the amount of computation implemented by an arbitrary physical system based on empirical time-series of its dynamics. This framework works by forming a statistical reconstruction of that dynamics, and then defining the amount of computation in terms of both the complexity and fidelity of this reconstruction. We validate our framework by showing that it appropriately distinguishes the relative amount of computation across different regimes of Lorenz dynamics and various computation classes of cellular automata. We then apply this framework to neural activity in \textit{Caenorhabditis elegans}, as captured by calcium imaging. By analyzing time-series neural data obtained from the fluorescent intensity of the calcium indicator GCaMP, we find that high and low amounts of computation are required, respectively, in the neural dynamics of freely moving and immobile worms. Our analysis further sheds light on the amount of computation performed when the system is in various locomotion states. In sum, our study refines the definition of computational amount from time-series data and highlights neural computation in a simple organism across distinct behavioral states.

q-bio.NC↗

The computational power of a human society: a new model of social evolution

Social evolutionary theory seeks to explain increases in the scale and complexity of human societies, from origins to present. Over the course of the twentieth century, social evolutionary theory largely fell out of favor as a way of investigating human history, just when advances in complex systems science and computer science saw the emergence of powerful new conceptions of complex systems, and in particular new methods of measuring complexity. We propose that these advances in our understanding of complex systems and computer science should be brought to bear on our investigations into human history. To that end, we present a new framework for modeling how human societies co-evolve with their biotic environments, recognizing that both a society and its environment are computers. This leads us to model the dynamics of each of those two systems using the same, new kind of computational machine, which we define here. For simplicity, we construe a society as a set of interacting occupations and technologies. Similarly, under such a model, a biotic environment is a set of interacting distinct ecological and environmental processes. This provides novel ways to characterize social complexity, which we hope will cast new light on the archaeological and historical records. Our framework also provides a natural way to formalize both the energetic (thermodynamic) costs required by a society as it runs, and the ways it can extract thermodynamic resources from the environment in order to pay for those costs -- and perhaps to grow with any left-over resources.

cs.MA↗

What can we know about that which we cannot even imagine?

In this essay I will consider a sequence of questions. The first questions concern the biological function of intelligence in general, and cognitive prostheses of human intelligence in particular. These will lead into questions concerning human language, perhaps the most important cognitive prosthesis humanity has ever developed. While it is traditional to rhapsodize about the cognitive power encapsulated in human language, I will emphasize how horribly limited human language is - and therefore how limited our cognitive abilities are, despite their being augmented with language. This will lead to questions of whether human mathematics, being ultimately formulated in terms of human language, is also deeply limited. I will then combine these questions to pose a partial, sort-of, sideways answer to the guiding concern of this essay: what we can ever discern about that we cannot even conceive?

physics.hist-ph↗

Strengthened second law for multi-dimensional systems coupled to multiple thermodynamic reservoirs

The second law of thermodynamics can be formulated as a restriction on the evolution of the entropy of any system undergoing Markovian dynamics. Here I show that this form of the second law is strengthened for multi-dimensional, complex systems, coupled to multiple thermodynamic reservoirs, if we have a set of \textit{a priori} constraints restricting how the dynamics of each coordinate can depend on the other coordinates. As an example, this strengthened second law (SSL) applies to complex systems composed of multiple physically separated, co-evolving subsystems, each identified as a coordinate of the overall system. In this example, the constraints concern how the dynamics of some subsystems are allowed to depend on the states of the other subsystems. Importantly the SSL applies to such complex systems even if some of its subsystems can change state simultaneously, which is prohibited in a multipartite process. The SSL also strengthens previously derived bounds on how much work can be extracted from a system using feedback control, if the system is multi-dimensional. Importantly, the SSL does not require local detailed balance. So it potentially applies to complex systems ranging from interacting economic agents to co-evolving biological species.

cond-mat.stat-mech↗

Entropy production in communication channels

In many complex systems, whether biological or artificial, the thermodynamic costs of communication among their components are large. These systems also tend to split information transmitted between any two components across multiple channels. A common hypothesis is that such inverse multiplexing strategies reduce total thermodynamic costs. So far, however, there have been no physics-based results supporting this hypothesis. This gap existed partially because we have lacked a theoretical framework that addresses the interplay of thermodynamics and information in off-equilibrium systems. Here we present the first study that rigorously combines such a framework, stochastic thermodynamics, with Shannon information theory. We develop a minimal model that captures the fundamental features common to a wide variety of communication systems, and study the relationship between the entropy production of the communication process and the channel capacity, the canonical measure of the communication capability of a channel. In contrast to what is assumed in previous works not based on first principles, we show that the entropy production is not always a convex and monotonically increasing function of the channel capacity. However, those two properties are recovered for sufficiently high channel capacity. These results clarify when and how to split a single communication stream across multiple channels.

cond-mat.stat-mech↗

Game Mining: How to Make Money from those about to Play a Game

It is known that a player in a noncooperative game can benefit by publicly restricting his possible moves before play begins. We show that, more generally, a player may benefit by publicly committing to pay an external party an amount that is contingent on the game's outcome. We explore what happens when external parties -- who we call ``game miners'' -- discover this fact and seek to profit from it by entering an outcome-contingent contract with the players. We analyze various structured bargaining games between miners and players for determining such an outcome-contingent contract. These bargaining games include playing the players against one another, as well as allowing the players to pay the miner(s) for exclusivity and first-mover advantage. We establish restrictions on the strategic settings in which a game miner can profit and bounds on the game miner's profit. We also find that game miners can lead to both efficient and inefficient equilibria.

econ.GN↗

Memory Systems, the Epistemic Arrow of Time, and the Second Law

The epistemic arrow of time is the fact that our knowledge of the past seems to be both of a different kind and more detailed than our knowledge of the future. Just like with the other arrows of time, it has often been speculated that the epistemic arrow arises due to the second law of thermodynamics. In this paper we investigate the epistemic arrow of time, using a fully formal framework. We begin by defining a memory system as any physical system whose present state can provide information about the state of the external world at some time other than the present. We then identify two types of memory systems in our universe, along with an important special case of the first type, which we distinguish as a third type of memory system. We show that two of these types of memory system are time-symmetric, able to provide knowledge about both the past and the future. However, the third type of memory systems exploits the second law of thermodynamics in all of its instances we find in our universe. The result is that in our universe, this type of memory system only ever provides information about the past. Finally, we argue that human memory is of this third type, completing the argument. Our analysis is indebted to prior work in Wolpert 1992, but expands and improves upon this work in several respects.

physics.hist-ph↗

Thermodynamics of deterministic finite automata operating locally and periodically

Real-world computers have operational constraints that cause nonzero entropy production (EP). In particular, almost all real-world computers are ``periodic'', iteratively undergoing the same physical process; and ``local", in that subsystems evolve whilst physically decoupled from the rest of the computer. These constraints are so universal because decomposing a complex computation into small, iterative calculations is what makes computers so powerful. We first derive the nonzero EP caused by the locality and periodicity constraints for deterministic finite automata (DFA), a foundational system of computer science theory. We then relate this minimal EP to the computational characteristics of the DFA. We thus divide the languages recognised by DFA into two classes: those that can be recognised with zero EP, and those that necessarily have non-zero EP. We also demonstrate the thermodynamic advantages of implementing a DFA with a physical process that is agnostic about the inputs that it processes.

cond-mat.stat-mech↗