arXiv subjects

James P. Crutchfield

Publications and source records attributed to James P. Crutchfield.

At least 37 records · Page 2Linked to original sources

arXiv2024

Quantum Information Dimension and Geometric Entropy

Geometric quantum mechanics, through its differential-geometric underpinning, provides additional tools of analysis and interpretation that bring quantum mechanics closer to classical mechanics: state spaces in both are equipped with symplectic geometry. This opens the door to revisiting foundational questions and issues, such as the nature of quantum entropy, from a geometric perspective. Central to this is the concept of geometric quantum state -- the probability measure on a system's space of pure states. This space's continuity leads us to introduce two analysis tools, inspired by Renyi's information theory, to characterize and quantify fundamental properties of geometric quantum states: the quantum information dimension that is the rate of geometric quantum state compression and the dimensional geometric entropy that monitors information stored in quantum states. We recount their classical definitions, information-theoretic meanings, and physical interpretations, and adapt them to quantum systems via the geometric approach. We then explicitly compute them in various examples and classes of quantum system. We conclude commenting on future directions for information in geometric quantum mechanics.

quant-ph↗

arXiv2024

Thermodynamic Overfitting and Generalization: Energetic Limits on Predictive Complexity

Efficiently harvesting thermodynamic resources requires a precise understanding of their structure. This becomes explicit through the lens of information engines -- thermodynamic engines that use information as fuel. Maximizing the work harvested using available information is a form of physically-instantiated machine learning that drives information engines to develop complex predictive memory to store an environment's temporal correlations. We show that an information engine's complex predictive memory poses both energetic benefits and risks. While increasing memory facilitates detection of hidden patterns in an environment, it also opens the possibility of thermodynamic overfitting, where the engine dissipates additional energy in testing. To address overfitting, we introduce thermodynamic regularizers that incur a cost to engine complexity in training due to the physical constraints on the information engine. We demonstrate that regularized thermodynamic machine learning generalizes effectively. In particular, the physical constraints from which regularizers are derived improve the performance of learned predictive models. This suggests that the laws of physics jointly create the conditions for emergent complexity and predictive intelligence.

cond-mat.stat-mech↗

arXiv2023

Whales in Space: Experiencing Aquatic Animals in Their Natural Place with the Hydroambiphone

Recording the undersea three-dimensional bioacoustic sound field in real-time promises major benefits to marine behavior studies. We describe a novel hydrophone array -- the hydroambiphone (HAP) -- that adapts ambisonic spatial-audio theory to sound propagation in ocean waters to realize many of these benefits through spatial localization and acoustic immersion. Deploying it to monitor the humpback whales (Megaptera novaeangliae) of southeast Alaska demonstrates that HAP recording provides a qualitatively-improved experience of their undersea behaviors; revealing, for example, new aspects of social coordination during bubble-net feeding. On the practical side, spatialized hydrophone recording greatly reduces post-field analytical and computational challenges -- such as the "cocktail party problem" of distinguishing single sources in a complicated and crowded auditory environment -- that are common to field recordings. On the scientific side, comparing the HAP's capabilities to single-hydrophone and nonspatialized recordings yields new insights into the spatial information that allows animals to thrive in complex acoustic environments. Spatialized bioacoustics markedly improves access to the humpbacks' undersea acoustic environment and expands our appreciation of their rich vocal lives.

q-bio.PE↗

arXiv2023

On Principles of Emergent Organization

After more than a century of concerted effort, physics still lacks basic principles of spontaneous self-organization. To appreciate why, we first state the problem, outline historical approaches, and survey the present state of the physics of self-organization. This frames the particular challenges arising from mathematical intractability and the resulting need for computational approaches, as well as those arising from a chronic failure to define structure. Then, an overview of two modern mathematical formulations of organization -- intrinsic computation and evolution operators -- lays out a way to overcome these challenges. Together, the vantage point they afford shows how to account for the emergence of structured states via a statistical mechanics of systems arbitrarily far from equilibrium. The result is a constructive path forward to principles of organization that builds on mathematical identification of structure.

cond-mat.stat-mech↗

arXiv2023

Unsupervised Discovery of Extreme Weather Events Using Universal Representations of Emergent Organization

Spontaneous self-organization is ubiquitous in systems far from thermodynamic equilibrium. While organized structures that emerge dominate transport properties, universal representations that identify and describe these key objects remain elusive. Here, we introduce a theoretically-grounded framework for describing emergent organization that, via data-driven algorithms, is constructive in practice. Its building blocks are spacetime lightcones that embody how information propagates across a system through local interactions. We show that predictive equivalence classes of lightcones -- local causal states -- capture organized behaviors and coherent structures in complex spatiotemporal systems. Employing an unsupervised physics-informed machine learning algorithm and a high-performance computing implementation, we demonstrate automatically discovering coherent structures in two real world domain science problems. We show that local causal states identify vortices and track their power-law decay behavior in two-dimensional fluid turbulence. We then show how to detect and track familiar extreme weather events -- hurricanes and atmospheric rivers -- and discover other novel coherent structures associated with precipitation extremes in high-resolution climate data at the grid-cell level.

physics.comp-ph↗

arXiv2023

First and Second Laws of Information Processing by Nonequilibrium Dynamical States

The averaged steady-state surprisal links a driven stochastic system's information processing to its nonequilibrium thermodynamic response. By explicitly accounting for the effects of nonequilibrium steady states, a decomposition of the surprisal results in an information processing First Law that extends and tightens -- to strict equalities -- various information processing Second Laws. Applying stochastic thermodynamics' integral fluctuation theorems then shows that the decomposition reduces to the second laws under appropriate limits. In unifying them, the First Law paves the way to identifying the mechanisms by which nonequilibrium steady-state systems extract work from information-bearing degrees of freedom. To illustrate, we analyze an autonomous Maxwellian information ratchet that tunably violates detailed balance in its effective dynamics. This demonstrates how the presence of nonequilibrium steady states qualitatively alters an information engine's allowed functionality.

cond-mat.stat-mech↗

arXiv2023

Complexity-calibrated Benchmarks for Machine Learning Reveal When Next-Generation Reservoir Computer Predictions Succeed and Mislead

Recurrent neural networks are used to forecast time series in finance, climate, language, and from many other domains. Reservoir computers are a particularly easily trainable form of recurrent neural network. Recently, a "next-generation" reservoir computer was introduced in which the memory trace involves only a finite number of previous symbols. We explore the inherent limitations of finite-past memory traces in this intriguing proposal. A lower bound from Fano's inequality shows that, on highly non-Markovian processes generated by large probabilistic state machines, next-generation reservoir computers with reasonably long memory traces have an error probability that is at least ~ 60% higher than the minimal attainable error probability in predicting the next observation. More generally, it appears that popular recurrent neural networks fall far short of optimally predicting such complex processes. These results highlight the need for a new generation of optimized recurrent neural network architectures. Alongside this finding, we present concentration-of-measure results for randomly-generated but complex processes. One conclusion is that large probabilistic state machines -- specifically, large $ε$-machines -- are key to generating challenging and structurally-unbiased stimuli for ground-truthing recurrent neural network architectures.

cs.LG↗

arXiv2023

Intrinsic and Measured Information in Separable Quantum Processes

Stationary quantum information sources emit sequences of correlated qudits -- that is, structured quantum stochastic processes. If an observer performs identical measurements on a qudit sequence, the outcomes are a realization of a classical stochastic process. We introduce quantum-information-theoretic properties for separable qudit sequences that serve as bounds on the classical information properties of subsequent measured processes. For sources driven by hidden Markov dynamics we describe how an observer can temporarily or permanently synchronize to the source's internal state using specific positive operator-valued measures or adaptive measurement protocols. We introduce a method for approximating an information source with an independent and identically-distributed, Markov, or larger memory model through tomographic reconstruction. We identify broad classes of separable processes based on their quantum information properties and the complexity of measurements required to synchronize to and accurately reconstruct them.

quant-ph↗

arXiv2022

Szilard Engines as Quantum Thermodynamical Systems

We analyze an engine whose working fluid consists of a single quantum particle, paralleling Szilard's construction of a classical single-particle engine. Following his resolution of Maxwell's Second Law paradox using the latter, which turned on physically instantiating the demon (control subsystem), the quantum engine's design mirrors the classically-chaotic Szilard Map that operates a thermodynamic cycle of measurement, thermal-energy extraction, and memory reset. Focusing on the thermodynamic costs to observe and control the particle and comparing these in the quantum and classical limits, we detail the thermodynamic tradeoffs behind Landauer's Principle for information-processing-induced thermodynamic dissipation in the quantum and classical regimes. In particular, and as found with the classical engine, we show that the sum of the thermodynamic costs over a cycle obeys a generalized Landauer Principle, exactly balancing energy extraction from the heat bath. Thus, the quantum engine obeys the Second Law. However, the quantum engine does so via substantially different mechanisms: classically measurement and erasure determine the thermodynamics, while in the quantum implementation the cost of partition insertion is key.

quant-ph↗

arXiv2022

Whale Casting: Remote mobile streaming humpback whale vocalizations to the world

Over several days in early August 2021, while at sea in Chatham Strait, Southeast Alaska, aboard M/Y Blue Pearl, an online twitch.tv stream broadcast in real-time humpback whale vocalizations monitored via hydrophone. Dozens on mainland North American and around the planet listened in and chatted via the stream. The webcasts demonstrated a proof-of-concept: only relatively inexpensive commercial-off-the-shelf equipment is required for remote mobile streaming at sea. These notes document what was required and make recommendations for higher-quality and larger-scale deployments. One conclusion is that real-time, automated audio documenting whale acoustic behavior is readily accessible and, using the cloud, it can be directly integrated into behavioral databases -- information sources that now often focus exclusively on nonreal-time visual-sighting narrative reports and photography.

cs.HC↗

arXiv2022

The Thermodynamic Uncertainty Theorem

Thermodynamic uncertainty relations (TURs) express a fundamental tradeoff between the precision (inverse scaled variance) of any thermodynamic current by functionals of the average entropy production. Relying on purely variational arguments, we significantly extend these inequalities by incorporating and analyzing the impact of higher statistical cumulants of entropy production within a general framework of time-symmetrically controlled computation. This allows us to derive an exact expression for the current that achieves the minimum scaled variance, for which the TUR bound tightens to an equality that we name Thermodynamic Uncertainty Theorem (TUT). Importantly, both the minimum scaled variance current and the TUT are functionals of the stochastic entropy production, thus retaining the impact of its higher moments. In particular, our results show that, beyond the average, the entropy production distribution's higher moments have a significant effect on any current's precision. This is made explicit via a thorough numerical analysis of swap and reset computations that quantitatively compares the TUT against previous generalized TURs. Our results demonstrate how to interpolate between previously-established bounds and how to identify the most relevant TUR bounds in different nonequilibrium regimes.

cond-mat.stat-mech↗

arXiv2022

Gigahertz Sub-Landauer Momentum Computing

We introduce a fast and highly-efficient physically-realizable bit swap. Employing readily available and scalable Josephson junction microtechnology, the design implements the recently introduced paradigm of momentum computing. Its nanosecond speeds and sub-Landauer thermodynamic efficiency arise from dynamically storing memory in momentum degrees of freedom. As such, during the swap, the microstate distribution is never near equilibrium and the memory-state dynamics fall far outside of stochastic thermodynamics that assumes detailed-balanced Markovian dynamics. The device implements a bit-swap operation -- a fundamental operation necessary to build reversible universal computing. Extensive, physically-calibrated simulations demonstrate that device performance is robust and that momentum computing can support thermodynamically-efficient, high-speed, large-scale general-purpose computing that circumvents Landauer's bound.

cond-mat.stat-mech↗

arXiv2022

Homeostatic and Adaptive Energetics: Nonequilibrium Fluctuations Beyond Detailed Balance in Voltage-Gated Ion Channels

Stochastic thermodynamics has largely succeeded in characterizing both equilibrium and far-from-equilibrium phenomena. Yet many opportunities remain for application to mesoscopic complex systems -- especially biological ones -- whose effective dynamics often violate detailed balance and whose microscopic degrees of freedom are often unknown or intractable. After reviewing excess and housekeeping energetics -- the adaptive and homeostatic components of a system's dissipation -- we extend stochastic thermodynamics with a trajectory class fluctuation theorem for nonequilibrium steady-state, nondetailed-balanced complex systems. We then take up the neurobiological examples of voltage-gated sodium and potassium ion channels to apply and illustrate the theory, elucidating their nonequilibrium behavior under a biophysically plausible action potential drive. These results uncover challenges for future experiments and highlight the progress possible understanding the thermodynamics of complex systems -- without exhaustive knowledge of every underlying degree of freedom.

cond-mat.stat-mech↗

arXiv2022

Algebraic Theory of Patterns as Generalized Symmetries

We generalize the exact predictive regularity of symmetry groups to give an algebraic theory of patterns, building from a core principle of future equivalence. For topological patterns in fully-discrete one-dimensional systems, future equivalence uniquely specifies a minimal semiautomaton. We demonstrate how the latter and its semigroup algebra generalizes translation symmetry to partial and hidden symmetries. This generalization is not as straightforward as previously considered. Here, though, we clarify the underlying challenges. A stochastic form of future equivalence, known as predictive equivalence, captures distinct statistical patterns supported on topological patterns. Finally, we show how local versions of future equivalence can be used to capture patterns in spacetime. As common when moving to higher dimensions, there is not a unique local approach, and we detail two local representations that capture different aspects of spacetime patterns. A previously-developed local spacetime variant of future equivalence captures patterns as generalized symmetries in higher dimensions, but we show this representation is not a faithful generator of its spacetime patterns. This motivates us to introduce a local representation that is a faithful generator, but we demonstrate that it no longer captures generalized spacetime symmetries. Taken altogether, building on future equivalence, the theory defines and quantifies patterns present in a wide range of classical field theories.

cond-mat.stat-mech↗

arXiv2022

Exploring Predictive States via Cantor Embeddings and Wasserstein Distance

Predictive states for stochastic processes are a nonparametric and interpretable construct with relevance across a multitude of modeling paradigms. Recent progress on the self-supervised reconstruction of predictive states from time-series data focused on the use of reproducing kernel Hilbert spaces. Here, we examine how Wasserstein distances may be used to detect predictive equivalences in symbolic data. We compute Wasserstein distances between distributions over sequences ("predictions"), using a finite-dimensional embedding of sequences based on the Cantor for the underlying geometry. We show that exploratory data analysis using the resulting geometry via hierarchical clustering and dimension reduction provides insight into the temporal structure of processes ranging from the relatively simple (e.g., finite-state hidden Markov models) to the very complex (e.g., infinite-state indexed grammars).

cond-mat.stat-mech↗

arXiv2022

Nonequilibrium Statistical Mechanics and Optimal Prediction of Partially-Observed Complex Systems

Only a subset of degrees of freedom are typically accessible or measurable in real-world systems. As a consequence, the proper setting for empirical modeling is that of partially-observed systems. Notably, data-driven models consistently outperform physics-based models for systems with few observable degrees of freedom; e.g., hydrological systems. Here, we provide an operator-theoretic explanation for this empirical success. To predict a partially-observed system's future behavior with physics-based models, the missing degrees of freedom must be explicitly accounted for using data assimilation and model parametrization. Data-driven models, in contrast, employ delay-coordinate embeddings and their evolution under the Koopman operator to implicitly model the effects of the missing degrees of freedom. We describe in detail the statistical physics of partial observations underlying data-driven models using novel Maximum Entropy and Maximum Caliber measures. The resulting nonequilibrium Wiener projections applied to the Mori-Zwanzig formalism reveal how data-driven models may converge to the true dynamics of the observable degrees of freedom. Additionally, this framework shows how data-driven models infer the effects of unobserved degrees of freedom implicitly, in much the same way that physics models infer the effects explicitly. This provides a unified implicit-explicit modeling framework for predicting partially-observed systems, with hybrid physics-informed machine learning methods combining implicit and explicit aspects.

cond-mat.stat-mech↗

arXiv2021

Discovering Causal Structure with Reproducing-Kernel Hilbert Space $ε$-Machines

We merge computational mechanics' definition of causal states (predictively-equivalent histories) with reproducing-kernel Hilbert space (RKHS) representation inference. The result is a widely-applicable method that infers causal structure directly from observations of a system's behaviors whether they are over discrete or continuous events or time. A structural representation -- a finite- or infinite-state kernel $ε$-machine -- is extracted by a reduced-dimension transform that gives an efficient representation of causal states and their topology. In this way, the system dynamics are represented by a stochastic (ordinary or partial) differential equation that acts on causal states. We introduce an algorithm to estimate the associated evolution operator. Paralleling the Fokker-Plank equation, it efficiently evolves causal-state distributions and makes predictions in the original data space via an RKHS functional mapping. We demonstrate these techniques, together with their predictive abilities, on discrete-time, discrete-value infinite Markov-order processes generated by finite-state hidden Markov models with (i) finite or (ii) uncountably-infinite causal states and (iii) continuous-time, continuous-value processes generated by thermally-driven chaotic flows. The method robustly estimates causal structure in the presence of varying external and measurement noise levels and for very high dimensional data.

cs.LG↗

arXiv2021

Nonequilibrium Thermodynamics in Measuring Carbon Footprints: Disentangling Structure and Artifact in Input-Output Accounting

Multiregional input-output (MRIO) tables, in conjunction with Leontief analysis, are widely-used to assess the geographical distribution of carbon emissions and the economic activities that cause them. Majorization, a tool originating in economics that has found utility in statistical mechanics, can provide insight into how Leontief analysis links disparities in emissions with global income inequality. We examine Leontief analysis as a model, drawing out similarities with modern nonequilibrium statistical mechanics. Paralleling the physical concept of thermo-majorization, we define the concept of eco-majorization and show it is a sufficient condition to determine the directionality of embodied emission flows. Surprisingly, relatively small trade deficits and a geographically heterogeneous emissions-per-dollar ratio greatly increases the appearance of eco-majorization, regardless of any further content in the MRIO tables used. Our results are bolstered by a statistical analysis of null models of MRIO tables, based on data provided by the Global Trade Aggregation Project9

physics.soc-ph↗