arXiv subjects

J. P. Crutchfield

Publications and source records attributed to J. P. Crutchfield.

At least 19 recordsLinked to original sources

arXiv2023

Optimality and Complexity in Measured Quantum-State Stochastic Processes

If an experimentalist observes a sequence of emitted quantum states via either projective or positive-operator-valued measurements, the outcomes form a time series. Individual time series are realizations of a stochastic process over the measurements' classical outcomes. We recently showed that, in general, the resulting stochastic process is highly complex in two specific senses: (i) it is inherently unpredictable to varying degrees that depend on measurement choice and (ii) optimal prediction requires using an infinite number of temporal features. Here, we identify the mechanism underlying this complicatedness as generator nonunifilarity -- the degeneracy between sequences of generator states and sequences of measurement outcomes. This makes it possible to quantitatively explore the influence that measurement choice has on a quantum process' degrees of randomness and structural complexity using recently introduced methods from ergodic theory. Progress in this, though, requires quantitative measures of structure and memory in observed time series. And, success requires accurate and efficient estimation algorithms that overcome the requirement to explicitly represent an infinite set of predictive features. We provide these metrics and associated algorithms, using them to design informationally-optimal measurements of open quantum dynamical systems.

quant-ph↗

arXiv2021

Thermodynamic Machine Learning through Maximum Work Production

Adaptive systems -- such as a biological organism gaining survival advantage, an autonomous robot executing a functional task, or a motor protein transporting intracellular nutrients -- must model the regularities and stochasticity in their environments to take full advantage of thermodynamic resources. Analogously, but in a purely computational realm, machine learning algorithms estimate models to capture predictable structure and identify irrelevant noise in training data. This happens through optimization of performance metrics, such as model likelihood. If physically implemented, is there a sense in which computational models estimated through machine learning are physically preferred? We introduce the thermodynamic principle that work production is the most relevant performance metric for an adaptive physical agent and compare the results to the maximum-likelihood principle that guides machine learning. Within the class of physical agents that most efficiently harvest energy from their environment, we demonstrate that an efficient agent's model explicitly determines its architecture and how much useful work it harvests from the environment. We then show that selecting the maximum-work agent for given environmental data corresponds to finding the maximum-likelihood model. This establishes an equivalence between nonequilibrium thermodynamics and dynamic learning. In this way, work maximization emerges as an organizing principle that underlies learning in adaptive thermodynamic systems.

cond-mat.stat-mech↗

arXiv2020

Network and Phase Symmetries Reveal That Amplitude Dynamics Stabilize Decoupled Oscillator Clusters

Oscillator networks display intricate synchronization patterns. Determining their stability typically requires incorporating the symmetries of the network coupling. Going beyond analyses that appeal only to a network's automorphism group, we explore synchronization patterns that emerge from the phase-shift invariance of the dynamical equations and symmetries in the nodes. We show that these nonstructural symmetries simplify stability calculations. We analyze a ring-network of phase-amplitude oscillators that exhibits a "decoupled" state in which physically-coupled nodes appear to act independently due to emergent cancellations in the equations of dynamical evolution. We establish that this state can be linearly stable for a ring of phase-amplitude oscillators, but not for a ring of phase-only oscillators that otherwise require explicit long-range, nonpairwise, or nonphase coupling. In short, amplitude-phase interactions are key to stable synchronization at a distance.

math.DS↗

arXiv2020

Fraudulent White Noise: Flat power spectra belie arbitrarily complex processes

Power spectral densities are a common, convenient, and powerful way to analyze signals. So much so that they are now broadly deployed across the sciences and engineering---from quantum physics to cosmology, and from crystallography to neuroscience to speech recognition. The features they reveal not only identify prominent signal-frequencies but also hint at mechanisms that generate correlation and lead to resonance. Despite their near-centuries-long run of successes in signal analysis, here we show that flat power spectra can be generated by highly complex processes, effectively hiding all inherent structure in complex signals. Historically, this circumstance has been widely misinterpreted, being taken as the renowned signature of "structureless" white noise---the benchmark of randomness. We argue, in contrast, to the extent that most real-world complex systems exhibit correlations beyond pairwise statistics their structures evade power spectra and other pairwise statistical measures. As concrete physical examples, we demonstrate that fraudulent white noise hides the predictable structure of both entangled quantum systems and chaotic crystals. To make these words of warning operational, we present constructive results that explore how this situation comes about and the high toll it takes in understanding complex mechanisms. First, we give the closed-form solution for the power spectrum of a very broad class of structurally-complex signal generators. Second, we demonstrate the close relationship between eigen-spectra of evolution operators and power spectra. Third, we characterize the minimal generative structure implied by any power spectrum. Fourth, we show how to construct arbitrarily complex processes with flat power spectra. Finally, leveraging this diagnosis of the problem, we point the way to developing more incisive tools for discovering structure in complex signals.

cond-mat.stat-mech↗

arXiv2020

Balancing Error and Dissipation in Computing

Modern digital electronics support remarkably reliable computing, especially given the challenge of controlling nanoscale logical components that interact in fluctuating environments. However, we demonstrate that the high-reliability limit is subject to a fundamental error-energy-efficiency tradeoff that arises from time-symmetric control: Requiring a low probability of error causes energy consumption to diverge as logarithm of the inverse error rate for nonreciprocal logical transitions. The reciprocity (self-invertibility) of a computation is a stricter condition for thermodynamic efficiency than logical reversibility (invertibility), the latter being the root of Landauer's work bound on erasing information. Beyond engineered computation, the results identify a generic error-dissipation tradeoff in steady-state transformations of genetic information carried out by biological organisms. The lesson is that computation under time-symmetric control cannot reach, and is often far above, the Landauer limit. In this way, time-asymmetry becomes a design principle for thermodynamically efficient computing.

cond-mat.stat-mech↗

arXiv2020

Inference, Prediction, and Entropy-Rate Estimation of Continuous-time, Discrete-event Processes

Inferring models, predicting the future, and estimating the entropy rate of discrete-time, discrete-event processes is well-worn ground. However, a much broader class of discrete-event processes operates in continuous-time. Here, we provide new methods for inferring, predicting, and estimating them. The methods rely on an extension of Bayesian structural inference that takes advantage of neural network's universal approximation power. Based on experiments with complex synthetic data, the methods are competitive with the state-of-the-art for prediction and entropy-rate estimation.

cond-mat.stat-mech↗

arXiv2019

Optimizing Quantum Models of Classical Channels: The reverse Holevo problem

Given a classical channel---a stochastic map from inputs to outputs---the input can often be transformed to an intermediate variable that is informationally smaller than the input. The new channel accurately simulates the original but at a smaller transmission rate. Here, we examine this procedure when the intermediate variable is a quantum state. We determine when and how well quantum simulations of classical channels may improve upon the minimal rates of classical simulation. This inverts Holevo's original question of quantifying the capacity of quantum channels with classical resources. We also show that this problem is equivalent to another, involving the local generation of a distribution from common entanglement.

quant-ph↗

arXiv2019

Probabilistic Deterministic Finite Automata and Recurrent Networks, Revisited

Reservoir computers (RCs) and recurrent neural networks (RNNs) can mimic any finite-state automaton in theory, and some workers demonstrated that this can hold in practice. We test the capability of generalized linear models, RCs, and Long Short-Term Memory (LSTM) RNN architectures to predict the stochastic processes generated by a large suite of probabilistic deterministic finite-state automata (PDFA). PDFAs provide an excellent performance benchmark in that they can be systematically enumerated, the randomness and correlation structure of their generated processes are exactly known, and their optimal memory-limited predictors are easily computed. Unsurprisingly, LSTMs outperform RCs, which outperform generalized linear models. Surprisingly, each of these methods can fall short of the maximal predictive accuracy by as much as 50% after training and, when optimized, tend to fall short of the maximal predictive accuracy by ~5%, even though previously available methods achieve maximal predictive accuracy with orders-of-magnitude less data. Thus, despite the representational universality of RCs and RNNs, using them can engender a surprising predictive gap for simple stimuli. One concludes that there is an important and underappreciated role for methods that infer "causal states" or "predictive state representations".

cs.LG↗

arXiv2018

Shortcuts to Thermodynamic Computing: The Cost of Fast and Faithful Erasure

Landauer's Principle states that the energy cost of information processing must exceed the product of the temperature and the change in Shannon entropy of the information-bearing degrees of freedom. However, this lower bound is achievable only for quasistatic, near-equilibrium computations -- that is, only over infinite time. In practice, information processing takes place in finite time, resulting in dissipation and potentially unreliable logical outcomes. For overdamped Langevin dynamics, we show that counterdiabatic potentials can be crafted to guide systems rapidly and accurately along desired computational paths, providing shortcuts that allows for the precise design of finite-time computations. Such shortcuts require additional work, beyond Landauer's bound, that is irretrievably dissipated into the environment. We show that this dissipated work is proportional to the computation rate as well as the square of the information-storing system's length scale. As a paradigmatic example, we design shortcuts to erase a bit of information metastably stored in a double-well potential. Though dissipated work generally increases with erasure fidelity, we show that it is possible perform perfect erasure in finite time with finite work. We also show that the robustness of information storage affects the energetic cost of erasure---specifically, the dissipated work scales as the information lifetime of the bistable system. Our analysis exposes a rich and nuanced relationship between work, speed, size of the information-bearing degrees of freedom, storage robustness, and the difference between initial and final informational statistics.

cond-mat.stat-mech↗

arXiv2017

A Physics-Based Approach to Unsupervised Discovery of Coherent Structures in Spatiotemporal Systems

Given that observational and numerical climate data are being produced at ever more prodigious rates, increasingly sophisticated and automated analysis techniques have become essential. Deep learning is quickly becoming a standard approach for such analyses and, while great progress is being made, major challenges remain. Unlike commercial applications in which deep learning has led to surprising successes, scientific data is highly complex and typically unlabeled. Moreover, interpretability and detecting new mechanisms are key to scientific discovery. To enhance discovery we present a complementary physics-based, data-driven approach that exploits the causal nature of spatiotemporal data sets generated by local dynamics (e.g. hydrodynamic flows). We illustrate how novel patterns and coherent structures can be discovered in cellular automata and outline the path from them to climate data.

physics.flu-dyn↗

arXiv2017

Prediction and Generation of Binary Markov Processes: Can a Finite-State Fox Catch a Markov Mouse?

Understanding the generative mechanism of a natural system is a vital component of the scientific method. Here, we investigate one of the fundamental steps toward this goal by presenting the minimal generator of an arbitrary binary Markov process. This is a class of processes whose predictive model is well known. Surprisingly, the generative model requires three distinct topologies for different regions of parameter space. We show that a previously proposed generator for a particular set of binary Markov processes is, in fact, not minimal. Our results shed the first quantitative light on the relative (minimal) costs of prediction and generation. We find, for instance, that the difference between prediction and generation is maximized when the process is approximately independently, identically distributed.

cond-mat.stat-mech↗

arXiv2017

Extreme Quantum Advantage for Rare-Event Sampling

We introduce a quantum algorithm for efficient biased sampling of the rare events generated by classical memoryful stochastic processes. We show that this quantum algorithm gives an extreme advantage over known classical biased sampling algorithms in terms of the memory resources required. The quantum memory advantage ranges from polynomial to exponential and when sampling the rare equilibrium configurations of spin systems the quantum advantage diverges.

quant-ph↗

arXiv2017

Islands in the Gap: Intertwined Transport and Localization in Structurally Complex Materials

Localized waves in disordered one-dimensional materials have been studied for decades, including white-noise and correlated disorder, as well as quasi-periodic disorder. How these wave phenomena relate to those in crystalline (periodic ordered) materials---arguably the better understood setting---has been a mystery ever since Anderson discovered disorder-induced localization. Nonetheless, together these revolutionized materials science and technology and led to new physics far beyond the solid state. We introduce a broad family of structurally complex materials---chaotic crystals---that interpolate between these organizational extremes---systematically spanning periodic structures and random disorder. Within the family one can tune the degree of disorder to sweep through an intermediate structurally disordered region between two periodic lattices. This reveals new transport and localization phenomena reflected in a rich array of energy-dependent localization degree and density of states. In particular, strong localization is observed even with a very low degree of disorder. Moreover, markedly enhanced localization and delocalization coexist in a very narrow range of energies. Most notably, beyond the simply smoothed bands found in previous disorder studies, islands of transport emerge in band gaps and sharp band boundaries persist in the presence of substantial disorder. Finally, the family of materials comes with rather direct specifications of how to assemble the requisite material organizations.

cond-mat.mtrl-sci↗

arXiv2017

Structure and Randomness of Continuous-Time Discrete-Event Processes

Loosely speaking, the Shannon entropy rate is used to gauge a stochastic process' intrinsic randomness; the statistical complexity gives the cost of predicting the process. We calculate, for the first time, the entropy rate and statistical complexity of stochastic processes generated by finite unifilar hidden semi-Markov models---memoryful, state-dependent versions of renewal processes. Calculating these quantities requires introducing novel mathematical objects (ε-machines of hidden semi-Markov processes) and new information-theoretic methods to stochastic processes.

cond-mat.stat-mech↗

arXiv2016

Thermodynamics of Random Number Generation

We analyze the thermodynamic costs of the three main approaches to generating random numbers via the recently introduced Information Processing Second Law. Given access to a specified source of randomness, a random number generator (RNG) produces samples from a desired target probability distribution. This differs from pseudorandom number generators (PRNG) that use wholly deterministic algorithms and from true random number generators (TRNG) in which the randomness source is a physical system. For each class, we analyze the thermodynamics of generators based on algorithms implemented as finite-state machines, as these allow for direct bounds on the required physical resources. This establishes bounds on heat dissipation and work consumption during the operation of three main classes of RNG algorithms---including those of von Neumann, Knuth and Yao, and Roche and Hoshi---and for PRNG methods. We introduce a general TRNG and determine its thermodynamic costs exactly for arbitrary target distributions. The results highlight the significant differences between the three main approaches to random number generation: One is work producing, one is work consuming, and the other is potentially dissipation neutral. Notably, TRNGs can both generate random numbers and convert thermal energy to stored work. These thermodynamic costs on information creation complement Landauer's limit on the irreducible costs of information destruction.

cond-mat.stat-mech↗

arXiv2016

Fluctuations When Driving Between Nonequilibrium Steady States

Maintained by environmental fluxes, biological systems are thermodynamic processes that operate far from equilibrium without detailed-balance dynamics. Yet, they often exhibit well defined nonequilibrium steady states (NESSs). More importantly, critical thermodynamic functionality arises directly from transitions among their NESSs, driven by environmental switching. Here, we identify constraints on excess thermodynamic quantities that ride above the NESS housekeeping background. We do this by extending the Crooks fluctuation theorem to transitions among NESSs, without invoking an unphysical dual dynamics. This and corresponding integral fluctuation theorems determine how much work must be expended when controlling systems maintained far from equilibrium. This generalizes feedback control theory, showing that Maxwellian Demons can leverage mesoscopic-state information to take advantage of the excess energetics in NESS transitions. Altogether, these point to universal thermodynamic laws that are immediately applicable to the accessible degrees of freedom within the effective dynamic at any emergent level of hierarchical organization. By way of illustration, this readily allows analyzing a voltage-gated sodium ion channel whose molecular conformational dynamics play a critical functional role in propagating action potentials in mammalian neuronal membranes.

cond-mat.stat-mech↗

arXiv2016

The Markov Memory for Generating Rare Events

We classify the rare events of structured, memoryful stochastic processes and use this to analyze sequential and parallel generators for these events. Given a stochastic process, we introduce a method to construct a new process whose typical realizations are a given process' rare events. This leads to an expression for the minimum memory required to generate rare events. We then show that the recently discovered classical-quantum ambiguity of simplicity also occurs when comparing the structure of process fluctuations.

cond-mat.stat-mech↗

arXiv2016

Extreme Quantum Advantage when Simulating Strongly Coupled Classical Systems

Classical stochastic processes can be generated by quantum simulators instead of the more standard classical ones, such as hidden Markov models. One reason for using quantum simulators is that they generally require less memory than their classical counterparts. Here, we examine this quantum advantage for strongly coupled spin systems---the Dyson-like one-dimensional Ising spin chain with variable interaction length. We find that the advantage scales with both interaction range and temperature, growing without bound as interaction increases. Thus, quantum systems can very efficiently simulate strongly coupled classical systems.

quant-ph↗