arXiv subjects

Sarah Marzen

Publications and source records attributed to Sarah Marzen.

22 records · Page 2Linked to original sources

arXiv2014

Circumventing the Curse of Dimensionality in Prediction: Causal Rate-Distortion for Infinite-Order Markov Processes

Predictive rate-distortion analysis suffers from the curse of dimensionality: clustering arbitrarily long pasts to retain information about arbitrarily long futures requires resources that typically grow exponentially with length. The challenge is compounded for infinite-order Markov processes, since conditioning on finite sequences cannot capture all of their past dependencies. Spectral arguments show that algorithms which cluster finite-length sequences fail dramatically when the underlying process has long-range temporal correlations and can fail even for processes generated by finite-memory hidden Markov models. We circumvent the curse of dimensionality in rate-distortion analysis of infinite-order processes by casting predictive rate-distortion objective functions in terms of the forward- and reverse-time causal states of computational mechanics. Examples demonstrate that the resulting causal rate-distortion theory substantially improves current predictive rate-distortion analyses.

cond-mat.stat-mech↗

arXiv2014

Informational and Causal Architecture of Discrete-Time Renewal Processes

Renewal processes are broadly used to model stochastic behavior consisting of isolated events separated by periods of quiescence, whose durations are specified by a given probability law. Here, we identify the minimal sufficient statistic for their prediction (the set of causal states), calculate the historical memory capacity required to store those states (statistical complexity), delineate what information is predictable (excess entropy), and decompose the entropy of a single measurement into that shared with the past, future, or both. The causal state equivalence relation defines a new subclass of renewal processes with a finite number of causal states despite having an unbounded interevent count distribution. We use these formulae to analyze the output of the parametrized Simple Nonunifilar Source, generated by a simple two-state hidden Markov model, but with an infinite-state epsilon-machine presentation. All in all, the results lay the groundwork for analyzing processes with infinite statistical complexity and infinite excess entropy.

cond-mat.stat-mech↗

arXiv2014

Information Anatomy of Stochastic Equilibria

A stochastic nonlinear dynamical system generates information, as measured by its entropy rate. Some---the ephemeral information---is dissipated and some---the bound information---is actively stored and so affects future behavior. We derive analytic expressions for the ephemeral and bound informations in the limit of small-time discretization for two classical systems that exhibit dynamical equilibria: first-order Langevin equations (i) where the drift is the gradient of a potential function and the diffusion matrix is invertible and (ii) with a linear drift term (Ornstein-Uhlenbeck) but a noninvertible diffusion matrix. In both cases, the bound information is sensitive only to the drift, while the ephemeral information is sensitive only to the diffusion matrix and not to the drift. Notably, this information anatomy changes discontinuously as any of the diffusion coefficients vanishes, indicating that it is very sensitive to the noise structure. We then calculate the information anatomy of the stochastic cusp catastrophe and of particles diffusing in a heat bath in the overdamped limit, both examples of stochastic gradient descent on a potential landscape. Finally, we use our methods to calculate and compare approximations for the so-called time-local predictive information for adaptive agents.

cond-mat.stat-mech↗

arXiv2010

An equivalence between a Maximum Caliber analysis of two-state kinetics and the Ising model

Application of the information-theoretic Maximum Caliber principle to the microtrajectories of a two-state system shows that the determination of key dynamical quantities can be mapped onto the evaluation of properties of the 1-D Ising model. The strategy described here is equivalent to an earlier Maximum Caliber formulation of the two-state problem, but reveals a different way of imposing the constraints which determine the probability distribution of allowed microtrajectories. The theoretical calculations of second moments, covariances, and correlation times that are obtained from Maximum Caliber agree well with simulated data of a particle diffusing on a double Gaussian surface, as well as with recent experiments on a particle trapped by a dual-well optical trap. The formalism reveals a new relationship between the average occupancy of the two states of the system, the average number of transitions between the two states that the system undergoes, Markov transition probabilities, and the discretization time step. In addition, Maxwell-like relations imply how measurements on one potential landscape can be used to make predictions about the dynamics on a different potential landscape, independent of further experiment.

physics.bio-ph↗