The typical set and entropy in stochastic systems with arbitrary phase space growth

Abstract

The existence of the {\em typical set} is key for data compression strategies and for the emergence of robust statistical observables in macroscopic physical systems. Standard approaches derive its existence from a restricted set of dynamical constraints. However, given the enormous consequences for the understanding of the system's dynamics, and its role underlying the presence of stable, almost deterministic statistical patterns, a question arises whether typical sets exist in much more general scenarios. We demonstrate here that the typical set can be defined and characterized from general forms of entropy for a much wider class of stochastic processes than it was previously thought. This includes processes showing arbitrary path dependence, long range correlations or dynamic sampling spaces; suggesting that typicality is a generic property of stochastic processes, regardless of their complexity. Our results impact directly in the understanding of the stability of complex systems, open the door to new data compression strategies and points to the existence of statistical mechanics-like approaches to systems arbitrarily away from equilibrium with dynamic phase spaces. We argue that the potential emergence of robust properties in complex stochastic systems provided by the existence of typical sets has special relevance to biological systems.