arXiv subjects

Rudolf Hanel

Publications and source records attributed to Rudolf Hanel.

At least 37 records · Page 2Linked to original sources

arXiv2018

Information geometric duality of $ϕ$-deformed exponential families

In the world of generalized entropies---which, for example, play a role in physical systems with sub- and super-exponential phasespace growth per degree of freedom---there are two ways for implementing constraints in the maximum entropy principle: linear- and escort constraints. Both appear naturally in different contexts. Linear constraints appear e.g. in physical systems, when additional information about the system is available through higher moments. Escort distributions appear naturally in the context of multifractals and information geometry. It was shown recently that there exists a fundamental duality that relates both approaches on the basis of the corresponding deformed logarithms (deformed-log duality). Here we show that there exists another duality that arises in the context of information geometry, relating the Fisher information of $ϕ$-deformed exponential families that correspond to linear constraints (as studied by J. Naudts), with those that are based on escort constraints (as studied by S.-I. Amari). We explicitly demonstrate this information geometric duality for the case of $(c,d)$-entropy that covers all situations that are compatible with the first three Shannon-Khinchin axioms, and that include Shannon, Tsallis, Anteneodo-Plastino entropy, and many more as special cases. Finally, we discuss the relation between the deformed-log duality and the information geometric duality, and mention that the escort distributions arising in the two dualities are generally different and only coincide for the case of the Tsallis deformation.

math-ph↗

arXiv2018

Maximum configuration principle for driven systems with arbitrary driving

Depending on context, the term entropy is used for a thermodynamic quantity, a~measure of available choice, a quantity to measure information, or, in the context of statistical inference, a maximum configuration predictor. For systems in equilibrium or processes without memory, the mathematical expression for these different concepts of entropy appears to be the so-called Boltzmann--Gibbs--Shannon entropy, H.For processes with memory, such as driven- or self-reinforcing-processes, this is no longer true: the different entropy concepts lead to distinct functionals that generally differ from H. Here we focus on the maximum configuration entropy (that predicts empirical distribution functions) in the context of driven dissipative systems. We develop the corresponding framework and derive the entropy functional that describes the distribution of observable states as a function of the details of the driving process. We do this for sample space reducing (SSR) processes, which provide an analytically tractable model for driven dissipative systems with controllable driving. The fact that a consistent framework for a maximum configuration entropy exists for arbitrarily driven non-equilibrium systems opens the possibility of deriving a full statistical theory of driven dissipative systems of this kind. This provides us with the technical means needed~to derive a thermodynamic theory of driven processes based on a statistical theory. We discuss the Legendre structure for driven systems.

cond-mat.stat-mech↗

arXiv2018

Classification of complex systems by their sample-space scaling exponents

The nature of statistics, statistical mechanics and consequently the thermodynamics of stochastic systems is largely determined by how the number of states $W(N)$ depends on the size $N$ of the system. Here we propose a scaling expansion of the phasespace volume $W(N)$ of a stochastic system. The corresponding expansion coefficients (exponents) define the universality class the system belongs to. Systems within the same universality class share the same statistics and thermodynamics. For sub-exponentially growing systems such expansions have been shown to exist. By using the scaling expansion this classification can be extended to all stochastic systems, including correlated, constraint and super-exponential systems. The extensive entropy of these systems can be easily expressed in terms of thee scaling exponents. Systems with super-exponential phasespace growth contain important systems, such as magnetic coins that combine combinatorial and structural statistics. We discuss other applications in the statistics of networks, aging, and cascading random walks.

cond-mat.stat-mech↗

arXiv2018

How driving rates determine the statistics of driven non-equilibrium systems with stationary distributions

Sample space reducing (SSR) processes offer a simple analytical way to understand of the origin and ubiquity of power-laws in many path-dependent complex systems. SRR processes show a wide range of applications that range from fragmentation processes, language formation to cascading pro- cesses. Here we argue that they also offer a natural framework to understand stationary distributions of generic driven non-equilibrium systems that are composed of a driving and a relaxing process. We show that the statistics of driven non-equilibrium systems can be derived from the understanding of the nature of the underlying driving process. For constant driving rates exact power-laws emerge with exponents that are related to the driving rate. If driving rates become state-dependent, or if they vary across the life-span of the process, the functional form of the state-dependence determines the statistics. Constant driving rates lead to exact power-laws, a linear state-dependence function yields exponential or Gamma distributions, a quadratic function gives the normal distribution. Logarithmic and power-law state dependence leads to log-normal and stretched exponential distribution functions, respectively. Also Weibull, Gompertz and Tsallis-Pareto distributions arise naturally from simple state-dependent driving rates. We discuss a simple physical example of consecutive elastic collisions that exactly represents a SSR process.

cond-mat.dis-nn↗

arXiv2017

Sample space reducing cascading processes produce the full spectrum of scaling exponents

Sample Space Reducing (SSR) processes are simple stochastic processes that offer a new route to understand scaling in path-dependent processes. Here we define a cascading process that generalises the recently defined SSR processes and is able to produce power laws with arbitrary exponents. We demonstrate analytically that the frequency distributions of states are power laws with exponents that coincide with the multiplication parameter of the cascading process. In addition, we show that imposing energy conservation in SSR cascades allows us to recover Fermi's classic result on the energy spectrum of cosmic rays, with the universal exponent -2, which is independent of the multiplication parameter of the cascade. Applications of the proposed process include fragmentation processes or directed cascading diffusion on networks, such as rumour or epidemic spreading.

cond-mat.stat-mech↗

arXiv2017

The three faces of entropy for complex systems -- information, thermodynamics and the maxent principle

There are three ways to conceptualize entropy: entropy as an extensive thermodynamic quantity of physical systems (Clausius, Boltzmann, Gibbs), entropy as a measure for information production of ergodic sources (Shannon), and entropy as a means for statistical inference on multinomial Bernoulli processes (Jaynes maximum entropy principle). Even though these notions are fundamentally different concepts, the functional form of the entropy for thermodynamic systems in equilibrium, for ergodic sources in information theory, and for independent sampling processes in statistical systems, is degenerate, $H(p)=-\sum_i p_i\log p_i$. For many complex systems, which are typically history-dependent, non-ergodic and non-multinomial, this is no longer the case. Here we show that for such processes the three entropy concepts lead to different functional forms of entropy. We explicitly compute these entropy functionals for three concrete examples. For Polya urn processes, which are simple self-reinforcing processes, the source information rate is $S_{\rm IT}=\frac{1}{1-c}\frac1N \log N$, the thermodynamical (extensive) entropy is $(c,d)$-entropy, $S_{\rm EXT}=S_{(c,0)}$, and the entropy in the maxent principle (MEP) is $S_{\rm MEP}(p)=-\sum_i \log p_i$. For sample space reducing (SSR) processes, which are simple path-dependent processes that are associated with power law statistics, the information rate is $S_{\rm IT}=1+ \frac12 \log W$, the extensive entropy is $S_{\rm EXT}=H(p)$, and the maxent result is $S_{\rm MEP}(p)=H(p/p_1)+H(1-p/p_1)$. Finally, for multinomial mixture processes, the information rate is given by the conditional entropy $\langle H\rangle_f$, with respect to the mixing kernel $f$, the extensive entropy is given by $H$, and the MEP functional corresponds one-to-one to the logarithm of the mixing kernel.

cond-mat.stat-mech↗

arXiv2016

Analytical computation of frequency distributions of path-dependent processes by means of a non-multinomial maximum entropy approach

Path-dependent stochastic processes are often non-ergodic and observables can no longer be computed within the ensemble picture. The resulting mathematical difficulties pose severe limits to the analytical understanding of path-dependent processes. Their statistics is typically non-multinomial in the sense that the multiplicities of the occurrence of states is not a multinomial factor. The maximum entropy principle is tightly related to multinomial processes, non-interacting systems, and to the ensemble picture; It loses its meaning for path-dependent processes. Here we show that an equivalent to the ensemble picture exists for path-dependent processes, such that the non-multinomial statistics of the underlying dynamical process, by construction, is captured correctly in a functional that plays the role of a relative entropy. We demonstrate this for self-reinforcing Pólya urn processes, which explicitly generalise multinomial statistics. We demonstrate the adequacy of this constructive approach towards non-multinomial pendants of entropy by computing frequency and rank distributions of Pólya urn processes. We show how microscopic update rules of a path-dependent process allow us to explicitly construct a non-multinomial entropy functional, that, when maximized, predicts the time-dependent distribution function.

cond-mat.stat-mech↗

arXiv2016

Fitting Power-laws in empirical data with estimators that work for all exponents

It has been repeatedly stated that maximum likelihood (ML) estimates of exponents of power-law distributions can only be reliably obtained for exponents smaller than minus one. The main argument that power laws are otherwise not normalizable, depends on the underlying sample space the data is drawn from, and is true only for sample spaces that are unbounded from above. Here we show that power-laws obtained from bounded sample spaces (as is the case for practically all data related problems) are always free of such limitations and maximum likelihood estimates can be obtained for arbitrary powers without restrictions. Here we first derive the appropriate ML estimator for arbitrary exponents of power-law distributions on bounded discrete sample spaces. We then show that an almost identical estimator also works perfectly for continuous data. We implemented this ML estimator and discuss its performance with previous attempts. We present a general recipe of how to use these estimators and present the associated computer codes.

physics.data-an↗

arXiv2016

Systemic stability, cell differentiation, and evolution - A dynamical systems perspective

Species or population that proliferate faster than others become dominant in numbers. Catalysis allows catalytic sets within a molecular reaction network to dominate the non catalytic parts of the network by processing most of the available substrate. As a consequence one may consider a 'catalytic fitness' of sets of molecular species. The fittest sets emerge as the expressed chemical backbone or sub-network of larger chemical reaction networks employed by organisms. However, catalytic fitness depends on the systemic context and the stability of systemic dynamics. Unstable reaction networks would easily be reshaped or destroyed by fluctuations of the chemical environment. In this paper we therefore focus on recognizing systemic stability as an evolutionary selection criterion. In fact, instabilities of regulatory systems dynamics become predictive for associated evolutionary forces driving the emergence large reaction networks that avoid or control inherent instabilities. Systemic instabilities can be identified and analyzed using relatively simple mathematical random networks models of complex regulatory systems. Using a statistical ensemble approach one can identify fundamental causes of instable dynamics, infer evolutionary preferred network properties, and predict evolutionary emergent control mechanisms and their entanglement with cell differentiation processes. Surprisingly, what systemic stability tells us here is that cells (or other non-linear regulatory systems) never had to learn how to differentiate, but rather how to avoid and control differentiation. For example, in this framework we can predict that regulatory systems will evolutionary favor networks where the number of catalytic enhancers is not larger than the number of suppressors.

q-bio.CB↗

arXiv2016

Extreme robustness of scaling in sample space reducing processes explains Zipf's law in diffusion on directed networks

It has been shown recently that a specific class of path-dependent stochastic processes, which reduce their sample space as they unfold, lead to exact scaling laws in frequency and rank distributions. Such Sample Space Reducing processes (SSRP) offer an alternative new mechanism to understand the emergence of scaling in countless processes. The corresponding power law exponents were shown to be related to noise levels in the process. Here we show that the emergence of scaling is not limited to the simplest SSRPs, but holds for a huge domain of stochastic processes that are characterized by non-uniform prior distributions. We demonstrate mathematically that in the absence of noise the scaling exponents converge to $-1$ (Zipf's law) for almost all prior distributions. As a consequence it becomes possible to fully understand targeted diffusion on weighted directed networks and its associated scaling laws law in node visit distributions. The presence of cycles can be properly interpreted as playing the same role as noise in SSRPs and, accordingly, determine the scaling exponents. The result that Zipf's law emerges as a generic feature of diffusion on networks, regardless of its details, and that the exponent of visiting times is related to the amount of cycles in a network could be relevant for a series of applications in traffic-, transport- and supply chain management.

physics.soc-ph↗

arXiv2015

Understanding Zipf's law of word frequencies through sample-space collapse in sentence formation

The formation of sentences is a highly structured and history-dependent process. The probability of using a specific word in a sentence strongly depends on the 'history' of word-usage earlier in that sentence. We study a simple history-dependent model of text generation assuming that the sample-space of word usage reduces along sentence formation, on average. We first show that the model explains the approximate Zipf law found in word frequencies as a direct consequence of sample-space reduction. We then empirically quantify the amount of sample-space reduction in the sentences of ten famous English books, by analysis of corresponding word-transition tables that capture which words can follow any given word in a text. We find a highly nested structure in these transition tables and show that this `nestedness' is tightly related to the power law exponents of the observed word frequency distributions. With the proposed model it is possible to understand that the nestedness of a text can be the origin of the actual scaling exponent, and that deviations from the exact Zipf law can be understood by variations of the degree of nestedness on a book-by-book basis. On a theoretical level we are able to show that in case of weak nesting, Zipf's law breaks down in a fast transition. Unlike previous attempts to understand Zipf's law in language the sample-space reducing model is not based on assumptions of multiplicative, preferential, or self-organised critical mechanisms behind language formation, but simply used the empirically quantifiable parameter 'nestedness' to understand the statistics of word frequencies.

physics.soc-ph↗

arXiv2015

Understanding scaling through history-dependent processes with collapsing sample space

History-dependent processes are ubiquitous in natural and social systems. Many such stochastic processes, especially those that are associated with complex systems, become more constrained as they unfold, meaning that their sample-space, or their set of possible outcomes, reduces as they age. We demonstrate that these sample-space reducing (SSR) processes necessarily lead to Zipf's law in the rank distributions of their outcomes. We show that by adding noise to SSR processes the corresponding rank distributions remain exact power-laws, $p(x)\sim x^{-λ}$, where the exponent directly corresponds to the mixing ratio of the SSR process and noise. This allows us to give a precise meaning to the scaling exponent in terms of the degree to how much a given process reduces its sample-space as it unfolds. Noisy SSR processes further allow us to explain a wide range of scaling exponents in frequency distributions ranging from $α= 2$ to $\infty$. We discuss several applications showing how SSR processes can be used to understand Zipf's law in word frequencies, and how they are related to diffusion processes in directed networks, or ageing processes such as in fragmentation processes. SSR processes provide a new alternative to understand the origin of scaling in complex systems without the recourse to multiplicative, preferential, or self-organised critical processes.

physics.soc-ph↗

arXiv2014

How multiplicity determines entropy and the derivation of the maximum entropy principle for complex systems

The maximum entropy principle (MEP) is a method for obtaining the most likely distribution functions of observables from statistical systems, by maximizing entropy under constraints. The MEP has found hundreds of applications in ergodic and Markovian systems in statistical mechanics, information theory, and statistics. For several decades there exists an ongoing controversy whether the notion of the maximum entropy principle can be extended in a meaningful way to non-extensive, non-ergodic, and complex statistical systems and processes. In this paper we start by reviewing how Boltzmann-Gibbs-Shannon entropy is related to multiplicities of independent random processes. We then show how the relaxation of independence naturally leads to the most general entropies that are compatible with the first three Shannon-Khinchin axioms, the (c,d)-entropies. We demonstrate that the MEP is a perfectly consistent concept for non-ergodic and complex statistical systems if their relative entropy can be factored into a generalized multiplicity and a constraint term. The problem of finding such a factorization reduces to finding an appropriate representation of relative entropy in a linear basis. In a particular example we show that path-dependent random processes with memory naturally require specific generalized entropies. The example is the first exact derivation of a generalized entropy from the microscopic properties of a path-dependent random process.

physics.class-ph↗

arXiv2013

Generalized (c,d)-entropy and aging random walks

Complex systems are often inherently non-ergodic and non-Markovian for which Shannon entropy loses its applicability. In particular accelerating, path-dependent, and aging random walks offer an intuitive picture for these non-ergodic and non-Markovian systems. It was shown that the entropy of non-ergodic systems can still be derived from three of the Shannon-Khinchin axioms, and by violating the fourth -- the so-called composition axiom. The corresponding entropy is of the form $S_{c,d} \sim \sum_i Γ(1+d,1-c\ln p_i)$ and depends on two system-specific scaling exponents, $c$ and $d$. This entropy contains many recently proposed entropy functionals as special cases, including Shannon and Tsallis entropy. It was shown that this entropy is relevant for a special class of non-Markovian random walks. In this work we generalize these walks to a much wider class of stochastic systems that can be characterized as `aging' systems. These are systems whose transition rates between states are path- and time-dependent. We show that for particular aging walks $S_{c,d}$ is again the correct extensive entropy. Before the central part of the paper we review the concept of $(c,d)$-entropy in a self-contained way.

cond-mat.stat-mech↗

arXiv2013

Statistical detection of systematic election irregularities

Democratic societies are built around the principle of free and fair elections, that each citizen's vote should count equal. National elections can be regarded as large-scale social experiments, where people are grouped into usually large numbers of electoral districts and vote according to their preferences. The large number of samples implies certain statistical consequences for the polling results which can be used to identify election irregularities. Using a suitable data collapse, we find that vote distributions of elections with alleged fraud show a kurtosis of hundred times more than normal elections on certain levels of data aggregation. As an example we show that reported irregularities in recent Russian elections are indeed well explained by systematic ballot stuffing and develop a parametric model quantifying to which extent fraudulent mechanisms are present. We show that if specific statistical properties are present in an election, the results do not represent the will of the people. We formulate a parametric test detecting these statistical properties in election results. Remarkably, this technique produces similar outcomes irrespective of the data resolution and thus allows for cross-country comparisons.

physics.soc-ph↗

arXiv2013

Reconstructing protein binding patterns from ChIP time-series

Motivation Gene transcription requires the orchestrated binding of various proteins to the promoter of a gene. The binding times and binding order of proteins allow to draw conclusions about the proteins' exact function in the recruitment process. Time-resolved ChIP experiments are being used to analyze the order of protein binding for these processes. However, these ChIP signals do not represent the exact protein binding patterns. Results We show that for promoter complexes that follow sequential recruitment dynamics the ChIP signal can be understood as a convoluted signal and propose the application of deconvolution methods to recover the protein binding patterns from experimental ChIP time-series. We analyze the suitability of four deconvolution methods: two non-blind deconvolution methods, Wiener deconvolution and Lucy-Richardson deconvolution, and two blind deconvolution methods, blind Lucy-Richardson deconvolution and binary blind deconvolution. We apply these methods to infer the protein binding pattern from ChIP time-series for the pS2 gene.

q-bio.MN↗

arXiv2013

The Transformation-Groupoid Structure of the q-Gaussian Family

The q-Gaussian function emerges naturally in various applications of statistical mechanics of non-ergodic and complex systems. In particular it was shown that in the theory of binary processes with correlations, the q-Gaussian can appear as a limiting distribution. Further, there exist several problems and situations where, depending on procedural or algorithmic details of data-processing, q-Gaussian distributions may yield distinct values of q, where one value is larger, the other smaller than one. To relate such pairs of q-Gaussians it would be convenient to map such distributions onto one another, ideally in a way, that any value of q can be mapped uniquely to any other value q'. So far a (duality) map from q -> q'=(7-5q)/(5-3q) was found, mapping q from the interval q\in [-\infty, 1] -> q'\in [1, 5/3]. Here we complete the theory of transformations of q-Gaussians by deriving a general map γ_{qq'}, that transforms normalizable q-Gaussian distributions onto one another for which q and q' are in the range of [1,3). By combining this with the previous result, a mapping from any value of q \in [-\infty,3) is possible to any other value q'\in [-\infty,3). We show that the action of γ_{qq'} on the set of q-Gaussian distributions is a transformation groupoid.

math-ph↗

arXiv2012

Generalized entropies and logarithms and their duality relations

For statistical systems that violate one of the four Shannon-Khinchin axioms, entropy takes a more general form than the Boltzmann-Gibbs entropy. The framework of superstatistics allows one to formulate a maximum entropy principle with these generalized entropies, making them useful for understanding distribution functions of non-Markovian or non-ergodic complex systems. For such systems where the composability axiom is violated there exist only two ways to implement the maximum entropy principle, one using escort probabilities, the other not. The two ways are connected through a duality. Here we show that this duality fixes a unique escort probability, which allows us to derive a complete theory of the generalized logarithms that naturally arise from the violation of this axiom. We then show how the functional forms of these generalized logarithms are related to the asymptotic scaling behavior of the entropy.

physics.class-ph↗