Characterization of the probability and information entropy of a process with an exponentially increasing sample space and its application to the Broad Money Supply

Abstract

There is a random variable (X) with a determined outcome (i.e., X = x0), p(x0) = 1. Consider x0 to have a discrete uniform distribution over the integer interval [1, s], where the size of the sample space (s) = 1, in the initial state, such that p(x0) = 1. What is the probability of x0 and the associated information entropy (H), as s increases exponentially? If the sample space expansion occurs at an exponential rate (rate constant = lambda) with time (t) and applying time scaling, such that T = lambda x t, gives: p(x0|T)=exp(-T) and H(T)=T. The characterization has also been extended to include exponential expansion by means of simultaneous, independent processes, as well as the more general multi-exponential case. The methodology was applied to the expansion of the broad money supply of US$ over the period 2001-2019, as a real-world example. At any given time, the information entropy is related to the rate at which the sample space is expanding. In the context of the expansion of the broad money supply, the information entropy could be considered to be related to the "velocity" of the expansion of the money supply.