Towards a Theory of Pragmatic Information

Abstract

Standard information theory says nothing about how much meaning is conveyed by a message. We fill this gap with a quantitative definition of ``pragmatic information'', the amount of meaning in a message relevant to a particular decision, a definition that is close to, but not identical with previous definitions. Here, we posit that such a message updates a random variable, $\omega$, that informs the decision. The pragmatic information of a single message is then defined as the Kulback-Leibler divergence between the prior and posterior probabilities of $\omega$; the pragmatic information of a message ensemble is the expected value of the pragmatic information of the ensemble's component messages. Canonical standard information theory results justify these definitions, most notably that that the pragmatic information of a single message is the expected difference between the shortest binary encoding of $\omega$ under the {\it a priori} and {\it a posteriori} distributions. We also show that the average of the pragmatic values of individual messages, when sampled a large number of times from the ensemble, approaches its expected value.\\ Pragmatic information is non-negative and additive for independent decisions and ``pragmatically independent'' messages. Also, pragmatic information is the information analogue of free energy: just as free energy quantifies the part of a system's total energy available to do useful work, so pragmatic information quantifies the information actually used in making a decision.\\ We sketch 3 applications: the single play of a slot machine, a.k.a. a ``one armed bandit'', with an unknown payout probability; a characterization of the rate of biological evolution in the so-called ``quasi-species'' model; and a reformulation of the efficient market hypothesis of finance. We note the importance of the computational capacity of the receiver in each case.