arXiv subjects

Keenan J. A. Down

Publications and source records attributed to Keenan J. A. Down.

4 recordsLinked to original sources

arXiv2026

The mathematical landscape of partial information decomposition: A comprehensive review of properties and measures

Partial Information Decomposition (PID) has become one of the most prominent information-theoretic frameworks for describing the structure and quality of information in complex systems. Despite its widespread utility, there exists no unique solution constraining precisely how a PID should be constructed, leading to a multiverse of different formalisms with different mathematical commitments. In this work, we provide a comprehensive overview of the mathematical landscape of PID. By integrating existing PID measures into a common language, we systematically examine all major approaches to the PID framework that have emerged so far, determining for each measure whether or not each known property holds. In addition, we derive a web of all known theorems mapping the relationships and incompatibilities between these properties, before also revealing some novel interdependency results. In doing so, we chart a brief history of the framework, promote a unified perspective for its discussions, and offer a path towards both theoretical refinement and informed empirical applications for the future of this powerful method.

cs.IT↗

arXiv2025

A Logarithmic Decomposition and a Signed Measure Space for Entropy

The Shannon entropy of a random variable has much behaviour analogous to a signed measure. Previous work has explored this connection by defining a signed measure on abstract sets, which are taken to represent the information that different random variables contain. This construction is sufficient to derive many measure-theoretical counterparts to information quantities such as the mutual information (the intersection of sets), the joint entropy (the union of sets), and the conditional entropy (the difference of sets). Here we provide concrete characterisations of these abstract sets and a corresponding signed measure by extending the approach used by Yeung to all possible outcomes in an outcome space $Ω$, and in doing so we demonstrate that there exists a much finer decomposition with intuitive properties which we call the logarithmic decomposition (LD). We show that this signed measure space has the useful property that its logarithmic atoms are easily characterised with negative or positive entropy, depending only on their structure, while also being consistent with Yeung's I-measure. We present the usability of our approach by re-examining the Gács-Körner common information and minimally sufficient statistics from this new geometric perspective and characterising it in terms of our logarithmic atoms -- a property we call logarithmic decomposability. We present possible extensions of this construction to continuous probability distributions before discussing implications for quality-led information theory. As a motivating example, we apply our new decomposition to the Dyadic and Triadic systems of James and Crutchfield and show that, in contrast to the I-measure alone, our decomposition is able to qualitatively distinguish between them.

cs.IT↗

arXiv2024

Algebraic Representations of Entropy and Fixed-Parity Information Quantities

Many information-theoretic quantities have corresponding representations in terms of sets. The prevailing signed measure space for characterising entropy, the $I$-measure of Yeung, is occasionally unable to discern between qualitatively distinct systems. In previous work, we presented a refinement of this signed measure space and demonstrated its capability to represent many quantities, which we called logarithmically decomposable quantities. In the present work we demonstrate that this framework has natural algebraic behaviour which can be expressed in terms of ideals (characterised here as upper-sets), and we show that this behaviour allows us to make various counting arguments and characterise many fixed-parity information quantity expressions. As an application, we give an algebraic proof that the only completely synergistic system of three finite variables $X$, $Y$ and $Z = f(X,Y)$ is the XOR gate.

cs.IT↗

arXiv2023

A Logarithmic Decomposition for Information

The Shannon entropy of a random variable $X$ has much behaviour analogous to a signed measure. Previous work has concretized this connection by defining a signed measure $μ$ on an abstract information space $\tilde{X}$, which is taken to represent the information that $X$ contains. This construction is sufficient to derive many measure-theoretical counterparts to information quantities such as the mutual information $I(X; Y) = μ(\tilde{X} \cap \tilde{Y})$, the joint entropy $H(X,Y) = μ(\tilde{X} \cup \tilde{Y})$, and the conditional entropy $H(X|Y) = μ(\tilde{X}\, \setminus \, \tilde{Y})$. We demonstrate that there exists a much finer decomposition with intuitive properties which we call the logarithmic decomposition (LD). We show that this signed measure space has the useful property that its logarithmic atoms are easily characterised with negative or positive entropy, while also being coherent with Yeung's $I$-measure. We present the usability of our approach by re-examining the Gács-Körner common information from this new geometric perspective and characterising it in terms of our logarithmic atoms. We then highlight that our geometric refinement can account for an entire class of information quantities, which we call logarithmically decomposable quantities.

cs.IT↗