Measuring the Redundancy of Information from a Source Failure Perspective

Abstract

In this paper, we define a new measure of the redundancy of information from a fault tolerance perspective. The partial information decomposition (PID) emerged last decade as a framework for decomposing the multi-source mutual information $I(T;X_1, ..., X_n)$ into atoms of redundant, synergistic, and unique information. It built upon the notion of redundancy/synergy from McGill's interaction information (McGill 1954). Separately, the redundancy of system components has served as a principle of fault tolerant engineering, for sensing, routing, and control applications. Here, redundancy is understood as the level of duplication necessary for the fault tolerant performance of a system. With these two perspectives in mind, we propose a new PID-based measure of redundancy $I_{\text{ft}}$, based upon the presupposition that redundant information is robust to individual source failures. We demonstrate that this new measure satisfies the common PID axioms from (Williams 2010). In order to do so, we establish an order-reversing correspondence between collections of source-fallible instantiations of a system, on the one hand, and the PID lattice from (Williams 2010), on the other.