arXiv subjects

David Sigtermans

Publications and source records attributed to David Sigtermans.

4 recordsLinked to original sources

arXiv2020

Is Information Theory Inherently a Theory of Causation?

Information theory gives rise to a novel method for causal skeleton discovery by expressing associations between variables as tensors. This tensor-based approach reduces the dimensionality of the data needed to test for conditional independence, e.g., for systems comprising three variables, the causal skeleton can be determined using pair-wise determined tensors. To arrive at this result, an additional information measure, path information, is proposed.

stat.ML↗

arXiv2020

A Partial Information Decomposition Based on Causal Tensors

We propose a partial information decomposition based on the newly introduced framework of causal tensors, i.e., multilinear stochastic maps that transform source data into destination data. This framework enables us to express an indirect association in terms of the constituting, direct associations. This is not possible when using average measures like mutual information or transfer entropy. From this, an intuitive definition of redundant and unique information arises. The proposed redundancy satisfies the three axioms stated by introduced by Williams and Beer. The symmetry and self-redundancy properties follow directly from our definition. The Data Processing Inequality ensures that the monotonicity axiom is satisfied. Additional, two other proposed axioms are satisfied: the identity property, and the left monotonicity axiom. Because causal tensors can describe both mutual information as transfer entropy, the proposed partial information decomposition applies to both measures. Results show that the decomposition closely resembles the decomposition of another approach that expresses associations in terms of mutual information a posteriori. It is furthermore demonstrated that negative contributions can arise when our assumptions about completeness of the data set, or what should be included as a source, are incorrect.

cs.IT↗

arXiv2020

Towards a Framework for Observational Causality From Time Series: When Shannon Meets Turing

We propose a novel tensor-based formalism for inferring causal structures from time series. An information theoretical analysis of transfer entropy, shows that transfer entropy results from transmission of information over a set of communication channels. Tensors are the mathematical equivalents of these multi-channel causal channels. A multi-channel causal channel is a generalization of a discrete memoryless channel. Investigation of a system comprising three variables shows that in our formalism, bivariate analysis suffices to differentiate between direct and indirect relations. For this to be true, we have to combine the output of multi-channel causal channels with the output of single-channel causal channels. We can understand this result when we consider the role of noise. Subsequent transmission of information over noisy channels can never result in less noisy transmission overall. This implies that a Data Processing Inequality exists for transfer entropy.

cs.IT↗

arXiv2019

Transfer Entropy: where Shannon meets Turing

Transfer entropy is capable of capturing nonlinear source-destination relations between multi-variate time series. It is a measure of association between source data that are transformed into destination data via a set of linear transformations between their probability mass functions. The resulting tensor formalism is used to show that in specific cases, e.g., in the case the system consists of three stochastic processes, bivariate analysis suffices to distinguish true relations from false relations. This allows us to determine the causal structure as far as encoded in the probability mass functions of noisy data. The tensor formalism was also used to derive the Data Processing Inequality for transfer entropy.

cs.IT↗