Searcharxiv⌕ Search

arXiv subjects

Johannes Rauh

Publications and source records attributed to Johannes Rauh.

At least 19 recordsLinked to original sources
arXiv2023

Continuity and Additivity Properties of Information Decompositions

Information decompositions quantify how the Shannon information about a given random variable is distributed among several other random variables. Various requirements have been proposed that such a decomposition should satisfy, leading to different candidate solutions. Curiously, however, only two of the original requirements that determined the Shannon information have been considered, namely monotonicity and normalization. Two other important properties, continuity and additivity, have not been considered. In this contribution, we focus on the mutual information of two finite variables $Y,Z$ about a third finite variable $S$ and check which of the decompositions satisfy these two properties. While most of them satisfy continuity, only one of them is both continuous and additive.

cs.IT↗
arXiv2022

Conditional independence ideals with hidden variables

We study a class of determinantal ideals that are related to conditional independence (CI) statements with hidden variables. Such CI statements correspond to determinantal conditions on a matrix whose entries are probabilities of events involving the observed random variables. We focus on an example that generalizes the CI ideals of the intersection axiom. In this example, the minimal primes are again determinantal ideals, which is not true in general.

math.AC↗
arXiv2022

Modelling volume-outcome relationships in health care

Despite the ongoing strong interest in associations between quality of care and the volume of health care providers, a unified statistical framework for analyzing them is missing, and many studies suffer from poor statistical modelling choices. We propose a flexible, additive mixed model for studying volume-outcome associations in health care that takes into account individual patient characteristics as well as provider-specific effects through a multi-level approach. More specifically, we treat volume as a continuous variable, and its effect on the considered outcome is modelled as a smooth function. We take account of different case-mixes by including patient-specific risk factors and of clustering on the provider level through random intercepts. This strategy enables us to extract a smooth volume effect as well as volume-independent provider effects. These two quantities can be compared directly in terms of their magnitude, which gives insight into the sources of variability of quality of care. Based on a causal DAG, we derive conditions under which the volume-effect can be interpreted as a causal effect. The paper provides confidence sets for each of the estimated quantities relying on joint estimation of all effects and parameters. Our approach is illustrated through simulation studies and an application to German health care data. Keywords: health care quality measurement, volume-outcome analysis, minimum provider volume, additive regression models, random intercept

stat.ME↗
arXiv2021

Properties of Unique Information

We study the measure of unique information $UI(T:X\setminus Y)$ defined by Bertschinger et al. (2014) within the framework of information decompositions. We study uniqueness and support of the solutions to the optimization problem underlying the definition of $UI$. We identify sufficient conditions for non-uniqueness of solutions with full support in terms of conditional independence constraints and in terms of the cardinalities of $T$, $X$ and $Y$. Our results are based on a reformulation of the first order conditions on the objective function as rank constraints on a matrix of conditional probabilities. These results help to speed up the computation of $UI(T:X\setminus Y)$, most notably when $T$ is binary. In the case that all variables are binary, we obtain a complete picture of where the optimizing probability distributions lie.

cs.IT↗
arXiv2019

Unique Informations and Deficiencies

Given two channels that convey information about the same random variable, we introduce two measures of the unique information of one channel with respect to the other. The two quantities are based on the notion of generalized weighted Le Cam deficiencies and differ on whether one channel can approximate the other by a randomization at either its input or output. We relate the proposed quantities to an existing measure of unique information which we call the minimum-synergy unique information. We give an operational interpretation of the latter in terms of an upper bound on the one-way secret key rate and discuss the role of the unique informations in the context of nonnegative mutual information decompositions into unique, redundant and synergistic components.

cs.IT↗
arXiv2019

Unique Information and Secret Key Decompositions

The unique information ($UI$) is an information measure that quantifies a deviation from the Blackwell order. We have recently shown that this quantity is an upper bound on the one-way secret key rate. In this paper, we prove a triangle inequality for the $UI$, which implies that the $UI$ is never greater than one of the best known upper bounds on the two-way secret key rate. We conjecture that the $UI$ lower bounds the two-way rate and discuss implications of the conjecture.

cs.IT↗
arXiv2018

Computing the Unique Information

Given a pair of predictor variables and a response variable, how much information do the predictors have about the response, and how is this information distributed between unique, redundant, and synergistic components? Recent work has proposed to quantify the unique component of the decomposition as the minimum value of the conditional mutual information over a constrained set of information channels. We present an efficient iterative divergence minimization algorithm to solve this optimization problem with convergence guarantees and evaluate its performance against other techniques.

cs.IT↗
arXiv2018

Cores, shell indices and the degeneracy of a graph limit

The $k$-core of a graph is the maximal subgraph in which every node has degree at least~$k$, the shell index of a node is the largest $k$ such that the $k$-core contains the node, and the degeneracy of a graph is the largest shell index of any node. After a suitable normalization, these three concepts generalize to limits of dense graphs (also called graphons). In particular, the degeneracy is continuous with respect to the cut metric.

math.CO↗
arXiv2018

Approximating faces of marginal polytopes in discrete hierarchical models

The existence of the maximum likelihood estimate in hierarchical loglinear models is crucial to the reliability of inference for this model. Determining whether the estimate exists is equivalent to finding whether the sufficient statistics vector $t$ belongs to the boundary of the marginal polytope of the model. The dimension of the smallest face $F_t$ containing $t$ determines the dimension of the reduced model which should be considered for correct inference. For higher-dimensional problems, it is not possible to compute $F_{t}$ exactly. Massam and Wang (2015) found an outer approximation to $F_t$ using a collection of sub-models of the original model. This paper refines the methodology to find an outer approximation and devises a new methodology to find an inner approximation. The inner approximation is given not in terms of a face of the marginal polytope, but in terms of a subset of the vertices of $F_t$. Knowing $F_t$ exactly indicates which cell probabilities have maximum likelihood estimates equal to $0$. When $F_t$ cannot be obtained exactly, we can use, first, the outer approximation $F_2$ to reduce the dimension of the problem and, then, the inner approximation $F_1$ to obtain correct estimates of cell probabilities corresponding to elements of $F_1$ and improve the estimates of the remaining probabilities corresponding to elements in $F_2\setminus F_1$. Using both real-world and simulated data, we illustrate our results, and show that our methodology scales to high dimensions.

math.ST↗
arXiv2017

On extractable shared information

We consider the problem of quantifying the information shared by a pair of random variables $X_{1},X_{2}$ about another variable $S$. We propose a new measure of shared information, called extractable shared information, that is left monotonic; that is, the information shared about $S$ is bounded from below by the information shared about $f(S)$ for any function $f$. We show that our measure leads to a new nonnegative decomposition of the mutual information $I(S;X_1X_2)$ into shared, complementary and unique components. We study properties of this decomposition and show that a left monotonic shared information is not compatible with a Blackwell interpretation of unique information. We also discuss whether it is possible to have a decomposition in which both shared and unique information are left monotonic.

cs.IT↗
arXiv2017

Coarse-graining and the Blackwell order

Suppose we have a pair of information channels, $κ_{1},κ_{2}$, with a common input. The Blackwell order is a partial order over channels that compares $κ_{1}$ and $κ_{2}$ by the maximal expected utility an agent can obtain when decisions are based on the channel outputs. Equivalently, $κ_{1}$ is said to be Blackwell-inferior to $κ_{2}$ if and only if $κ_{1}$ can be constructed by garbling the output of $κ_{2}$. A related partial order stipulates that $κ_{2}$ is more capable than $κ_{1}$ if the mutual information between the input and output is larger for $κ_{2}$ than for $κ_{1}$ for any distribution over inputs. A Blackwell-inferior channel is necessarily less capable. However, examples are known where $κ_{1}$ is less capable than $κ_{2}$ but not Blackwell-inferior. We show that this may even happen when $κ_{1}$ is constructed by coarse-graining the inputs of $κ_{2}$. Such a coarse-graining is a special kind of "pre-garbling" of the channel inputs. This example directly establishes that the expected value of the shared utility function for the coarse-grained channel is larger than it is for the non-coarse-grained channel. This contradicts the intuition that coarse-graining can only destroy information and lead to inferior channels. We also discuss our results in the context of information decompositions.

cs.IT↗
arXiv2017

Secret Sharing and Shared Information

Secret sharing is a cryptographic discipline in which the goal is to distribute information about a secret over a set of participants in such a way that only specific authorized combinations of participants together can reconstruct the secret. Thus, secret sharing schemes are systems of variables in which it is very clearly specified which subsets have information about the secret. As such, they provide perfect model systems for information decompositions. However, following this intuition too far leads to an information decomposition with negative partial information terms, which are difficult to interpret. One possible explanation is that the partial information lattice proposed by Williams and Beer is incomplete and has to be extended to incorporate terms corresponding to higher order redundancy. These results put bounds on information decompositions that follow the partial information framework, and they hint at where the partial information lattice needs to be improved.

cs.IT↗
arXiv2017

Prime splittings of Determinantal Ideals

We consider determinantal ideals, where the generating minors are encoded in a hypergraph. We study when the generating minors form a Gröbner basis. In this case, the ideal is radical, and we can describe algebraic and numerical invariants of these ideals in terms of combinatorial data of their hypergraphs, such as the clique decomposition. In particular, we can construct a minimal free resolution as a tensor product of the minimal free resolution of their cliques. For several classes of hypergraphs we find a combinatorial description of the minimal primes in terms of a prime splitting. That is, we write the determinantal ideal as a sum of smaller determinantal ideals such that each minimal prime is a sum of minimal primes of the summands.

math.AC↗
arXiv2017

Algebraic Aspects of Conditional Independence and Graphical Models

This chapter of the forthcoming Handbook of Graphical Models contains an overview of basic theorems and techniques from algebraic geometry and how they can be applied to the study of conditional independence and graphical models. It also introduces binomial ideals and some ideas from real algebraic geometry. When random variables are discrete or Gaussian, tools from computational algebraic geometry can be used to understand implications between conditional independence statements. This is accomplished by computing primary decompositions of conditional independence ideals. As examples the chapter presents in detail the graphical model of a four cycle and the intersection axiom, a certain implication of conditional independence statements. Another important problem in the area is to determine all constraints on a graphical model, for example, equations determined by trek separation. The full set of equality constraints can be determined by computing the model's vanishing ideal. The chapter illustrates these techniques and ideas with examples from the literature and provides references for further reading.

math.ST↗
arXiv2017

Geometry of Policy Improvement

We investigate the geometry of optimal memoryless time independent decision making in relation to the amount of information that the acting agent has about the state of the system. We show that the expected long term reward, discounted or per time step, is maximized by policies that randomize among at most $k$ actions whenever at most $k$ world states are consistent with the agent's observation. Moreover, we show that the expected reward per time step can be studied in terms of the expected discounted reward. Our main tool is a geometric version of the policy improvement lemma, which identifies a polyhedral cone of policy changes in which the state value function increases for all states.

cs.AI↗
arXiv2017

Semigroups --- A Computational Approach

The question whether there exists an integral solution to the system of linear equations with non-negative constraints, $A\x = \b, \, \x \ge 0$, where $A \in \Z^{m\times n}$ and ${\mathbf b} \in \Z^m$, finds its applications in many areas, such as operation research, number theory and statistics. In order to solve this problem, we have to understand the semigroup generated by the columns of the matrix $A$ and the structure of the "holes" which are the difference between the semigroup generated by the columns of the matrix $A$ and its saturation. In this paper, we discuss the implementation of an algorithm by Hemmecke, Takemura, and Yoshida that computes the set of holes of a semigroup, % generated by the columns of $A$ and we discuss applications to problems in combinatorics. Moreover, we compute the set of holes for the common diagonal effect model, and we show that the $n$th linear ordering polytope has the integer-decomposition property for $n\leq 7$. The software is available at \url{http://ehrhart.math.fu-berlin.de/People/fkohl/HASE/}.

math.CO↗
arXiv2017

On the Number of Non-zero Elements of Joint Degree Vectors

Joint degree vectors give the number of edges between vertices of degree $i$ and degree $j$ for $1\le i\le j\le n-1$ in an $n$-vertex graph. We find lower and upper bounds for the maximum number of nonzero elements in a joint degree vector as a function of $n$. This provides an upper bound on the number of estimable parameters in the exponential random graph model with bidegree-distribution as its sufficient statistics.

math.CO↗