arXiv subjects

Venkat Anantharam

Publications and source records attributed to Venkat Anantharam.

At least 37 records · Page 2Linked to original sources

arXiv2019

Optimal Chernoff and Hoeffding Bounds for Finite State Markov Chains

This paper develops an optimal Chernoff type bound for the probabilities of large deviations of sums $\sum_{k=1}^n f (X_k)$ where $f$ is a real-valued function and $(X_k)_{k \in \mathbb{Z}_{\ge 0}}$ is a finite state Markov chain with an arbitrary initial distribution and an irreducible transition probability matrix satisfying a mild assumption on its positivity pattern, related to the function $f$ being considered. The novelty lies in this being a non-asymptotic finite sample bound. Further, our bound is optimal in the large deviations sense, attaining a constant prefactor and an exponential decay with the optimal large deviations rate. Moreover, through a Pinsker type inequality and a Hoeffding type lemma, we are able to loosen up our Chernoff type bound to a Hoeffding type bound and reveal the sub-Gaussian nature of the sums. Finally, under the same mild assumption on the positivity pattern of the transition probability matrix, we prove a uniform multiplicative ergodic theorem for the exponential family of tilted transition probability matrices corresponding to $f$.

math.PR↗

arXiv2019

Pairwise Near-maximal Grand Coupling of Brownian Motions

The well-known reflection coupling gives a maximal coupling of two one-dimensional Brownian motions with different starting points. Nevertheless, the reflection coupling does not generalize to more than two Brownian motions. In this paper, we construct a coupling of all Brownian motions with all possible starting points (i.e., a grand coupling), such that the coupling for any pair of the coupled processes is close to being maximal, that is, the distribution of the coupling time of the pair approaches that of the maximal coupling as the time tends to $0$ or $\infty$, and the coupling time of the pair is always within a multiplicative factor $2e^{2}$ from the maximal one. We also show that a grand coupling that is pairwise exactly maximal does not exist.

math.PR↗

arXiv2019

Pairwise Multi-marginal Optimal Transport and Embedding for Earth Mover's Distance

We investigate the problem of pairwise multi-marginal optimal transport, that is, given a collection of probability distributions $\{P_α\}$ on a Polish space $\mathcal{X}$, to find a coupling $\{X_α\}$, $X_α\sim P_α$, such that $\mathbf{E}[c(X_α,X_β)]\le r\inf_{X\sim P_α,Y\sim P_β}\mathbf{E}[c(X,Y)]$ for all $α,β$, where $c$ is a cost function and $r\ge1$. In other words, every pair $(X_α,X_β)$ has an expected cost at most a factor of $r$ from its lowest possible value. This can be regarded as a locality sensitive hash function for probability distributions, and has applications such as robust and distributed computation of transport plans. It can also be considered as a bi-Lipschitz embedding of the collection of probability distributions into the space of random variables taking values on $\mathcal{X}$. For $c(x,y)=\Vert x-y\Vert_2^q$ on $\mathbb{R}^n$, where $q>0$, we show that a finite $r$ is attainable if and only if either $n=1$ or $0<q<1$. As $n\to\infty$, the growth rate of the smallest possible $r$ is exactly $Θ(n^{q/2})$ if $0<q<1$. Hence, the metric space of probability distributions on $\mathbb{R}^n$ with finite $q$-th absolute moments, $0<q<1$, with the earth mover's distance (or 1-Wasserstein distance) with respect to the snowflake metric $c(x,y)=\Vert x-y\Vert_2^q$, is bi-Lipschitz embeddable into $L_1$ with distortion $O(n^{q/2})$. If we consider $c(x,y)=\Vert x-y\Vert_2$ (i.e., $q=1$) on the grid $[0..s]^n$ instead of $\mathbb{R}^n$, then $r=O(\sqrt{n}\log s)$ is attainable, which implies the embeddability of the space of probability distributions on $[0..s]^n$ into $L_1$ with distortion $O(\sqrt{n}\log s)$, and improves upon the $O(n\log s)$ result by Indyk and Thaper. The case of the discrete metric cost $c(x,y)=\mathbf{1}\{x\neq y\}$ and more general metric and ultrametric costs are also investigated.

math.PR↗

arXiv2019

Universal Lossless Compression of Graphical Data

Graphical data is comprised of a graph with marks on its edges and vertices. The mark indicates the value of some attribute associated to the respective edge or vertex. Examples of such data arise in social networks, molecular and systems biology, and web graphs, as well as in several other application areas. Our goal is to design schemes that can efficiently compress such graphical data without making assumptions about its stochastic properties. Namely, we wish to develop a universal compression algorithm for graphical data sources. To formalize this goal, we employ the framework of local weak convergence, also called the objective method, which provides a technique to think of a marked graph as a kind of stationary stochastic processes, stationary with respect to movement between vertices of the graph. In recent work, we have generalized a notion of entropy for unmarked graphs in this framework, due to Bordenave and Caputo, to the case of marked graphs. We use this notion to evaluate the efficiency of a compression scheme. The lossless compression scheme we propose in this paper is then proved to be universally optimal in a precise technical sense. It is also capable of performing local data queries in the compressed form.

math.PR↗

arXiv2019

A Notion of Entropy for Stochastic Processes on Marked Rooted Graphs

In this document, we introduce a notion of entropy for stochastic processes on marked rooted graphs. For this, we employ the framework of local weak limit theory for sparse marked graphs, also known as the objective method, due to Benjamini, Schramm, Aldous, Steele and Lyons. Our contribution is a generalization of the notion of entropy introduced by Bordenave and Caputo to graphs which carry marks on their vertices and edges. The theory of time series is the engine driving an enormous range of applications in areas such as control theory, communications, information theory and signal processing. It is to be expected that a theory of stationary stochastic processes indexed by combinatorial structures, in particular graphs, would eventually have a similarly wide-ranging impact.

cs.IT↗

arXiv2019

A Unified Framework for One-shot Achievability via the Poisson Matching Lemma

We introduce a fundamental lemma called the Poisson matching lemma, and apply it to prove one-shot achievability results for various settings, namely channels with state information at the encoder, lossy source coding with side information at the decoder, joint source-channel coding, broadcast channels, distributed lossy source coding, multiple access channels, channel resolvability and wiretap channels. Our one-shot bounds improve upon the best known one-shot bounds in most of the aforementioned settings (except multiple access channels, channel resolvability and wiretap channels, where we recover bounds comparable to the best known bounds), with shorter proofs in some settings even when compared to the conventional asymptotic approach using typicality. The Poisson matching lemma replaces both the packing and covering lemmas, greatly simplifying the error analysis. This paper extends the work of Li and El Gamal on Poisson functional representation, which mainly considered variable-length source coding settings, whereas this paper studies fixed-length settings, and is not limited to source coding, showing that the Poisson functional representation is a viable alternative to typicality for most problems in network information theory.

cs.IT↗

arXiv2018

Optimal Resource Allocation over Networks via Lottery-Based Mechanisms

We show that, in a resource allocation problem, the ex ante aggregate utility of players with cumulative-prospect-theoretic preferences can be increased over deterministic allocations by implementing lotteries. We formulate an optimization problem, called the system problem, to find the optimal lottery allocation. The system problem exhibits a two-layer structure comprised of a permutation profile and optimal allocations given the permutation profile. For any fixed permutation profile, we provide a market-based mechanism to find the optimal allocations and prove the existence of equilibrium prices. We show that the system problem has a duality gap, in general, and that the primal problem is NP-hard. We then consider a relaxation of the system problem and derive some qualitative features of the optimal lottery structure.

econ.TH↗

arXiv2018

One-Shot Variable-Length Secret Key Agreement Approaching Mutual Information

This paper studies an information-theoretic one-shot variable-length secret key agreement problem with public discussion. Let $X$ and $Y$ be jointly distributed random variables, each taking values in some measurable space. Alice and Bob observe $X$ and $Y$ respectively, can communicate interactively through a public noiseless channel, and want to agree on a key length and a key that is approximately uniformly distributed over all bit sequences with the agreed key length. The public discussion is observed by an eavesdropper, Eve. The key should be approximately independent of the public discussion, conditional on the key length. We show that the optimal expected key length is close to the mutual information $I(X;Y)$ within a logarithmic gap. Moreover, an upper bound and a lower bound on the optimal expected key length can be written down in terms of $I(X;Y)$ only. This means that the optimal one-shot performance is always within a small gap of the optimal asymptotic performance regardless of the distribution of the pair $(X,Y)$. This one-shot result may find applications in situations where the components of an i.i.d. pair source $(X^{n},Y^{n})$ are observed sequentially and the key is output bit by bit with small delay, or in situations where the random source is not an i.i.d. or ergodic process.

cs.IT↗

arXiv2017

On the Geometry of Nash and Correlated Equilibria with Cumulative Prospect Theoretic Preferences

It is known that the set of all correlated equilibria of an n-player non-cooperative game is a convex polytope and includes all the Nash equilibria. Further, the Nash equilibria all lie on the boundary of this polytope. We study the geometry of both these equilibrium notions when the players have cumulative prospect theoretic (CPT) preferences. The set of CPT correlated equilibria includes all the CPT Nash equilibria but it need not be a convex polytope. We show that it can, in fact, be disconnected. However, all the CPT Nash equilibria continue to lie on its boundary. We also characterize the sets of CPT correlated equilibria and CPT Nash equilibria for all 2x2 games.

cs.GT↗

arXiv2017

Load Balancing in Hypergraphs

Consider a simple locally finite hypergraph on a countable vertex set, where each edge represents one unit of load which should be distributed among the vertices defining the edge. An allocation of load is called balanced if load cannot be moved from a vertex to another that is carrying less load. We analyze the properties of balanced allocations of load. We extend the concept of balancedness from finite hypergraphs to their local weak limits in the sense of Benjamini and Schramm (2001) and Aldous and Steele (2004). To do this, we define a notion of unimodularity for hypergraphs which could be considered an extension of unimodularity in graphs. We give a variational formula for the balanced load distribution and, in particular, we characterize it in the special case of unimodular hypergraph Galton Watson processes. Moreover, we prove the convergence of the maximum load under some conditions. Our work is an extension to hypergraphs of Anantharam and Salez (2016), which considered load balancing in graphs, and is aimed at more comprehensively resolving conjectures of Hajek (1990).

math.PR↗

arXiv2017

A combinatorial proof of a formula of Biane and Chapuy

Let $G$ be a simple strongly connected weighted directed graph. Let $\mathcal{G}$ denote the spanning tree graph of $G$. That is, the vertices of $\mathcal{G}$ consist of the directed rooted spanning trees on $G$, and the edges of $\mathcal{G}$ consist of pairs of trees $(t_i, t_j)$ such that $t_j$ can be obtained from $t_i$ by adding the edge from the root of $t_i$ to the root of $t_j$ and deleting the outgoing edge from the root of $t_j$. A formula for the ratio of the sum of the weights of the directed rooted spanning trees on $\mathcal{G}$ to the sum of the weights of the directed rooted spanning trees on $G$ was recently given by Biane and Chapuy. We provide an alternative proof of this formula, which is both simple and combinatorial. The proof involves working with the stochastic zeta function of an irreducible Markov chain. By generalizing the stochastic zeta function we also recover the general result of Biane and Chapuy which gives a formula for the determinant of the Schrödinger matrix on $\mathcal{G}$ corresponding to a given Schrödinger matrix on $G$, in terms of the minors of the latter matrix.

math.CO↗

arXiv2017

Intrinsic entropies of log-concave distributions

The entropy of a random variable is well-known to equal the exponential growth rate of the volumes of its typical sets. In this paper, we show that for any log-concave random variable $X$, the sequence of the $\lfloor nθ\rfloor^{\text{th}}$ intrinsic volumes of the typical sets of $X$ in dimensions $n \geq 1$ grows exponentially with a well-defined rate. We denote this rate by $h_X(θ)$, and call it the $θ^{\text{th}}$ intrinsic entropy of $X$. We show that $h_X(θ)$ is a continuous function of $θ$ over the range $[0,1]$, thereby providing a smooth interpolation between the values 0 and $h(X)$ at the endpoints 0 and 1, respectively.

cs.IT↗

arXiv2017

A Variational Characterization of Rényi Divergences

Atar, Chowdhary and Dupuis have recently exhibited a variational formula for exponential integrals of bounded measurable functions in terms of Rényi divergences. We develop a variational characterization of the Rényi divergences between two probability distributions on a measurable sace in terms of relative entropies. When combined with the elementary variational formula for exponential integrals of bounded measurable functions in terms of relative entropy, this yields the variational formula of Atar, Chowdhary and Dupuis as a corollary. We also develop an analogous variational characterization of the Rényi divergence rates between two stationary finite state Markov chains in terms of relative entropy rates. When combined with Varadhan's variational characterization of the spectral radius of square matrices with nonnegative entries in terms of relative entropy, this yields an analog of the variational formula of Atar, Chowdary and Dupuis in the framework of finite state Markov chains.

cs.IT↗

arXiv2016

On Non-Interactive Simulation of Joint Distributions

We consider the following non-interactive simulation problem: Alice and Bob observe sequences $X^n$ and $Y^n$ respectively where $\{(X_i, Y_i)\}_{i=1}^n$ are drawn i.i.d. from $P(x,y),$ and they output $U$ and $V$ respectively which is required to have a joint law that is close in total variation to a specified $Q(u,v).$ It is known that the maximal correlation of $U$ and $V$ must necessarily be no bigger than that of $X$ and $Y$ if this is to be possible. Our main contribution is to bring hypercontractivity to bear as a tool on this problem. In particular, we show that if $P(x,y)$ is the doubly symmetric binary source, then hypercontractivity provides stronger impossibility results than maximal correlation. Finally, we extend these tools to provide impossibility results for the $k$-agent version of this problem.

cs.IT↗

arXiv2016

The densest subgraph problem in sparse random graphs

We determine the asymptotic behavior of the maximum subgraph density of large random graphs with a prescribed degree sequence. The result applies in particular to the Erdős-Rényi model, where it settles a conjecture of Hajek [IEEE Trans. Inform. Theory 36 (1990) 1398-1414]. Our proof consists in extending the notion of balanced loads from finite graphs to their local weak limits, using unimodularity. This is a new illustration of the objective method described by Aldous and Steele [In Probability on Discrete Structures (2004) 1-72 Springer].

math.PR↗

arXiv2015

The two-unicast problem

We consider the communication capacity of wireline networks for a two-unicast traffic pattern. The network has two sources and two destinations with each source communicating a message to its own destination, subject to the capacity constraints on the directed edges of the network. We propose a simple outer bound for the problem that we call the Generalized Network Sharing (GNS) bound. We show this bound is the tightest edge-cut bound for two-unicast networks and is tight in several bottleneck cases, though it is not tight in general. We also show that the problem of computing the GNS bound is NP-complete. Finally, we show that despite its seeming simplicity, the two-unicast problem is as hard as the most general network coding problem. As a consequence, linear coding is insufficient to achieve capacity for general two-unicast networks, and non-Shannon inequalities are necessary for characterizing capacity of general two-unicast networks.

cs.IT↗

arXiv2015

A Geometric Analysis of the AWGN channel with a $(σ, ρ)$-Power Constraint

In this paper, we consider the AWGN channel with a power constraint called the $(σ, ρ)$-power constraint, which is motivated by energy harvesting communication systems. Given a codeword, the constraint imposes a limit of $σ+ k ρ$ on the total power of any $k\geq 1$ consecutive transmitted symbols. Such a channel has infinite memory and evaluating its exact capacity is a difficult task. Consequently, we establish an $n$-letter capacity expression and seek bounds for the same. We obtain a lower bound on capacity by considering the volume of ${\cal S}_n(σ, ρ) \subseteq \mathbb{R}^n$, which is the set of all length $n$ sequences satisfying the $(σ, ρ)$-power constraints. For a noise power of $ν$, we obtain an upper bound on capacity by considering the volume of ${\cal S}_n(σ, ρ) \oplus B_n(\sqrt{nν})$, which is the Minkowski sum of ${\cal S}_n(σ, ρ)$ and the $n$-dimensional Euclidean ball of radius $\sqrt{nν}$. We analyze this bound using a result from convex geometry known as Steiner's formula, which gives the volume of this Minkowski sum in terms of the intrinsic volumes of ${\cal S}_n(σ, ρ)$. We show that as the dimension $n$ increases, the logarithm of the sequence of intrinsic volumes of $\{{\cal S}_n(σ, ρ)\}$ converges to a limit function under an appropriate scaling. The upper bound on capacity is then expressed in terms of this limit function. We derive the asymptotic capacity in the low and high noise regime for the $(σ, ρ)$-power constrained AWGN channel, with strengthened results for the special case of $σ= 0$, which is the amplitude constrained AWGN channel.

cs.IT↗

arXiv2014

The Boolean Model in the Shannon Regime: Three Thresholds and Related Asymptotics

Consider a family of Boolean models, indexed by integers $n \ge 1$, where the $n$-th model features a Poisson point process in ${\mathbb{R}}^n$ of intensity $e^{n ρ_n}$ with $ρ_n \to ρ$ as $n \to \infty$, and balls of independent and identically distributed radii distributed like $\bar X_n \sqrt{n}$, with $\bar X_n$ satisfying a large deviations principle. It is shown that there exist three deterministic thresholds: $τ_d$ the degree threshold; $τ_p$ the percolation threshold; and $τ_v$ the volume fraction threshold; such that asymptotically as $n$ tends to infinity, in a sense made precise in the paper: (i) for $ρ< τ_d$, almost every point is isolated, namely its ball intersects no other ball; (ii) for $τ_d< ρ< τ_p$, almost every ball intersects an infinite number of balls and nevertheless there is no percolation; (iii) for $τ_p< ρ< τ_v$, the volume fraction is 0 and nevertheless percolation occurs; (iv) for $τ_d< ρ< τ_v$, almost every ball intersects an infinite number of balls and nevertheless the volume fraction is 0; (v) for $ρ> τ_v$, the whole space covered. The analysis of this asymptotic regime is motivated by related problems in information theory, and may be of interest in other applications of stochastic geometry.

math.PR↗