arXiv subjects

Vincent Y. F. Tan

Publications and source records attributed to Vincent Y. F. Tan.

At least 163 records · Page 9Linked to original sources

arXiv2018

Time-Division is Optimal for Covert Communication over Some Broadcast Channels

We consider a covert communication scenario where a transmitter wishes to communicate simultaneously to two legitimate receivers while ensuring that the communication is not detected by an adversary, the warden. The legitimate receivers and the adversary observe the transmission from the transmitter via a three-user discrete or Gaussian memoryless broadcast channel. We focus on the case where the "no-input" symbol is not redundant, i.e., the output distribution at the warden induced by the no-input symbol is not a mixture of the output distributions induced by other input symbols, so that the covert communication is governed by the square root law, i.e., at most $Θ(\sqrt{n})$ bits can be transmitted over $n$ channel uses. We show that for such a setting, a simple time-division strategy achieves the optimal throughputs for a non-trivial class of broadcast channels; this is not true for communicating over broadcast channels without the covert communication constraint. Our result implies that a code that uses two separate optimal point-to-point codes each designed for the constituent channels and each used for a fraction of the time is optimal in the sense that it achieves the best constants of the $\sqrt{n}$-scaling for the throughputs. Our proof strategy combines several elements in the network information theory literature, including concave envelope representations of the capacity regions of broadcast channels and El Gamal's outer bound for more capable broadcast channels.

cs.IT↗

arXiv2018

Refined Asymptotics for Rate-Distortion using Gaussian Codebooks for Arbitrary Sources

The rate-distortion saddle-point problem considered by Lapidoth (1997) consists in finding the minimum rate to compress an arbitrary ergodic source when one is constrained to use a random Gaussian codebook and minimum (Euclidean) distance encoding is employed. We extend Lapidoth's analysis in several directions in this paper. Firstly, we consider refined asymptotics. In particular, when the source is stationary and memoryless, we establish the second-order, moderate, and large deviation asymptotics of the problem. Secondly, by "random Gaussian codebook", Lapidoth referred to a collection of random codewords, each of which is drawn independently and uniformly from the surface of an $n$-dimensional sphere. To be more precise, we term this as a spherical codebook. We also consider i.i.d.\ Gaussian codebooks in which each random codeword is drawn independently from a product Gaussian distribution. We derive the second-order, moderate, and large deviation asymptotics when i.i.d.\ Gaussian codebooks are employed. In contrast to the recent work on the channel coding counterpart by Scarlett, Tan and Durisi (2017), the dispersions for spherical and i.i.d.\ Gaussian codebooks are identical. The ensemble excess-distortion exponents for both spherical and i.i.d.\ Gaussian codebooks are established for all rates. Furthermore, we show that the i.i.d.\ Gaussian codebook has a strictly larger excess-distortion exponent than its spherical counterpart for any rate greater than the ensemble rate-distortion function derived by Lapidoth.

cs.IT↗

arXiv2018

The Dispersion of Mismatched Joint Source-Channel Coding for Arbitrary Sources and Additive Channels

We consider a joint source channel coding (JSCC) problem in which we desire to transmit an arbitrary memoryless source over an arbitrary additive channel. We propose a mismatched coding architecture that consists of Gaussian codebooks for both the source reproduction sequences and channel codewords. The natural nearest neighbor encoder and decoder, however, need to be judiciously modified to obtain the highest communication rates at finite blocklength. In particular, we consider a unequal error protection (UEP) scheme in which all sources are partitioned into disjoint power type classes. We also regularize the nearest neighbor decoder so that an appropriate measure of the size of each power type class is taken into account in the decoding strategy. For such an architecture, we derive ensemble-tight second-order and moderate deviations results. Our first-order (optimal bandwidth expansion ratio) result generalizes the seminal results by Lapidoth (1996, 1997). The dispersion of our JSCC scheme is a linear combination of the mismatched dispersions for the channel coding saddle-point problem by Scarlett, Tan and Durisi (2017) and the rate-distortion saddle-point problem by the present authors, thus also generalizing these results.

cs.IT↗

arXiv2018

Asymptotically Optimal Codes Correcting Fixed-Length Duplication Errors in DNA Storage Systems

A (tandem) duplication of length $ k $ is an insertion of an exact copy of a substring of length $ k $ next to its original position. This and related types of impairments are of relevance in modeling communication in the presence of synchronization errors, as well as in several information storage applications. We demonstrate that Levenshtein's construction of binary codes correcting insertions of zeros is, with minor modifications, applicable also to channels with arbitrary alphabets and with duplication errors of arbitrary (but fixed) length $ k $. Furthermore, we derive bounds on the cardinality of optimal $ q $-ary codes correcting up to $ t $ duplications of length $ k $, and establish the following corollaries in the asymptotic regime of growing block-length: 1.) the presented family of codes is optimal for every $ q, t, k $, in the sense of the asymptotic scaling of code redundancy; 2.) the upper bound, when specialized to $ q = 2 $, $ k = 1 $, improves upon Levenshtein's bound for every $ t \geq 3 $; 3.) the bounds coincide for $ t = 1 $, thus yielding the exact asymptotic behavior of the size of optimal single-duplication-correcting codes.

cs.IT↗

arXiv2018

Strong Converse for Hypothesis Testing Against Independence over a Two-Hop Network

By proving a strong converse, we strengthen the weak converse result by Salehkalaibar, Wigger and Wang (2017) concerning hypothesis testing against independence over a two-hop network with communication constraints. Our proof follows by judiciously combining two recently proposed techniques for proving strong converse theorems, namely the strong converse technique via reverse hypercontractivity by Liu, van Handel, and Verdú (2017) and the strong converse technique by Tyagi and Watanabe (2018), in which the authors used a change-of-measure technique and replaced hard Markov constraints with soft information costs. The techniques used in our paper can also be applied to prove strong converse theorems for other multiterminal hypothesis testing against independence problems.

cs.IT↗

arXiv2018

On the Throughput of Channels that Wear Out

This work investigates the fundamental limits of communication over a noisy discrete memoryless channel that wears out, in the sense of signal-dependent catastrophic failure. In particular, we consider a channel that starts as a memoryless binary-input channel and when the number of transmitted ones causes a sufficient amount of damage, the channel ceases to convey signals. Constant composition codes are adopted to obtain an achievability bound and the left-concave right-convex inequality is then refined to obtain a converse bound on the log-volume throughput for channels that wear out. Since infinite blocklength codes will always wear out the channel for any finite threshold of failure and therefore cannot convey information at positive rates, we analyze the performance of finite blocklength codes to determine the maximum expected transmission volume at a given level of average error probability. We show that this maximization problem has a recursive form and can be solved by dynamic programming. Numerical results demonstrate that a sequence of block codes is preferred to a single block code for streaming sources.

cs.IT↗

arXiv2018

Codes in the Space of Multisets---Coding for Permutation Channels with Impairments

Motivated by communication channels in which the transmitted sequences are subject to random permutations, as well as by certain DNA storage systems, we study the error control problem in settings where the information is stored/transmitted in the form of multisets of symbols from a given finite alphabet. A general channel model is assumed in which the transmitted multisets are potentially impaired by insertions, deletions, substitutions, and erasures of symbols. Several constructions of error-correcting codes for this channel are described, and bounds on the size of optimal codes correcting any given number of errors derived. The construction based on the notion of Sidon sets in finite Abelian groups is shown to be optimal, in the sense of the asymptotic scaling of code redundancy, for any "error radius" and any alphabet size. It is also shown to be optimal in the stronger sense of maximal code cardinality in various cases.

cs.IT↗

arXiv2018

Distributed Hypothesis Testing with Privacy Constraints

We revisit the distributed hypothesis testing (or hypothesis testing with communication constraints) problem from the viewpoint of privacy. Instead of observing the raw data directly, the transmitter observes a sanitized or randomized version of it. We impose an upper bound on the mutual information between the raw and randomized data. Under this scenario, the receiver, which is also provided with side information, is required to make a decision on whether the null or alternative hypothesis is in effect. We first provide a general lower bound on the type-II exponent for an arbitrary pair of hypotheses. Next, we show that if the distribution under the alternative hypothesis is the product of the marginals of the distribution under the null (i.e., testing against independence), then the exponent is known exactly. Moreover, we show that the strong converse property holds. Using ideas from Euclidean information theory, we also provide an approximate expression for the exponent when the communication rate is low and the privacy level is high. Finally, we illustrate our results with a binary and a Gaussian example.

cs.IT↗

arXiv2018

Moderate Deviation Asymptotics for Variable-Length Codes with Feedback

We consider data transmission across discrete memoryless channels (DMCs) using variable-length codes with feedback. We consider the family of such codes whose rates are $ρ_N$ below the channel capacity $C$, where $ρ_N$ is a positive sequence that tends to zero slower than the reciprocal of the square root of the expectation of the (random) blocklength $N$. This is known as the moderate deviations regime and we establish the optimal moderate deviations constant. We show that in this scenario, the error probability decays sub-exponentially with speed $\exp(-(B/C)Nρ_N)$, where $B$ is the maximum relative entropy between output distributions of the DMC.

cs.IT↗

arXiv2018

On the Maximum Size of Block Codes Subject to a Distance Criterion

We establish a general formula for the maximum size of finite length block codes with minimum pairwise distance no less than $d$. The achievability argument involves an iterative construction of a set of radius-$d$ balls, each centered at a codeword. We demonstrate that the number of such balls that cover the entire code alphabet cannot exceed this maximum size. Our approach can be applied to codes $i)$ with elements over arbitrary code alphabets, and $ii)$ under a broad class of distance measures, thereby ensuring the generality of our formula. Our formula indicates that the maximum code size can be fully characterized by the cumulative distribution function of the distance measure evaluated at two independent and identically distributed random codewords. When the two random codewords assume a uniform distribution over the entire code alphabet, our formula recovers and obtains a natural generalization of the Gilbert-Varshamov (GV) lower bound. We also establish a general formula for the zero-error capacity of any sequence of channels. Finally, we extend our study to the asymptotic setting, where we establish first- and second-order bounds on the asymptotic code rate subject to a normalized minimum distance constraint.

cs.IT↗

arXiv2018

Lower Bounds on the Bayes Risk of the Bayesian BTL Model with Applications to Comparison Graphs

We consider the problem of aggregating pairwise comparisons to obtain a consensus ranking order over a collection of objects. We use the popular Bradley-Terry-Luce (BTL) model which allows us to probabilistically describe pairwise comparisons between objects. In particular, we employ the Bayesian BTL model which allows for meaningful prior assumptions and to cope with situations where the number of objects is large and the number of comparisons between some objects is small or even zero. For the conventional Bayesian BTL model, we derive information-theoretic lower bounds on the Bayes risk of estimators for norm-based distortion functions. We compare the information-theoretic lower bound with the Bayesian Cramér-Rao lower bound we derive for the case when the Bayes risk is the mean squared error. We illustrate the utility of the bounds through simulations by comparing them with the error performance of an expectation-maximization based inference algorithm proposed for the Bayesian BTL model. We draw parallels between pairwise comparisons in the BTL model and inter-player games represented as edges in a comparison graph and analyze the effect of various graph structures on the lower bounds. We also extend the information-theoretic and Bayesian Cramér-Rao lower bounds to the more general Bayesian BTL model which takes into account home-field advantage.

cs.IT↗

arXiv2018

The Optimal Compression Rate of Variable-to-Fixed Length Source Coding with a Non-Vanishing Excess-Distortion Probability

We consider the variable-to-fixed length lossy source coding (VFSC) problem. The optimal compression rate of the average length of variable-to-fixed source coding, allowing a non-vanishing probability of excess-distortion $\varepsilon$, is shown to be equal to $(1-\varepsilon)R(D)$, where $R(D)$ is the rate-distortion function of the source. In comparison to the related results of Koga and Yamamoto as well as Kostina, Polyanskiy, and Verdú for fixed-to-variable length source coding, our results demonstrate an interesting feature that variable-to-fixed length source coding has the same first-order compression rate as fixed-to-variable length source coding.

cs.IT↗

arXiv2018

On Gaussian MACs with Variable-Length Feedback and Non-Vanishing Error~Probabilities

We characterize the fundamental limits of transmission of information over a Gaussian multiple access channel (MAC) with the use of variable-length feedback codes and under a non-vanishing error probability formalism. We develop new achievability and converse techniques to handle the continuous nature of the channel and the presence of expected power constraints. We establish the $\varepsilon$-capacity regions and bounds on the second-order asymptotics of the Gaussian MAC with variable-length feedback with termination (VLFT) codes and stop-feedback codes. We show that the former outperforms the latter significantly. Due to the multi-terminal nature of the channel model, we leverage tools from renewal theory developed by Lai and Siegmund to bound the asymptotic behavior of the maximum of a finite number of stopping times.

cs.IT↗

arXiv2017

Exponential Strong Converse for Content Identification with Lossy Recovery

We revisit the high-dimensional content identification with lossy recovery problem (Tuncel and Gündüz, 2014) and establish an exponential strong converse theorem. As a corollary of the exponential strong converse theorem, we derive an upper bound on the joint identification-error and excess-distortion exponent for the problem. Our main results can be specialized to the biometrical identification problem~(Willems, 2003) and the content identification problem~(Tuncel, 2009) since these two problems are both special cases of the content identification with lossy recovery problem. We leverage the information spectrum method introduced by Oohama and adapt the strong converse techniques therein to be applicable to the problem at hand.

cs.IT↗

arXiv2017

Minimum Rates of Approximate Sufficient Statistics

Given a sufficient statistic for a parametric family of distributions, one can estimate the parameter without access to the data. However, the memory or code size for storing the sufficient statistic may nonetheless still be prohibitive. Indeed, for $n$ independent samples drawn from a $k$-nomial distribution with $d=k-1$ degrees of freedom, the length of the code scales as $d\log n+O(1)$. In many applications, we may not have a useful notion of sufficient statistics (e.g., when the parametric family is not an exponential family) and we also may not need to reconstruct the generating distribution exactly. By adopting a Shannon-theoretic approach in which we allow a small error in estimating the generating distribution, we construct various {\em approximate sufficient statistics} and show that the code length can be reduced to $\frac{d}{2}\log n+O(1)$. We consider errors measured according to the relative entropy and variational distance criteria. For the code constructions, we leverage Rissanen's minimum description length principle, which yields a non-vanishing error measured according to the relative entropy. For the converse parts, we use Clarke and Barron's formula for the relative entropy of a parametrized distribution and the corresponding mixture distribution. However, this method only yields a weak converse for the variational distance. We develop new techniques to achieve vanishing errors and we also prove strong converses. The latter means that even if the code is allowed to have a non-vanishing error, its length must still be at least $\frac{d}{2}\log n$.

cs.IT↗

arXiv2017

Stochastic L-BFGS: Improved Convergence Rates and Practical Acceleration Strategies

We revisit the stochastic limited-memory BFGS (L-BFGS) algorithm. By proposing a new framework for the convergence analysis, we prove improved convergence rates and computational complexities of the stochastic L-BFGS algorithms compared to previous works. In addition, we propose several practical acceleration strategies to speed up the empirical performance of such algorithms. We also provide theoretical analyses for most of the strategies. Experiments on large-scale logistic and ridge regression problems demonstrate that our proposed strategies yield significant improvements vis-à-vis competing state-of-the-art algorithms.

math.OC↗

arXiv2017

Improved Bounds on Sidon Sets via Lattice Packings of Simplices

A $ B_h $ set (or Sidon set of order $ h $) in an Abelian group $ G $ is any subset $ \{b_0, b_1, \ldots,b_{n}\} $ of $ G $ with the property that all the sums $ b_{i_1} + \cdots + b_{i_h} $ are different up to the order of the summands. Let $ ϕ(h,n) $ denote the order of the smallest Abelian group containing a $ B_h $ set of cardinality $ n + 1 $. It is shown that \[ \lim_{h \to \infty} \frac{ ϕ(h,n) }{ h^n } = \frac{1}{n! δ_L(\triangle^n)} , \] where $ δ_L(\triangle^n) $ is the lattice packing density of an $ n $-simplex in Euclidean space. This determines the asymptotics exactly in cases where this density is known ($ n \leq 3 $) and gives improved bounds on $ ϕ(h,n) $ in the remaining cases. The corresponding geometric characterization of bases of order $ h $ in finite Abelian groups in terms of lattice coverings by simplices is also given.

math.CO↗

arXiv2017

Achievable Moderate Deviations Asymptotics for Streaming Compression of Correlated Sources

Motivated by streaming multi-view video coding and wireless sensor networks, we consider the problem of blockwise streaming compression of a pair of correlated sources, which we term streaming Slepian-Wolf coding. We study the moderate deviations regime in which the rate pairs of a sequence of codes converge, along a straight line, to various points on the boundary of the Slepian-Wolf region at a speed slower than the inverse square root of the blocklength $n$, while the error probability decays subexponentially fast in $n$. Our main result focuses on directions of approaches to corner points of the Slepian-Wolf region. It states that for each correlated source and all corner points, there exists a non-empty subset of directions of approaches such that the moderate deviations constant (the constant of proportionality for the subexponential decay of the error probability) is enhanced (over the non-streaming case) by at least a factor of $T$, the block delay of decoding source block pairs. We specialize our main result to the setting of streaming lossless source coding and generalize this result to the setting where we have different delay requirements for each of the two source blocks. The proof of our main result involves the use of various analytical tools and amalgamates several ideas from the recent information-theoretic streaming literature. We adapt the so-called truncated memory encoding idea from Draper and Khisti (2011) and Lee, Tan, and Khisti (2016) to ensure that the effect of error accumulation is nullified in the limit of large blocklengths. We also adapt the use of the so-called minimum weighted empirical suffix entropy decoder which was used by Draper, Chang, and Sahai (2014) to derive achievable error exponents for symbolwise streaming Slepian-Wolf coding.

cs.IT↗