arXiv subjects

Vincent Y. F. Tan

Publications and source records attributed to Vincent Y. F. Tan.

At least 145 records · Page 8Linked to original sources

arXiv2020

Distributed Detection with Empirically Observed Statistics

Consider a distributed detection problem in which the underlying distributions of the observations are unknown; instead of these distributions, noisy versions of empirically observed statistics are available to the fusion center. These empirically observed statistics, together with source (test) sequences, are transmitted through different channels to the fusion center. The fusion center decides which distribution the source sequence is sampled from based on these data. For the binary case, we derive the optimal type-II error exponent given that the type-I error decays exponentially fast. The type-II error exponent is maximized over the proportions of channels for both source and training sequences. We conclude that as the ratio of the lengths of training to test sequences $α$ tends to infinity, using only one channel is optimal. By calculating the derived exponents numerically, we conjecture that the same is true when $α$ is finite. We relate our results to the classical distributed detection problem studied by Tsitsiklis, in which the underlying distributions are known. Finally, our results are extended to the case of $m$-ary distributed detection with a rejection option.

cs.IT↗

arXiv2020

Economy Statistical Recurrent Units For Inferring Nonlinear Granger Causality

Granger causality is a widely-used criterion for analyzing interactions in large-scale networks. As most physical interactions are inherently nonlinear, we consider the problem of inferring the existence of pairwise Granger causality between nonlinearly interacting stochastic processes from their time series measurements. Our proposed approach relies on modeling the embedded nonlinearities in the measurements using a component-wise time series prediction model based on Statistical Recurrent Units (SRUs). We make a case that the network topology of Granger causal relations is directly inferrable from a structured sparse estimate of the internal parameters of the SRU networks trained to predict the processes$'$ time series measurements. We propose a variant of SRU, called economy-SRU, which, by design has considerably fewer trainable parameters, and therefore less prone to overfitting. The economy-SRU computes a low-dimensional sketch of its high-dimensional hidden state in the form of random projections to generate the feedback for its recurrent processing. Additionally, the internal weight parameters of the economy-SRU are strategically regularized in a group-wise manner to facilitate the proposed network in extracting meaningful predictive features that are highly time-localized to mimic real-world causal events. Extensive experiments are carried out to demonstrate that the proposed economy-SRU based time series prediction model outperforms the MLP, LSTM and attention-gated CNN-based time series models considered previously for inferring Granger causality.

cs.LG↗

arXiv2019

Wyner's Common Information under Rényi Divergence Measures

We study a generalized version of Wyner's common information problem (also coined the distributed source simulation problem). The original common information problem consists in understanding the minimum rate of the common input to independent processors to generate an approximation of a joint distribution when the distance measure used to quantify the discrepancy between the synthesized and target distributions is the normalized relative entropy. Our generalization involves changing the distance measure to the unnormalized and normalized Rényi divergences of order $α=1+s\in[0,2]$. We show that the minimum rate needed to ensure the Rényi divergences between the distribution induced by a code and the target distribution vanishes remains the same as the one in Wyner's setting, except when the order $α=1+s=0$. This implies that Wyner's common information is rather robust to the choice of distance measure employed. As a byproduct of the proofs used to the establish the above results, the exponential strong converse for the common information problem under the total variation distance measure is established.

cs.IT↗

arXiv2019

On the Capacity of Channels with Deletions and States

We consider the class of channels formed from the concatenation of a deletion channel and a finite-state channel. For this class of channels, we show that the operational capacity is equal to the stationary capacity, which can be approached by a sequence of Markov processes with increasing Markovian orders. As a by-product, we show that the polar coding scheme constructed by Tal, Pfister, Fazeli and Vardy achieves the capacity of the deletion channel.

cs.IT↗

arXiv2019

On the Bee-Identification Error Exponent with Absentee Bees

The "bee-identification problem" was formally defined by Tandon, Tan and Varshney [IEEE Trans. Commun. (2019) [Online early access]], and the error exponent was studied. This work extends the results for the "absentee bees" scenario, where a small fraction of the bees are absent in the beehive image used for identification. For this setting, we present an exact characterization of the bee-identification error exponent, and show that independent barcode decoding is optimal, i.e., joint decoding of the bee barcodes does not result in a better error exponent relative to independent decoding of each noisy barcode. This is in contrast to the result without absentee bees, where joint barcode decoding results in a significantly higher error exponent than independent barcode decoding. We also define and characterize the "capacity" for the bee-identification problem with absentee bees, and prove the strong converse for the same.

cs.IT↗

arXiv2019

An Improved Linear Programming Bound on the Average Distance of a Binary Code

Ahlswede and Katona (1977) posed the following isodiametric problem in Hamming spaces: For every $n$ and $1\le M\le2^{n}$, determine the minimum average Hamming distance of binary codes with length $n$ and size $M$. Fu, Wei, and Yeung (2001) used linear programming duality to derive a lower bound on the minimum average distance. However, their linear programming approach was not completely exploited. In this paper, we improve Fu-Wei-Yeung's bound by finding a better feasible solution to their dual program. For fixed $0<a\le1/2$ and for $M=\left\lceil a2^{n}\right\rceil $, our feasible solution attains the asymptotically optimal value of Fu-Wei-Yeung's dual program as $n\to\infty$. Hence for $0<a\le1/2$, all possible asymptotic bounds that can be derived by Fu-Wei-Yeung's linear program have been characterized. Furthermore, noting that the average distance of a code is closely related to weights of Fourier coefficients of a Boolean function, we also apply the linear programming technique to prove bounds on Fourier weights of a Boolean function of various degrees.

math.CO↗

arXiv2019

Exact Channel Synthesis

We consider the exact channel synthesis problem. This problem concerns the determination of the minimum amount of information required to create exact correlation remotely when there is a certain rate of randomness shared by two terminals. This problem generalizes an existing approximate version, in which the generated joint distribution is required to be close to a target distribution under the total variation (TV) distance measure (instead being exactly equal to the target distribution). We provide single-letter inner and outer bounds on the admissible region of the shared randomness rate and the communication rate for the exact channel synthesis problem. These two bounds coincide for doubly symmetric binary sources. We observe that for such sources, the admissible rate region for exact channel synthesis is strictly included in that for the TV-approximate version. We also extend the exact and TV-approximate channel synthesis problems to sources with countably infinite alphabets and continuous sources; the latter includes Gaussian sources. As by-products, lemmas concerning soft-covering under Rényi divergence measures are derived.

cs.IT↗

arXiv2019

Throughput Scaling of Covert Communication over Wireless Adhoc Networks

We consider the problem of covert communication over wireless adhoc networks in which (roughly) $n$ legitimate nodes (LNs) and $n^κ$ for $0<κ<1$ non-communicating warden nodes (WNs) are randomly distributed in a square of unit area. Each legitimate source wants to communicate with its intended destination node while ensuring that every WN is unable to detect the presence of the communication. In this scenario, we study the throughput scaling law. Due the covert communication constraint, the transmit powers are necessarily limited. Under this condition, we introduce a preservation region around each WN. This region serves to prevent transmission from the LNs and to increase the transmit power of the LNs outside the preservation regions. For the achievability results, multi-hop (MH), hierarchical cooperation (HC), and hybrid HC-MH schemes are utilized with some appropriate modifications. In the proposed MH and hybrid schemes, because the preservation regions may impede communication along direct data paths, the data paths are suitably modified by taking a detour around each preservation region. To avoid the concentration of detours resulting extra relaying burdens, we distribute the detours evenly over a wide region. In the proposed HC scheme, we control the symbol power and the scheduling of distributed multiple-input multiple-output transmission. We also present matching upper bounds on the throughput scaling under the assumption that every active LN consumes the same average transmit power over the time period in which the WNs observe the channel outputs.

cs.IT↗

arXiv2019

Covert Communication Over a Compound Channel

In this paper, we consider fundamental communication limits over a compound channel. Covert communication in the information-theoretic context has been primarily concerned with fundamental limits when the transmitter wishes to communicate to legitimate receiver(s) while ensuring that the communication is not detected by an adversary. This paper, however, considers an alternative, and no less important, setting in which the object to be masked is the state of the compound channel. Such a communication model has applications in the prevention of malicious parties seeking to jam the communication signal when, for example, the signal-to-noise ratio of a wireless channel is found to be low. Our main contribution is the establishment of bounds on the throughput-key region when the covertness constraint is defined in terms of the total variation distance. In addition, for the scenario in which the key length is infinite, we provide a sufficient condition for when the bounds coincide for the scaling of the throughput, which follows the square-root law. Numerical examples, including that of a Gaussian channel, are provided to illustrate our results.

cs.IT↗

arXiv2019

A Ranking Model Motivated by Nonnegative Matrix Factorization with Applications to Tennis Tournaments

We propose a novel ranking model that combines the Bradley-Terry-Luce probability model with a nonnegative matrix factorization framework to model and uncover the presence of latent variables that influence the performance of top tennis players. We derive an efficient, provably convergent, and numerically stable majorization-minimization-based algorithm to maximize the likelihood of datasets under the proposed statistical model. The model is tested on datasets involving the outcomes of matches between 20 top male and female tennis players over 14 major tournaments for men (including the Grand Slams and the ATP Masters 1000) and 16 major tournaments for women over the past 10 years. Our model automatically infers that the surface of the court (e.g., clay or hard court) is a key determinant of the performances of male players, but less so for females. Top players on various surfaces over this longitudinal period are also identified in an objective manner.

cs.LG↗

arXiv2019

Exact Error and Erasure Exponents for the Asymmetric Broadcast Channel

Consider the asymmetric broadcast channel with a random superposition codebook, which may be comprised of constant composition or \iid codewords. By applying Forney's optimal decoder for individual messages and the message pair for the receiver that decodes both messages, exact (ensemble-tight) error and erasure exponents are derived. It is shown that the optimal decoder designed to decode the pair of messages achieves the optimal trade-off between the total and undetected exponents associated with the optimal decoder for the private message. Convex optimization-based procedures to evaluate the exponents efficiently are proposed. Finally, numerical examples are presented to illustrate the results.

cs.IT↗

arXiv2019

Error Exponent Bounds for the Bee-Identification Problem

Consider the problem of identifying a massive number of bees, uniquely labeled with barcodes, using noisy measurements. We formally introduce this `bee-identification problem', define its error exponent, and derive efficiently computable upper and lower bounds for this exponent. We show that joint decoding of barcodes provides a significantly better exponent compared to separate decoding followed by permutation inference. For low rates, we prove that the lower bound on the bee-identification exponent obtained using typical random codes (TRC) is strictly better than the corresponding bound obtained using a random code ensemble (RCE). Further, as the rate approaches zero, we prove that the upper bound on the bee-identification exponent meets the lower bound obtained using TRC with joint barcode decoding.

cs.IT↗

arXiv2019

The Reliability Function of Lossy Source-Channel Coding of Variable-Length Codes with Feedback

We consider transmission of discrete memoryless sources (DMSes) across discrete memoryless channels (DMCs) using variable-length lossy source-channel codes with feedback. The reliability function (optimum error exponent) is shown to be equal to $\max\{0, B(1-R(D)/C)\},$ where $R(D)$ is the rate-distortion function of the source, $B$ is the maximum relative entropy between output distributions of the DMC, and $C$ is the Shannon capacity of the channel. We show that, in this setting and in this asymptotic regime, separate source-channel coding is, in fact, optimal.

cs.IT↗

arXiv2019

Strong Converse Theorems for Multimessage Networks with Tight Cut-Set Bound

This paper considers a multimessage network where each node may send a message to any other node in the network. Under the discrete memoryless model, we prove the strong converse theorem for any network whose cut-set bound is tight, i.e., achievable. Our result implies that for any fixed rate vector that resides outside the capacity region, the average error probabilities of any sequence of length-$n$ codes operated at the rate vector must tend to $1$ as $n$ approaches infinity. The proof is based on the method of types and is inspired by the work of Csiszár and Körner in 1982 which fully characterized the reliability function of any discrete memoryless channel (DMC) with feedback for rates above capacity. In addition, we generalize the strong converse theorem to the Gaussian model where each node is subject to an almost-sure power constraint. Important consequences of our results are new strong converses for the Gaussian multiple access channel (MAC) with feedback and the following relay channels under both models: The degraded relay channel (RC), the RC with orthogonal sender components, and the general RC with feedback.

cs.IT↗

arXiv2019

Fundamental Limits of Communication Over State-Dependent Channels With Feedback

The fundamental limits of communication over state-dependent discrete memoryless channels with noiseless feedback are studied, under the assumption that the communicating parties are allowed to use variable-length coding schemes. Various cases are analyzed, with the employed coding schemes having either bounded or unbounded codeword lengths, and with state information revealed to the encoder and/or decoder in a strictly causal, causal, or non-causal manner. In each of these settings, necessary and sufficient conditions for positivity of the zero-error capacity are obtained and it is shown that, whenever the zero-error capacity is positive, it equals the conventional vanishing-error capacity. Moreover, it is shown that the vanishing-error capacity of state-dependent channels is not increased by the use of feedback and variable-length coding. Both these kinds of capacities of state-dependent channels with feedback are thus fully characterized.

cs.IT↗

arXiv2018

Simulation of Random Variables under Rényi Divergence Measures of All Orders

The random variable simulation problem consists in using a $k$-dimensional i.i.d. random vector $X^{k}$ with distribution $P_{X}^{k}$ to simulate an $n$-dimensional i.i.d. random vector $Y^{n}$ so that its distribution is approximately $Q_{Y}^{n}$. In contrast to previous works, in this paper we consider the standard Rényi divergence and two variants of all orders to measure the level of approximation. These two variants are the max-Rényi divergence $D_α^{\mathsf{max}}(P,Q)$ and the sum-Rényi divergence $D_α^{+}(P,Q)$. When $α=\infty$, these two measures are strong because for any $ε>0$, $D_{\infty}^{\mathsf{max}}(P,Q)\leqε$ or $D_{\infty}^{+}(P,Q)\leqε$ implies $e^{-ε}\leq\frac{P(x)}{Q(x)}\leq e^ε$ for all $x$. Under these Rényi divergence measures, we characterize the asymptotics of normalized divergences as well as the Rényi conversion rates. The latter is defined as the supremum of $\frac{n}{k}$ such that the Rényi divergences vanish asymptotically. In addition, when the Rényi parameter is in the interval $(0,1)$, the Rényi conversion rates equal the ratio of the Shannon entropies $\frac{H\left(P_{X}\right)}{H\left(Q_{Y}\right)}$, which is consistent with traditional results in which the total variation measure was adopted. When the Rényi parameter is in the interval $(1,\infty]$, the Rényi conversion rates are, in general, smaller than $\frac{H\left(P_{X}\right)}{H\left(Q_{Y}\right)}$. When specialized to the case in which either $P_{X}$ or $Q_{Y}$ is uniform, the simulation problem reduces to the source resolvability and intrinsic randomness problems. The preceding results are used to characterize the asymptotics of Rényi divergences and the Rényi conversion rates for these two cases.

cs.IT↗

arXiv2018

Second-Order Asymptotically Optimal Statistical Classification

Motivated by real-world machine learning applications, we analyze approximations to the non-asymptotic fundamental limits of statistical classification. In the binary version of this problem, given two training sequences generated according to two {\em unknown} distributions $P_1$ and $P_2$, one is tasked to classify a test sequence which is known to be generated according to either $P_1$ or $P_2$. This problem can be thought of as an analogue of the binary hypothesis testing problem but in the present setting, the generating distributions are unknown. Due to finite sample considerations, we consider the second-order asymptotics (or dispersion-type) tradeoff between type-I and type-II error probabilities for tests which ensure that (i) the type-I error probability for {\em all} pairs of distributions decays exponentially fast and (ii) the type-II error probability for a {\em particular} pair of distributions is non-vanishing. We generalize our results to classification of multiple hypotheses with the rejection option.

cs.IT↗

arXiv2018

Rényi Resolvability and Its Applications to the Wiretap Channel

The conventional channel resolvability problem refers to the determination of the minimum rate required for an input process so that the output distribution approximates a target distribution in either the total variation distance or the relative entropy. In contrast to previous works, in this paper, we use the (normalized or unnormalized) Rényi divergence (with the Rényi parameter in $[0,2]\cup\{\infty\}$) to measure the level of approximation. We also provide asymptotic expressions for normalized Rényi divergence when the Rényi parameter is larger than or equal to $1$ as well as (lower and upper) bounds for the case when the same parameter is smaller than $1$. We characterize the Rényi resolvability, which is defined as the minimum rate required to ensure that the Rényi divergence vanishes asymptotically. The Rényi resolvabilities are the same for both the normalized and unnormalized divergence cases. In addition, when the Rényi parameter smaller than~$1$, consistent with the traditional case where the Rényi parameter is equal to~$1$, the Rényi resolvability equals the minimum mutual information over all input distributions that induce the target output distribution. When the Rényi parameter is larger than $1$ the Rényi resolvability is, in general, larger than the mutual information. The optimal Rényi divergence is proven to vanish at least exponentially fast for both of these two cases, as long as the code rate is larger than the Rényi resolvability. The optimal exponential rate of decay for i.i.d.\ random codes is also characterized exactly. We apply these results to the wiretap channel, and completely characterize the optimal tradeoff between the rates of the secret and non-secret messages when the leakage measure is given by the (unnormalized) Rényi divergence.

cs.IT↗