arXiv subjects

Anelia Somekh-Baruch

Publications and source records attributed to Anelia Somekh-Baruch.

At least 19 recordsLinked to original sources

arXiv2025

The Shannon Upper Bound for the Error Exponent

For the discrete-time additive white generalized Gaussian noise channel with a generalized input power constraint, with the respective shape and power parameters >= 1, we derive an upper bound on the optimal block error exponent. Explicit asymptotic upper bounds in the limit of a large block length n are given for three special cases: the Laplace noise channel and the Gaussian noise channel with the average absolute value constraint, and for the Laplace noise channel with the second power constraint. The derivation uses the method of types with finite alphabets of sizes depending on the block length n and with the number of types sub-exponential in n.

cs.IT↗

arXiv2024

Pre-Decoder Processing Functions for a DMC with Mismatched Decoding

This paper analyzes the effect of adding a pre-decoder processing function to a receiver that contains a fixed mismatched decoder at the output of a discrete memoryless channel. We study properties of the symbolwise pre-processing function and show that it is a simple yet very powerful tool which enables to obtain reliable transmission at a positive rate for almost every metric. We present lower and upper bounds on the capacity of a channel with mismatched decoding and symbolwise(scalar-to-scalar) pre-processing, and show that the optimal pre-processing function for random coding is deterministic. We also characterize achievable error exponents. Finally, we prove that a separation principle holds for vectorwise(vector-to-vector) pre-processing functions and further, that deterministic functions maximize the reliably transmitted rate in this case.

cs.IT↗

arXiv2023

The Method of Types for the AWGN Channel

For the discrete-time AWGN channel with a power constraint, we give an alternative derivation of Shannon's sphere-packing upper bound on the optimal block error exponent and prove for the first time an analogous lower bound on the optimal correct-decoding exponent. The derivations use the method of types with finite alphabets of sizes depending on the block length n and with the number of types sub-exponential in n.

cs.IT↗

arXiv2023

Information-Theoretic Foundations of Mismatched Decoding

Shannon's channel coding theorem characterizes the maximal rate of information that can be reliably transmitted over a communication channel when optimal encoding and decoding strategies are used. In many scenarios, however, practical considerations such as channel uncertainty and implementation constraints rule out the use of an optimal decoder. The mismatched decoding problem addresses such scenarios by considering the case that the decoder cannot be optimized, but is instead fixed as part of the problem statement. This problem is not only of direct interest in its own right, but also has close connections with other long-standing theoretical problems in information theory. In this monograph, we survey both classical literature and recent developments on the mismatched decoding problem, with an emphasis on achievable random-coding rates for memoryless channels. We present two widely-considered achievable rates known as the generalized mutual information (GMI) and the LM rate, and overview their derivations and properties. In addition, we survey several improved rates via multi-user coding techniques, as well as recent developments and challenges in establishing upper bounds on the mismatch capacity, and an analogous mismatched encoding problem in rate-distortion theory. Throughout the monograph, we highlight a variety of applications and connections with other prominent information theory problems.

cs.IT↗

arXiv2022

A Generalization of the DMC

We consider a generalization of the discrete memoryless channel, in which the channel probability distribution is replaced by a uniform distribution over clouds of channel output sequences. For a random ensemble of such channels, we derive an achievable error exponent, as well as its converse together with the optimal correct-decoding exponent, all as functions of information rate. As a corollary of these results, we obtain the channel ensemble capacity.

cs.IT↗

arXiv2022

An Upper Bound on the Reliability Function of the DMC

We derive a new upper bound on the reliability function for channel coding over discrete memoryless channels. Our bounding technique relies on two main elements: (i) adding an auxiliary genie-receiver that reveals to the original receiver a list of codewords including the transmitted one, which satisfy a certain type property, and (ii) partitioning (most of) the list into subsets of codewords that satisfy a certain pairwise-symmetry property, which facilitates lower bounding of the average error probability by the pairwise error probability within a subset. We compare the obtained bound to the Shannon-Gallager-Berlekamp straight-line bound, the sphere-packing bound, and an amended version of Blahut's bound. Our bound is shown to be at least as tight for all rates, with cases of stricter tightness in a certain range of low rates, compared to all three aforementioned bounds. Our derivation is performed in a unified manner which is valid for any rate, as well as for a wide class of additive decoding metrics, whenever the corresponding zero-error capacity is zero. We further present a relatively simple function that may be regarded as an approximation to the reliability function in some cases. We also present a dual form of the bound, and discuss a looser bound of a simpler form, which is analyzed for the case of the binary symmetric channel with maximum likelihood decoding.

cs.IT↗

arXiv2022

Upper Bounds on the Mismatched Reliability Function and Capacity Using a Genie Receiver

We develop a novel framework for proving converse theorems for channel coding, which is based on the analysis technique of multicast transmission with an additional auxiliary receiver, which serves as a genie to the original receiver. The genie provides the original receiver a certain narrowed list of codewords to choose from that includes the transmitted one. This technique is used to derive upper bounds on the mismatch capacity of discrete memoryless channels as well as the reliability function with a mismatched decoding metric. Unlike previous works, our bounding technique exploits also the inherent symmetric requirement from the codewords, leading to these new upper bounds. Since the computations of most of the known bounds on the mismatch capacity are rather complicated, we further present a method to obtain relaxed bounds that are easier to compute. As an example, we analyze the obtained bounds in the binary-input channels case. We conclude by presenting simpler bounds on the reliability function, and provide sufficient conditions for their tightness in certain ranges of rates.

cs.IT↗

arXiv2021

A Single-Letter Upper Bound on the Mismatch Capacity via Multicast Transmission

We introduce a new analysis technique to derive a single-letter upper bound on the mismatch capacity of a stationary, single-user, memoryless channel with a decoding metric $q$. Our bound is obtained by considering a multicast transmission over a two-user broadcast channel with decoding metrics $q$ and $ρ$ at the receivers, referred to as $(q,ρ)$-surely degraded. This channel has the property that the intersection event of correct $q$-decoding of receiver $1$ and erroneous $ρ$-decoding of receiver $2$ has zero probability for any fixed-composition codebook of a certain composition $P$. Our bound holds in the strong converse sense of an exponential decay of the probability of correct decoding at rates above the bound. Further, we refine the proof and present a bound that is at least as tight as that of any choice of $ρ$. Several examples that demonstrate the strict improvement of our bound compared to previous results are analyzed. Finally, we detect equivalence classes of isomorphic channel-metric pairs $(W,q)$ that share the same mismatch capacity. We prove that if the class contains a matched pair, then our bound is tight and the mismatch capacity of the entire class is fully characterized and is equal to the LM rate, which is achievable by random coding, and may be strictly lower that the matched capacity.

cs.IT↗

arXiv2021

The Restless Hidden Markov Bandit with Linear Rewards and Side Information

In this paper we present a model for the hidden Markovian bandit problem with linear rewards. As opposed to current work on Markovian bandits, we do not assume that the state is known to the decision maker before making the decision. Furthermore, we assume structural side information where the decision maker knows in advance that there are two types of hidden states; one is common to all arms and evolves according to a Markovian distribution, and the other is unique to each arm and is distributed according to an i.i.d. process that is unique to each arm. We present an algorithm and regret analysis to this problem. Surprisingly, we can recover the hidden states and maintain logarithmic regret in the case of a convex polytope action set. Furthermore, we show that the structural side information leads to expected regret that does not depend on the number of extreme points in the action space. Therefore, we obtain practical solutions even in high dimensional problems.

cs.LG↗

arXiv2020

Proof of Convergence for Correct-Decoding Exponent Computation

For a discrete memoryless channel with finite input and output alphabets, we prove convergence of a parametric family of iterative computations of the optimal correct-decoding exponent. The exponent, as a function of communication rate, is computed for a fixed rate and for a fixed slope.

cs.IT↗

arXiv2019

Generalized Random Gilbert-Varshamov Codes

We introduce a random coding technique for transmission over discrete memoryless channels, reminiscent of the basic construction attaining the Gilbert-Varshamov bound for codes in Hamming spaces. The code construction is based on drawing codewords recursively from a fixed type class, in such a way that a newly generated codeword must be at a certain minimum distance from all previously chosen codewords, according to some generic distance function. We derive an achievable error exponent for this construction, and prove its tightness with respect to the ensemble average. We show that the exponent recovers the Csiszár and K{ö}rner exponent as a special case, which is known to be at least as high as both the random-coding and expurgated exponents, and we establish the optimality of certain choices of the distance function. In addition, for additive distances and decoding metrics, we present an equivalent dual expression, along with a generalization to infinite alphabets via cost-constrained random coding.

cs.IT↗

arXiv2019

Broadcasting Information subject to State Masking over a MIMO State Dependent Gaussian Channel

The problem of channel coding over the Gaussian multiple-input multiple-output (MIMO) broadcast channel (BC) with additive independent Gaussian states is considered. The states are known in a noncausal manner to the encoder, and it wishes to minimize the amount of information that the receivers can learn from the channel outputs about the state sequence. The state leakage rate is measured as a normalized blockwise mutual information between the state sequence and the channel outputs' sequences. We employ a new version of a state-dependent extremal inequality and show that Gaussian input maximizes the state-dependent version of Marton's outer bound. Further we show that our inner bound coincides with the outer bound. Our result generalizes previously studied scalar Gaussian BC with state and MIMO BC without state.

cs.IT↗

arXiv2018

Ratio List Decoding

We extend the notion of list decoding to {\em ratio list decoding} which involves a list decoder whose list size is specified as a function of the number of messages $M_n$ and the block length $n$. We present necessary and sufficient conditions on $M_n$ for the existence of code sequences which enable reliable list decoding with respect to the desired list size $L(M_n,n)$. It is shown that the ratio-capacity, defined as the supremum of achievable normalized logarithms of the ratio $r(M_n,n)=M_n/L(M_n,n)$ is equal to the Shannon channel capacity $C$, for both stochastic and deterministic encoding. Allowing for random list size, we are able to deduce some properties of identification codes, where the decoder's output can be viewed as a list of messages corresponding to decision regions that include the channel output. We further address the regime of mismatched list decoding, in which the list constitutes of the codewords that accumulate the highest score values (jointly with the channel output) according to some given function. We study the case of deterministic encoding and mismatched ratio list decoding. We establish similar necessary and sufficient conditions for the existence of code sequences which enable reliable mismatched list decoding with respect to the desired list size $L(M_n,n)$, and we show that the ratio-capacity with mismatched decoding is equal to the mismatch capacity. Focusing on the case of an exponential list size $L_n=e^{n Θ}$, its comparison with ordinary mismatched decoding shows that the increase in capacity is by $Θ$ bits per channel use for all channels and decoding metrics. Several properties of the average error probability in the setup of mismatched list decoding with deterministic list size are provided.

cs.IT↗

arXiv2018

Converse Theorems for the DMC with Mismatched Decoding

The problem of mismatched decoding with an additive metric $q$ for a discrete memoryless channel $W$ is addressed. The "product-space" improvement of the random coding lower bound on the mismatch capacity, $C_q^{(\infty)}(W)$, was introduced by Csiszár and Narayan. We study two kinds of decoders. The {\it $δ$-margin mismatched decoder} outputs a message whose metric with the channel output exceeds that of all the other codewords by at least $δ$. The {\it $τ$-threshold decoder} outputs a single message whose metric with the channel output exceeds a threshold $τ$. Both decoders declare an error if they fail to find a message that meets the requirement. It is assumed that $q$ is bounded. It is proved that $C_q^{(\infty)}(W)$ is equal to the mismatch capacity with a constant margin decoder. We next consider sequences of $P$-constant composition codebooks, whose empirical distribution of the codewords are at least $o(n^{-1/2})$ close in the $L_1$ distance sense to $P$. Using the Central Limit Theorem, it is shown that for such sequences of codebooks the supremum of achievable rates with constant threshold decoding is upper bounded by the supremum of the achievable rates with a constant margin decoder, and therefore also by $C_q^{(\infty)}(W)$. Further, a soft converse is proved stating that if the average probability of error of a sequence of codebooks converges to zero sufficiently fast, the rate of the code sequence is upper bounded by $C_q^{(\infty)}(W)$. In particular, if $q$ is a bounded rational metric, and the average probability of error converges to zero faster than $O(n^{-1})$, then $R\leq C_q^{(\infty)}(W)$. Finally, a max-min multi-letter upper bound on the mismatch capacity that bears some resemblance to $C_q^{(\infty)}(W)$ is presented.

cs.IT↗

arXiv2016

On the Non-Existence of Unbiased Estimators in Constrained Estimation Problems

We address the problem of existence of unbiased constrained parameter estimators. We show that if the constrained set of parameters is compact and the hypothesized distributions are absolutely continuous with respect to one another, then there exists no unbiased estimator. Weaker conditions for the absence of unbiased constrained estimators are also specified. We provide several examples which demonstrate the utility of these conditions.

math.ST↗

arXiv2016

On the Multiple Access Channel with Asynchronous Cognition

In this paper we introduce the two-user asynchronous cognitive multiple access channel (ACMAC). This channel model includes two transmitters, an uninformed one, and an informed one which knows prior to the beginning of a transmission the message which the uninformed transmitter is about to send. We assume that the channel from the uninformed transmitter to the receiver suffers a fixed but unknown delay. We further introduce a modified model, referred to as the asynchronous codeword cognitive multiple access channel (ACC-MAC), which differs from the ACMAC in that the informed user knows the signal that is to be transmitted by the other user, rather than the message that it is about to transmit. We state inner and outer bounds on the ACMAC and the ACC-MAC capacity regions, and we specialize the results to the Gaussian case. Further, we characterize the capacity regions of these channels in terms of multi-letter expressions. Finally, we provide an example which instantiates the difference between message side-information and codeword side-information.

cs.IT↗

arXiv2016

On Simultaneous Percolation with Two Disk Types

In this paper we consider the simultaneous percolation of two Gilbert disk models. The two models are connected through excluding disks, which prevent elements of the second model to be in the vicinity of the first model. Under these assumptions we characterize the region of densities in which the two models both have a unique infinite connected component. The motivation for this work is the co-existence of two cognitive radio networks.

cs.NI↗

arXiv2015

A Counter-Example to the Mismatched Decoding Converse for Binary-Input Discrete Memoryless Channels

This paper studies the mismatched decoding problem for binary-input discrete memoryless channels. An example is provided for which an achievable rate based on superposition coding exceeds the LM rate (Hui, 1983; Csiszár-Körner, 1981), thus providing a counter-example to a previously reported converse result (Balakirsky, 1995). Both numerical evaluations and theoretical results are used in establishing this claim.

cs.IT↗