arXiv subjects

Haim H. Permuter

Publications and source records attributed to Haim H. Permuter.

At least 37 records · Page 2Linked to original sources

arXiv2017

Feedback Capacity and Coding for the $(0,k)$-RLL Input-Constrained BEC

The input-constrained binary erasure channel (BEC) with strictly causal feedback is studied. The channel input sequence must satisfy the $(0,k)$-runlength limited (RLL) constraint, i.e., no more than $k$ consecutive `$0$'s are allowed. The feedback capacity of this channel is derived for all $k\geq 1$, and is given by $$C^\mathrm{fb}_{(0,k)}(\varepsilon) = \max\frac{\overline{\varepsilon}H_2(δ_0)+\sum_{i=1}^{k-1}\left(\overline{\varepsilon}^{i+1}H_2(δ_i)\prod_{m=0}^{i-1}δ_m\right)}{1+\sum_{i=0}^{k-1}\left(\overline{\varepsilon}^{i+1} \prod_{m=0}^{i}δ_m\right)},$$ where $\varepsilon$ is the erasure probability, $\overline{\varepsilon}=1-\varepsilon$ and $H_2(\cdot)$ is the binary entropy function. The maximization is only over $δ_{k-1}$, while the parameters $δ_i$ for $i\leq k-2$ are straightforward functions of $δ_{k-1}$. The lower bound is obtained by constructing a simple coding for all $k\geq1$. It is shown that the feedback capacity can be achieved using zero-error, variable length coding. For the converse, an upper bound on the non-causal setting, where the erasure is available to the encoder just prior to the transmission, is derived. This upper bound coincides with the lower bound and concludes the search for both the feedback capacity and the non-causal capacity. As a result, non-causal knowledge of the erasures at the encoder does not increase the feedback capacity for the $(0,k)$-RLL input-constrained BEC. This property does not hold in general: the $(2,\infty)$-RLL input-constrained BEC, where every `$1$' is followed by at least two `$0$'s, is used to show that the feedback capacity can be strictly greater than the non-causal capacity.

cs.IT↗

arXiv2017

Extracting analytic proofs from numerically solved Shannon-type Inequalities

A class of information inequalities, called Shannon-type inequalities (STIs), can be proven via a computer software called ITIP. In previous work, we have shown how this technique can be utilized to Fourier-Motzkin elimination algorithm for Information Theoretic Inequalities. Here, we provide an algorithm for extracting \emph{analytic} proofs of information inequalities. Shannon-type inequalities are proven by solving an optimization problem. We will show how to extract a \emph{formal} proof of numerically solved information inequality. Such proof may become useful when an inequality is implied by several constraints due to the PMF, and the proof is not apparent easily. More complicated are cases where an inequality holds due to both constraints from the PMF and due to other constraints that arise from the statistical model. Such cases include information theoretic capacity regions, rate-distortion functions and lossless compression rates. We begin with formal definition of Shannon-type information inequalities. We then review the optimal solution of the optimization problem and how to extract a proof that is readable to the user.

cs.IT↗

arXiv2017

Broadcast Channels with Privacy Leakage Constraints

The broadcast channel (BC) with one common and two private messages with leakage constraints is studied, where leakage rate refers to the normalized mutual information between a message and a channel symbol string. Each private message is destined for a different user and the leakage rate to the other receiver must satisfy a constraint. This model captures several scenarios concerning secrecy, i.e., when both, either or neither of the private messages are secret. Inner and outer bounds on the leakage-capacity region are derived when the eavesdropper knows the codebook. The inner bound relies on a Marton-like code construction and the likelihood encoder. A Uniform Approximation Lemma is established that states that the marginal distribution induced by the encoder on each of the bins in the Marton codebook is approximately uniform. Without leakage constraints the inner bound recovers Marton's region and the outer bound reduces to the UVW-outer bound. The bounds match for semi-deterministic (SD) and physically degraded (PD) BCs, as well as for BCs with a degraded message set. The leakage-capacity regions of the SD-BC and the BC with a degraded message set recover past results for different secrecy scenarios. A Blackwell BC example illustrates the results and shows how its leakage-capacity region changes from the capacity region without secrecy to the secrecy-capacity regions for different secrecy scenarios.

cs.IT↗

arXiv2017

Cooperative Binning for Semi-deterministic Channels with Non-causal State Information

The capacity of the semi-deterministic relay channel (SD-RC) with non-causal channel state information (CSI) only at the encoder and decoder is characterized. The capacity is achieved by a scheme based on cooperative-bin-forward. This scheme allows cooperation between the transmitter and the relay without the need to decode a part of the message by the relay. The transmission is divided into blocks and each deterministic output of the channel (observed by the relay) is mapped to a bin. The bin index is used by the encoder and the relay to choose the cooperation codeword in the next transmission block. In causal settings the cooperation is independent of the state. In \emph{non-causal} settings dependency between the relay's transmission and the state can increase the transmission rates. The encoder implicitly conveys partial state information to the relay. In particular, it uses the states of the next block and selects a cooperation codeword accordingly and the relay transmission depends on the cooperation codeword and therefore also on the states. We also consider the multiple access channel with partial cribbing as a semi-deterministic channel. The capacity region of this channel with non-causal CSI is achieved by the new scheme. Examining the result in several cases, we introduce a new problem of a point-to-point (PTP) channel where the state is provided to the transmitter by a state encoder. Interestingly, even though the CSI is also available at the receiver, we provide an example which shows that the capacity with non-causal CSI at the state encoder is strictly larger than the capacity with causal CSI.

cs.IT↗

arXiv2016

Capacity of Remotely Powered Communication

Motivated by recent developments in wireless power transfer, we study communication with a remotely powered transmitter. We propose an information-theoretic model where a charger can dynamically decide on how much power to transfer to the transmitter based on its side information regarding the communication, while the transmitter needs to dynamically adapt its coding strategy to its instantaneous energy state, which in turn depends on the actions previously taken by the charger. We characterize the capacity as an $n$-letter mutual information rate under various levels of side information available at the charger. When the charger is finely tunable to different energy levels, referred to as a "precision charger", we show that these expressions reduce to single-letter form and there is a simple and intuitive joint charging and coding scheme achieving capacity. The precision charger scenario is motivated by the observation that in practice the transferred energy can be controlled by simply changing the amplitude of the beamformed signal. When the charger does not have sufficient precision, for example when it is restricted to use a few discrete energy levels, we show that the computation of the $n$-letter capacity can be cast as a Markov decision process if the channel is noiseless. This allows us to numerically compute the capacity for specific cases and obtain insights on the corresponding optimal policy, or even to obtain closed form analytical solutions by solving the corresponding Bellman equations, as we demonstrate through examples. Our findings provide some surprising insights on how side information at the charger can be used to increase the overall capacity of the system.

cs.IT↗

arXiv2016

Arbitrarily Varying Wiretap Channels with Type Constrained States

An arbitrarily varying wiretap channel (AVWTC) with a type constraint on the allowed state sequences is considered, and a single-letter characterization of its correlated-random (CR) assisted semantic-security (SS) capacity is derived. The allowed state sequences are the ones in a typical set around a single constraining type. SS is established by showing that the mutual information between the message and the eavesdropper's observations is negligible even when maximized over all message distributions, choices of state sequences and realizations of the CR-code. Both the achievability and the converse proofs of the type constrained coding theorem rely on stronger claims than actually required. The direct part establishes a novel single-letter lower bound on the CR-assisted SS-capacity of an AVWTC with state sequences constrained by any convex and closed set of state probability mass functions. This bound achieves the best known single-letter secrecy rates for a corresponding compound wiretap channel over the same constraint set. In contrast to other single-letter results in the AVWTC literature, this work does not assume the existence of a best channel to the eavesdropper. Instead, SS follows by leveraging the heterogeneous version of the stronger soft-covering lemma and a CR-code reduction argument. Optimality is a consequence of an max-inf upper bound on the CR-assisted SS-capacity of an AVWTC with state sequences constrained to any collection of type-classes. When adjusted to the aforementioned compound WTC, the upper bound simplifies to a max-min structure, thus strengthening the previously best known single-letter upper bound by Liang et al. that has a min-max form. The proof of the upper bound uses a novel distribution coupling argument.

cs.IT↗

arXiv2016

Fourier-Motzkin Elimination Software for Information Theoretic Inequalities

We provide open-source software implemented in MATLAB, that performs Fourier-Motzkin elimination (FME) and removes constraints that are redundant due to Shannon-type inequalities (STIs). The FME is often used in information theoretic contexts to simplify rate regions, e.g., by eliminating auxiliary rates. Occasionally, however, the procedure becomes cumbersome, which makes an error-free hand-written derivation an elusive task. Some computer software have circumvented this difficulty by exploiting an automated FME process. However, the outputs of such software often include constraints that are inactive due to information theoretic properties. By incorporating the notion of STIs (a class of information inequalities provable via a computer program), our algorithm removes such redundant constraints based on non-negativity properties, chain-rules and probability mass function factorization. This newsletter first illustrates the program's abilities, and then reviews the contribution of STIs to the identification of redundant constraints.

cs.IT↗

arXiv2016

Semantic-Security Capacity for Wiretap Channels of Type II

The secrecy capacity of the type II wiretap channel (WTC II) with a noisy main channel is currently an open problem. Herein its secrecy-capacity is derived and shown to be equal to its semantic-security (SS) capacity. In this setting, the legitimate users communicate via a discrete-memoryless (DM) channel in the presence of an eavesdropper that has perfect access to a subset of its choosing of the transmitted symbols, constrained to a fixed fraction of the blocklength. The secrecy criterion is achieved simultaneously for all possible eavesdropper subset choices. The SS criterion demands negligible mutual information between the message and the eavesdropper's observations even when maximized over all message distributions. A key tool for the achievability proof is a novel and stronger version of Wyner's soft covering lemma. Specifically, a random codebook is shown to achieve the soft-covering phenomenon with high probability. The probability of failure is doubly-exponentially small in the blocklength. Since the combined number of messages and subsets grows only exponentially with the blocklength, SS for the WTC II is established by using the union bound and invoking the stronger soft-covering lemma. The direct proof shows that rates up to the weak-secrecy capacity of the classic WTC with a DM erasure channel (EC) to the eavesdropper are achievable. The converse follows by establishing the capacity of this DM wiretap EC as an upper bound for the WTC II. From a broader perspective, the stronger soft-covering lemma constitutes a tool for showing the existence of codebooks that satisfy exponentially many constraints, a beneficial ability for many other applications in information theoretic security.

cs.IT↗

arXiv2016

Duality of a Source Coding Problem and the Semi-Deterministic Broadcast Channel with Rate-Limited Cooperation

The Wyner-Ahlswede-Körner (WAK) empirical-coordination problem where the encoders cooperate via a finite-capacity one-sided link is considered. The coordination-capacity region is derived by combining several source coding techniques, such as Wyner-Ziv (WZ) coding, binning and superposition coding. Furthermore, a semi-deterministic (SD) broadcast channel (BC) with one-sided decoder cooperation is considered. Duality principles relating the two problems are presented, and the capacity region for the SD-BC setting is derived. The direct part follows from an achievable region for a general BC that is tight for the SD scenario. A converse is established by using telescoping identities. The SD-BC is shown to be operationally equivalent to a class of relay-BCs (RBCs) and the correspondence between their capacity regions is established. The capacity region of the SD-BC is transformed into an equivalent region that is shown to be dual to the admissible region of the WAK problem in the sense that the information measures defining the corner points of both regions coincide. Achievability and converse proofs for the equivalent region are provided. For the converse, we use a probabilistic construction of auxiliary random variables that depends on the distribution induced by the codebook. Several examples illustrate the results.

cs.IT↗

arXiv2016

A Single-Letter Upper Bound on the Feedback Capacity of Unifilar Finite-State Channels

An upper bound on the feedback capacity of unifilar finite-state channels (FSCs) is derived. A new technique, called the $Q$-contexts, is based on a construction of a directed graph that is used to quantize recursively the receiver's output sequences to a finite set of contexts. For any choice of $Q$-graph, the feedback capacity is bounded by a single-letter expression, $C_\text{fb}\leq \sup I(X,S;Y|Q)$, where the supremum is over $P_{X|S,Q}$ and the distribution of $(S,Q)$ is their stationary distribution. It is shown that the bound is tight for all unifilar FSCs where feedback capacity is known: channels where the state is a function of the outputs, the trapdoor channel, Ising channels, the no-consecutive-ones input-constrained erasure channel and for the memoryless channel. Its efficiency is also demonstrated by deriving a new capacity result for the dicode erasure channel (DEC); the upper bound is obtained directly from the above expression and its tightness is concluded with a general sufficient condition on the optimality of the upper bound. This sufficient condition is based on a fixed point principle of the BCJR equation and, indeed, formulated as a simple lower bound on feedback capacity of unifilar FSCs for arbitrary $Q$-graphs. This upper bound indicates that a single-letter expression might exist for the capacity of finite-state channels with or without feedback based on a construction of auxiliary random variable with specified structure, such as $Q$-graph, and not with i.i.d distribution. The upper bound also serves as a non-trivial bound on the capacity of channels without feedback, a problem that is still open.

cs.IT↗

arXiv2016

Analogy Between Gambling and Measurement-Based Work Extraction

In information theory, one area of interest is gambling, where mutual information characterizes the maximal gain in wealth growth rate due to knowledge of side information; the betting strategy that achieves this maximum is named the Kelly strategy. In the field of physics, it was recently shown that mutual information can characterize the maximal amount of work that can be extracted from a single heat bath using measurement-based control protocols, i.e., using "information engines". However, to the best of our knowledge, no relation between gambling and information engines has been presented before. In this paper, we briefly review the two concepts and then demonstrate an analogy between gambling, where bits are converted into wealth, and information engines, where bits representing measurements are converted into energy. From this analogy follows an extension of gambling to the continuous-valued case, which is shown to be useful for investments in currency exchange rates or in the stock market using options. Moreover, the analogy enables us to use well-known methods and results from one field to solve problems in the other. We present three such cases: maximum work extraction when the probability distributions governing the system and measurements are unknown, work extraction when some energy is lost in each cycle, e.g., due to friction, and an analysis of systems with memory. In all three cases, the analogy enables us to use known results in order to obtain new ones.

cs.IT↗

arXiv2015

The Feedback Capacity of the $(1,\infty)$-RLL Input-Constrained Erasure Channel

The input-constrained erasure channel with feedback is considered, where the binary input sequence contains no consecutive ones, i.e., it satisfies the $(1,\infty)$-RLL constraint. We derive the capacity for this setting, which can be expressed as $C_ε=\max_{0 \leq p \leq \frac{1}{2}}\frac{H_{b}(p)}{p+\frac{1}{1-ε}}$, where $ε$ is the erasure probability and $ H_{b}(\cdot)$ is the binary entropy function. Moreover, we prove that a-priori knowledge of the erasure at the encoder does not increase the feedback capacity. The feedback capacity was calculated using an equivalent dynamic programming (DP) formulation with an optimal average-reward that is equal to the capacity. Furthermore, we obtained an optimal encoding procedure from the solution of the DP, leading to a capacity-achieving, zero-error coding scheme for our setting. DP is thus shown to be a tool not only for solving optimization problems such as capacity calculation, but also for constructing optimal coding schemes. The derived capacity expression also serves as the only non-trivial upper bound known on the capacity of the input-constrained erasure channel without feedback, a problem that is still open.

cs.IT↗

arXiv2015

On the Capacity of Wireless Networks with Random Transmission Delay

In this paper, we introduce novel coding schemes for wireless networks with random transmission delays. These coding schemes obviate the need for synchronicity, reduce the number of transmissions and achieve the optimal rate region in the corresponding wired model for both multiple unicast and multicast cases with up to three users under the equal rate constraint. The coding schemes are presented in two phases; first, coding schemes for line, star and line-star topologies with random transmission delays are provided. Second, any general topology with multiple bidirectional unicast and multicast sessions is shown to be decomposable into these canonical topologies to reduce the number of transmissions without rate redundancy. As a result, the coding schemes developed for the line, star and line-star topologies serve as building blocks for the construction of more general coding schemes for all networks. The proposed schemes are proved to be Real Time (RT) for wireless networks in the sense that they achieve the minimal decoding delay. With a negligible size header, these coding schemes are shown to be applicable to unsynchronized networks, i.e., networks with random transmission delays. Finally, we demonstrate the applicability of these schemes by extensive simulations. The implementation of such coding schemes on a wireless network with random transmission delay can improve performance and power efficiency.

cs.IT↗

arXiv2015

The Finite State MAC with Cooperative Encoders and Delayed CSI

In this paper, we consider the finite-state multiple access channel (MAC) with partially cooperative encoders and delayed channel state information (CSI). Here partial cooperation refers to the communication between the encoders via finite-capacity links. The channel states are assumed to be governed by a Markov process. Full CSI is assumed at the receiver, while at the transmitters, only delayed CSI is available. The capacity region of this channel model is derived by first solving the case of the finite-state MAC with a common message. Achievability for the latter case is established using the notion of strategies, however, we show that optimal codes can be constructed directly over the input alphabet. This results in a single codebook construction that is then leveraged to apply simultaneous joint decoding. Simultaneous decoding is crucial here because it circumvents the need to rely on the capacity region's corner points, a task that becomes increasingly cumbersome with the growth in the number of messages to be sent. The common message result is then used to derive the capacity region for the case with partially cooperating encoders. Next, we apply this general result to the special case of the Gaussian vector MAC with diagonal channel transfer matrices, which is suitable for modeling, e.g., orthogonal frequency division multiplexing (OFDM)-based communication systems. The capacity region of the Gaussian channel is presented in terms of a convex optimization problem that can be solved efficiently using numerical tools. The region is derived by first presenting an outer bound on the general capacity region and then suggesting a specific input distribution that achieves this bound. Finally, numerical results are provided that give valuable insight into the practical implications of optimally using conferencing to maximize the transmission rates.

cs.IT↗

arXiv2014

Initialization Algorithms For Convolutional Network Coding

We present algorithms for initializing a convolutional network coding scheme in networks that may contain cycles. An initialization process is needed if the network is unknown or if local encoding kernels are chosen randomly. During the initialization process every source node transmits basis vectors and every sink node measures the impulse response of the network. The impulse response is then used to find a relationship between the transmitted and the received symbols, which is needed for a decoding algorithm and to find the set of all achievable rates. Unlike acyclic networks, for which it is enough to transmit basis vectors one after another, the initialization of cyclic networks is more involved, as pilot symbols interfere with each other and the impulse response is of infinite duration.

cs.IT↗

arXiv2014

On State Dependent Broadcast Channels with Cooperation

In this paper, we investigate problems of communication over physically degraded, state-dependent broadcast channels (BCs) with cooperating decoders. Two different setups are considered and their capacity regions are characterized. First, we study a setting in which one decoder can use a finite capacity link to send the other decoder information regarding the messages or the channel states. In this scenario we analyze two cases: one where noncausal state information is available to the encoder and the strong decoder and the other where state information is available only to the encoder in a causal manner. Second, we examine a setting in which the cooperation between the decoders is limited to taking place before the outputs of the channel are given. In this case, one decoder, which is informed of the state sequence noncausally, can cooperate only to send the other decoder rate-limited information about the state sequence. The proofs of the capacity regions introduce a new method of coding for channels with cooperation between different users, where we exploit the link between the decoders for multiple-binning. Finally, we discuss the optimality of using rate splitting techniques when coding for cooperative BCs. In particular, we show that rate splitting is not necessarily optimal when coding for cooperative BCs by solving an example in which our method of coding outperforms rate splitting.

cs.IT↗

arXiv2014

Lossless Coding of Correlated Sources with Actions

This work studies the problem of distributed compression of correlated sources with an action-dependent joint distribution. This class of problems is, in fact, an extension of the Slepian-Wolf model, but where cost-constrained actions taken by the encoder or the decoder affect the generation of one of the sources. The purpose of this work is to study the implications of actions on the achievable rates. In particular, two cases where transmission occurs over a rate-limited link are studied; case A for actions taken at the decoder and case B where actions are taken at the encoder. A complete single-letter characterization of the set of achievable rates is given in both cases. Furthermore, a network coding setup is investigated for the case where actions are taken at the encoder. The sources are generated at different nodes of the network and are required at a set of terminal nodes, yet transmission occurs over a general, acyclic, directed network. For this setup, generalized cut-set bounds are derived, and a full characterization of the set of achievable rates using single-letter expressions is provided. For this scenario, random linear network coding is proved to be optimal, even though this is not a classical multicast problem. Additionally, two binary examples are investigated and demonstrate how actions taken at different nodes of the system have a significant affect on the achievable rate region in comparison to a naive time-sharing strategy.

cs.IT↗

arXiv2013

Capacity of a POST Channel with and without Feedback

We consider finite state channels where the state of the channel is its previous output. We refer to these as POST (Previous Output is the STate) channels. We first focus on POST($α$) channels. These channels have binary inputs and outputs, where the state determines if the channel behaves as a $Z$ or an $S$ channel, both with parameter $α$. %with parameter $α.$ We show that the non feedback capacity of the POST($α$) channel equals its feedback capacity, despite the memory of the channel. The proof of this surprising result is based on showing that the induced output distribution, when maximizing the directed information in the presence of feedback, can also be achieved by an input distribution that does not utilize of the feedback. We show that this is a sufficient condition for the feedback capacity to equal the non feedback capacity for any finite state channel. We show that the result carries over from the POST($α$) channel to a binary POST channel where the previous output determines whether the current channel will be binary with parameters $(a,b)$ or $(b,a)$. Finally, we show that, in general, feedback may increase the capacity of a POST channel.

cs.IT↗