arXiv subjects

Neri Merhav

Publications and source records attributed to Neri Merhav.

At least 37 records · Page 2Linked to original sources

arXiv2024

Modulation and Estimation with a Helper

The problem of transmitting a parameter value over an additive white Gaussian noise (AWGN) channel is considered, where, in addition to the transmitter and the receiver, there is a helper that observes the noise non-causally and provides a description of limited rate $R_h$ to the transmitter and/or the receiver. We derive upper and lower bounds on the optimal achievable $α$-th moment of the estimation error and show that they coincide for small values of $α$ and for high values of $R_h$. The upper bound relies on a recently proposed channel-coding scheme that effectively conveys $R_h$ bits essentially error-free and the rest of the rate - over the same AWGN channel without help, with the error-free bits being allocated to the most significant bits of the quantized parameter. We then concentrate on the setting with a total transmit energy constraint, for which we derive achievability results for both channel coding and parameter modulation for several scenarios: when the helper assists only the transmitter or only the receiver and knows the noise, and when the helper assists the transmitter and/or the receiver and knows both the noise and the message. In particular, for the message-informed helper that assists both the receiver and the transmitter, it is shown that the error probability in the channel-coding task decays doubly exponentially. Finally, we translate these results to those for continuous-time power-limited AWGN channels with unconstrained bandwidth. As a byproduct, we show that the capacity with a message-informed helper that is available only at the transmitter can exceed the sum of the capacity without help and the help rate $R_h$, when the helper knows only the noise but not the message.

cs.IT↗

arXiv2024

Universal Slepian-Wolf coding for individual sequences

We establish a coding theorem and a matching converse theorem for separate encodings and joint decoding of individual sequences using finite-state machines. The achievable rate region is characterized in terms of the Lempel-Ziv (LZ) complexities, the conditional LZ complexities and the joint LZ complexity of the two source sequences. An important feature that is needed to this end, which may be interesting on its own right, is a certain asymptotic form of a chain rule for LZ complexities, which we establish in this work. The main emphasis in the achievability scheme is on the universal decoder and its properties. We then show that the achievable rate region is universally attainable by a modified version of Draper's universal incremental Slepian-Wolf (SW) coding scheme, provided that there exists a low-rate reliable feedback link.

cs.IT↗

arXiv2024

Lossy Compression of Individual Sequences Revisited: Fundamental Limits of Finite-State Encoders

We extend Ziv and Lempel's model of finite-state encoders to the realm of lossy compression of individual sequences. In particular, the model of the encoder includes a finite-state reconstruction codebook followed by an information lossless finite-state encoder that compresses the reconstruction codeword with no additional distortion. We first derive two different lower bounds to the compression ratio that depend on the number of states of the lossless encoder. Both bounds are asymptotically achievable by conceptually simple coding schemes. We then show that when the number of states of the lossless encoder is large enough in terms of the reconstruction block-length, the performance can be improved, sometimes significantly so. In particular, the improved performance is achievable using a random-coding ensemble that is universal, not only in terms of the source sequence, but also in terms of the distortion measure.

cs.IT↗

arXiv2023

Trade-offs Between Weak-Noise Performance and Probability of Anomaly in Parameter Estimation from Noisy Chaotic Signals

We consider the problem of parameter estimation, based on noisy chaotic signals, from the viewpoint of twisted modulation for waveform communication. In particular, we study communication systems where the parameter to be estimated is conveyed as the initial condition of a chaotic dynamical system of a certain class and we examine its estimation performance in terms of the expectation of a given convex function of the estimation error at high SNR, under the demand that the probability of anomaly is kept small. We derive a lower bound on the weak-noise estimation error for this class of chaotic modulators, and argue that it can be outperformed by using the itinerary signal associated with the chaotic system instead of the main chaotic output signal.

cs.IT↗

arXiv2023

Some families of Jensen-like inequalities with application to information theory

It is well known that the traditional Jensen inequality is proved by lower bounding the given convex function, $f(x)$, by the tangential affine function that passes through the point $(E\{X\},f(E\{X\}))$, where $E\{X\}$ is the expectation of the random variable $X$. While this tangential affine function yields the tightest lower bound among all lower bounds induced by affine functions that are tangential to $f$, it turns out that when the function $f$ is just part of a more complicated expression whose expectation is to be bounded, the tightest lower bound might belong to a tangential affine function that passes through a point different than $(E\{X\},f(E\{X\}))$. In this paper, we take advantage of this observation, by optimizing the point of tangency with regard to the specific given expression, in a variety of cases, and thereby derive several families of inequalities, henceforth referred to as ``Jensen-like'' inequalities, which are new to the best knowledge of the author. The degree of tightness and the potential usefulness of these inequalities is demonstrated in several application examples related to information theory.

cs.IT↗

arXiv2022

A Universal Random Coding Ensemble for Sample-wise Lossy Compression

We propose a universal ensemble for random selection of rate-distortion codes, which is asymptotically optimal in a sample-wise sense. According to this ensemble, each reproduction vector, $\hbx$, is selected independently at random under the probability distribution that is proportional to $2^{-LZ(\hbx)}$, where $LZ(\hbx)$ is the code-length of $\hbx$ pertaining to the 1978 version of the Lempel-Ziv (LZ) algorithm. We show that, with high probability, the resulting codebook gives rise to an asymptotically optimal variable-rate lossy compression scheme under an arbitrary distortion measure, in the sense that a matching converse theorem also holds. According to the converse theorem, even if the decoder knew $\ell$-th order type of source vector in advance ($\ell$ being a large but fixed positive integer), the performance of the above-mentioned code could not have been improved essentially, for the vast majority of codewords that represent all source vectors in the same type. Finally, we provide a discussion of our results, which includes, among other things, a comparison to a coding scheme that selects the reproduction vector with the shortest LZ code length among all vectors that are within the allowed distortion from the source vector.

cs.IT↗

arXiv2022

The Secrecy Capacity of The Gaussian Wiretap Channel with Rate-Limited Help

The Gaussian wiretap channel with rate-limited help, available at the legitimate receiver (Rx) or/and transmitter (Tx), is studied under various channel configurations (degraded, reversely degraded and non-degraded). In the case of Rx help and all channel configurations, the rate-limited help results in a secrecy capacity boost equal to the help rate irrespective of whether the help is secure or not, so that the secrecy of help does not provide any capacity increase. The secrecy capacity is positive for the reversely-degraded channel (where the no-help secrecy capacity is zero) and no wiretap coding is needed to achieve it. More noise at the legitimate receiver can sometimes result in higher secrecy capacity. The secrecy capacity with Rx help is not increased even if the helper is aware of the message being transmitted. The same secrecy capacity boost also holds if non-secure help is available to the transmitter (encoder), in addition to or instead of the same Rx help, so that, in the case of the joint Tx/Rx help, one help link can be omitted without affecting the capacity. If Rx/Tx help links are independent of each other, then the boost in the secrecy capacity is the sum of help rates and no link can be omitted without a loss in the capacity. Non-singular correlation of the receiver and eavesdropper noises does not affect the secrecy capacity and non-causal help does not bring in any capacity increase over the causal one.

cs.IT↗

arXiv2022

Codebook Mismatch Can Be Fully Compensated by Mismatched Decoding

We consider an ensemble of constant composition codes that are subsets of linear codes: while the encoder uses only the constant-composition subcode, the decoder operates as if the full linear code was used, with the motivation of simultaneously benefiting both from the probabilistic shaping of the channel input and from the linear structure of the code. We prove that the codebook mismatch can be fully compensated by using a mismatched additive decoding metric that achieves the random coding error exponent of (non-linear) constant composition codes. As the coding rate tends to the mutual information, the optimal mismatched metric approaches the maximum a posteriori probability (MAP) metric, showing that codebook mismatch with mismatched MAP metric is capacity-achieving for the optimal input assignment.

cs.IT↗

arXiv2022

Error Exponents of the Dirty-Paper and Gel'fand-Pinsker Channels

We derive various error exponents for communication channels with random states, which are available non-causally at the encoder only. For both the finite-alphabet Gel'fand-Pinsker channel and its Gaussian counterpart, the dirty-paper channel, we derive random coding exponents, error exponents of the typical random codes (TRCs), and error exponents of expurgated codes. For the two channel models, we analyze some sub-optimal bin-index decoders, which turn out to be asymptotically optimal, at least for the random coding error exponent. For the dirty-paper channel, we show explicitly via a numerical example, that both the error exponent of the TRC and the expurgated exponent strictly improve upon the random coding exponent, at relatively low coding rates, which is a known fact for discrete memoryless channels without random states. We also show that at rates below capacity, the optimal values of the dirty-paper design parameter $α$ in the random coding sense and in the TRC exponent sense are different from one another, and they are both different from the optimal $α$ that is required for attaining the channel capacity. For the Gel'fand-Pinsker channel, we allow for a variable-rate random binning code construction, and prove that the previously proposed maximum penalized mutual information decoder is asymptotically optimal within a given class of decoders, at least for the random coding error exponent.

cs.IT↗

arXiv2022

$D$-semifaithful codes that are universal over both memoryless sources and distortion measures

We prove the existence of codebooks for d-semifaithful lossy compression that are simultaneously universal with respect to both the class of finite-alphabet memoryless sources and the class of all bounded additive distortion measures. By applying independent random selection of the codewords according to a mixture of all memoryless sources, we achieve redundancy rates that are within O(log n/n) close to the empirical rate-distortion function of every given source vector with respect to every bounded distortion measure. As outlined in the last section, the principal ideas can also be extended significantly beyond the class of memoryless sources, namely, to the setting of individual sequences encoded by finite-state machines.

cs.IT↗

arXiv2022

The DNA Storage Channel: Capacity and Error Probability

The DNA storage channel is considered, in which the $M$ Deoxyribonucleic acid (DNA) molecules comprising each codeword are stored without order, sampled $N$ times with replacement, and then sequenced over a discrete memoryless channel. For a constant coverage depth $M/N$ and molecule length scaling $Θ(\log M)$, lower (achievability) and upper (converse) bounds on the capacity of the channel, as well as a lower (achievability) bound on the reliability function of the channel are provided. Both the lower and upper bounds on the capacity generalize a bound which was previously known to hold only for the binary symmetric sequencing channel, and only under certain restrictions on the molecule length scaling and the crossover probability parameters. When specified to binary symmetric sequencing channel, these restrictions are completely removed for the lower bound and are significantly relaxed for the upper bound in the high-noise regime. The lower bound on the reliability function is achieved under a universal decoder, and reveals that the dominant error event is that of outage -- the event in which the capacity of the channel induced by the DNA molecule sampling operation does not support the target rate.

cs.IT↗

arXiv2022

Optimal Correlators and Waveforms for Mismatched Detection

We consider the classical Neymann-Pearson hypothesis testing problem of signal detection, where under the null hypothesis ($\calH_0$), the received signal is white Gaussian noise, and under the alternative hypothesis ($\calH_1$), the received signal includes also an additional non-Gaussian random signal, which in turn can be viewed as a deterministic waveform plus zero-mean, non-Gaussian noise. However, instead of the classical likelihood ratio test detector, which might be difficult to implement, in general, we impose a (mismatched) correlation detector, which is relatively easy to implement, and we characterize the optimal correlator weights in the sense of the best trade-off between the false-alarm error exponent and the missed-detection error exponent. Those optimal correlator weights depend (non-linearly, in general) on the underlying deterministic waveform under $\calH_1$. We then assume that the deterministic waveform may also be free to be optimized (subject to a power constraint), jointly with the correlator, and show that both the optimal waveform and the optimal correlator weights may take on values in a small finite set of typically no more than two to four levels, depending on the distribution of the non-Gaussian noise component. Finally, we outline an extension of the scope to a wider class of detectors that are based on linear combinations of the correlation and the energy of the received signal.

cs.IT↗

arXiv2021

Universal Randomized Guessing Subjected to Distortion

In this paper, we consider the problem of guessing a sequence subject to a distortion constraint. Specifically, we assume the following game between Alice and Bob: Alice has a sequence $\bx$ of length $n$. Bob wishes to guess $\bx$, yet he is satisfied with finding any sequence $\hat{\bx}$ which is within a given distortion $D$ from $\bx$. Thus, he successively submits queries to Alice, until receiving an affirmative answer, stating that his guess was within the required distortion. Finding guessing strategies which minimize the number of guesses (the \emph{guesswork}), and analyzing its properties (e.g., its $ρ$--th moment) has several applications in information security, source and channel coding. Guessing subject to a distortion constraint is especially useful when considering contemporary biometrically--secured systems, where the "password" which protects the data is not a single, fixed vector but rather a \emph{ball of feature vectors} centered at some $\bx$, and any feature vector within the ball results in acceptance. We formally define the guessing problem under distortion in \emph{four different setups}: memoryless sources, guessing through a noisy channel, sources with memory and individual sequences. We suggest a randomized guessing strategy which is asymptotically optimal for all setups and is \emph{five--fold universal}, as it is independent of the source statistics, the channel, the moment to be optimized, the distortion measure and the distortion level.

cs.IT↗

arXiv2021

Encoding Individual Source Sequences for the Wiretap Channel

We consider the problem of encoding a deterministic source sequence (a.k.a.\ individual sequence) for the degraded wiretap channel by means of an encoder and decoder that can both be implemented as finite--state machines. Our first main result is a necessary condition for both reliable and secure transmission in terms of the given source sequence, the bandwidth expansion factor, the secrecy capacity, the number of states of the encoder and the number of states of the decoder. Equivalently, this necessary condition can be presented as a converse bound (i.e., a lower bound) on the smallest achievable bandwidth expansion factor. The bound is asymptotically achievable by Lempel-Ziv compression followed by good channel coding for the wiretap channel. Given that the lower bound is saturated, we also derive a lower bound on the minimum necessary rate of purely random bits needed for local randomness at the encoder in order to meet the security constraint. This bound too is achieved by the same achievability scheme. Finally, we extend the main results to the case where the legitimate decoder has access to a side information sequence, which is another individual sequence that may be related to the source sequence, and a noisy version of the side information sequence leaks to the wiretapper.

cs.IT↗

arXiv2021

Finite-state source-channel coding for individual sequences with source side information at the decoder

We study the following semi-deterministic setting of the joint source-channel coding problem: a deterministic source sequence (a.k.a. individual sequence) is transmitted via a memoryless channel, using delay-limited encoder and decoder, which are both implementable by periodically-varying finite-state machines, and the decoder is granted with access to side information, which is a noisy version of the source sequence. We first derive a lower bound on the achievable expected distortion in terms of the empirical statistics of the source sequence, the number of states of the encoder, the number of states of the decoder, their period, and the overall delay. The bound is shown to be asymptotically achievable by universal block codes in the limit of long blocks. We also derive a lower bound to the best achievable excess--distortion probability and discuss situations where it is achievable. Here, of course, source coding and channel coding cannot be completely separated without loss of optimality. Finally, we outline a few extensions of the model considered, such as: (i) incorporating a common reconstruction constraint, (ii) availability of side information at both ends, and (iii) extension to the Shannon channel with causal state information at the encoder. This work both extends and improves on earlier work of the same flavor (Ziv 1980, Merhav 2014), which focused only on the expected distortion, without side information at either end, and without the above mentioned additional ingredients.

cs.IT↗

arXiv2021

Guessing Based on Compressed Side Information

A source sequence is to be guessed with some fidelity based on a rate-limited description of an observed sequence with which it is correlated. The trade-off between the description rate and the exponential growth rate of the least power mean of the number of guesses is characterized.

cs.IT↗

arXiv2021

Trade-offs Between Error Exponents and Excess-Rate Exponents of Typical Slepian-Wolf Codes

Typical random codes (TRC) in a communication scenario of source coding with side information at the decoder is the main subject of this work. We study the semi-deterministic code ensemble, which is a certain variant of the ordinary random binning code ensemble. In this code ensemble, the relatively small type classes of the source are deterministically partitioned into the available bins in a one-to-one manner. As a consequence, the error probability decreases dramatically. The random binning error exponent and the error exponent of the TRC are derived and proved to be equal to one another in a few important special cases. We show that the performance under optimal decoding can be attained also by certain universal decoders, e.g., the stochastic likelihood decoder with an empirical entropy metric. Moreover, we discuss the trade-offs between the error exponent and the excess-rate exponent for the typical random semi-deterministic code and characterize its optimal rate function. We show that for any pair of correlated information sources, both error and excess-rate probabilities are exponentially vanishing when the blocklength tends to infinity.

cs.IT↗

arXiv2020

On Error Exponents of Encoder-Assisted Communication Systems

We consider a point-to-point communication system, where in addition to the encoder and the decoder, there is a helper that observes non-causally the realization of the noise vector and provides a (lossy) rate-$R_{\mbox{\tiny h}}$ description of it to the encoder ($R_{\mbox{\tiny h}} < \infty$). While Lapidoth and Marti (2020) derived coding theorems, associated with achievable channel-coding rates (of the main encoder) for this model, here our focus is on error exponents. We consider both continuous-alphabet, additive white Gaussian channels and finite-alphabet, modulo-additive channels, and for each one of them, we study the cases of both fixed-rate and variable-rate noise descriptions by the helper. Our main finding is that, as long as the channel-coding rate, $R$, is below the helper-rate, $R_{\mbox{\tiny h}}$, the achievable error exponent is unlimited (i.e., it can be made arbitrarily large), and in some of the cases, it is even strictly infinite (i.e., the error probability can be made strictly zero). However, in the range of coding rates $(R_{\mbox{\tiny h}},R_{\mbox{\tiny h}}+C_0)$, $C_0$ being the ordinary channel capacity (without help), the best achievable error exponent is finite and strictly positive, although there is a certain gap between our upper bound (converse bound) and lower bound (achievability) on the highest achievable error exponent. This means that the model of encoder-assisted communication is essentially equivalent to a model, where in addition to the noisy channel between the encoder and decoder, there is also a parallel noiseless bit-pipe of capacity $R_{\mbox{\tiny h}}$. We also extend the scope to the Gaussian multiple access channel (MAC) and characterize the rate sub-region, where the achievable error exponent is unlimited or even infinite.

cs.IT↗