arXiv subjects

Mladen Kovačević

Publications and source records attributed to Mladen Kovačević.

36 records · Page 2Linked to original sources

arXiv2019

Zero-Error Capacity of Duplication Channels

This paper is concerned with the problem of error-free communication over the i.i.d. duplication channel which acts on a transmitted sequence $ x_1 \cdots x_n $ by inserting a random number of copies of each symbol $ x_i $ next to the original symbol. The random variables representing the numbers of inserted copies at each position $ i $ are independent and take values in $ \{0, 1, \ldots, r\} $, where $ r $ is a fixed parameter. A more general model in which blocks of $ \ell $ consecutive symbols are being duplicated, and which is inspired by DNA-based data storage systems wherein the stored molecules are subject to tandem-duplication mutations, is also analyzed. A construction of optimal codes correcting all patterns of errors of this type is described, and the zero-error capacity of the duplication channel---the largest rate at which information can be transmitted through it in an error-free manner---is determined for each $ \ell $ and $ r $.

cs.IT↗

arXiv2019

Some Enumeration Problems in the Duplication-Loss Model of Genome Rearrangement

Tandem-duplication-random-loss (TDRL) is an important genome rearrangement operation studied in evolutionary biology. This paper investigates some of the formal properties of TDRL operations on the symmetric group (the space of permutations over an $ n $-set). In particular, the cardinality of `balls' of radius one in the TDRL metric, as well as the cardinality of the maximum intersection of two such balls, are determined. The corresponding problems for the so-called mirror (or palindromic) TDRL rearrangement operations are also solved. The results represent an initial step in the study of error correction and reconstruction problems in this context and are of potential interest in DNA-based data storage applications.

cs.DM↗

arXiv2019

Runlength-Limited Sequences and Shift-Correcting Codes: Asymptotic Analysis

This work is motivated by the problem of error correction in bit-shift channels with the so-called $ (d,k) $ input constraints (where successive $ 1 $'s are required to be separated by at least $ d $ and at most $ k $ zeros, $ 0 \leq d < k \leq \infty $). Bounds on the size of optimal $ (d,k) $-constrained codes correcting a fixed number of bit-shifts are derived, with a focus on their asymptotic behavior in the large block-length limit. The upper bound is obtained by a packing argument, while the lower bound follows from a construction based on a family of integer lattices. Several properties of $ (d, k) $-constrained sequences that may be of independent interest are established as well; in particular, the exponential growth-rate of the number of $ (d, k) $-constrained constant-weight sequences is characterized. The results are relevant for magnetic and optical information storage systems, reader-to-tag RFID channels, and other communication models where bit-shift errors are dominant and where $ (d, k) $-constrained sequences are used for modulation.

cs.IT↗

arXiv2019

Fundamental Limits of Communication Over State-Dependent Channels With Feedback

The fundamental limits of communication over state-dependent discrete memoryless channels with noiseless feedback are studied, under the assumption that the communicating parties are allowed to use variable-length coding schemes. Various cases are analyzed, with the employed coding schemes having either bounded or unbounded codeword lengths, and with state information revealed to the encoder and/or decoder in a strictly causal, causal, or non-causal manner. In each of these settings, necessary and sufficient conditions for positivity of the zero-error capacity are obtained and it is shown that, whenever the zero-error capacity is positive, it equals the conventional vanishing-error capacity. Moreover, it is shown that the vanishing-error capacity of state-dependent channels is not increased by the use of feedback and variable-length coding. Both these kinds of capacities of state-dependent channels with feedback are thus fully characterized.

cs.IT↗

arXiv2018

Asymptotically Optimal Codes Correcting Fixed-Length Duplication Errors in DNA Storage Systems

A (tandem) duplication of length $ k $ is an insertion of an exact copy of a substring of length $ k $ next to its original position. This and related types of impairments are of relevance in modeling communication in the presence of synchronization errors, as well as in several information storage applications. We demonstrate that Levenshtein's construction of binary codes correcting insertions of zeros is, with minor modifications, applicable also to channels with arbitrary alphabets and with duplication errors of arbitrary (but fixed) length $ k $. Furthermore, we derive bounds on the cardinality of optimal $ q $-ary codes correcting up to $ t $ duplications of length $ k $, and establish the following corollaries in the asymptotic regime of growing block-length: 1.) the presented family of codes is optimal for every $ q, t, k $, in the sense of the asymptotic scaling of code redundancy; 2.) the upper bound, when specialized to $ q = 2 $, $ k = 1 $, improves upon Levenshtein's bound for every $ t \geq 3 $; 3.) the bounds coincide for $ t = 1 $, thus yielding the exact asymptotic behavior of the size of optimal single-duplication-correcting codes.

cs.IT↗

arXiv2018

Codes in the Space of Multisets---Coding for Permutation Channels with Impairments

Motivated by communication channels in which the transmitted sequences are subject to random permutations, as well as by certain DNA storage systems, we study the error control problem in settings where the information is stored/transmitted in the form of multisets of symbols from a given finite alphabet. A general channel model is assumed in which the transmitted multisets are potentially impaired by insertions, deletions, substitutions, and erasures of symbols. Several constructions of error-correcting codes for this channel are described, and bounds on the size of optimal codes correcting any given number of errors derived. The construction based on the notion of Sidon sets in finite Abelian groups is shown to be optimal, in the sense of the asymptotic scaling of code redundancy, for any "error radius" and any alphabet size. It is also shown to be optimal in the stronger sense of maximal code cardinality in various cases.

cs.IT↗

arXiv2017

A Note on Parallel Asynchronous Channels with Arbitrary Skews

A zero-error coding scheme of asymptotic rate $ \log_2 (1+\sqrt{5}) - 1 $ was recently described for a communication channel composed of parallel asynchronous lines satisfying the so-called no switch assumption. We prove that this is in fact the highest rate attainable, i.e., the zero-error capacity of this channel.

cs.IT↗

arXiv2017

Improved Bounds on Sidon Sets via Lattice Packings of Simplices

A $ B_h $ set (or Sidon set of order $ h $) in an Abelian group $ G $ is any subset $ \{b_0, b_1, \ldots,b_{n}\} $ of $ G $ with the property that all the sums $ b_{i_1} + \cdots + b_{i_h} $ are different up to the order of the summands. Let $ ϕ(h,n) $ denote the order of the smallest Abelian group containing a $ B_h $ set of cardinality $ n + 1 $. It is shown that \[ \lim_{h \to \infty} \frac{ ϕ(h,n) }{ h^n } = \frac{1}{n! δ_L(\triangle^n)} , \] where $ δ_L(\triangle^n) $ is the lattice packing density of an $ n $-simplex in Euclidean space. This determines the asymptotics exactly in cases where this density is known ($ n \leq 3 $) and gives improved bounds on $ ϕ(h,n) $ in the remaining cases. The corresponding geometric characterization of bases of order $ h $ in finite Abelian groups in terms of lattice coverings by simplices is also given.

math.CO↗

arXiv2017

Zero-Error Capacity of $P$-ary Shift Channels and FIFO Queues

The objects of study of this paper are communication channels in which the dominant type of noise are symbol shifts, the main motivating examples being timing and bit-shift channels. Two channel models are introduced and their zero-error capacities and zero-error-detection capacities determined by explicit constructions of optimal codes. Model A can be informally described as follows: 1) The information is stored in an $ n $-cell register, where each cell is either empty or contains a particle of one of $ P $ possible types, and 2) due to the imperfections of the device each of the particles may be shifted several cells away from its original position over time. Model B is an abstraction of a single-server queue: 1) The transmitter sends packets from a $ P $-ary alphabet through a queuing system with an infinite buffer and a First-In-First-Out (FIFO) service procedure, and 2) each packet is being processed by the server for a random number of time slots. More general models including additional types of noise that the particles/packets can experience are also studied, as are the continuous-time versions of these problems.

cs.IT↗

arXiv2017

On Error Detection in Asymmetric Channels

We study the error detection problem in $ q $-ary asymmetric channels wherein every input symbol $ x_i $ is mapped to an output symbol $ y_i $ satisfying $ y_i \geq x_i $. A general setting is assumed where the noise vectors are (potentially) restricted in: 1) the amplitude, $ y_i - x_i \leq a $, 2) the Hamming weight, $ \sum_{i=1}^n 1_{\{y_i \neq x_i\}} \leq h $, and 3) the total weight, $ \sum_{i=1}^n (y_i - x_i) \leq t $. Optimal codes detecting these types of errors are described for certain sets of parameters $ a, h, t $, both in the standard and in the cyclic ($ \operatorname{mod}\, q $) version of the problem. It is also demonstrated that these codes are optimal in the large alphabet limit for every $ a, h, t $ and every block-length $ n $.

cs.IT↗

arXiv2015

Information-Geometric Equivalence of Transportation Polytopes

This paper deals with transportation polytopes in the probability simplex (that is, sets of categorical bivariate probability distributions with prescribed marginals). Information projections between such polytopes are studied, and a sufficient condition is described under which these mappings are homeomorphisms.

cs.IT↗

arXiv2015

On the Entropy of Couplings

In this paper, some general properties of Shannon information measures are investigated over sets of probability distributions with restricted marginals. Certain optimization problems associated with these functionals are shown to be NP-hard, and their special cases are found to be essentially information-theoretic restatements of well-known computational problems, such as the SUBSET SUM and the 3-PARTITION. The notion of minimum entropy coupling is introduced and its relevance is demonstrated in information-theoretic, computational, and statistical contexts. Finally, a family of pseudometrics (on the space of discrete probability distributions) defined by these couplings is studied, in particular their relation to the total variation distance, and a new characterization of the conditional entropy is given.

cs.IT↗

arXiv2014

Zero-Error Capacity of a Class of Timing Channels

We analyze the problem of zero-error communication through timing channels that can be interpreted as discrete-time queues with bounded waiting times. The channel model includes the following assumptions: 1) Time is slotted, 2) at most $ N $ "particles" are sent in each time slot, 3) every particle is delayed in the channel for a number of slots chosen randomly from the set $ \{0, 1, \ldots, K\} $, and 4) the particles are identical. It is shown that the zero-error capacity of this channel is $ \log r $, where $ r $ is the unique positive real root of the polynomial $ x^{K+1} - x^{K} - N $. Capacity-achieving codes are explicitly constructed, and a linear-time decoding algorithm for these codes devised. In the particular case $ N = 1 $, $ K = 1 $, the capacity is equal to $ \log ϕ$, where $ ϕ= (1 + \sqrt{5}) / 2 $ is the golden ratio, and the constructed codes give another interpretation of the Fibonacci sequence.

cs.IT↗

arXiv2013

Perfect Codes in the Discrete Simplex

We study the problem of existence of (nontrivial) perfect codes in the discrete $ n $-simplex $ Δ_{\ell}^n := \left\{ \begin{pmatrix} x_0, \ldots, x_n \end{pmatrix} : x_i \in \mathbb{Z}_{+}, \sum_i x_i = \ell \right\} $ under $ \ell_1 $ metric. The problem is motivated by the so-called multiset codes, which have recently been introduced by the authors as appropriate constructs for error correction in the permutation channels. It is shown that $ e $-perfect codes in the $ 1 $-simplex $ Δ_{\ell}^1 $ exist for any $ \ell \geq 2e + 1 $, the $ 2 $-simplex $ Δ_{\ell}^2 $ admits an $ e $-perfect code if and only if $ \ell = 3e + 1 $, while there are no perfect codes in higher-dimensional simplices. In other words, perfect multiset codes exist only over binary and ternary alphabets.

cs.IT↗

arXiv2013

Subset Codes for Packet Networks

In this paper, we present a coding-theoretic framework for message transmission over packet-switched networks. Network is modeled as a channel which can induce packet errors, deletions, insertions, and out of order delivery of packets. The proposed approach can be viewed as an extension of the one introduced by Koetter and Kschischang for networks based on random linear network coding. Namely, while their framework is based on subspace codes and designed for networks in which network nodes perform random linear combining of the packets, ours is based on the so-called subset codes, and is designed for networks employing routing in network nodes.

cs.IT↗

arXiv2013

Some Properties of Rényi Entropy over Countably Infinite Alphabets

In this paper we study certain properties of Rényi entropy functionals $H_α(\mathcal{P})$ on the space of probability distributions over $\mathbb{Z}_+$. Primarily, continuity and convergence issues are addressed. Some properties shown parallel those known in the finite alphabet case, while others illustrate a quite different behaviour of Rényi entropy in the infinite case. In particular, it is shown that, for any distribution $\mathcal P$ and any $r\in[0,\infty]$, there exists a sequence of distributions $\mathcal{P}_n$ converging to $\mathcal{P}$ with respect to the total variation distance, such that $\lim_{n\to\infty}\lim_{α\to{1+}} H_α(\mathcal{P}_n) = \lim_{α\to{1+}}\lim_{n\to\infty} H_α(\mathcal{P}_n) + r$.

cs.IT↗

arXiv2013

Multiset Codes for Permutation Channels

This paper introduces the notion of multiset codes as relevant to the problem of reliable information transmission over permutation channels. The motivation for studying permutation channels comes from the effect of out of order delivery of packets in some types of packet networks. The proposed codes are a generalization of the so-called subset codes, recently proposed by the authors. Some of the basic properties of multiset codes are established, among which their equivalence to integer codes under the Manhattan metric. The presented coding-theoretic framework follows closely the one proposed by Koetter and Kschischang for the operator channels. The two mathematical models are similar in many respects, and the basic idea is presented in a way which admits a unified view on coding for these types of channels.

cs.IT↗

arXiv2012

On the Hardness of Entropy Minimization and Related Problems

We investigate certain optimization problems for Shannon information measures, namely, minimization of joint and conditional entropies $H(X,Y)$, $H(X|Y)$, $H(Y|X)$, and maximization of mutual information $I(X;Y)$, over convex regions. When restricted to the so-called transportation polytopes (sets of distributions with fixed marginals), very simple proofs of NP-hardness are obtained for these problems because in that case they are all equivalent, and their connection to the well-known \textsc{Subset sum} and \textsc{Partition} problems is revealed. The computational intractability of the more general problems over arbitrary polytopes is then a simple consequence. Further, a simple class of polytopes is shown over which the above problems are not equivalent and their complexity differs sharply, namely, minimization of $H(X,Y)$ and $H(Y|X)$ is trivial, while minimization of $H(X|Y)$ and maximization of $I(X;Y)$ are strongly NP-hard problems. Finally, two new (pseudo)metrics on the space of discrete probability distributions are introduced, based on the so-called variation of information quantity, and NP-hardness of their computation is shown.

cs.IT↗