arXiv subjects
Publications and source records attributed to Ram Zamir.
We utilize Lempel-Ziv universal compression for music note generation. We control the algorithm's tendency to over-copy or under-copy training data by manipulating the average sequence length saved in the dictionary.
Universal compression can learn the source and adapt to it either in a batch mode (forward adaptation), or in a sequential mode (backward adaptation). We recast the sequential mode as a multi-armed bandit problem, a fundamental model in reinforcement-learning, and study the trade-off between exploration and exploitation in the lossy compression case. We show that a previously proposed "natural type selection" scheme can be cast as a reconstruction-directed MAB algorithm, for sequential lossy compression, and explain its limitations in terms of robustness and short-block performance. We then derive and analyze robust cost-directed MAB algorithms, which work at any block length.
The inherent trade-off in on-line learning is between exploration and exploitation. A good balance between these two (conflicting) goals can achieve a better long-term performance. Can we define an optimal balance? We propose to study this question through a backward-adaptive lossy compression system, which exhibits a "natural" trade-off between exploration and exploitation.
Analog codes add redundancy by expanding the dimension using real/complex-valued operations. Frame theory provides a mathematical basis for constructing such codes, with diverse applications in non-orthogonal code-division multiple access (NOMA-CDMA), distributed computation, multiple description source coding, space-time coding (STC), and more. The channel model corresponding to these applications is a combination of noise and erasures. Recent analyses showed a useful connection between spectral random-matrix theory and large equiangular tight frames (ETFs) under random uniform erasures. In this work we generalize this model to a channel where the erasures come in blocks. This particularly fits NOMA-CDMA with multiple transmit antennas for each user and STC with known spatial grouping. We present a method to adjust ETF codes to suit block erasures, and find minimum intra-block-correlation frames which outperform ETFs in this setting.
Non-Orthogonal Multiple Access (NOMA) is a concept which is gaining a big popularity in multiuser networks. It's due to its advantages in sense of total network throughput. It becomes especially significant in large networks such as Internet of Things (IoT) networks or 5G networks. One of the known NOMA techniques is DS-CDMA NOMA, which make use of non-orthogonal coding schemes to optimize capacity at multiuser networks. Equiangular Tight Frames (ETF) are known to be an optimal sequences' sets (in sense of capacity) for this technique. Unfortunately, ETFs are limited to very specific pairs of users' number and sequence lengths which put undesirable constraints on practical systems. In this paper our goal is to break those constraints by proposing alternative family of non-orthogonal sequences which on the one hand, possess similar properties to those of ETFs (for that reason we'll denote them as Almost ETFs) and on the other hand, doesn't have those limitation on users' number and sequence length. We're basing our approach by starting with known technique of building standard ETFs, and extending it by slight modifications to technique of building AETFs. In this paper we'll concentrate on bipolar (+/-1 valued) and relatively short (up to length of 100) sequences, since we're interested in sequences which will be of practical value in real systems.
It is well known that independent (separate) encoding of K correlated sources may incur some rate loss compared to joint encoding, even if the decoding is done jointly. This loss is particularly evident in the multiple descriptions problem, where the sources are repetitions of the same source, but each description must be individually good. We observe that under mild conditions about the source and distortion measure, the rate ratio Rindependent(K)/Rjoint goes to one in the limit of small rate/high distortion. Moreover, we consider the excess rate with respect to the rate-distortion function, Rindependent(K, M) - R(D), in M rounds of K independent encodings with a final distortion level D. We provide two examples - a Gaussian source with mean-squared error and an exponential source with one-sided error - for which the excess rate vanishes in the limit as the number of rounds M goes to infinity, for any fixed D and K. This result has an interesting interpretation for a multi-round variant of the multiple descriptions problem, where after each round the encoder gets a (block) feedback regarding which of the descriptions arrived: In the limit as the number of rounds M goes to infinity (i.e., many incremental rounds), the total rate of received descriptions approaches the rate-distortion function. We provide theoretical and experimental evidence showing that this phenomenon is in fact more general than in the two examples above.
Over-complete systems of vectors, or in short, frames, play the role of analog codes in many areas of communication and signal processing. To name a few, spreading sequences for code-division multiple access (CDMA), over-complete representations for multiple-description (MD) source coding, space-time codes, sensing matrices for compressed sensing (CS), and more recently, codes for unreliable distributed computation. In this survey paper we observe an information-theoretic random-like behavior of frame subsets. Such sub-frames arise in setups involving erasures (communication), random user activity (multiple access), or sparsity (signal processing), in addition to channel or quantization noise. The goodness of a frame as an analog code is a function of the eigenvalues of a sub-frame, averaged over all sub-frames. Within the highly symmetric class of Equiangular Tight Frames (ETF), as well as other "near ETF" families, we show a universal behavior of the empirical eigenvalue distribution (ESD) of a randomly-selected sub-frame: (i) the ESD is asymptotically indistinguishable from Wachter's MANOVA distribution; and (ii) it exhibits a convergence rate to this limit that is indistinguishable from that of a matrix sequence drawn from MANOVA (Jacobi) ensembles of corresponding dimensions. Some of these results follow from careful statistical analysis of empirical evidence, and some are proved analytically using random matrix theory arguments of independent interest. The goodness measures of the MANOVA limit distribution are better, in a concrete formal sense, than those of the Marchenko-Pastur distribution at the same aspect ratio, implying that deterministic analog codes are better than random (i.i.d.) analog codes. We further give evidence that the ETF (and near ETF) family is in fact superior to any other frame family in terms of its typical sub-frame goodness.
Distributed computation is a framework used to break down a complex computational task into smaller tasks and distributing them among computational nodes. Erasure correction codes have recently been introduced and have become a popular workaround to the well known ``straggling nodes'' problem, in particular, by matching linear coding for linear computation tasks. It was observed that decoding tends to amplify the computation ``noise'', i.e., the numerical errors at the computation nodes. We propose taking advantage of the case that more nodes return than minimally required. We show how a clever construction of a polynomial code, inspired by recent results on robust frames, can significantly reduce the amplification of noise, and achieves graceful degradation with the number of straggler nodes.
Construction $C^\star$ was recently introduced as a generalization of the multilevel Construction C (or Forney's code-formula), such that the coded levels may be dependent. Both constructions do not produce a lattice in general, hence the central idea of this paper is to present a 3-level lattice Construction $C^\star$ scheme that admits an efficient nearest-neighborhood decoding. In order to achieve this objective, we choose coupled codes for levels 1 and 3, and set the second level code C2 as an independent linear binary self-dual code, which is known to have a rich mathematical structure among families of linear codes. Our main result states a necessary and sufficient condition for this construction to generate a lattice. We then present examples of efficient lattices and also non-lattice constellations with good packing properties.
This note outlines the steps for proving that the moments of a randomly-selected subset of a general ETF (complex, with aspect ratio $0<γ<1$) converge to the corresponding MANOVA moments. We bring here an extension for the proof of the 'Kesten-Mckay' moments (real ETF, $γ=1/2$) \cite{magsino2020kesten}. In particular, we establish a recursive computation of the $r$th moment, for $r = 1,2,\ldots$, and verify, using a symbolic program, that the recursion output coincides with the MANOVA moments.
Non-orthogonal multiple-access (NOMA) is a leading technology which gain a lot of interest this past several years. It enables larger user density and therefore is suited for modern systems such as 5G and IoT. In this paper we examined different frame-based codes for a partially active NOMA system. It is a more realistic setting where only part of the users, in an overly populated system, are active simultaneously. We introduce a new analysis approach were the active user ratio, a system's feature, is kept constant and different sized frames are employed. The frame types were partially derived from previous papers on the subject [1][2] and partially novel such as the LPF and the Steiner ETF. We learned the best capacity achieving frame depends on the active user ratio and three distinct ranges where defined. In addition, we introduced a measure called practical capacity which is the maximal rate achieved by simple coding scheme. ETFs always achieve the best practical capacity while LPFs and sparse frames are worse than a random one.
It was recently proposed to encode the one-sided exponential source X via K parallel channels, Y1, ..., YK , such that the error signals X - Yi, i = 1,...,K, are one-sided exponential and mutually independent given X. Moreover, it was shown that the optimal estimator \hat{Y} of the source X with respect to the one-sided error criterion, is simply given by the maximum of the outputs, i.e., \hat{Y} = max{Y1,..., YK}. In this paper, we show that the distribution of the resulting estimation error X - \hat{Y} , is equivalent to that of the optimum noise in the backward test-channel of the one-sided exponential source, i.e., it is one-sided exponentially distributed and statistically independent of the joint output Y1,...,YK.
It is common to assess the "memory strength" of a stationary process looking at how fast the normalized log-determinant of its covariance submatrices (i.e., entropy rate) decreases. In this work, we propose an alternative characterization in terms of the normalized trace-inverse of the covariance submatrices. We show that this sequence is monotonically non-decreasing and is constant if and only if the process is white. Furthermore, while the entropy rate is associated with one-sided prediction errors (present from past), the new measure is associated with two-sided prediction errors (present from past and future). This measure can be used as an alternative to Burg's maximum-entropy principle for spectral estimation. We also propose a counterpart for non-stationary processes, by looking at the average trace-inverse of subsets.
For a discrete memoryless channel with finite input and output alphabets, we prove convergence of a parametric family of iterative computations of the optimal correct-decoding exponent. The exponent, as a function of communication rate, is computed for a fixed rate and for a fixed slope.
Lattice and special nonlattice multilevel constellations constructed from binary codes, such as Constructions A, C, and D, have relevant applications in Mathematics (sphere packing) and in Communication (multi-stage decoding and efficient vector quantization). In this work, we explore some properties of Construction C, in particular its geometric uniformity. We then propose a new multilevel construction, inspired by bit interleaved coded modulation (BICM), that we call Construction C*. We investigate the geometric uniformity, laticeness, and minimum distance properties of Construction C* and discuss its superior packing efficiency when compared to Construction C.
The matrix version of the entropy-power inequality for real or complex coefficients and variables is proved using a transportation argument that easily settles the equality case. An application to blind source extraction is given.
We consider a channel-independent decoder which is for i.i.d. random codes what the maximum mutual-information decoder is for constant composition codes. We show that this decoder results in exactly the same i.i.d. random coding error exponent and almost the same correct-decoding exponent for a given codebook distribution as the maximum-likelihood decoder. We propose an algorithm for computation of the optimal correct-decoding exponent which operates on the corresponding expression for the channel-independent decoder. The proposed algorithm comes in two versions: computation at a fixed rate and for a fixed slope. The fixed-slope version of the algorithm presents an alternative to the Arimoto algorithm for computation of the random coding exponent function in the correct-decoding regime. The fixed-rate version of the computation algorithm translates into a stochastic iterative algorithm for adaptation of the i.i.d. codebook distribution to a discrete memoryless channel in the limit of large block length. The adaptation scheme uses i.i.d. random codes with the channel-independent decoder and relies on one bit of feedback per transmitted block. The communication itself is assumed reliable at a constant rate $R$. In the end of the iterations the resulting codebook distribution guarantees reliable communication for all rates below $R + Δ$ for some predetermined parameter of decoding confidence $Δ> 0$, provided that $R + Δ$ is less than the channel capacity.
For the model of communication through a discrete memoryless channel using i.i.d. random block codes, where the channel is changing slowly from block to block, we propose a stochastic algorithm for adaptation of the generating distribution of the code in the process of continuous reliable communication. The purpose of the algorithm is to match the generating distribution $Q(x)$ to the changing channel $P(y\,|\,x)$, so that reliable communication is maintained at some constant rate $R$. This is achieved by a feedback of one bit per transmitted block. The feedback bit is determined by the joint type of the last transmitted codeword and the received block, a constant threshold $T>R$, and some conditional distribution $Φ(x\,|\,y)$. Depending on the value of the feedback bit, the system parameters $Q(x)$ and $Φ(x\,|\,y)$ are both updated according to the joint type of the last transmitted and received blocks, or remain unchanged. We show that, under certain technical conditions, the iterations of the algorithm lead to a distribution $Q(x)$, which guarantees reliable communication for all rates below the threshold $T$, provided that the discrete memoryless channel capacity of $P(y\,|\,x)$ stays above $T$.