Searcharxiv⌕ Search

arXiv subjects

Aaron B. Wagner

Publications and source records attributed to Aaron B. Wagner.

At least 37 records · Page 2Linked to original sources

Functional Covering of Point Processes

We introduce a new distortion measure for point processes called functional-covering distortion. It is inspired by intensity theory and is related to both the covering of point processes and logarithmic loss distortion. We obtain the distortion-rate function with feedforward under this distortion measure for a large class of point processes. For Poisson processes, the rate-distortion function is obtained under a general condition called constrained functional-covering distortion, of which both covering and functional-covering are special cases. Also for Poisson processes, we characterize the rate-distortion region for a two-encoder CEO problem and show that feedforward does not enlarge this region.

cs.IT↗

On One-Bit Quantization

We consider the one-bit quantizer that minimizes the mean squared error for a source living in a real Hilbert space. The optimal quantizer is a projection followed by a thresholding operation, and we provide methods for identifying the optimal direction along which to project. As an application of our methods, we characterize the optimal one-bit quantizer for a continuous-time random process that exhibits low-dimensional structure. We numerically show that this optimal quantizer is found by a neural-network-based compressor trained via stochastic gradient descent.

cs.IT↗

Efficiently Computable Converses for Finite-Blocklength Communication

This paper presents a method for computing a finite-blocklength converse for the rate of fixed-length codes with feedback used on discrete memoryless channels (DMCs). The new converse is expressed in terms of a stochastic control problem whose solution can be efficiently computed using dynamic programming and Fourier methods. For channels such as the binary symmetric channel (BSC) and binary erasure channel (BEC), the accuracy of the proposed converse is similar to that of existing special-purpose converse bounds, but the new converse technique can be applied to arbitrary DMCs. We provide example applications of the new converse technique to the binary asymmetric channel (BAC) and the quantized amplitude-constrained AWGN channel.

cs.IT↗

The Rate-Distortion-Perception Tradeoff: The Role of Common Randomness

A rate-distortion-perception (RDP) tradeoff has recently been proposed by Blau and Michaeli and also Matsumoto. Focusing on the case of perfect realism, which coincides with the problem of distribution-preserving lossy compression studied by Li et al., a coding theorem for the RDP tradeoff that allows for a specified amount of common randomness between the encoder and decoder is provided. The existing RDP tradeoff is recovered by allowing for the amount of common randomness to be infinite. The quadratic Gaussian case is examined in detail.

cs.IT↗

Strong Asymptotic Composition Theorems for Mutual Information Measures

We characterize the growth of the Sibson and Arimoto mutual informations and $α$-maximal leakage, of any order that is at least unity, between a random variable and a growing set of noisy, conditionally independent and identically-distributed observations of the random variable. Each of these measures increases exponentially fast to a limit that is order- and measure-dependent, with an exponent that is order- and measure-independent.

cs.IT↗

Principal Bit Analysis: Autoencoding with Schur-Concave Loss

We consider a linear autoencoder in which the latent variables are quantized, or corrupted by noise, and the constraint is Schur-concave in the set of latent variances. Although finding the optimal encoder/decoder pair for this setup is a nonconvex optimization problem, we show that decomposing the source into its principal components is optimal. If the constraint is strictly Schur-concave and the empirical covariance matrix has only simple eigenvalues, then any optimal encoder/decoder must decompose the source in this way. As one application, we consider a strictly Schur-concave constraint that estimates the number of bits needed to represent the latent variables under fixed-rate encoding, a setup that we call \emph{Principal Bit Analysis (PBA)}. This yields a practical, general-purpose, fixed-rate compressor that outperforms existing algorithms. As a second application, we show that a prototypical autoencoder-based variable-rate compressor is guaranteed to decompose the source into its principal components.

cs.IT↗

A coding theorem for the rate-distortion-perception function

The rate-distortion-perception function (RDPF; Blau and Michaeli, 2019) has emerged as a useful tool for thinking about realism and distortion of reconstructions in lossy compression. Unlike the rate-distortion function, however, it is unknown whether encoders and decoders exist that achieve the rate suggested by the RDPF. Building on results by Li and El Gamal (2018), we show that the RDPF can indeed be achieved using stochastic, variable-length codes. For this class of codes, we also prove that the RDPF lower-bounds the achievable rate

cs.IT↗

A Practical Coding Scheme for the BSC with Feedback

We provide a practical implementation of the rubber method of Ahlswede et al. for binary alphabets. The idea is to create the "skeleton" sequence therein via an arithmetic decoder designed for a particular $k$-th order Markov chain. For the stochastic binary symmetric channel, we show that the scheme is nearly optimal in a strong sense for certain parameters.

cs.IT↗

Neural Networks Optimally Compress the Sawbridge

Neural-network-based compressors have proven to be remarkably effective at compressing sources, such as images, that are nominally high-dimensional but presumed to be concentrated on a low-dimensional manifold. We consider a continuous-time random process that models an extreme version of such a source, wherein the realizations fall along a one-dimensional "curve" in function space that has infinite-dimensional linear span. We precisely characterize the optimal entropy-distortion tradeoff for this source and show numerically that it is achieved by neural-network-based compressors trained via stochastic gradient descent. In contrast, we show both analytically and experimentally that compressors based on the classical Karhunen-Loève transform are highly suboptimal at high rates.

cs.IT↗

A Case for Maximal Leakage as a Side Channel Leakage Metric

Side channels represent a broad class of security vulnerabilities that have been demonstrated to exist in many applications. Because completely eliminating side channels often leads to prohibitively high overhead, there is a need for a principled trade-off between cost and leakage. In this paper, we make a case for the use of maximal leakage to analyze such trade-offs. Maximal leakage is an operationally interpretable leakage metric designed for side channels. We present the most useful theoretical properties of maximal leakage from previous work and demonstrate empirically that conventional metrics such as mutual information and channel capacity underestimate the threat posed by side channels whereas maximal leakage does not. We also study the cost-leakage trade-off as an optimization problem using maximal leakage. We demonstrate that not only can this problem be represented as a linear program, but also that optimal protection can be achieved using a combination of at most two deterministic schemes.

cs.IT↗

The Stochastic-Calculus Approach to Multi-Receiver Poisson Channels

We study two-receiver Poisson channels using tools derived from stochastic calculus. We obtain a general formula for the mutual information over the Poisson channel that allows for conditioning and the use of auxiliary random variables. We then use this formula to compute necessary and sufficient conditions under which one Poisson channel is less noisy and/or more capable than another, which turn out to be distinct from the conditions under which this ordering holds for the discretized versions of the channels. We also use general formula to determine the capacity region of the more capable Poisson broadcast channel with independent message sets, the more capable Poisson wiretap channel, and the general two-decoder Poisson broadcast channel with degraded message sets.

cs.IT↗

On exact asymptotics of the error probability in channel coding: symmetric channels

The exact order of the optimal sub-exponentially decaying factor in the classical bounds on the error probability of fixed-length codes over a Gallager-symmetric discrete memoryless channel with and without ideal feedback is determined. Regardless of the availability of feedback, it is shown that the order of the optimal sub-exponential factor exhibits a dichotomy. Moreover, the proof technique is used to establish the third-order term in the normal approximation for symmetric channels, where a similar dichotomy is shown to exist.

cs.IT↗

A New Method for Employing Feedback to Improve Coding Performance

We introduce a novel mechanism, called timid/bold coding, by which feedback can be used to improve coding performance. For a certain class of DMCs, called compound-dispersion channels, we show that timid/bold coding allows for an improved second-order coding rate compared with coding without feedback. For DMCs that are not compound dispersion, we show that feedback does not improve the second-order coding rate. Thus we completely determine the class of DMCs for which feedback improves the second-order coding rate. An upper bound on the second-order coding rate is provided for compound-dispersion DMCs. We also show that feedback does not improve the second-order coding rate for very noisy DMCs. The main results are obtained by relating feedback codes to certain controlled diffusions.

cs.IT↗

An Operational Approach to Information Leakage

Given two random variables $X$ and $Y$, an operational approach is undertaken to quantify the ``leakage'' of information from $X$ to $Y$. The resulting measure $\mathcal{L}(X \!\! \to \!\! Y)$ is called \emph{maximal leakage}, and is defined as the multiplicative increase, upon observing $Y$, of the probability of correctly guessing a randomized function of $X$, maximized over all such randomized functions. A closed-form expression for $\mathcal{L}(X \!\! \to \!\! Y)$ is given for discrete $X$ and $Y$, and it is subsequently generalized to handle a large class of random variables. The resulting properties are shown to be consistent with an axiomatic view of a leakage measure, and the definition is shown to be robust to variations in the setup. Moreover, a variant of the Shannon cipher system is studied, in which performance of an encryption scheme is measured using maximal leakage. A single-letter characterization of the optimal limit of (normalized) maximal leakage is derived and asymptotically-optimal encryption schemes are demonstrated. Furthermore, the sample complexity of estimating maximal leakage from data is characterized up to subpolynomial factors. Finally, the \emph{guessing} framework used to define maximal leakage is used to give operational interpretations of commonly used leakage measures, such as Shannon capacity, maximal correlation, and local differential privacy.

cs.IT↗

A New Stable Peer-to-Peer Protocol with Non-persistent Peers

Recent studies have suggested that the stability of peer-to-peer networks may rely on persistent peers, who dwell on the network after they obtain the entire file. In the absence of such peers, one piece becomes extremely rare in the network, which leads to instability. Technological developments, however, are poised to reduce the incidence of persistent peers, giving rise to a need for a protocol that guarantees stability with non-persistent peers. We propose a novel peer-to-peer protocol, the group suppression protocol, to ensure the stability of peer-to-peer networks under the scenario that all the peers adopt non-persistent behavior. Using a suitable Lyapunov potential function, the group suppression protocol is proven to be stable when the file is broken into two pieces, and detailed experiments demonstrate the stability of the protocol for arbitrary number of pieces. We define and simulate a decentralized version of this protocol for practical applications. Straightforward incorporation of the group suppression protocol into BitTorrent while retaining most of BitTorrent's core mechanisms is also presented. Subsequent simulations show that under certain assumptions, BitTorrent with the official protocol cannot escape from the missing piece syndrome, but BitTorrent with group suppression does.

cs.NI↗

Measuring Quantum Entropy

The entropy of a quantum system is a measure of its randomness, and has applications in measuring quantum entanglement. We study the problem of measuring the von Neumann entropy, $S(ρ)$, and Rényi entropy, $S_α(ρ)$ of an unknown mixed quantum state $ρ$ in $d$ dimensions, given access to independent copies of $ρ$. We provide an algorithm with copy complexity $O(d^{2/α})$ for estimating $S_α(ρ)$ for $α<1$, and copy complexity $O(d^{2})$ for estimating $S(ρ)$, and $S_α(ρ)$ for non-integral $α>1$. These bounds are at least quadratic in $d$, which is the order dependence on the number of copies required for learning the entire state $ρ$. For integral $α>1$, on the other hand, we provide an algorithm for estimating $S_α(ρ)$ with a sub-quadratic copy complexity of $O(d^{2-2/α})$. We characterize the copy complexity for integral $α>1$ up to constant factors by providing matching lower bounds. For other values of $α$, and the von Neumann entropy, we show lower bounds on the algorithm that achieves the upper bound. This shows that we either need new algorithms for better upper bounds, or better lower bounds to tighten the results. For non-integral $α$, and the von Neumann entropy, we consider the well known Empirical Young Diagram (EYD) algorithm, which is the analogue of empirical plug-in estimator in classical distribution estimation. As a corollary, we strengthen a lower bound on the copy complexity of the EYD algorithm for learning the maximally mixed state by showing that the lower bound holds with exponential probability (which was previously known to hold with a constant probability). For integral $α>1$, we provide new concentration results of certain polynomials that arise in Kerov algebra of Young diagrams.

quant-ph↗

Measuring Secrecy by the Probability of a Successful Guess

The secrecy of a communication system in which both the legitimate receiver and an eavesdropper are allowed some distortion is investigated. The secrecy metric considered is the exponent of the probability that the eavesdropper estimates the source sequence successfully within an acceptable distortion level. The problem is first studied when the transmitter and the legitimate receiver do not share any key and the transmitter is not subject to a rate constraint, which corresponds to a stylized model of a side channel and reveals connections to source coding with side information. The setting is then generalized to include a shared secret key between the transmitter and the legitimate receiver and a rate constraint on the transmitter, which corresponds to the Shannon cipher system. A single-letter characterization of the highest achievable exponent is provided, and asymptotically-optimal strategies for both the primary user and the eavesdropper are demonstrated.

cs.IT↗

Vector Gaussian Rate-Distortion with Variable Side Information

We consider rate-distortion with two decoders, each with distinct side information. This problem is well understood when the side information at the decoders satisfies a certain degradedness condition. We consider cases in which this degradedness condition is violated but the source and the side information consist of jointly Gaussian vectors. We provide a hierarchy of four lower bounds on the optimal rate. These bounds are then used to determine the optimal rate for several classes of instances.

cs.IT↗