Searcharxiv⌕ Search

arXiv subjects

Tamas Linder

Publications and source records attributed to Tamas Linder.

14 recordsLinked to original sources
arXiv2023

Reinforcement Learning for Near-Optimal Design of Zero-Delay Codes for Markov Sources

In the classical lossy source coding problem, one encodes long blocks of source symbols that enables the distortion to approach the ultimate Shannon limit. Such a block-coding approach introduces large delays, which is undesirable in many delay-sensitive applications. We consider the zero-delay case, where the goal is to encode and decode a finite-alphabet Markov source without any delay. It has been shown that this problem lends itself to stochastic control techniques, which lead to existence, structural, and general structural approximation results. However, these techniques so far have resulted only in computationally prohibitive algorithmic implementations for code design. To address this problem, we present a reinforcement learning design algorithm and rigorously prove its asymptotic optimality. In particular, we show that a quantized Q-learning algorithm can be used to obtain a near-optimal coding policy for this problem. The proof builds on recent results on quantized Q-learning for weakly Feller controlled Markov chains whose application necessitates the development of supporting technical results on regularity and stability properties, and relating the optimal solutions for discounted and average cost infinite horizon criteria problems. These theoretical results are supported by simulations.

cs.IT↗
arXiv2022

Logarithmic Regret in Adaptive Control of Noisy Linear Quadratic Regulator Systems Using Hints

The problem of regret minimization for online adaptive control of linear-quadratic systems is studied. In this problem, the true system transition parameters (matrices $A$ and $B$) are unknown, and the objective is to design and analyze algorithms that generate control policies with sublinear regret. Recent studies show that when the system parameters are fully unknown, there exists a choice of these parameters such that any algorithm that only uses data from the past system trajectory at best achieves a square root of time horizon regret bound, providing a hard fundamental limit on the achievable regret in general. However, it is also known that (poly)-logarithmic regret is achievable when only matrix $A$ or only matrix $B$ is unknown. We present a result, encompassing both scenarios, showing that (poly)-logarithmic regret is achievable when both of these matrices are unknown, but a hint is periodically given to the controller.

math.OC↗
arXiv2022

Zero-Delay Lossy Coding of Linear Vector Markov Sources: Optimality of Stationary Codes and Near Optimality of Finite Memory Codes

Optimal zero-delay coding (quantization) of $\mathbb{R}^d$-valued linearly generated Markov sources is studied under quadratic distortion. The structure and existence of deterministic and stationary coding policies that are optimal for the infinite horizon average cost (distortion) problem are established. Prior results studying the optimality of zero-delay codes for Markov sources for infinite horizons either considered finite alphabet sources or, for the $\mathbb{R}^d$-valued case, only showed the existence of deterministic and non-stationary Markov coding policies or those which are randomized. In addition to existence results, for finite blocklength (horizon) $T$ the performance of an optimal coding policy is shown to approach the infinite time horizon optimum at a rate $O(\frac{1}{T})$. This gives an explicit rate of convergence that quantifies the near-optimality of finite window (finite-memory) codes among all optimal zero-delay codes.

cs.IT↗
arXiv2021

An Iterative Riccati Algorithm for Online Linear Quadratic Control

An online policy learning problem of linear control systems is studied. In this problem, the control system is known and linear, and a sequence of quadratic cost functions is revealed to the controller in hindsight, and the controller updates its policy to achieve a sublinear regret, similar to online optimization. A modified online Riccati algorithm is introduced that under some boundedness assumption leads to logarithmic regret bound. In particular, the logarithmic regret for the scalar case is achieved without boundedness assumption. Our algorithm, while achieving a better regret bound, also has reduced complexity compared to earlier algorithms which rely on solving semi-definite programs at each stage.

math.OC↗
arXiv2018

Estimation Efficiency Under Privacy Constraints

We investigate the problem of estimating a random variable $Y\in \mathcal{Y}$ under a privacy constraint dictated by another random variable $X\in \mathcal{X}$, where estimation efficiency and privacy are assessed in terms of two different loss functions. In the discrete case, we use the Hamming loss function and express the corresponding utility-privacy tradeoff in terms of the privacy-constrained guessing probability $h(P_{XY}, ε)$, the maximum probability $\mathsf{P}_\mathsf{c}(Y|Z)$ of correctly guessing $Y$ given an auxiliary random variable $Z\in \mathcal{Z}$, where the maximization is taken over all $P_{Z|Y}$ ensuring that $\mathsf{P}_\mathsf{c}(X|Z)\leq ε$ for a given privacy threshold $ε\geq 0$. We prove that $h(P_{XY}, \cdot)$ is concave and piecewise linear, which allows us to derive its expression in closed form for any $ε$ when $X$ and $Y$ are binary. In the non-binary case, we derive $h(P_{XY}, ε)$ in the high utility regime (i.e., for sufficiently large values of $ε$) under the assumption that $Z$ takes values in $\mathcal{Y}$. We also analyze the privacy-constrained guessing probability for two binary vector scenarios. When $X$ and $Y$ are continuous random variables, we use the squared-error loss function and express the corresponding utility-privacy tradeoff in terms of $\mathsf{sENSR}(P_{XY}, ε)$, which is the smallest normalized minimum mean squared-error (mmse) incurred in estimating $Y$ from its Gaussian perturbation $Z$, such that the mmse of $f(X)$ given $Z$ is within $ε$ of the variance of $f(X)$ for any non-constant real-valued function $f$. We derive tight upper and lower bounds for $\mathsf{sENSR}$ when $Y$ is Gaussian. We also obtain a tight lower bound for $\mathsf{sENSR}(P_{XY}, ε)$ for general absolutely continuous random variables when $ε$ is sufficiently small.

cs.IT↗
arXiv2017

Push-sum on random graphs

In this paper, we study the problem of achieving average consensus over a random time-varying sequence of directed graphs by extending the class of so-called push-sum algorithms to such random scenarios. Provided that an ergodicity notion, which we term the directed infinite flow property, holds and the auxiliary states of agents are uniformly bounded away from zero infinitely often, we prove the almost sure convergence of the evolutions of this class of algorithms to the average of initial states. Moreover, for a random sequence of graphs generated using a time-varying B-irreducible probability matrix, we establish convergence rates for the proposed push-sum algorithm.

math.OC↗
arXiv2016

Almost Perfect Privacy for Additive Gaussian Privacy Filters

We study the maximal mutual information about a random variable $Y$ (representing non-private information) displayed through an additive Gaussian channel when guaranteeing that only $ε$ bits of information is leaked about a random variable $X$ (representing private information) that is correlated with $Y$. Denoting this quantity by $g_ε(X,Y)$, we show that for perfect privacy, i.e., $ε=0$, one has $g_0(X,Y)=0$ for any pair of absolutely continuous random variables $(X,Y)$ and then derive a second-order approximation for $g_ε(X,Y)$ for small $ε$. This approximation is shown to be related to the strong data processing inequality for mutual information under suitable conditions on the joint distribution $P_{XY}$. Next, motivated by an operational interpretation of data privacy, we formulate the privacy-utility tradeoff in the same setup using estimation-theoretic quantities and obtain explicit bounds for this tradeoff when $ε$ is sufficiently small using the approximation formula derived for $g_ε(X,Y)$.

cs.IT↗
arXiv2016

Adaptation is Useless for Two Discrete Additive-Noise Two-Way Channels

In two-way channels, each user transmits and receives at the same time. This allows each encoder to interactively adapt the current input to its own message and all previously received signals. Such coding approach can introduce correlation between inputs of different users, since all the users' outputs are correlated by the nature of the channel. However, for some channels, such adaptation in the coding scheme and its induced correlation among users are useless in the sense that they do not help enlarge the capacity region with respect to the standard coding method (where each user encodes only based on its own message). In this paper, it is shown that adaptation is not helpful for enlarging the capacity region of two classes of two-way discrete channels: the modulo additive-noise channel with memory and the multiple access/degraded broadcast channel.

cs.IT↗
arXiv2012

Entropy Density and Mismatch in High-Rate Scalar Quantization with Renyi Entropy Constraint

Properties of scalar quantization with $r$th power distortion and constrained Rényi entropy of order $α\in (0,1)$ are investigated. For an asymptotically (high-rate) optimal sequence of quantizers, the contribution to the Rényi entropy due to source values in a fixed interval is identified in terms of the "entropy density" of the quantizer sequence. This extends results related to the well-known point density concept in optimal fixed-rate quantization. A dual of the entropy density result quantifies the distortion contribution of a given interval to the overall distortion. The distortion loss resulting from a mismatch of source densities in the design of an asymptotically optimal sequence of quantizers is also determined. This extends Bucklew's fixed-rate ($α=0$) and Gray \emph{et al.}'s variable-rate ($α=1$) mismatch results to general values of the entropy order parameter $α$.

cs.IT↗
arXiv2011

High-resolution scalar quantization with Rényi entropy constraint

We consider optimal scalar quantization with $r$th power distortion and constrained Rényi entropy of order $α$. For sources with an absolutely continuous distribution the high rate asymptotics of the quantizer distortion has long been known for $α=0$ (fixed-rate quantization) and $\al pha=1$ (entropy-constrained quantization). For a large class of absolutely continuous source distributions we determine the sharp asymptotics of the optimal quantization distortion for $α\in [-\infty,0)\cup (0,1)$. The achievability proof is based on finding (asymptotically) optimal quantizers via the companding approach, and is thus constructive.

cs.IT↗
arXiv2011

Rate-Constrained Simulation and Source Coding IID Sources

Necessary conditions for asymptotically optimal sliding-block or stationary codes for source coding and rate-constrained simulation of memoryless sources are presented and used to motivate a design technique for trellis-encoded source coding and rate-constrained simulation. The code structure has intuitive similarities to classic random coding arguments as well as to ``fake process'' methods and alphabet-constrained methods. Experimental evidence shows that the approach provides comparable or superior performance in comparison with previously published methods on common examples, sometimes by significant margins.

cs.IT↗
arXiv2010

Inner and Outer Bounds for the Public Information Embedding Capacity Region Under Multiple Access Attacks

We consider a public multi-user information embedding (watermarking) system in which two messages (watermarks) are independently embedded into two correlated covertexts and are transmitted through a multiple-access attack channel. The tradeoff between the achievable embedding rates and the average distortions for the two embedders is studied. For given distortion levels, inner and outer bounds for the embedding capacity region are obtained in single-letter form. Tighter bounds are also given for independent covertexts.

cs.IT↗
arXiv2007

The on-line shortest path problem under partial monitoring

The on-line shortest path problem is considered under various models of partial monitoring. Given a weighted directed acyclic graph whose edge weights can change in an arbitrary (adversarial) way, a decision maker has to choose in each round of a game a path between two distinguished vertices such that the loss of the chosen path (defined as the sum of the weights of its composing edges) be as small as possible. In a setting generalizing the multi-armed bandit problem, after choosing a path, the decision maker learns only the weights of those edges that belong to the chosen path. For this problem, an algorithm is given whose average cumulative loss in n rounds exceeds that of the best path, matched off-line to the entire sequence of the edge weights, by a quantity that is proportional to 1/\sqrt{n} and depends only polynomially on the number of edges of the graph. The algorithm can be implemented with linear complexity in the number of rounds n and in the number of edges. An extension to the so-called label efficient setting is also given, in which the decision maker is informed about the weights of the edges corresponding to the chosen path at a total of m << n time instances. Another extension is shown where the decision maker competes against a time-varying path, a generalization of the problem of tracking the best expert. A version of the multi-armed bandit setting for shortest path is also discussed where the decision maker learns only the total weight of the chosen path but not the weights of the individual edges on the path. Applications to routing in packet switched networks along with simulation results are also presented.

cs.LG↗