Simulation of Random Variables under R\'enyi Divergence Measures of All Orders

Abstract

The random variable simulation problem consists in using a $k$-dimensional i.i.d. random vector $X^{k}$ with distribution $P_{X}^{k}$ to simulate an $n$-dimensional i.i.d. random vector $Y^{n}$ so that its distribution is approximately $Q_{Y}^{n}$. In contrast to previous works, in this paper we consider the standard R\'enyi divergence and two variants of all orders to measure the level of approximation. These two variants are the max-R\'enyi divergence $D_{\alpha}^{\mathsf{max}}(P,Q)$ and the sum-R\'enyi divergence $D_{\alpha}^{+}(P,Q)$. When $\alpha=\infty$, these two measures are strong because for any $\epsilon>0$, $D_{\infty}^{\mathsf{max}}(P,Q)\leq\epsilon$ or $D_{\infty}^{+}(P,Q)\leq\epsilon$ implies $e^{-\epsilon}\leq\frac{P(x)}{Q(x)}\leq e^{\epsilon}$ for all $x$. Under these R\'enyi divergence measures, we characterize the asymptotics of normalized divergences as well as the R\'enyi conversion rates. The latter is defined as the supremum of $\frac{n}{k}$ such that the R\'enyi divergences vanish asymptotically. In addition, when the R\'enyi parameter is in the interval $(0,1)$, the R\'enyi conversion rates equal the ratio of the Shannon entropies $\frac{H\left(P_{X}\right)}{H\left(Q_{Y}\right)}$, which is consistent with traditional results in which the total variation measure was adopted. When the R\'enyi parameter is in the interval $(1,\infty]$, the R\'enyi conversion rates are, in general, smaller than $\frac{H\left(P_{X}\right)}{H\left(Q_{Y}\right)}$. When specialized to the case in which either $P_{X}$ or $Q_{Y}$ is uniform, the simulation problem reduces to the source resolvability and intrinsic randomness problems. The preceding results are used to characterize the asymptotics of R\'enyi divergences and the R\'enyi conversion rates for these two cases.