arXiv subjects

Spencer Hill

Publications and source records attributed to Spencer Hill.

4 recordsLinked to original sources

arXiv2026

Rejection Sampling is Optimal for Relative Entropy Coding

In relative entropy coding, a sender aims to design a stochastic code such that, on input $X \sim P_X$, the receiver can generate a sample $Y \sim P_{Y \mid X}$. It is a standard result that (1) this requires at least $I(X; Y)$ bits, (2) the lower bound is achievable within a logarithmic gap, and (3) this gap cannot be reduced in general. The necessity of the gap suggests that the mutual information is not the correct information measure to quantify the rate of relative entropy coding. A potential alternative emerged in the work of Flamich et al. (2025), who proved a tighter lower bound of $I_F(X \to Y)$, a quantity we call the functional information. In this paper, we construct the ring toss code, an encoding method for rejection sampling which uses at most $I_F(X \to Y) + \log e$ bits, hence obtaining a tight one-shot characterization of relative entropy coding. For the trivial channel $Y = X$, our result recovers the noiseless source coding theorem within a small constant. For a general channel, it implies that the classical mutual information lower bound is achievable within $\log(I(X; Y) + 1) + 2.45$ bits in general and within $1.45$ bits for singular channels, which are both the tightest bounds of their kind to date. Moreover, our one-shot result also recovers Sriramu and Wagner's asymptotic results on the second-order redundancy of relative entropy codes.

cs.IT↗

arXiv2026

Singular Relative Entropy Coding with Bits-Back Rejection Sampling

A relative entropy code for a source $X \sim P_X$ is a stochastic code that encodes random samples from a prescribed $P_{Y \mid X}$ using as few bits as possible. A generalisation of entropy coding, it is a standard result that the minimum number of bits required to achieve this is at least the mutual information $I[X\,\Vert\,Y]$. However, a particularly fascinating feature of relative entropy coding compared to entropy coding is that, in general, this lower bound is only achievable to within an additional logarithmic factor. As such, an important research direction is to identify channels where we can reduce this gap. Sriramu and Wagner achieved such success by exhibiting a relative entropy code for so-called singular channels with sub-logarithmic asymptotic redundancy. However, their code is quite involved and, sadly, cannot be implemented in practice. In this paper, we construct the bits-back rejection sampler (BBRS), a relative entropy code that combines ideas from bits-back coding and (greedy) rejection sampling. Our analysis of BBRS reveals that the algorithm achieves the same asymptotic efficiency as Sriramu and Wagner's sampler, but with much simpler analysis and better constants. Moreover, BBRS can be implemented using standard relative entropy coding methods.

cs.IT↗

arXiv2026

Communication Complexity of Exact Sampling under Rényi Information

We study the problem of exact sampling under an exponential communication cost, specifically Campbell's average codeword length $L(t)$ of order $t$, and Rényi's entropy. We provide a lower bound on the Campbell cost of exact sampling that grows approximately as $D_{1/α}(P||Q)$, the Rényi divergence of order $1/α$, with $α= \frac{1}{1+t}$. Using the Poisson functional representation of Li and El Gamal, we prove an upper bound on $L(t)$ whose leading Rényi divergence term has order within $ε$ of that of the lower bound. Our results reduce to the bounds of Harsha et al. as $α\to 1$. We also provide numerical examples comparing the bounds in the cases of normal and Laplacian distributions, demonstrating that the upper and lower bounds are typically within 5-10 bits of each other. Our results characterize exactly the optimal asymptotic Campbell cost $L(t)$ per sample as the number of independent and identically distributed (i.i.d.) samples grows to infinity. We show that under the exponential cost, any causal sampler performs strictly worse asymptotically than noncausal samplers. This contrasts with the case of expected message length, where both causal and noncausal samplers have the same optimal asymptotic cost.

cs.IT↗

arXiv2019

Axisymmetric constraints on cross-equatorial Hadley cell extent

We consider the relevance of known constraints from each of Hide's theorem, the angular momentum conserving (AMC) model, and the equal-area model on the extent of cross-equatorial Hadley cells. These theories respectively posit that a Hadley circulation must span: all latitudes where the radiative convective equilibrium (RCE) absolute angular momentum ($M_\mathrm{rce}$) satisfies $M_\mathrm{rce}>Ωa^2$ or $M_\mathrm{rce}<0$ or where the RCE absolute vorticity ($η_\mathrm{rce}$) satisfies $fη_\mathrm{rce}<0$; all latitudes where the RCE zonal wind exceeds the AMC zonal wind; and over a range such that depth-averaged potential temperature is continuous and that energy is conserved. The AMC model requires knowledge of the ascent latitude $φ_\mathrm{a}$, which need not equal the RCE forcing maximum latitude $φ_\mathrm{m}$. Whatever the value of $φ_\mathrm{a}$, we demonstrate that an AMC cell must extend at least as far into the winter hemisphere as the summer hemisphere. The equal-area model predicts $φ_\mathrm{a}$, always placing it poleward of $φ_\mathrm{m}$. As $φ_\mathrm{m}$ is moved poleward (at a given thermal Rossby number), the equal-area predicted Hadley circulation becomes implausibly large, while both $φ_\mathrm{m}$ and $φ_\mathrm{a}$ become increasingly displaced poleward of the minimal cell extent based on Hide's theorem (i.e. of supercritical forcing). In an idealized dry general circulation model, cross-equatorial Hadley cells are generated, some spanning nearly pole-to-pole. All homogenize angular momentum imperfectly, are roughly symmetric in extent about the equator, and appear in extent controlled by the span of supercritical forcing.

physics.ao-ph↗