SearcharxivSearch

arXiv subjects

Jiange Li

Publications and source records attributed to Jiange Li.

16 recordsLinked to original sources

Multiplicative comparisons of R\'enyi entropies for weighted Bernoulli sums

We establish improved multiplicative bounds relating the R\'enyi entropies of different orders for weighted sums of independent Bernoulli random variables. In particular, we prove a logarithmic bound between the zeroth-order and infinity-order R\'enyi entropies, which yields a polynomial improvement over the square-root bound of Jain, Sah, and Sawhney. Additionally, we obtain explicit constant-factor bounds for comparisons among R\'enyi entropies of nonzero orders.

math.PR

Central limit theorem in R\'enyi divergence for lattice random variables

We establish a central limit theorem in R\'enyi divergence for independent and identically distributed lattice random variables $X_1, \cdots, X_n$ with zero mean, unit variance, and maximal span $h>0$. Let $S_n=(X_1+\cdots+X_n)/\sqrt n$. Let $Z_n$ denote the standard Gaussian distribution quantized on the support lattice of $S_n$. For every $\alpha>1$, with $\beta=\alpha/(\alpha-1)$, we prove that the R\'enyi divergence $D_\alpha(S_n\|Z_n)\to 0$ if and only if the divergence is finite at some convolution level and the strict sub-Gaussian condition $$ \mathbb E e^{tX}<e^{\beta t^2/2},\quad t\in\mathbb R,~ t\ne0 $$ holds. Under these conditions, we further derive an Edgeworth-type asymptotic expansion of the divergence to arbitrary order. These results provide a lattice counterpart of the R\'enyi entropic central limit theorem for continuous random variables due to Bobkov, Chisyakov and G\"{o}tze (\emph{Ann. Probab.} \textbf{47} (2019), 270--323).

math.PR

Symmetrization resistance for exponential and geometric distributions

Given a random variable $X$, an independent random variable $Y$ is called a symmetrizer of $X$ if their sum $X+Y$ is symmetric about the origin. The study of symmetrization resistance asks whether every such $Y$ must contain at least as much randomness as $X$. This problem was previously investigated for binary random variables in terms of variance and Shannon entropy. In this paper, we establish sharp symmetrization resistance results for exponential and geometric distributions. We prove that every independent symmetrizer $Y$ of an exponential random variable $X$ has variance, R\'enyi and Tsallis entropies of every positive order at least as large as that of $X$. Parallel results are obtained for integer-valued independent symmetrizers of geometric random variables. Equality in each comparison holds precisely when $Y$ is an independent copy of $-X$. Furthermore, we show the majorization of $X$ over $Y$ via establishing sharp concentration inequalities for $Y$. The proofs crucially rely on a differential/difference inversion formula for exponential/geometric convolution, which converts the symmetrization constraint into a hazard-rate inequality and enables us to identity exponential/geometric distributions as extremal distributions of a corresponding optimization problem.

math.PR

A reverse entropy power inequality for i.i.d. log-concave random variables

We show that $h_\infty(X+Y)\leq h_\infty(Z+W)$, where $X, Y$ are independent log-concave random variables, and $Z, W$ are exponential random variables having the same respective $\infty$-R\'enyi entropies. Analogs for integer-valued monotone log-concave random variables are also obtained. Our main tools are decreasing rearrangement, majorization, and the change of measure.

math.PR

Asymptotic Normality and Concentration Inequalities of Statistics of Core Partitions with Bounded Perimeters

Core partitions have attracted much attention since Anderson's work (2002) on the number of $(s,t)$-core partitions for coprime $s,t$. Recently, there has been a growing interest in studying the limiting distributions of the sizes of random simultaneous core partitions. In this paper, we prove the asymptotic normality of certain statistics of uniform random core partitions with bounded perimeters in the Kolmogorov and Wasserstein $W_1$ distances, including the length and size of a random (strict) $n$-core partition, the length of the Durfee square and the size of a random self-conjugate $n$-core partition. Accordingly, we prove that these statistics are subgaussian. This contrasts with the asymptotic behavior of the size of a random $(s, t)$-core partition for coprime $s,t$ studied by Even-Zohar (2022), which converges in law to Watson's $U^2$ distribution. Our results show that the distribution of the size of a random strict $(n, dn+1)$-core partition is asymptotically normal when $d \ge 3$ is fixed and $n$ tends to infinity, which is an analog of Zaleski's conjecture (2017) and covers Komlós, Sergel, and Tusnády's result (2020) as a special case. Our proof integrates a variety of combinatorial and probabilistic tools, including Stein's method based on Hoeffding decomposition, Hoeffding's combinatorial central limit theorem, the Efron-Stein inequalities on product spaces and slices, and asymptotics of Pólya frequency sequences. Furthermore, our approach is potentially applicable to the study of the asymptotic normality of functionals of random variables with certain global dependence structures that can be decomposed into appropriate mixture forms.

math.PR

Long-term balanced allocation via thinning

We study the long-term behavior of the two-thinning variant of the classical balls-and-bins model. In this model, an overseer is provided with uniform random allocation of $m$ balls into $n$ bins in an on-line fashion. For each ball, the overseer could reject its allocation and place the ball into a new bin drawn independently at random. The purpose of the overseer is to reduce the maximum load of the bins, which is defined as the difference between the maximum number of balls in a single bin and $m/n$, i.e., the average number of balls among all bins. We provide tight estimates for three quantities: the lowest maximum load that could be achieved at time $m$, the lowest maximum load that could be achieved uniformly over the entire time interval $[m]:=\{1, 2, \cdots, m\}$, and the lowest \emph{typical} maximum load that could be achieved over the interval $[m]$, where the typicality means that the maximum load holds for $1-o(1)$ portion of the times in $[m]$. We show that when $m$ and $n$ are sufficiently large, a typical maximum load of $(\log n)^{1/2+o(1)}$ can be achieved with high probability, asymptotically the same as the optimal maximum load that could be achieved at time $m$. However, for any strategy, the maximal load among all times in the interval $[m]$ is $Ω\big(\frac{\log n}{\log\log n}\big)$ with high probability. A strategy achieving this bound is provided. An explanation for this gap is provided by our optimal strategies as follows. To control the typical load, we restrain the maximum load for some time, during which we accumulate more and more bins with relatively high load. After a while, we have to employ for a short time a different strategy to reduce the number of relatively heavily loaded bins, at the expanse of temporarily inducing high load in a few bins.

math.PR

Boolean functions: noise stability, non-interactive correlation distillation, and mutual information

Let $T_ε$ be the noise operator acting on Boolean functions $f:\{0, 1\}^n\to \{0, 1\}$, where $ε\in[0, 1/2]$ is the noise parameter. Given $α>1$ and fixed mean $\mathbb{E} f$, which Boolean function $f$ has the largest $α$-th moment $\mathbb{E}(T_εf)^α$? This question has close connections with noise stability of Boolean functions, the problem of non-interactive correlation distillation, and Courtade-Kumar's conjecture on the most informative Boolean function. In this paper, we characterize maximizers in some extremal settings, such as low noise ($ε=ε(n)$ is close to 0), high noise ($ε=ε(n)$ is close to 1/2), as well as when $α=α(n)$ is large. Analogous results are also established in more general contexts, such as Boolean functions defined on discrete torus $(\mathbb{Z}/p\mathbb{Z})^n$ and the problem of noise stability in a tree model.

math.PR

Concentration of information content for convex measures

We establish sharp exponential deviation estimates of the information content as well as a sharp bound on the varentropy for the class of convex measures on Euclidean spaces. This generalizes a similar development for log-concave measures in the recent work of Fradelizi, Madiman and Wang (2016). In particular, our results imply that convex measures in high dimensions are concentrated in an annulus between two convex sets (as in the log-concave case) despite their possibly having much heavier tails. Various tools and consequences are developed, including a sharp comparison result for Rényi entropies, inequalities of Kahane-Khinchine type for convex measures that extend those of Koldobsky, Pajor and Yaskin (2008) for log-concave measures, and an extension of Berwald's inequality (1947).

math.PR

Load balancing under $d$-thinning

In the classical balls-and-bins model, $m$ balls are allocated into $n$ bins one by one uniformly at random. In this note, we consider the $d$-thinning variant of this model, in which the process is regulated in an on-line fashion as follows. For each ball, after a random bin has been selected, an overseer may decide, based on all previous history, whether to accept this bin or not. However, one of every $d$ consecutive suggested bins must be accepted. The maximum load of this setting is the number of balls in the most loaded bin. We show that after $Θ(n)$ balls have been allocated, the least maximum load achievable with high probability is $(d+o(1))\sqrt[d]{\frac{d\log n}{\log\log n}}$. This should be compared with the related $d$-choice setting, in which the optimal maximum load achievable with high probability is $\frac{\log\log n}{\log d}+O(1)$.

math.PR

Further investigations of Rényi entropy power inequalities and an entropic characterization of s-concave densities

We investigate the role of convexity in Rényi entropy power inequalities. After proving that a general Rényi entropy power inequality in the style of Bobkov-Chistyakov (2015) fails when the Rényi parameter $r\in(0,1)$, we show that random vectors with $s$-concave densities do satisfy such a Rényi entropy power inequality. Along the way, we establish the convergence in the Central Limit Theorem for Rényi entropies of order $r\in(0,1)$ for log-concave densities and for compactly supported, spherically symmetric and unimodal densities, complementing a celebrated result of Barron (1986). Additionally, we give an entropic characterization of the class of $s$-concave densities, which extends a classical result of Cover and Zhang (1994).

math.PR

Large deviations for conditional guesswork

The guesswork problem was originally studied by Massey to quantify the number of guesses needed to ascertain a discrete random variable. It has been shown that for a large class of random processes the rescaled logarithm of the guesswork satisfies the large deviation principle and this has been extended to the case where $k$ out $m$ sequences are guessed. The study of conditional guesswork, where guessing of a sequence is aided by the observation of another one, was initiated by Arıkan in his simple derivation of the upper bound of the cutoff rate for sequential decoding. In this note, we extend these large deviation results to the setting of conditional guesswork.

math.PR

Capacity-achieving Guessing Random Additive Noise Decoding (GRAND)

We introduce a new algorithm for realizing Maximum Likelihood (ML) decoding in discrete channels with or without memory. In it, the receiver rank orders noise sequences from most likely to least likely. Subtracting noise from the received signal in that order, the first instance that results in a member of the code-book is the ML decoding. We name this algorithm GRAND for Guessing Random Additive Noise Decoding. We establish that GRAND is capacity-achieving when used with random code-books. For rates below capacity we identify error exponents, and for rates beyond capacity we identify success exponents. We determine the scheme's complexity in terms of the number of computations the receiver performs. For rates beyond capacity, this reveals thresholds for the number of guesses by which if a member of the code-book is identified it is likely to be the transmitted code-word. We introduce an approximate ML decoding scheme where the receiver abandons the search after a fixed number of queries, an approach we dub GRANDAB, for GRAND with ABandonment. While not an ML decoder, we establish that the algorithm GRANDAB is also capacity-achieving for an appropriate choice of abandonment threshold, and characterize its complexity, error and success exponents. Worked examples are presented for Markovian noise that indicate these decoding schemes substantially out-perform the brute force decoding approach.

cs.IT

A combinatorial approach to small ball inequalities for sums and differences

Small ball inequalities have been extensively studied in the setting of Gaussian processes and associated Banach or Hilbert spaces. In this paper, we focus on studying small ball probabilities for sums or differences of independent, identically distributed random elements taking values in very general sets. Depending on the setting--abelian or nonabelian groups, or vector spaces, or Banach spaces--we provide a collection of inequalities relating different small ball probabilities that are sharp in many cases of interest. We prove these distribution-free probabilistic inequalities by showing that underlying them are inequalities of extremal combinatorial nature, related among other things to classical packing problems such as the kissing number problem. Applications are given to moment inequalities.

math.PR

Rényi entropy power inequality and a reverse

This paper is twofold. In the first part, we present a refinement of the Rényi Entropy Power Inequality (EPI) recently obtained in \cite{BM16}. The proof largely follows the approach in \cite{DCT91} of employing Young's convolution inequalities with sharp constants. In the second part, we study the reversibility of the Rényi EPI, and confirm a conjecture in \cite{BNT15, MMX16} in two cases. Connections with various $p$-th mean bodies in convex geometry are also explored.

math.PR

Entropies of weighted sums in cyclic groups and an application to polar codes

In this note, the following basic question is explored: in a cyclic group, how are the Shannon entropies of the sum and difference of i.i.d. random variables related to each other? For the integer group, we show that they can differ by any real number additively, but not too much multiplicatively; on the other hand, for $\mathbb{Z}/3\mathbb{Z}$, the entropy of the difference is always at least as large as that of the sum. These results are closely related to the study of more-sum-than-difference (i.e. MSTD) sets in additive combinatorics. We also investigate polar codes for $q$-ary input channels using non-canonical kernels to construct the generator matrix, and present applications of our results to constructing polar codes with significantly improved error probability compared to the canonical construction.

cs.IT

A Note on Distribution Free Symmetrization Inequalities

Let $X, Y$ be two independent identically distributed (i.i.d.) random variables taking values from a separable Banach space $(\mathcal{X}, \|\cdot\|)$. Given two measurable subsets $F, K\subseteq\cal{X}$, we established distribution free comparison inequalities between $\mathbb{P}(X\pm Y \in F)$ and $\mathbb{P}(X-Y\in K)$. These estimates are optimal for real random variables as well as when $\mathcal{X}=\mathbb{R}^d$ is equipped with the $\|\cdot\|_\infty$ norm. Our approach for both problems extends techniques developed by Schultze and Weizsächer (2007).

math.PR