arXiv subjects

Ian Zieder

Publications and source records attributed to Ian Zieder.

3 recordsLinked to original sources

arXiv2026

The Binomial Channel: On Capacity, Optimal Inputs, and Beta-Binomial Approximation

We study the binomial channel with input alphabet $[0,1]$ and output alphabet ${0,\ldots,n}$. We investigate its capacity and the structure of the capacity-achieving input and output distributions. Since the output alphabet is finite whereas the input alphabet is continuous, different input distributions may induce the same output distribution; hence, uniqueness and support properties of optimal inputs do not follow from strict concavity arguments. We first establish structural properties of the capacity-achieving input distribution. In particular, we show that it is discrete, unique, symmetric around $1/2$, and contains the endpoints ${0,1}$ in its support. We also derive location constraints and bounds on the probability masses of support points, and improve the Witsenhausen-type upper bound on the support size from order $n$ to order $n/2$. We derive explicit nonasymptotic upper and lower bounds on the capacity $C(n)$. These bounds imply $C(n)=\frac{1}{2}\log(\frac{n\pi}{2e})+o(1).$ The lower bound is obtained by evaluating the mutual information at the reference input $X_r\sim \mathrm{Beta}(1/2,1/2)$, which induces a beta-binomial output distribution, while the upper bound follows from a minimax redundancy construction. Finally, we prove an improved lower bound on the support size of the capacity-achieving input distribution. We show that the beta-binomial output induced by $X_r$ is asymptotically optimal and close to the capacity-achieving output distribution in relative entropy and $\chi^2$ divergence. We also prove a finite-mixture approximation lower bound showing that the beta-binomial output cannot be approximated too accurately by binomial mixtures with few components. Combining these results yields a support-size lower bound of order $\Omega(\sqrt{n\log\log n})$, with explicit constants. Numerical results illustrate the capacity bounds and optimal input.

cs.IT↗

arXiv2024

Binomial Channel: On the Capacity-Achieving Distribution and Bounds on the Capacity

This work considers a binomial noise channel. The paper can be roughly divided into two parts. The first part is concerned with the properties of the capacity-achieving distribution. In particular, for the binomial channel, it is not known if the capacity-achieving distribution is unique since the output space is finite (i.e., supported on integers $0, \ldots, n)$ and the input space is infinite (i.e., supported on the interval $[0,1]$), and there are multiple distributions that induce the same output distribution. This paper shows that the capacity-achieving distribution is unique by appealing to the total positivity property of the binomial kernel. In addition, we provide upper and lower bounds on the cardinality of the support of the capacity-achieving distribution. Specifically, an upper bound of order $ \frac{n}{2}$ is shown, which improves on the previous upper bound of order $n$ due to Witsenhausen. Moreover, a lower bound of order $\sqrt{n}$ is shown. Finally, additional information about the locations and probability values of the support points is established. The second part of the paper focuses on deriving upper and lower bounds on capacity. In particular, firm bounds are established for all $n$ that show that the capacity scales as $\frac{1}{2} \log(n)$.

cs.IT↗

arXiv2022

An MMSE Lower Bound via Poincaré Inequality

This paper studies the minimum mean squared error (MMSE) of estimating $\mathbf{X} \in \mathbb{R}^d$ from the noisy observation $\mathbf{Y} \in \mathbb{R}^k$, under the assumption that the noise (i.e., $\mathbf{Y}|\mathbf{X}$) is a member of the exponential family. The paper provides a new lower bound on the MMSE. Towards this end, an alternative representation of the MMSE is first presented, which is argued to be useful in deriving closed-form expressions for the MMSE. This new representation is then used together with the Poincaré inequality to provide a new lower bound on the MMSE. Unlike, for example, the Cramér-Rao bound, the new bound holds for all possible distributions on the input $\mathbf{X}$. Moreover, the lower bound is shown to be tight in the high-noise regime for the Gaussian noise setting under the assumption that $\mathbf{X}$ is sub-Gaussian. Finally, several numerical examples are shown which demonstrate that the bound performs well in all noise regimes.

cs.IT↗