A Statistical-Physics Refinement of Soft Covering

Abstract

We study the channel output distribution induced by a rate-$R$ random code via statistical physics. The partition function is $Z_n(\beta|\mathcal{C}) = \sum_{y^n}[P_{Y^n|\mathcal{C}}(y^n)]^\beta$, where $\mathcal{C}$ is the code and $\beta > 0$ is inverse temperature. Our focus is on the free energy which is the normalized logarithm of this quantity, which encodes the full R\'{e}nyi spectrum of the output distribution. The single-letter formula derived for the annealed free energy decomposes into two branches which reflect a ``competition'' between two populations of codewords. One is the \emph{bulk branch}, $\psi_{\mbox{\tiny b}}(\beta,R)$, which is driven by typical codewords and the other one is the \emph{sparse branch} $\psi_{\mbox{\tiny s}}(\beta,R)$, which is driven by a-typical codewords, where the qualifiers `typical' and `atypical' are in a sense to become apparent later. We analyze the phase structure of each branch separately and characterize their competition. Both branches are derived for all $\beta > 0$. The phase boundary $R^\star(\beta)$, where the two branches are equal, is analyzed for $\beta \geq 1$, where it has an explicit closed-form expression. The phase diagram in the first quadrant of the $(\beta, R)$ plane has four regions separated by three boundaries: $R = I^{\mbox{\tiny b}}(\beta)$ (bulk branch transition), $R = R^\star(\beta)$ (bulk--sparse competition boundary), and $R = I^{\mbox{\tiny s}}(\beta)$ (sparse branch transition), all meeting at the point $(\beta, R) = (1, I(X;Y))$, where $I(X;Y)$ is the mutual information induced by the input type and the channel. Applications to guesswork, channel resolvability, and hypothesis testing are discussed, and all results are illustrated with a numerical example of a Z-channel.