A Lower Bound on the Relative Entropy with Respect to a Symmetric Probability

Abstract

Let $ρ$ and $μ$ be two probability measures on $\mathbb{R}$ which are not the Dirac mass at $0$. We denote by $H(μ|ρ)$ the relative entropy of $μ$ with respect to $ρ$. We prove that, if $ρ$ is symmetric and $μ$ has a finite first moment, then \[ H(μ|ρ)\geq \frac{\displaystyle{(\int_{\mathbb{R}}z\,dμ(z))^2}}{\displaystyle{2\int_{\mathbb{R}}z^2\,dμ(z)}}\,,\] with equality if and only if $μ=ρ$.