Geometry of R\'enyi Entropy on the Majorization Lattice

Abstract

Majorization is a stochastic ordering relation that compares the relative diversity of probability distributions with numerous applications in econometrics, spectral theory, and ecology. It is well-known that the majorization partial order forms a complete lattice on the set of ordered probability distributions. In this work, we study the properties of R\'enyi entropy on the majorization lattice. We establish a fundamental relation between the comonotone coupling and the independent coupling associated with a collection of marginal distributions. Consequently, we show that, for every order $\alpha \in [0,\infty]$, the R\'enyi entropy is subadditive on the majorization lattice. We further characterize the supermodular regime, showing that R\'enyi entropy is supermodular on the majorization lattice for $\alpha \in \{0\} \,\cup \, [1,\infty]$. For the Tsallis entropy, we show that it also satisfies subadditivity on the majorization lattice, for every order $\alpha \in [0,\infty)$. Finally, we show that, unlike the R\'enyi entropy, the Tsallis entropy is supermodular on the majorization lattice for every $\alpha \in [0,\infty)$.