An Explicit Description of Extreme Points of the Set of Couplings with Given Marginals: with Application to Minimum-Entropy Coupling Problems

Abstract

Given probability distributions ${\bf p}=(p_1,p_2,\ldots,p_m)$ and ${\bf q}=(q_1,q_2,\ldots, q_n)$ with $m,n\geq 2$, denote by ${\cal C}(\bf p,q)$ the set of all couplings of $\bf p,q$, a convex subset of $\R^{mn}$. Denote by ${\cal C}_e({\bf p},{\bf q})$ the finite set of all extreme points of ${\cal C}(\bf p,q)$. It is well known that, as a strictly concave function, the Shannan entropy $H$ on ${\cal C}(\bf p,q)$ takes its minimal value in ${\cal C}_e({\bf p},{\bf q})$. In this paper, first, the detailed structure of ${\cal C}_e({\bf p},{\bf q})$ is well specified and all extreme points are enumerated by a special algorithm. As an application, the exact solution of the minimum-entropy coupling problem is obtained. Second, it is proved that for any strict Schur-concave function $\Psi$ on ${\cal C}(\bf p,q)$, $\Psi$ also takes its minimal value on ${\cal C}_e({\bf p},{\bf q})$. As an application, the exact solution of the minimum-entropy coupling problem is obtained for $(\Phi,\hbar)$-entropy, a large class of entropy including Shannon entropy, R\'enyi entropy and Tsallis entropy etc. Finally, all the above are generalized to multi-marginal case.