A Unifying View of Distributionally Robust Optimization for Probability Smoothing via Maximum Entropy

Abstract

The probability smoothing problem seeks estimators that assign positive probability to all categories, including those unobserved in a finite sample. In our previous work, we established that the estimator derived from distributionally robust optimization (DRO) with a $q$-norm ambiguity set ($q$-DRO) satisfies the axioms of Positivity, Symmetry, and Order Preservation, while relaxing the rigid Ratio Preservation axiom characterizing Laplace smoothing. In this paper, we observe that Symmetry itself imposes a strong constraint: it forces all unobserved categories to receive identical probability regardless of their relation to observed categories. Motivated by this observation, we study DRO with an optimal transport ambiguity set (OT-DRO), which incorporates inter-category dissimilarities through a cost matrix and thereby enables structure-respecting smoothing. We derive a tractable dual reformulation of OT-DRO as an empirical cross-entropy minimization with a Lipschitz-type penalty on the log-probability vector, and we identify an explicit threshold on the robustness radius at which the optimal estimator reduces to the uniform distribution, paralleling the threshold established for $q$-DRO. We then show, through Sion's minimax theorem, that DRO for probability smoothing over any compact convex ambiguity set that contains a strictly positive distribution is equivalent to entropy maximization over that set. This unifying perspective explains the parallel threshold structure in $q$-DRO and OT-DRO.