De Finetti + Sanov = Bayes: Exchangeable Prediction under Moment Constraints

Abstract

We study exchangeable prediction when empirical-moment constraints define each active finite horizon N. The relevant law is the de Finetti mixture conditioned on E_N = {Phat_N in E_{eps_N}}. By permutation invariance, the target may be any fixed block of m coordinates within the active horizon, regardless of whether those coordinates are labeled past, held out, or future relative to any finite cut. Conditionally on the directing measure mu, the Gibbs-conditioning principle sends the law of such a block to the m-fold product of the I-projection P*_mu = argmin_{Q in E} D(Q || mu). On a finite alphabet we give an elementary master inequality for general polyhedral moment windows. After mixing over the constraint posterior Pi_{N,E}, and under weak convergence plus posterior-averaged component control, the finite-dimensional marginals converge to a consistent exchangeable law whose random directing measure is the I-projection P*_mu, with mu drawn from the weak limit Pi_E. Sequential prediction under this limiting law is therefore Bayesian prediction from a mixture of componentwise I-projections. The limiting behavior depends on reachability. For a reachable constraint, the projection is asymptotically the identity on the selected subfamily. Under additional regularity, an unreachable constraint makes the constraint posterior concentrate on the rate-minimizing subfamily, while the projections remain nontrivial. In our examples, at least one operation is asymptotically inactive, though both enter the finite-horizon construction. The master bound also reads as an equivalence of ensembles. We reserve "maximum entropy" for a uniform or flat baseline and use "minimum relative entropy" or "I-projection" in general.