Orthogonal Decomposition of Discretization-Induced Transport-Information Cost under Rank-Deficient Parametrizations

Abstract

When we consider discretization of continuous probability distributions, it inevitably induces irreversible geometric distortion of local measure on the discretized support. While such discretziation-induced distortion is extrinsic to information geometry (IG) alone, we recently demonstrate that the discretization cost can be naturally characterized by the standard Kullback-Leibler (KL) divergence between continuous distributions as expectation of their infinitesimal parameter variations. The framework is based on the correspondence between optimal transport (OT) and IG, primarily requring the selected parameters directly identifiable with support coordinates. The present work extends the framework to more generalized parametrization theta, particularly the Jacobian between theta and support coordinates is rank-deficient, which generally results in breaking down the interpretation of the discretization-induced costs as information-geometric quantities. To address the problem, we here introduce an orthogonal decomposition of the second-moment tensor onto linear subspace for the covariance matrices generated by parameter fluctuations, based on Frobenius projection. The decomposition naturally separates the discretization cost into observable and unobservable components relative to the chosen parametrization. The present formulation provides a geometric framework for analyzing partial observability of discretization-induced transport-information costs. The present framework thus clarifies the role of parametrization-dependent information loss.