Iso-Riemannian Optimization on Learned Data Manifolds

Abstract

We develop a theory of iso-Riemannian optimization for problems constrained to learned data manifolds, a setting in which classical Riemannian optimization - and Riemannian gradient descent in particular - can be poorly suited. That is, favorable Euclidean properties of an objective need not translate into geodesic convexity or L-smoothness, and the Riemannian gradient may provide an unsuitable search direction. We instead combine the manifold mappings induced by the iso-connection, whose geodesics have constant Euclidean speed, with the l2-projected Euclidean gradient, resulting in l2-projected gradient iso-Riemannian descent, in an attempt to alleviate these issues. We analyze this scheme from two complementary perspectives. First, we introduce iso-g-convexity and iso-L-smoothness, relate strong iso-g-convexity to a Polyak-Lojasiewicz-type condition, and establish general convergence guarantees. This function-based theory addresses the conditioning and search-direction issues that motivate our approach. Second, we introduce iso-monotonicity and iso-Lipschitzness for vector fields. However, while these notions yield analogous convergence results in one dimension and enable applications such as the computation of iso-Riemannian barycentres, the resulting theory need not extend meaningfully to higher dimensions. So unlike in the classical Levi-Civita setting, these function- and vector field-based perspectives need not be equivalent in the iso-Riemannian setting. Consequently, distinct extensions of classical Riemannian optimization lead to different assumptions and convergence guarantees. The theory developed here suggests that, for function optimization, the function-based perspective provides the more general, tractable and overall suitable framework, whereas the vector field perspective is naturally reserved for problems without a direct function-based formulation.