Riemannian Regression

Abstract

Classical linear regression assumes that the relevant geometry of the predictor space is Euclidean and that all centered observations contribute to the least-squares fit in the same geometric scale. This paper proposes \emph{Riemannian Regression}, a regression framework in which the usual vector differences are replaced by locally weighted differences induced by a data-dependent similarity structure. We introduce a generalized framework, termed {\em Riemannian Regression}, extending classic regression to any data endowed with a local distance structure. By equipping data tables with local metrics, we adapt regression model to incorporate manifold geometry. Given a similarity matrix $S=(S_{ij})$, obtained from UMAP, ISOMAP, or DBSCAN \cite{mcinnes,isomap,dbscan}, we define the dissimilarity coefficient $ρ_{ij}=1-S_{ij}$ and the induced subtraction $ x_i\ominus x_j=ρ_{ij}(x_i-x_j). $ A Riemannian center $g=x_λ$ is selected as a discrete Fréchet mean, and regression is performed on the Riemannian-centered variables $X_R=W X_{c,λ}$ and $y_R=W y_{c,λ}$, where $W=\operatorname{diag}(ρ_{1λ},\ldots,ρ_{nλ})$. The resulting estimator has the weighted least-squares form $ \widehatβ_R=(X_{c,λ}^{t}W^2X_{c,λ})^{-1}X_{c,λ}^{t}W^2y_{c,λ}. $ The proposed approach preserves the linear form of the regression model while changing the geometry of the fit. The paper develops three ways to construct the local metric: UMAP-based fuzzy similarities, ISOMAP-based normalized geodesic distances, and DBSCAN-based density similarities. Simulated examples and the Abalone data set illustrate how Riemannian Regression can reduce the influence of locally anomalous observations and adapt to regions with different local densities.