Features Extraction Across Multi-Source Heterogeneous Data

Abstract

High-dimensional data from multiple sources often combine shared structure with source-specific variation. We study heterogeneous principal component analysis, which seeks to recover common and private principal subspaces when their dimensions are unknown. {We propose \emph{Feature Extraction (FE)}, a novel data-driven framework with two steps. First, the Eigenvector-based Spike-Count (ESC) estimator determines the number of principal components (PC) in each source using only sample eigenvector information, rather than eigenvalue-based criteria. Second, FE identifies the locations of common features by comparing bias-corrected inner products between sample eigenvectors across sources. These matches yield estimates of the common and source-specific subspaces without iterative optimization. We establish the consistency of our estimators under a spiked covariance model in a high-dimensional regime where the dimension and sample size grow proportionally. Simulations and applications to federated image data and U.S. housing prices demonstrate the performance of FE in rank recovery, subspace estimation, and data reconstruction.} Code is available at https://github.com/liwenbo-research/HeterogeneousPCA