Biweighted Poisson Subsampling for Convoluted Rank Regression with Massive Data

Abstract

Optimal subsampling efficiently selects the most informative data points, enabling accurate statistical inference while significantly reducing computational burden for massive datasets. However, the existing relevant methods can not directly be applied to pairwise loss problems, particularly for convoluted rank regression (CRR), due to the double summation structure in objective function. To this end, we first propose a new BIweighted Poisson Subsampling (BIPS) framework for such problems through designing a proper weight for a pair of observations instead of for a single observation for objective function. Two concrete inverse probability weighting strategies are considered. Secondly, we focus on the CRR models, under which the BIPS estimator (BIPS-CRR) is formulated. We establish consistency and asymptotic normality for BIPS-CRR, derive its optimal Poisson subsampling probabilities under the L-optimality criterion, and provide a practical algorithm to facilitate implementation. Thirdly, we develop a distributed estimator for CRR that incorporates BIPS as a pilot subsampling strategy. This estimation is globally efficient and is robust to both randomly and non-randomly distributed datasets in distributed computing environments. Extensive simulations and a real-world application demonstrate the excellent finite-sample performance of proposed methodology. Additionally, our BIPS can be readily extended to other U-statistics optimization problems and pairwise learning tasks.