Subset selection for matrices by volume sampling

Abstract

We address the Subset selection problem for matrices, where the goal is to select a subset $\mathcal{S}$ of $k$ column indices from a \enquote{short-and-fat} matrix $X \in \mathbb{R}^{m \times n}$, such that the sampled submatrix $X_{\mathcal{S}}$ has $\|X_{\mathcal{S}}^†X\|_F$ as small as possible. Our approach is centered on volume sampling, which attains the tightest known bound on this objective in expectation. As our primary contribution, we propose a new deterministic algorithm, Forward derandomized volume sampling (FDVS), which provably attains this bound and has asymptotic complexity $O(nkm)$. In contrast, the complexity of all previously known algorithms with this guarantee scales at least quadratically in $n$ when $k \ll n$, making FDVS particularly attractive for very wide matrices. In addition, we systematically structure the landscape of volume sampling methods: we propose a simple $O(nm^2)$ algorithm for exact forward volume sampling, show that a known deterministic method is in fact a derandomization of Reverse iterative volume sampling, and derive a modification of the latter that avoids repeated SVD downdating, along with a fast greedy forward variant. The algorithms are verified and compared in numerical experiments involving optimal experimental design, sensor placement, and DLRA-DEIM.