arXiv subjects

Ivan Kozyrev

Publications and source records attributed to Ivan Kozyrev.

3 recordsLinked to original sources

arXiv2026

Subset selection for matrices by column exchange

The paper considers the problem of finding a submatrix $X_{\mathcal{S}} \in \mathbb{R}^{m \times k}$ in a matrix $X \in \mathbb{R}^{m \times n}$, such that the spectral or Frobenius norm of $X_{\mathcal{S}}^{\dag} X$ is limited, which guarantees it provides a good representation of the whole matrix. Such bounds can be reached by applying greedy algorithms, maximizing the submatrix volume. We suggest a modification of a greedy volume maximization, which performs column exchanges asymptotically faster for $n \gg m$ than the known alternatives, while guaranteeing the same bounds on $X_{\mathcal{S}}^{\dag} X$. In addition, we prove a new upper bound on the number of required exchanges, which is applicable to the new algorithm as well as to other greedy volume maximization algorithms.

math.NA↗

arXiv2025

Ky Fan Norms and Beyond: Dual Norms and Combinations for Matrix Optimization

In this article, we explore the use of various matrix norms for optimizing functions of weight matrices, a crucial problem in deep learning. Moving beyond the spectral norm that underlies the Muon update, we leverage the duals of the Ky Fan norms to introduce the Fanion family of linear minimization oracle (LMO) algorithms, which are closely related to Muon, $\nu$-SAM, and Dion. Staying inside the LMO, we construct the families of F-Fanions and S-Fanions, whose updates are convex combinations of the updates of Fanions and Normalized SGD or SignSGD, respectively. The most promising algorithms in these families are F-Muon and S-Muon. By conducting an extensive empirical study of all three algorithm families across a wide range of tasks and settings, we demonstrate that F-Muon and S-Muon consistently match Muon's performance, while outperforming Muon on a synthetic smooth convex problem.

math.OC↗

arXiv2025

Subset selection for matrices in spectral norm

We address the subset selection problem for matrices, where the goal is to select a subset of $k$ columns from a "short-and-fat" matrix $X \in \mathbb{R}^{m \times n}$, such that the pseudoinverse of the sampled submatrix has as small spectral or Frobenius norm as possible. For the NP-hard spectral norm variant, we propose a new deterministic approximation algorithm. Our method refines the potential-based framework of spectral sparsification by specializing it to a single barrier function. This key modification enables direct, unweighted column selection, bypassing the intermediate weighting step required by previous approaches. It also allows for a novel adaptive update strategy for the barrier. This approach yields a new, explicit bound on the approximation quality that improves upon existing guarantees in key parameter regimes, without increasing the asymptotic computational complexity. Furthermore, numerical experiments demonstrate that the proposed method consistently outperforms its direct competitors. A complete C++ implementation is provided to support our findings and facilitate future research.

math.NA↗