SearcharxivSearch

arXiv subjects

Simon Mataigne

Publications and source records attributed to Simon Mataigne.

11 recordsLinked to original sources

Halving the size of skew-symmetric eigenvalue problems via the polar decomposition

This paper introduces a novel algorithm for computing eigenvalues and eigenvectors of a dense real skew-symmetric matrix $A$. Its main ingredient is the computation of a polar factor of $A$ that is both skew-symmetric and orthogonal. This polar factor is then used to transform the original problem into a Hermitian eigenvalue problem of half the size, which can be solved accurately and efficiently with standard software such as LAPACK. Numerical experiments demonstrate the stability of the method and show that its running time is competitive with existing approaches for skew-symmetric eigenvalue problems. Finally, we show that the same principle can be used to reduce an orthogonal eigenvalue problem to a unitary eigenvalue problem of half the size.

math.NA

The sketched landing method for large-scale optimization under orthogonality constraints

We propose the sketched landing method, a randomized variant of the landing method for optimization under orthogonality constraints. Each landing step consists of the sum of a normal component, which reduces infeasibility, and a tangent component, which decreases the objective function. Our main contribution is the introduction of low-dimensional random sketch matrices to reduce the computational cost of these directions. We consider both dense (Gaussian) and sparse (subsampling) sketch matrices, and show how they reduce the per-iteration cost while preserving convergence guarantees in expectation.

math.OC

A Jacobi-like algorithm for normal matrices by the skew-symmetric part

We present a fast Jacobi-like algorithm for computing the eigenvalues, and optionally the eigenvectors, of a real normal matrix. The method gains a computational advantage by using Paardekooper's method for skew-symmetric matrices The method is most efficient for matrices where most eigenvalues are complex, such as random orthogonal matrices arising in the context of statistics on manifolds. In this case, the method is faster than the other Jacobi-like algorithms. In the last section of this paper, we also give explicit formulas for the nearest symmetric skew-Hamiltonian and the nearest ortho-symplectic matrix. These problems arise in the design and the analysis of the algorithm.

math.NA

bispectrum: Selective $G$-Bispectra Made Practical

Many machine learning tasks are invariant under the action of a group $G$ of transformations: signal classification can be invariant under translations, image classification under 2D rotations, and spherical-image classification under 3D rotations. The $G$-bispectrum is a principled complete invariant of a signal (retaining all all signal's information up to the group action) with proven benefits in machine learning and as a pooling layer in deep networks. However, its deployment has been hampered by high computational cost and a patchwork of group-specific implementations. We present bispectrum, an open-source, fully unit-tested PyTorch library that implements selective $G$-bispectra for seven different group actions, as differentiable modules that can be directly incorporated into machine learning pipelines and deep learning architectures. For finite groups $G$, selectivity reduces the computational cost from $O(|G|^2)$ to $O(|G|)$. For planar rotations, we leverage the disk bispectrum. For spherical 3D rotations, we introduce an augmented selective bispectrum at band-limit $L$ which reduces the cost from $O(L^3)$ to $\Theta(L^2)$ coefficients. We profile the entire library (for which we implemented various compute optimizations), showing that it delivers near-exact $G$-invariance with its selective $G$-bispectra computed in sub-millisecond time on GPU (up to commonly used bandlimits). We evaluate the benefits of incorporating $G$-bispectra as pooling layers into deep learning architectures on three classical benchmark datasets --comparing against norm pooling, gated pooling, Fourier-ELU pooling, max pooling, and (non-equivariant) data-augmented convolutional baselines. Results show that $G$-bispectra consistently outperform alternatives in the low-data, moderate-capacity regime.

cs.LG

Shortest Geodesic Loops, Sectional Curvature, and Injectivity Radius of the Stiefel Manifold

We determine the length of the shortest nontrivial geodesic loops on the Stiefel manifold endowed with any member of the one-parameter family of Riemannian metrics introduced by H\"uper et al. (2021). This family includes, in particular, the canonical and Euclidean metrics. By combining existing and new bounds on the sectional curvature, we determine the exact value of the injectivity radius of the Stiefel manifold under a wide range of members of the metric family.

math.DG

On the approximation of the Riemannian barycenter

We present a method for computing an approximate Riemannian barycenter of a collection of points lying on a Riemannian manifold. Our approach relies on the use of theoretically proven under- and over-approximations of the Riemannian distance function. We compare it to Riemannian steepest descent on the exact objective function of the Riemannian barycenter and to an approach that approximates the Riemannian logarithm using lifting maps. Experiments are conducted on the Stiefel manifold.

math.DG

The eigenvalue decomposition of normal matrices by the skew-symmetric part

We propose a new method for computing the eigenvalue decomposition of a dense real normal matrix $A$ through the decomposition of its skew-symmetric part. The method relies on algorithms that are known to be efficiently implemented, such as the bidiagonal singular value decomposition and the symmetric eigenvalue decomposition. The advantages of this method stand for normal matrices with few real eigenvalues, such as random orthogonal matrices. We provide a stability and a complexity analysis of the method. The numerical performance is compared with existing algorithms. In most cases, the method has the same operation count as the Hessenberg factorization of a dense matrix. Finally, we provide experiments for the application of computing a Riemannian barycenter on the special orthogonal group.

math.NA

Bounds on the geodesic distances on the Stiefel manifold for a family of Riemannian metrics

We give bounds on geodesic distances on the Stiefel manifold, derived from new geometric insights. The considered geodesic distances are induced by the one-parameter family of Riemannian metrics introduced by Hüper et al. (2021), which contains the well-known Euclidean and canonical metrics. First, we give the best Lipschitz constants between the distances induced by any two members of the family of metrics. Then, we give a lower and an upper bound on the geodesic distance by the easily computable Frobenius distance. We give explicit families of pairs of matrices that depend on the parameter of the metric and the dimensions of the manifold, where the lower and the upper bound are attained. These bounds aim at improving the theoretical guarantees and performance of minimal geodesic computation algorithms by reducing the initial velocity search space. In addition, these findings contribute to advancing the understanding of geodesic distances on the Stiefel manifold and their applications.

math.DG

The Selective G-Bispectrum and its Inversion: Applications to G-Invariant Networks

An important problem in signal processing and deep learning is to achieve \textit{invariance} to nuisance factors not relevant for the task. Since many of these factors are describable as the action of a group $G$ (e.g. rotations, translations, scalings), we want methods to be $G$-invariant. The $G$-Bispectrum extracts every characteristic of a given signal up to group action: for example, the shape of an object in an image, but not its orientation. Consequently, the $G$-Bispectrum has been incorporated into deep neural network architectures as a computational primitive for $G$-invariance\textemdash akin to a pooling mechanism, but with greater selectivity and robustness. However, the computational cost of the $G$-Bispectrum ($\mathcal{O}(|G|^2)$, with $|G|$ the size of the group) has limited its widespread adoption. Here, we show that the $G$-Bispectrum computation contains redundancies that can be reduced into a \textit{selective $G$-Bispectrum} with $\mathcal{O}(|G|)$ complexity. We prove desirable mathematical properties of the selective $G$-Bispectrum and demonstrate how its integration in neural networks enhances accuracy and robustness compared to traditional approaches, while enjoying considerable speeds-up compared to the full $G$-Bispectrum.

cs.LG

An efficient algorithm for the Riemannian logarithm on the Stiefel manifold for a family of Riemannian metrics

Since the popularization of the Stiefel manifold for numerical applications in 1998 in a seminal paper from Edelman et al., it has been exhibited to be a key to solve many problems from optimization, statistics and machine learning. In 2021, H\"uper et al. proposed a one-parameter family of Riemannian metrics on the Stiefel manifold, subsuming the well-known Euclidean and canonical metrics. Since then, several methods have been proposed to obtain a candidate for the Riemannian logarithm given any metric from the family. Most of these methods are based on the shooting method or rely on optimization approaches. For the canonical metric, Zimmermann proposed in 2017 a particularly efficient method based on a pure matrix-algebraic approach. In this paper, we derive a generalization of this algorithm that works for the one-parameter family of Riemannian metrics. The algorithm is proposed in two versions, termed backward and forward, for which we prove that it conserves the local linear convergence previously exhibited in Zimmermann's algorithm for the canonical metric.

math.NA

The ultimate upper bound on the injectivity radius of the Stiefel manifold

We exhibit conjugate points on the Stiefel manifold endowed with any member of the family of Riemannian metrics introduced by H\"uper et al. (2021). This family contains the well-known canonical and Euclidean metrics. An upper bound on the injectivity radius of the Stiefel manifold in the considered metric is then obtained as the minimum between the length of the geodesic along which the points are conjugate and the length of certain geodesic loops. Numerical experiments support the conjecture that the obtained upper bound is in fact equal to the injectivity radius.

math.DG