SearcharxivSearch

arXiv subjects

Alberto Bucci

Publications and source records attributed to Alberto Bucci.

9 recordsLinked to original sources

Randomized Tucker-Sketched GMRES

We address the problem of solving large-scale tensor-structured linear systems in the Tucker format. In this setting, standard iterative solvers such as GMRES face a fundamental bottleneck: the multilinear ranks of the Krylov basis vectors grow with the iteration count, leading to rapidly increasing tensor operation costs and memory requirements. To overcome these challenges, we propose two randomized algorithms within the sketched GMRES framework that replace full Arnoldi orthogonalization with short recurrences. The first, RHOSVD-Tucker sGMRES, uses randomized HOSVD with per-iteration rank selection, providing robustness across a wide range of problems. The second method, MLN-Tucker sGMRES, leverages the multilinear Nystr\"om approximation with a fixed rank, enabling streaming computations; the streamability of the approximation further allows, at no additional cost, a memory-efficient reconstruction of the solution from a compact sketched representation of the Krylov basis. Both methods outperform standard low-rank Tucker solvers in symmetric and non-symmetric settings. Applied to inverse problems, the low-rank Tucker constraint acts as an implicit regularizer; combined with adaptive projected Tikhonov penalization and automatic regularization parameter selection, the methods yield stable reconstructions.

math.NA

An Equivalence result for sketched Anderson Acceleration and sketched GMRES

In this paper we present an equivalence result between a randomized version of Anderson Acceleration and of randomized GMRES for linear problems. Namely, we extend the classical result of Walker and Ni (2011) to the case in which the least-squares problem in Anderson Acceleration is solved in a sketched space defined by a random projection. This equivalence suggests potential avenues for further research in the design and analysis of randomized acceleration methods.

math.NA

Randomized Flexible LSQR and LSMR with applications to inverse problems

LSQR and LSMR are iterative methods, based on the Golub-Kahan bidiagonalization algorithm, widely used for large-scale linear least squares problems. FLSQR and FLSMR are flexible variants of LSQR and LSMR, respectively, based on a flexible Golub-Kahan (Arnoldi-like) factorization algorithm, which naturally allow modifications of the solution approximation subspace and/or handling inexact matrix-vector multiplications with the (transpose of the) coefficient matrix, thereby enabling to enforce prior information into the computed solution. The goal of this paper is to introduce sFLSQR and sFLSMR, i.e., sketched variants of FLSQR and FLSMR, respectively, where randomization becomes particularly effective, as it allows to recover short recurrences for the solution approximation. In particular, this paper explores applications to large-scale inverse problems, showing the ability of the new randomized solvers to alleviate computational bottlenecks while preserving reconstruction quality. A theoretical analysis of sFLSQR and sFLSMR is provided, and their performance is validated through numerical experiments.

math.NA

Numerical Stability of the Nystr\"om Method

The Nystr\"om method is a widely used technique for improving the scalability of kernel-based algorithms, including kernel ridge regression, spectral clustering, and Gaussian processes. Despite its popularity, the numerical stability of the method has remained largely an unresolved problem. In particular, the pseudo-inversion of the submatrix involved in the Nystr\"om method may pose stability issues as the submatrix is likely to be ill-conditioned, resulting in numerically poor approximation. In this work, we establish conditions under which the Nystr\"om method is numerically stable. We show that stability can be achieved through an appropriate choice of column subsets and a careful implementation of the pseudoinverse. Our results and experiments provide theoretical justification and practical guidance for the stable application of the Nystr\"om method in large-scale kernel computations.

math.NA

Randomized algorithms for streaming low-rank approximation in tree tensor network format

In this work, we present the tree tensor network Nystr\"om (TTNN), an algorithm that extends recent research on streamable tensor approximation, such as for Tucker and tensor-train formats, to the more general tree tensor network format, enabling a unified treatment of various existing methods. Our method retains the key features of the generalized Nystr\"om approximation for matrices, that is randomized, single-pass, streamable, and cost-effective. Additionally, the structure of the sketching allows for parallel implementation. We provide a deterministic error bound for the algorithm and, in the specific case of Gaussian dimension reduction maps, also a probabilistic one. We also introduce a sequential variant of the algorithm, referred to as sequential tree tensor network Nystr\"om (STTNN), which offers better performance for dense tensors. Furthermore, both algorithms are well-suited for the recompression or rounding of tensors in the tree tensor network format. Numerical experiments highlight the efficiency and effectiveness of the proposed methods.

math.NA

Randomized sketched TT-GMRES for linear systems with tensor structure

In the last decade, tensors have shown their potential as valuable tools for various tasks in numerical linear algebra. While most of the research has been focusing on how to compress a given tensor in order to maintain information as well as reducing the storage demand for its allocation, the solution of linear tensor equations is a less explored venue. Even if many of the routines available in the literature are based on alternating minimization schemes (ALS), we pursue a different path and utilize Krylov methods instead. The use of Krylov methods in the tensor realm is not new. However, these routines often turn out to be rather expensive in terms of computational cost and ALS procedures are preferred in practice. We enhance Krylov methods for linear tensor equations with a panel of diverse randomization-based strategies which remarkably increase the efficiency of these solvers making them competitive with state-of-the-art ALS schemes. The up-to-date randomized approaches we employ range from sketched Krylov methods with incomplete orthogonalization and structured sketching transformations to streaming algorithms for tensor rounding. The promising performance of our new solver for linear tensor equations is demonstrated by many numerical results.

math.NA

A sequential multilinear Nystr\"om algorithm for streaming low-rank approximation of tensors in Tucker format

We present a sequential version of the multilinear Nystr\"om algorithm which is suitable for the low-rank Tucker approximation of tensors given in a streaming format. Accessing the tensor $\mathcal{A}$ exclusively through random sketches of the original data, the algorithm effectively leverages structures in $\mathcal{A}$, such as low-rankness, and linear combinations. We present a deterministic analysis of the algorithm and demonstrate its superior speed and efficiency in numerical experiments including an application in video processing.

math.NA

A multilinear Nystr\"om algorithm for low-rank approximation of tensors in Tucker format

The Nystr\"om method offers an effective way to obtain low-rank approximation of SPD matrices, and has been recently extended and analyzed to nonsymmetric matrices (leading to the generalized Nystr\"om method). It is a randomized, single-pass, streamable, cost-effective, and accurate alternative to the randomized SVD, and it facilitates the computation of several matrix low-rank factorizations. In this paper, we take these advancements a step further by introducing a higher-order variant of Nystr\"om's methodology tailored to approximating low-rank tensors in the Tucker format: the multilinear Nystr\"om technique. We show that, by introducing appropriate small modifications in the formulation of the higher-order method, strong stability properties can be obtained. This algorithm retains the key attributes of the generalized Nystr\"om method, positioning it as a viable substitute for the randomized higher-order SVD algorithm.

math.NA

A continuation method for computing the multilinear Pagerank

The multilinear Pagerank model [Gleich, Lim and Yu, 2015] is a tensor-based generalization of the Pagerank model. Its computation requires solving a system of polynomial equations that contains a parameter $\alpha \in [0,1)$. For $\alpha \approx 1$, this computation remains a challenging problem, especially since the solution may be non-unique. Extrapolation strategies that start from smaller values of $\alpha$ and `follow' the solution by slowly increasing this parameter have been suggested; however, there are known cases where these strategies fail, because a globally continuous solution curve cannot be defined as a function of $\alpha$. In this paper, we improve on this idea, by employing a predictor-corrector continuation algorithm based on a more general representation of the solutions as a curve in $\mathbb{R}^{n+1}$. We prove several global properties of this curve that ensure the good behavior of the algorithm, and we show in our numerical experiments that this method is significantly more reliable than the existing alternatives.

math.NA