arXiv subjects

Karl Meerbergen

Publications and source records attributed to Karl Meerbergen.

30 records · Page 2Linked to original sources

arXiv2020

Subspace method for multiparameter-eigenvalue problems based on tensor-train representations

In this paper we solve $m$-parameter eigenvalue problems ($m$EPs), with $m$ any natural number by representing the problem using Tensor-Trains (TT) and designing a method based on this format. $m$EPs typically arise when separation of variables is applied to separable boundary value problems. Often, methods for solving $m$EP are restricted to $m = 3$, due to the fact that, to the best of our knowledge, no available solvers exist for $m>3$ and reasonable size of the involved matrices. In this paper, we prove that computing the eigenvalues of a $m$EP can be recast into computing the eigenvalues of TT-operators. We adapted the algorithm in \cite{Dolgov2014a} for symmetric eigenvalue problems in TT-format to an algorithm for solving generic $m$EPs. This leads to a subspace method whose subspace dimension does not depend on $m$, in contrast to other subspace methods for $m$EPS. This allows us to tackle $m$EPs with $m > 3$ and reasonable size of the matrices. We provide theoretical results and report numerical experiments. The MATLAB code is publicly available.

math.NA↗

arXiv2020

Two-level preconditioning for Ridge Regression

Solving linear systems is often the computational bottleneck in real-life problems. Iterative solvers are the only option due to the complexity of direct algorithms or because the system matrix is not explicitly known. Here, we develop a two-level preconditioner for regularized least squares linear systems involving a feature or data matrix. Variants of this linear system may appear in machine learning applications, such as ridge regression, logistic regression, support vector machines and Bayesian regression. We use clustering algorithms to create a coarser level that preserves the principal components of the covariance or Gram matrix. This coarser level approximates the dominant eigenvectors and is used to build a subspace preconditioner accelerating the Conjugate Gradient method. We observed speed-ups for artificial and real-life data.

math.NA↗

arXiv2020

Multilevel Gibbs Sampling for Bayesian Regression

Bayesian regression remains a simple but effective tool based on Bayesian inference techniques. For large-scale applications, with complicated posterior distributions, Markov Chain Monte Carlo methods are applied. To improve the well-known computational burden of Markov Chain Monte Carlo approach for Bayesian regression, we developed a multilevel Gibbs sampler for Bayesian regression of linear mixed models. The level hierarchy of data matrices is created by clustering the features and/or samples of data matrices. Additionally, the use of correlated samples is investigated for variance reduction to improve the convergence of the Markov Chain. Testing on a diverse set of data sets, speed-up is achieved for almost all of them without significant loss in predictive performance.

stat.CO↗

arXiv2020

A multishift, multipole rational QZ method with aggressive early deflation

The rational QZ method generalizes the QZ method by implicitly supporting rational subspace iteration. In this paper we extend the rational QZ method by introducing shifts and poles of higher multiplicity in the Hessenberg pencil, which is a pencil consisting of two Hessenberg matrices. The result is a multishift, multipole iteration on block Hessenberg pencils which allows one to stick to real arithmetic for a real input pencil. In combination with optimally packed shifts and aggressive early deflation as an advanced deflation technique we obtain an efficient method for the dense generalized eigenvalue problem. In the numerical experiments we compare the results with state-of-the-art routines for the generalized eigenvalue problem and show that we are competitive in terms of speed and accuracy.

math.NA↗

arXiv2019

Calculating the minimal/maximal eigenvalue of symmetric parametrized matrices using projection

In applications of linear algebra including nuclear physics and structural dynamics, there is a need to deal with uncertainty in the matrices. We focus on matrices that depend on a set of parameters $ω$ and we are interested in the minimal eigenvalue of a matrix pencil $(A,B)$ with $A,B$ symmetric and $B$ positive definite. If $ω$ can be interpreted as the realisation of random variables, one may be interested in statistical moments of the minimal eigenvalue. In order to obtain statistical moments, we need a fast evaluation of the eigenvalue as a function of $ω$. Since this is costly for large matrices,we are looking for a small parametrized eigenvalue problem whose minimal eigenvalue makes a small error with the minimal eigenvalue of the large eigenvalue problem. The advantage, in comparison with a global polynomial approximation (on which, e.g., the polynomial chaos approximation relies), is that we do not suffer from the possible non-smoothness of the minimal eigenvalue. The small scale eigenvalue problem is obtained by projection of the large scale problem. Our main contribution is that for constructing the subspace we use multiple eigenvectors as well as derivatives of eigenvectors.We provide theoretical results and document numerical experiments regarding the beneficial effect of adding multiple eigenvectors and derivatives.

math.NA↗

arXiv2019

Subspace Methods for 3-Parameter Eigenvalue Problems

We propose subspace methods for 3-parameter eigenvalue problems. Such problems arise when separation of variables is applied to separable boundary value problems; a particular example is the Helmholtz equation in ellipsoidal and paraboloidal coordinates. While several subspace methods for 2-parameter eigenvalue problems exist, their extensions to three parameter setting seem to be challenging. An inherent difficulty is that, while for 2-parameter eigenvalue problems we can exploit a relation to Sylvester equations to obtain a fast Arnoldi type method, such a relation does not seem to exist when there are three or more parameters. Instead, we introduce a subspace iteration method with projections onto generalized Krylov subspaces that are constructed from scratch at every iteration using certain Ritz vectors as the initial vectors. Another possibility is a Jacobi--Davidson type method for three or more parameters, which we generalize from its 2-parameter counterpart. For both approaches, we introduce a selection criterion for deflation that is based on the angles between left and right eigenvectors. The Jacobi--Davidson approach is devised to locate eigenvalues close to a prescribed target, yet it often also performs well when eigenvalues are sought based on the proximity of one of the components to a prescribed target. The subspace iteration method is devised specifically for the latter task. The proposed approaches are suitable especially for problems where the computation of several eigenvalues is required with high accuracy. Matlab implementations of both methods have been made available in the package MultiParEig.

math.NA↗

arXiv2018

A rational QZ method

We propose a rational QZ method for the solution of the dense, unsymmetric generalized eigenvalue problem. This generalization of the classical QZ method operates implicitly on a Hessenberg, Hessenberg pencil instead of on a Hessenberg, triangular pencil. Whereas the QZ method performs nested subspace iteration driven by a polynomial, the rational QZ method allows for nested subspace iteration driven by a rational function, this creates the additional freedom of selecting poles. In this article we study Hessenberg, Hessenberg pencils, link them to rational Krylov subspaces, propose a direct reduction method to such a pencil, and introduce the implicit rational QZ step. The link with rational Krylov subspaces allows us to prove essential uniqueness (implicit Q theorem) of the rational QZ iterates as well as convergence of the proposed method. In the proofs, we operate directly on the pencil instead of rephrasing it all in terms of a single matrix. Numerical experiments are included to illustrate competitiveness in terms of speed and accuracy with the classical approach. Two other types of experiments exemplify new possibilities. First we illustrate that good pole selection can be used to deflate the original problem during the reduction phase, and second we use the rational QZ method to implicitly filter a rational Krylov subspace in an iterative method.

math.NA↗

arXiv2018

Fast semi-supervised discriminant analysis for binary classification of large data-sets

High-dimensional data requires scalable algorithms. We propose and analyze three scalable and related algorithms for semi-supervised discriminant analysis (SDA). These methods are based on Krylov subspace methods which exploit the data sparsity and the shift-invariance of Krylov subspaces. In addition, the problem definition was improved by adding centralization to the semi-supervised setting. The proposed methods are evaluated on a industry-scale data set from a pharmaceutical company to predict compound activity on target proteins. The results show that SDA achieves good predictive performance and our methods only require a few seconds, significantly improving computation time on previous state of the art.

cs.AI↗

arXiv2018

Automatic rational approximation and linearization of nonlinear eigenvalue problems

We present a method for solving nonlinear eigenvalue problems using rational approximation. The method uses the AAA method by Nakatsukasa, Sète, and Trefethen to approximate the nonlinear eigenvalue problem by a rational eigenvalue problem and is embedded in the state space representation of a rational polynomial by Su and Bai. The advantage of the method, compared to related techniques such as NLEIGS and infinite Arnoldi, is the efficient computation by an automatic procedure. In addition, a set-valued approach is developed that allows building a low degree rational approximation of a nonlinear eigenvalue problem. The method perfectly fits the framework of the Compact rational Krylov methods (CORK and TS-CORK), allowing to efficiently solve large scale nonlinear eigenvalue problems. Numerical examples show that the presented framework is competitive with NLEIGS and usually produces smaller linearizations with the same accuracy but with less effort for the user.

math.NA↗

arXiv2017

Mixed forward-backward stability of the two-level orthogonal Arnoldi method for quadratic problems

We revisit the numerical stability of the two-level orthogonal Arnoldi (TOAR) method for computing an orthonormal basis of a second--order Krylov subspace associated with two given matrices. We show that the computed basis is close (on certain subspace metric sense) to a basis for a second-order Krylov subspace associated with nearby coefficient matrices, provided that the norms of the given matrices are not too large or too small. Thus, the results in this work provide for the first time conditions that guarantee the numerical stability of the TOAR method in computing orthonormal bases of second-order Krylov subspaces. We also study scaling the quadratic problem for improving the numerical stability of the TOAR procedure when the norms of the matrices are too large or too small. We show that in many cases the TOAR procedure applied to scaled matrices is numerically stable when the scaling introduced by Fan, Lin and Van Dooren is used.

math.NA↗

arXiv2017

A Subspace Method for Large Scale Eigenvalue Optimization

We consider the minimization or maximization of the $J$th largest eigenvalue of an analytic and Hermitian matrix-valued function, and build on Mengi et al. (2014, SIAM J. Matrix Anal. Appl., 35, 699-724). This work addresses the setting when the matrix-valued function involved is very large. We describe subspace procedures that convert the original problem into a small-scale one by means of orthogonal projections and restrictions to certain subspaces, and that gradually expand these subspaces based on the optimal solutions of small-scale problems. Global convergence and superlinear rate-of-convergence results with respect to the dimensions of the subspaces are presented in the infinite dimensional setting, where the matrix-valued function is replaced by a compact operator depending on parameters. In practice, it suffices to solve eigenvalue optimization problems involving matrices with sizes on the scale of tens, instead of the original problem involving matrices with sizes on the scale of thousands.

math.NA↗

arXiv2012

Computing a partial Schur factorization of nonlinear eigenvalue problems using the infinite Arnoldi method

The partial Schur factorization can be used to represent several eigenpairs of a matrix in a numerically robust way. Different adaptions of the Arnoldi method are often used to compute partial Schur factorizations. We propose here a technique to compute a partial Schur factorization of a nonlinear eigenvalue problem (NEP). The technique is inspired by the algorithm in [8], now called the infinite Arnoldi method. The infinite Arnoldi method is a method designed for NEPs, and can be interpreted as Arnoldi's method applied to a linear infinite-dimensional operator, whose reciprocal eigenvalues are the solutions to the NEP. As a first result we show that the invariant pairs of the operator are equivalent to invariant pairs of the NEP. We characterize the structure of the invariant pairs of the operator and show how one can carry out a modification of the infinite Arnoldi method by respecting the structure. This also allows us to naturally add the feature known as locking. We nest this algorithm with an outer iteration, where the infinite Arnoldi method for a particular type of structured functions is appropriately restarted. The restarting exploits the structure and is inspired by the well-known implicitly restarted Arnoldi method for standard eigenvalue problems. The final algorithm is applied to examples from a benchmark collection, showing that both processing time and memory consumption can be considerably reduced with the restarting technique.

math.NA↗