arXiv subjects

Petr Tichý

Publications and source records attributed to Petr Tichý.

15 recordsLinked to original sources

arXiv2025

On finite precision block Lanczos computations

In her seminal 1989 work, Greenbaum demonstrated that the results produced by the finite precision Lanczos algorithm after $k$ iterations can be interpreted as exact Lanczos results applied to a larger matrix, whose eigenvalues lie in small intervals around those of the original matrix. This establishes a mathematical model for finite precision Lanczos computations. In this paper, we extend these ideas to the block Lanczos algorithm. We generalize the continuation process and show that it can be completed in a finite number of iterations using carefully constructed perturbations. The block tridiagonal matrices produced after $k$ iterations can then be interpreted as arising from the exact block Lanczos algorithm applied to a larger model matrix. We derive sufficient conditions under which the required perturbations remain small, ensuring that the eigenvalues of the model matrix stay close to those of the original matrix. While in the single-vector case these conditions are always satisfiable, as shown by Greenbaum based on results by Paige, the question of whether they can always be satisfied in the block case remains open. Finally, we present numerical experiments demonstrating a practical implementation of the continuation process and empirically assess the validity of the sufficient conditions and the size of the perturbations.

math.NA↗

arXiv2025

Matrix best approximation in the spectral norm

We derive, similar to Lau and Riha, a matrix formulation of a general best approximation theorem of Singer for the special case of spectral approximations of a given matrix from a given subspace. Using our matrix formulation we describe the relation of the spectral approximation problem to semidefinite programming, and we present a simple MATLAB code to solve the problem numerically. We then obtain geometric characterizations of spectral approximations that are based on the $k$-dimensional field of $k$ matrices, which we illustrate with several numerical examples. The general spectral approximation problem is a min-max problem, whose value is bounded from below by the corresponding max-min problem. Using our geometric characterizations of spectral approximations, we derive several necessary and sufficient as well as sufficient conditions for equality of the max-min and min-max values. Finally, we prove that the max-min and min-max values are always equal when we ``double'' the problem. Several results in this paper generalize results that have been obtained in the convergence analysis of the GMRES method for solving linear algebraic systems.

math.NA↗

arXiv2025

Block CG algorithms revisited

Our goal in this paper is to clarify the relationship between the block Lanczos and the block conjugate gradient (BCG) algorithms. Under the full rank assumption for the block vectors, we show the one-to-one correspondence between the algorithms. This allows, for example, the computation of the block Lanczos coefficients in BCG. The availability of block Jacobi matrices in BCG opens the door for further development, e.g., for error estimation in BCG based on (modified) block Gauss quadrature rules. Driven by the need to get a practical variant of the BCG algorithm well suited for computations in finite precision arithmetic, we also discuss some important variants of BCG due to Dubrulle. These variants avoid the troubles with a possible rank deficiency within the block vectors. We show how to incorporate preconditioning and computation of Lanczos coefficients into these variants. We hope to help clarify which variant of the block conjugate gradient algorithm should be used for computations in finite precision arithmetic. Numerical results illustrate the performance of different variants of BCG on some examples.

math.NA↗

arXiv2025

Error norm estimates for the block conjugate gradient algorithm

In the book [Meurant and Tichy, SIAM, 2024] we discussed the estimation of error norms in the conjugate gradient (CG) algorithm for solving linear systems $Ax=b$ with a symmetric positive definite matrix $A$, where $b$ and $x$ are vectors. In this paper, we generalize the most important formulas for estimating the $A$-norm of the error to the block case. First, we discuss in detail the derivation of various variants of the block CG (BCG) algorithm from the block Lanczos algorithm. We then consider BCG and derive the related block Gauss and block Gauss-Radau quadrature rules. We show how to obtain lower and upper bounds on the $A$-norm of the error of each system, both in terms of the quantities computed in BCG and in terms of the underlying block Lanczos algorithm. Numerical experiments demonstrate the behavior of the bounds in practical computations.

math.NA↗

arXiv2023

Estimating the error in CG-like algorithms for least-squares and least-norm problems

In [Meurant, Papež, Tichý; Numerical Algorithms 88, 2021], we presented an adaptive estimate for the energy norm of the error in the conjugate gradient (CG) method. In this paper, we extend the estimate to algorithms for solving linear approximation problems with a general, possibly rectangular matrix that are based on applying CG to a system with a positive (semi-)definite matrix build from the original matrix. We show that the resulting estimate preserves its key properties: it can be very cheaply evaluated, and it is numerically reliable in finite-precision arithmetic under some mild assumptions. We discuss algorithms based on Hestenes-Stiefel-like implementation (often called CGLS and CGNE in the literature) as well as on bidiagonalization (LSQR and CRAIG), and both unpreconditioned and preconditioned variants. The numerical experiments confirm the robustness and very satisfactory behaviour of the estimate.

math.NA↗

arXiv2023

The behaviour of the Gauss-Radau upper bound of the error norm in CG

Consider the problem of solving systems of linear algebraic equations $Ax=b$ with a real symmetric positive definite matrix $A$ using the conjugate gradient (CG) method. To stop the algorithm at the appropriate moment, it is important to monitor the quality of the approximate solution. One of the most relevant quantities for measuring the quality of the approximate solution is the $A$-norm of the error. This quantity cannot be easily computed, however, it can be estimated. In this paper we discuss and analyze the behaviour of the Gauss-Radau upper bound on the $A$-norm of the error, based on viewing CG as a procedure for approximating a certain Riemann-Stieltjes integral. This upper bound depends on a prescribed underestimate $μ$ to the smallest eigenvalue of $A$. We concentrate on explaining a phenomenon observed during computations showing that, in later CG iterations, the upper bound loses its accuracy, and is almost independent of $μ$. We construct a model problem that is used to demonstrate and study the behaviour of the upper bound in dependence of $μ$, and developed formulas that are helpful in understanding this behavior. We show that the above mentioned phenomenon is closely related to the convergence of the smallest Ritz value to the smallest eigenvalue of $A$. It occurs when the smallest Ritz value is a better approximation to the smallest eigenvalue than the prescribed underestimate $μ$. We also suggest an adaptive strategy for improving the accuracy of the upper bounds in the previous iterations.

math.NA↗

arXiv2022

On the Forsythe conjecture

Forsythe formulated a conjecture about the asymptotic behavior of the restarted conjugate gradient method in 1968. We translate several of his results into modern terms, and generalize the conjecture (originally formulated only for symmetric positive definite matrices) to symmetric and nonsymmetric matrices. Our generalization is based on a two-sided or cross iteration with the given matrix and its transpose, which is based on the projection process used in the Arnoldi (or for symmetric matrices the Lanczos) algorithm. We prove several new results about the limiting behavior of this iteration, but the conjecture still remains largely open.

math.NA↗

arXiv2021

When does the Lanczos algorithm compute exactly?

In theory, the Lanczos algorithm generates an orthogonal basis of the corresponding Krylov subspace. However, in finite precision arithmetic, the orthogonality and linear independence of the computed Lanczos vectors is usually lost quickly. In this paper we study a class of matrices and starting vectors having a special nonzero structure that guarantees exact computations of the Lanczos algorithm whenever floating point arithmetic satisfying the IEEE 754 standard is used. Analogous results are formulated also for a variant of the conjugate gradient method that produces then almost exact results. The results are extended to the Arnoldi algorithm, the nonsymmetric Lanczos algorithm, the Golub-Kahan bidiagonalization, the block-Lanczos algorithm and their counterparts for solving linear systems.

math.NA↗

arXiv2021

Accurate error estimation in CG

In practical computations, the (preconditioned) conjugate gradient (P)CG method is the iterative method of choice for solving systems of linear algebraic equations $Ax=b$ with a real symmetric positive definite matrix $A$. During the iterations it is important to monitor the quality of the approximate solution $x_k$ so that the process could be stopped whenever $x_k$ is accurate enough. One of the most relevant quantities for monitoring the quality of $x_k$ is the squared $A$-norm of the error vector $x-x_k$. This quantity cannot be easily evaluated, however, it can be estimated. Many of the existing estimation techniques are inspired by the view of CG as a procedure for approximating a certain Riemann--Stieltjes integral. The most natural technique is based on the Gauss quadrature approximation and provides a lower bound on the quantity of interest. The bound can be cheaply evaluated using terms that have to be computed anyway in the forthcoming CG iterations. If the squared $A$-norm of the error vector decreases rapidly, then the lower bound represents a tight estimate. In this paper we suggest a heuristic strategy aiming to answer the question of how many forthcoming CG iterations are needed to get an estimate with the prescribed accuracy. Numerical experiments demonstrate that the suggested strategy is efficient and robust.

math.NA↗

arXiv2020

The field of values bound on ideal GMRES

A widely known result of Elman, and its improvements due to Starke, Eiermann and Ernst, gives a bound on the worst-case GMRES residual norm using quantities related to the field of values of the given matrix and its inverse. We prove that these bounds also hold for the ideal GMRES approximation, and we derive and discuss some improvements of the bounds.

math.NA↗

arXiv2020

On efficient numerical solution of linear algebraic systems arising in goal-oriented error estimates

We deal with the numerical solution of linear partial differential equations (PDEs) with focus on the goal-oriented error estimates including algebraic errors arising by an inaccurate solution of the corresponding algebraic systems. The goal-oriented error estimates require the solution of the primal as well as dual algebraic systems. We solve both systems simultaneously using the bi-conjugate gradient method which allows to control the algebraic errors of both systems. We develop a stopping criterion which is cheap to evaluate and guarantees that the estimation of the algebraic error is smaller than the estimation of the discretization error. Using this criterion and an adaptive mesh refinement technique, we obtain an efficient and robust method for the numerical solution of PDEs, which is demonstrated by several numerical experiments.

math.NA↗

arXiv2019

Analysis of the multiplicative Schwarz method for matrices with a special block structure

We analyze the convergence of the (algebraic) multiplicative Schwarz method applied to linear algebraic systems with matrices having a special block structure that arises, for example, when a (partial) differential equation is posed and discretized on a domain that consists of two subdomains with an overlap. This is a basic situation in the context of domain decomposition methods. Our analysis is based on the algebraic structure of the Schwarz iteration matrices, and we derive error bounds that are based on the block diagonal dominance of the given system matrix. Our analysis does not assume that the system matrix is symmetric (positive definite), or has the $M$- or $H$-matrix property. Our approach is motivated by and significantly generalizes an analysis for a special one-dimensional model problem given in [4].

math.NA↗

arXiv2018

Approximating the extreme Ritz values and upper bounds for the $A$-norm of the error in CG

In practical conjugate gradient (CG) computations it is important to monitor the quality of the approximate solution to $Ax=b$ so that the CG algorithm can be stopped when the required accuracy is reached. The relevant convergence characteristics, like the $A$-norm of the error or the normwise backward error, cannot be easily computed. However, they can be estimated. Such estimates often depend on approximations of the smallest or largest eigenvalue of~$A$. In the paper we introduce a new upper bound for the $A$-norm of the error, which is closely related to the Gauss-Radau upper bound, and discuss the problem of choosing the parameter $μ$ which should represent a lower bound for the smallest eigenvalue of $A$.The new bound has several practical advantages, the most important one is that it can be used as an approximation to the $A$-norm of the error even if $μ$ is not exactly a lower bound for the smallest eigenvalue of $A$. In this case, $μ$ can be chosen, e.g., as the smallest Ritz value or its approximation. We also describe a very cheap algorithm, based on the incremental norm estimation technique, which allows to estimate the smallest and largest Ritz values during the CG computations. An improvement of the accuracy of these estimates of extreme Ritz values is possible, at the cost of storing the CG coefficients and solving a linear system with a tridiagonal matrix at each CG iteration. Finally, we discuss how to cheaply approximate the normwise backward error. The numerical experiments demonstrate the efficiency of the estimates of the extreme Ritz values, and show their practical use in error estimation in CG.

math.NA↗

arXiv2013

Max-min and min-max approximation problems for normal matrices revisited

We give a new proof for an equality of certain max-min and min-max approximation problems involving normal matrices. The previously published proofs of this equality apply tools from matrix theory, (analytic) optimization theory and constrained convex optimization. Our proof uses a classical characterization theorem from approximation theory and thus exploits the link between the two approximation problems with normal matrices on the one hand and approximation problems on compact sets in the complex plane on the other.

math.NA↗

arXiv2013

Characterization of worst-case GMRES

Given a matrix $A$ and iteration step $k$, we study a best possible attainable upper bound on the GMRES residual norm that does not depend on the initial vector $b$. This quantity is called the worst-case GMRES approximation. We show that the worst case behavior of GMRES for the matrices $A$ and $A^T$ is the same, and we analyze properties of initial vectors for which the worst-case residual norm is attained. In particular, we show that such vectors satisfy a certain "cross equality", and we characterize them as right singular vectors of the corresponding GMRES residual matrix. We show that the worst-case GMRES polynomial may not be uniquely determined, and we consider the relation between the worst-case and the ideal GMRES approximations, giving new examples in which the inequality between the two quantities is sharp at all iteration steps $k\geq 3$. Finally, we give a complete characterization of how the values of the approximation problems in the context of worst-case and ideal GMRES for a real matrix change, when one considers complex (rather than real) polynomials and initial vectors in these problems.

math.NA↗