arXiv subjects

Zhongxiao Jia

Publications and source records attributed to Zhongxiao Jia.

48 records · Page 3Linked to original sources

arXiv2015

Robust Dropping Criteria for F-norm Minimization Based Sparse Approximate Inverse Preconditioning

Dropping tolerance criteria play a central role in Sparse Approximate Inverse preconditioning. Such criteria have received, however, little attention and have been treated heuristically in the following manner: If the size of an entry is below some empirically small positive quantity, then it is set to zero. The meaning of "small" is vague and has not been considered rigorously. It has not been clear how dropping tolerances affect the quality and effectiveness of a preconditioner $M$. In this paper, we focus on the adaptive Power Sparse Approximate Inverse algorithm and establish a mathematical theory on robust selection criteria for dropping tolerances. Using the theory, we derive an adaptive dropping criterion that is used to drop entries of small magnitude dynamically during the setup process of $M$. The proposed criterion enables us to make $M$ both as sparse as possible as well as to be of comparable quality to the potentially denser matrix which is obtained without dropping. As a byproduct, the theory applies to static F-norm minimization based preconditioning procedures, and a similar dropping criterion is given that can be used to sparsify a matrix after it has been computed by a static sparse approximate inverse procedure. In contrast to the adaptive procedure, dropping in the static procedure does not reduce the setup time of the matrix but makes the application of the sparser $M$ for Krylov iterations cheaper. Numerical experiments reported confirm the theory and illustrate the robustness and effectiveness of the dropping criteria.

math.NA↗

arXiv2015

On the Convergence of Ritz Pairs and Refined Ritz Vectors for Quadratic Eigenvalue Problems

For a given subspace, the Rayleigh-Ritz method projects the large quadratic eigenvalue problem (QEP) onto it and produces a small sized dense QEP. Similar to the Rayleigh-Ritz method for the linear eigenvalue problem, the Rayleigh-Ritz method defines the Ritz values and the Ritz vectors of the QEP with respect to the projection subspace. We analyze the convergence of the method when the angle between the subspace and the desired eigenvector converges to zero. We prove that there is a Ritz value that converges to the desired eigenvalue unconditionally but the Ritz vector converges conditionally and may fail to converge. To remedy the drawback of possible non-convergence of the Ritz vector, we propose a refined Ritz vector that is mathematically different from the Ritz vector and is proved to converge unconditionally. We construct examples to illustrate our theory.

math.NA↗

arXiv2015

On Convergence of the Inexact Rayleigh Quotient Iteration with the Lanczos Method Used for Solving Linear Systems

For the Hermitian inexact Rayleigh quotient iteration (RQI), the author has established new local general convergence results, independent of iterative solvers for inner linear systems. The theory shows that the method locally converges quadratically under a new condition, called the uniform positiveness condition. In this paper we first consider the local convergence of the inexact RQI with the unpreconditioned Lanczos method for the linear systems. Some attractive properties are derived for the residuals, whose norms are $ξ_{k+1}$'s, of the linear systems obtained by the Lanczos method. Based on them and the new general convergence results, we make a refined analysis and establish new local convergence results. It is proved that the inexact RQI with Lanczos converges quadratically provided that $ξ_{k+1}\leqξ$ with a constant $ξ\geq 1$. The method is guaranteed to converge linearly provided that $ξ_{k+1}$ is bounded by a small multiple of the reciprocal of the residual norm $\|r_k\|$ of the current approximate eigenpair. The results are fundamentally different from the existing convergence results that always require $ξ_{k+1}<1$, and they have a strong impact on effective implementations of the method. We extend the new theory to the inexact RQI with a tuned preconditioned Lanczos for the linear systems. Based on the new theory, we can design practical criteria to control $ξ_{k+1}$ to achieve quadratic convergence and implement the method more effectively than ever before. Numerical experiments confirm our theory.

math.NA↗

arXiv2015

On Inner Iterations in the Shift-Invert Residual Arnoldi Method and the Jacobi--Davidson Method

Using a new analysis approach, we establish a general convergence theory of the Shift-Invert Residual Arnoldi (SIRA) method for computing a simple eigenvalue nearest to a given target $σ$ and the associated eigenvector. In SIRA, a subspace expansion vector at each step is obtained by solving a certain inner linear system. We prove that the inexact SIRA method mimics the exact SIRA well, that is, the former uses almost the same outer iterations to achieve the convergence as the latter does if all the inner linear systems are iteratively solved with {\em low} or {\em modest} accuracy during outer iterations. Based on the theory, we design practical stopping criteria for inner solves. Our analysis is on one step expansion of subspace and the approach applies to the Jacobi--Davidson (JD) method with the fixed target $σ$ as well, and a similar general convergence theory is obtained for it. Numerical experiments confirm our theory and demonstrate that the inexact SIRA and JD are similarly effective and are considerably superior to the inexact SIA.

math.NA↗

arXiv2015

Harmonic and Refined Harmonic Shift-Invert Residual Arnoldi and Jacobi--Davidson Methods for Interior Eigenvalue Problems

This paper concerns the harmonic shift-invert residual Arnoldi (HSIRA) and Jacobi--Davidson (HJD) methods as well as their refined variants RHSIRA and RHJD for the interior eigenvalue problem. Each method needs to solve an inner linear system to expand the subspace successively. When the linear systems are solved only approximately, we are led to the inexact methods. We prove that the inexact HSIRA, RHSIRA, HJD and RHJD methods mimic their exact counterparts well when the inner linear systems are solved with only low or modest accuracy. We show that (i) the exact HSIRA and HJD expand subspaces better than the exact SIRA and JD and (ii) the exact RHSIRA and RHJD expand subspaces better than the exact HSIRA and HJD. Based on the theory, we design stopping criteria for inner solves. To be practical, we present restarted HSIRA, HJD, RHSIRA and RHJD algorithms. Numerical results demonstrate that these algorithms are much more efficient than the restarted standard SIRA and JD algorithms and furthermore the refined harmonic algorithms outperform the harmonic ones very substantially.

math.NA↗

arXiv2015

Implicitly Restarted Generalized Second-order Arnoldi Type Algorithms for the Quadratic Eigenvalue Problem

We investigate the generalized second-order Arnoldi (GSOAR) method, a generalization of the SOAR method proposed by Bai and Su [{\em SIAM J. Matrix Anal. Appl.}, 26 (2005): 640--659.], and the Refined GSOAR (RGSOAR) method for the quadratic eigenvalue problem (QEP). The two methods use the GSOAR procedure to generate an orthonormal basis of a given generalized second-order Krylov subspace, and with such basis they project the QEP onto the subspace and compute the Ritz pairs and the refined Ritz pairs, respectively. We develop implicitly restarted GSOAR and RGSOAR algorithms, in which we propose certain exact and refined shifts for respective use within the two algorithms. Numerical experiments on real-world problems illustrate the efficiency of the restarted algorithms and the superiority of the restarted RGSOAR to the restarted GSOAR. The experiments also demonstrate that both IGSOAR and IRGSOAR generally perform much better than the implicitly restarted Arnoldi method applied to the corresponding linearization problems, in terms of the accuracy and the computational efficiency.

math.NA↗

arXiv2015

Some Results on Regularization of LSQR and CGLS for Large-Scale Discrete Ill-Posed Problems

For large-scale discrete ill-posed problems, LSQR, a Lanczos bidiagonalization process based Krylov method, is most often used. It is well known that LSQR has natural regularizing properties, where the number of iterations plays the role of the regularization parameter. In this paper, for severely and moderately ill-posed problems, we establish quantitative bounds for the distance between the $k$-dimensional Krylov subspace and the subspace spanned by $k$ dominant right singular vectors. They show that the $k$-dimensional Krylov subspace may capture the $k$ dominant right singular vectors for severely and moderately ill-posed problems, but it seems not the case for mildly ill-posed problems. These results should be the first step towards to estimating the accuracy of the rank-$k$ approximation generated by Lanczos bidiagonalization. We also derive some other results, which help further understand the regularization effects of LSQR. We draw to a conclusion that a hybrid LSQR should generally be used for mildly ill-posed problems. We report numerical experiments to confirm our theory. We present more definitive and general observed phenomena, which will derive more research.

math.NA↗

arXiv2014

A positivity preserving inexact Noda iteration for computing the smallest eigenpair of a large irreducible M-matrix

In this paper, based on the Noda iteration, we present inexact Noda iterations (INI), to find the smallest eigenvalue and the associated positive eigenvector of a large irreducible nonsingular M-matrix. The positivity of approximations is critical in applications, and if the approximations lose the positivity then they will be physically meaningless. We propose two different inner tolerance strategies for solving the inner linear systems involved, and prove that the resulting INI algorithms are globally linear and superlinear with the convergence order $\frac{1+\sqrt{5}}{2}$, respectively. The proposed INI algorithms are structure preserving and maintains the positivity of approximate eigenvectors. We also revisit the exact Noda iteration and establish a new quadratic convergence result. All the above is first done for the problem of computing the Perron root and the positive Perron vector of an irreducible nonnegative matrix and then adapted to that of computing the smallest eigenpair of the irreducible nonsingular M-matrix. Numerical examples illustrate that the proposed INI algorithms are practical, and they always preserve the positivity of approximate eigenvectors. We compare them with the positivity non-preserving Jacobi--Davidson method and implicitly restarted Arnoldi method, which often compute physically meaningless eigenvectors, and illustrate that the overall efficiency of the INI algorithms is competitive with and can be considerably higher than the latter two methods.

math.NA↗

arXiv2014

A Posteriori Error Estimates of Krylov Subspace Approximations to Matrix Functions

Krylov subspace methods for approximating a matrix function $f(A)$ times a vector $v$ are analyzed in this paper. For the Arnoldi approximation to $e^{-τA}v$, two reliable a posteriori error estimates are derived from the new bounds and generalized error expansion we establish. One of them is similar to the residual norm of an approximate solution of the linear system, and the other one is determined critically by the first term of the error expansion of the Arnoldi approximation to $e^{-τA}v$ due to Saad. We prove that each of the two estimates is reliable to measure the true error norm, and the second one theoretically justifies an empirical claim by Saad. In the paper, by introducing certain functions $ϕ_k(z)$ defined recursively by the given function $f(z)$ for certain nodes, we obtain the error expansion of the Krylov-like approximation for $f(z)$ sufficiently smooth, which generalizes Saad's result on the Arnoldi approximation to $e^{-τA}v$. Similarly, it is shown that the first term of the generalized error expansion can be used as a reliable a posteriori estimate for the Krylov-like approximation to some other matrix functions times $v$. Numerical examples are reported to demonstrate the effectiveness of the a posteriori error estimates for the Krylov-like approximations to $e^{-τA}v$, $\cos(A)v$ and $\sin(A)v$.

math.NA↗

arXiv2011

A contribution to the condition number of the total least squares problem

This paper concerns cheaply computable formulas and bounds for the condition number of the TLS problem. For a TLS problem with data $A$, $b$, two formulas are derived that are simpler and more compact than the known results in the literature. One is derived by exploiting the properties of Kronecker products of matrices. The other is obtained by making use of the singular value decomposition (SVD) of $[A \,\,b]$, which allows us to compute the condition number cheaply and accurately. We present lower and upper bounds for the condition number that involve the singular values of $[A \,\, b]$ and the last entries of the right singular vectors of $[A \,\, b]$. We prove that they are always sharp and can estimate the condition number accurately by no more than four times. Furthermore, we establish a few other lower and upper bounds that involve only a few singular values of $A$ and $[A \,\, b]$. We discuss how tight the bounds are. These bounds are particularly useful for large scale TLS problems since for them any formulas and bounds for the condition number involving all the singular values of $A$ and/or $[A \ b]$ are too costly to be computed. Numerical experiments illustrate that our bounds are sharper than a known approximate condition number in the literature.

math.NA↗

arXiv2010

On Convergence of the Inexact Rayleigh Quotient Iteration with MINRES

For the Hermitian inexact Rayleigh quotient iteration (RQI), we present a new general theory, independent of iterative solvers for shifted inner linear systems. The theory shows that the method converges at least quadratically under a new condition, called the uniform positiveness condition, that may allow inner tolerance $ξ_k\geq 1$ at outer iteration $k$ and can be considerably weaker than the condition $ξ_k\leqξ<1$ with $ξ$ a constant not near one commonly used in literature. We consider the convergence of the inexact RQI with the unpreconditioned and tuned preconditioned MINRES method for the linear systems. Some attractive properties are derived for the residuals obtained by MINRES. Based on them and the new general theory, we make a more refined analysis and establish a number of new convergence results. Let $\|r_k\|$ be the residual norm of approximating eigenpair at outer iteration $k$. Then all the available cubic and quadratic convergence results require $ξ_k=O(\|r_k\|)$ and $ξ_k\leqξ$ with a fixed $ξ$ not near one, respectively. Fundamentally different from these, we prove that the inexact RQI with MINRES generally converges cubically, quadratically and linearly provided that $ξ_k\leqξ$ with a constant $ξ<1$ not near one, $ξ_k=1-O(\|r_k\|)$ and $ξ_k=1-O(\|r_k\|^2)$, respectively. Therefore, the new convergence conditions are much more relaxed than ever before. The theory can be used to design practical stopping criteria to implement the method more effectively. Numerical experiments confirm our results.

math.NA↗

arXiv2009

A Refined Harmonic Lanczos Bidiagonalization Method and an Implicitly Restarted Algorithm for Computing the Smallest Singular Triplets of Large Matrices

The harmonic Lanczos bidiagonalization method can be used to compute the smallest singular triplets of a large matrix $A$. We prove that for good enough projection subspaces harmonic Ritz values converge if the columns of $A$ are strongly linearly independent. On the other hand, harmonic Ritz values may miss some desired singular values when the columns of $A$ almost linearly dependent. Furthermore, harmonic Ritz vectors may converge irregularly and even may fail to converge. Based on the refined projection principle for large matrix eigenproblems due to the first author, we propose a refined harmonic Lanczos bidiagonalization method that takes the Rayleigh quotients of the harmonic Ritz vectors as approximate singular values and extracts the best approximate singular vectors, called the refined harmonic Ritz approximations, from the given subspaces in the sense of residual minimizations. The refined approximations are shown to converge to the desired singular vectors once the subspaces are sufficiently good and the Rayleigh quotients converge. An implicitly restarted refined harmonic Lanczos bidiagonalization algorithm (IRRHLB) is developed. We study how to select the best possible shifts, and suggest refined harmonic shifts that are theoretically better than the harmonic shifts used within the implicitly restarted Lanczos bidiagonalization algorithm (IRHLB). We propose a novel procedure that can numerically compute the refined harmonic shifts efficiently and accurately. Numerical experiments are reported that compare IRRHLB with five other algorithms based on the Lanczos bidiagonalization process. It appears that IRRHLB is at least competitive with them and can be considerably more efficient when computing the smallest singular triplets.

math.NA↗