arXiv subjects

Keiichi Morikuni

Publications and source records attributed to Keiichi Morikuni.

17 recordsLinked to original sources

arXiv2025

Error estimate for regularized optimal transport problems via Bregman divergence

Regularization by the Shannon entropy enables us to efficiently and approximately solve optimal transport problems on a finite set. This paper is concerned with regularized optimal transport problems via Bregman divergence. We introduce the required properties for Bregman divergences, provide a non-asymptotic error estimate for the regularized problem, and show that the error estimate becomes faster than exponentially.

math.OC↗

arXiv2024

Block cross-interactive residual smoothing for Lanczos-type solvers for linear systems with multiple right-hand sides

Lanczos-type solvers for large sparse linear systems often exhibit large oscillations in the residual norms. In finite precision arithmetic, large oscillations increase the residual gap (the difference between the recursively updated residual and the explicitly computed residual) and a loss of attainable accuracy of the approximations. This issue is addressed using cross-interactive residual smoothing (CIRS). This approach improves convergence behavior and reduces the residual gap. Similar to how the standard Lanczos-type solvers have been extended to global and block versions for solving systems with multiple right-hand sides, CIRS can also be extended to these versions. While we have developed a global CIRS scheme (Gl-CIRS) in our previous study [K. Aihara, A. Imakura, and K. Morikuni, SIAM J. Matrix Anal. Appl., 43 (2022), pp.1308--1330], in this study, we propose a block version (Bl-CIRS). Subsequently, we demonstrate the effectiveness of Bl-CIRS from various perspectives, such as theoretical insights into the convergence behaviors of the residual and approximation norms, numerical experiments on model problems, and a detailed rounding error analysis for the residual gap. For Bl-CIRS, orthonormalizing the columns of direction matrices is crucial in effectively reducing the residual gap. This analysis also complements our previous study and evaluates the residual gap of the block Lanczos-type solvers.

math.NA↗

arXiv2022

Verified eigenvalue and eigenvector computations using complex moments and the Rayleigh$\unicode{x2013}$Ritz procedure for generalized Hermitian eigenvalue problems

We propose a verified computation method for eigenvalues in a region and the corresponding eigenvectors of generalized Hermitian eigenvalue problems. The proposed method uses complex moments to extract the eigencomponents of interest from a random matrix and uses the Rayleigh$\unicode{x2013}$Ritz procedure to project a given eigenvalue problem into a reduced eigenvalue problem. The complex moment is given by contour integral and approximated using numerical quadrature. We split the error in the complex moment into the truncation error of the quadrature and rounding errors and evaluate each. This idea for error evaluation inherits our previous Hankel matrix approach, whereas the proposed method enables verification of eigenvectors and requires half the number of quadrature points for the previous approach to reduce the truncation error to the same order. Moreover, the Rayleigh$\unicode{x2013}$Ritz procedure approach forms a transformation matrix that enables verification of the eigenvectors. Numerical experiments show that the proposed method is faster than previous methods while maintaining verification performance and works even for nearly singular matrix pencils and in the presence of multiple and nearly multiple eigenvalues.

math.NA↗

arXiv2022

Complex moment-based methods for differential eigenvalue problems

This paper considers computing partial eigenpairs of differential eigenvalue problems (DEPs) such that eigenvalues are in a certain region on the complex plane. Recently, based on a "solve-then-discretize" paradigm, an operator analogue of the FEAST method has been proposed for DEPs without discretization of the coefficient operators. Compared to conventional "discretize-then-solve" approaches that discretize the operators and solve the resulting matrix problem, the operator analogue of FEAST exhibits much higher accuracy; however, it involves solving a large number of ordinary differential equations (ODEs). In this paper, to reduce the computational costs, we propose operation analogues of Sakurai-Sugiura-type complex moment-based eigensolvers for DEPs using higher-order complex moments and analyze the error bound of the proposed methods. We show that the number of ODEs to be solved can be reduced by a factor of the degree of complex moments without degrading accuracy, which is verified by numerical results. Numerical results demonstrate that the proposed methods are over five times faster compared with the operator analogue of FEAST for several DEPs while maintaining almost the same high accuracy. This study is expected to promote the "solve-then-discretize" paradigm for solving DEPs and contribute to faster and more accurate solutions in real-world applications.

math.NA↗

arXiv2022

Cross-interactive residual smoothing for global and block Lanczos-type solvers for linear systems with multiple right-hand sides

Global and block Krylov subspace methods are efficient iterative solvers for large sparse linear systems with multiple right-hand sides. However, global or block Lanczos-type solvers often exhibit large oscillations in the residual norms and may have a large residual gap relating to the loss of attainable accuracy of the approximations. Conventional residual smoothing schemes suppress these oscillations but cannot improve the attainable accuracy, whereas a recent residual smoothing scheme enables the improvement of the attainable accuracy for single right-hand side Lanczos-type solvers. The underlying concept of this scheme is that the primary and smoothed sequences of the approximations and residuals influence one another, thereby avoiding the severe propagation of rounding errors. In the present study, we extend this cross-interactive residual smoothing to the case of solving linear systems with multiple right-hand sides. The resulting smoothed methods can reduce the residual gap with a low additional cost compared to their original counterparts. We demonstrate the effectiveness of the proposed approach through rounding error analysis and numerical experiments.

math.NA↗

arXiv2022

A stabilized GMRES method for singular and severely ill-conditioned systems of linear equations

Consider using the right-preconditioned GMRES (AB-GMRES) for obtaining the minimum-norm solution of inconsistent underdetermined systems of linear equations. Morikuni (Ph.D. thesis, 2013) showed that for some inconsistent and ill-conditioned problems, the iterates may diverge. This is mainly because the Hessenberg matrix in the GMRES method becomes very ill-conditioned so that the backward substitution of the resulting triangular system becomes numerically unstable. We propose a stabilized GMRES based on solving the normal equations corresponding to the above triangular system using the standard Cholesky decomposition. This has the effect of shifting upwards the tiny singular values of the Hessenberg matrix which lead to an inaccurate solution. We analyze why the method works. Numerical experiments show that the proposed method is robust and efficient, not only for applying AB-GMRES to underdetermined systems, but also for applying GMRES to severely ill-conditioned range-symmetric systems of linear equations.

math.NA↗

arXiv2022

GMRES Methods for Tomographic Reconstruction with an Unmatched Back Projector

Unmatched pairs of forward and back projectors are common in X-ray CT computations for large-scale problems; they are caused by the need for fast algorithms that best utilize the computer hardware, and it is an interesting and challenging task to develop fast and easy-to-use algorithms for these cases. Our approach is to use preconditioned GMRES, in the form of the AB- and BA-GMRES algorithms, to handle the unmatched normal equations associated with an unmatched pair. These algorithms are simple to implement, they rely only on computations with the available forward and back projectors, and they do not require the tuning of any algorithm parameters. We show that these algorithms are equivalent to well-known LSQR and LSMR algorithms in the case of a matched projector. Our numerical experiments demonstrate that AB- and BA-GMRES exhibit a desired semi-convergence behavior that is comparable with LSQR/LSMR and that standard stopping rules work well. Hence, AB- and BA-GMRES are suited for large-scale CT reconstruction problems with noisy data and unmatched projector pairs.

math.NA↗

arXiv2021

Projection method for eigenvalue problems of linear nonsquare matrix pencils

Eigensolvers involving complex moments can determine all the eigenvalues in a given region in the complex plane and the corresponding eigenvectors of a regular linear matrix pencil. The complex moment acts as a filter for extracting eigencomponents of interest from random vectors or matrices. This study extends a projection method for regular eigenproblems to the singular nonsquare case, thus replacing the standard matrix inverse in the resolvent with the pseudoinverse. The extended method involves complex moments given by the contour integrals of generalized resolvents associated with nonsquare matrices. We establish conditions such that the method gives all finite eigenvalues in a prescribed region in the complex plane. In numerical computations, the contour integrals are approximated using numerical quadratures. The primary cost lies in the solutions of linear least squares problems that arise from quadrature points, and they can be readily parallelized in practice. Numerical experiments on large matrix pencils illustrate this method. The new method is more robust and efficient than previous methods, and based on experimental results, it is conjectured to be more efficient in parallelized settings. Notably, the proposed method does not fail in cases involving pairs of extremely close eigenvalues, and it overcomes the issue of problem size.

math.NA↗

arXiv2021

Kaczmarz-type inner-iteration preconditioned flexible GMRES methods for consistent linear systems

We propose using greedy and randomized Kaczmarz inner-iterations as preconditioners for the right-preconditioned flexible GMRES method to solve consistent linear systems, with a parameter tuning strategy for adjusting the number of inner iterations and the relaxation parameter. We also present theoretical justifications of the right-preconditioned flexible GMRES for solving consistent linear systems. Numerical experiments on overdetermined and underdetermined linear systems show that the proposed method is superior to the GMRES method preconditioned by NE-SOR inner iterations in terms of total CPU time.

math.NA↗

arXiv2020

Shifted Lanczos method for quadratic forms with Hermitian matrix resolvents

Quadratic forms of Hermitian matrix resolvents involve the solutions of shifted linear systems. Efficient iterative solutions use the shift-invariance property of Krylov subspaces The Hermitian Lanczos method reduces a given vector and matrix to a Jacobi matrix (real symmetric tridiagonal matrix with positive super and sub-diagonal entries) and approximates the quadratic form using the Jacobi matrix. This study develops a shifted Lanczos method that deals directly with the Hermitian matrix resolvent. We derive a matrix representation of a linear operator that approximates the resolvent by solving a Vorobyev moment problem associated with the shifted Lanczos method. We show that an entry of the Jacobi matrix resolvent can approximate the quadratic form, matching the moments. We give a sufficient condition such that the method does not break down, an error bound, and error estimates. Numerical experiments on matrices drawn from real-world applications compare the proposed method with previous methods and show that the proposed method outperforms well-established methods in solving some problems.

math.NA↗

arXiv2019

Multiclass spectral feature scaling method for dimensionality reduction

Irregular features disrupt the desired classification. In this paper, we consider aggressively modifying scales of features in the original space according to the label information to form well-separated clusters in low-dimensional space. The proposed method exploits spectral clustering to derive scaling factors that are used to modify the features. Specifically, we reformulate the Laplacian eigenproblem of the spectral clustering as an eigenproblem of a linear matrix pencil whose eigenvector has the scaling factors. Numerical experiments show that the proposed method outperforms well-established supervised dimensionality reduction methods for toy problems with more samples than features and real-world problems with more features than samples.

cs.LG↗

arXiv2019

Verified partial eigenvalue computations using contour integrals for Hermitian generalized eigenproblems

We propose a verified computation method for partial eigenvalues of a Hermitian generalized eigenproblem. The block Sakurai-Sugiura Hankel method, a contour integral-type eigensolver, can reduce a given eigenproblem into a generalized eigenproblem of block Hankel matrices whose entries consist of complex moments. In this study, we evaluate all errors in computing the complex moments. We derive a truncation error bound of the quadrature. Then, we take numerical errors of the quadrature into account and rigorously enclose the entries of the block Hankel matrices. Each quadrature point gives rise to a linear system, and its structure enables us to develop an efficient technique to verify the approximate solution. Numerical experiments show that the proposed method outperforms a standard method and infer that the proposed method is potentially efficient in parallel.

math.NA↗

arXiv2019

Implementation of Interior-point Methods for LP based on Krylov Subspace Iterative Solvers with Inner-iteration Preconditioning

We apply novel inner-iteration preconditioned Krylov subspace methods to the interior-point algorithm for linear programming (LP). Inner-iteration preconditioners recently proposed by Morikuni and Hayami enable us to overcome the severe ill-conditioning of linear equations solved in the final phase of interior-point iterations. The Krylov subspace methods do not suffer from rank-deficiency and therefore no preprocessing is necessary even if rows of the constraint matrix are not linearly independent. By means of these methods, a new interior-point recurrence is proposed in order to omit one matrix-vector product at each step. Extensive numerical experiments are conducted over diverse instances of 138 LP problems including the Netlib, QAPLIB, Mittelmann and Atomizer Basis Pursuit collections. The largest problem has 434,580 unknowns. It turns out that our implementation is more robust than the standard public domain solvers SeDuMi (Self-Dual Minimization), SDPT3 (Semidefinite Programming Toh-Todd-Tütüncü) and the LSMR iterative solver in PDCO (Primal-Dual Barrier Method for Convex Objectives) without increasing CPU time. The proposed interior-point method based on iterative solvers succeeds in solving a fairly large number of LP instances from benchmark libraries under the standard stopping criteria. The work also presents a fairly extensive benchmark test for several renowned solvers including direct and iterative solvers.

math.OC↗

arXiv2018

Spectral feature scaling method for supervised dimensionality reduction

Spectral dimensionality reduction methods enable linear separations of complex data with high-dimensional features in a reduced space. However, these methods do not always give the desired results due to irregularities or uncertainties of the data. Thus, we consider aggressively modifying the scales of the features to obtain the desired classification. Using prior knowledge on the labels of partial samples to specify the Fiedler vector, we formulate an eigenvalue problem of a linear matrix pencil whose eigenvector has the feature scaling factors. The resulting factors can modify the features of entire samples to form clusters in the reduced space, according to the known labels. In this study, we propose new dimensionality reduction methods supervised using the feature scaling associated with the spectral clustering. Numerical experiments show that the proposed methods outperform well-established supervised methods for toy problems with more samples than features, and are more robust regarding clustering than existing methods. Also, the proposed methods outperform existing methods regarding classification for real-world problems with more features than samples of gene expression profiles of cancer diseases. Furthermore, the feature scaling tends to improve the clustering and classification accuracies of existing unsupervised methods, as the proportion of training data increases.

stat.ML↗

arXiv2017

On GMRES for singular EP and GP systems

In this contribution, we study the numerical behavior of the Generalized Minimal Residual (GMRES) method for solving singular linear systems. It is known that GMRES determines a least squares solution without breakdown if the coefficient matrix is range-symmetric (EP), or if its range and nullspace are disjoint (GP) and the system is consistent. We show that the accuracy of GMRES iterates may deteriorate in practice due to three distinct factors: (i) the inconsistency of the linear system; (ii) the distance of the initial residual to the nullspace of the coefficient matrix; (iii) the extremal principal angles between the ranges of the coefficient matrix and its transpose. These factors lead to poor conditioning of the extended Hessenberg matrix in the Arnoldi decomposition and affect the accuracy of the computed least squares solution. We also compare GMRES with the range restricted GMRES (RR-GMRES) method. Numerical experiments show typical behaviors of GMRES for small problems with EP and GP matrices.

math.NA↗

arXiv2015

Multistep matrix splitting iteration preconditioning for singular linear systems

Multistep matrix splitting iterations serve as preconditioning for Krylov subspace methods for solving singular linear systems. The preconditioner is applied to the generalized minimal residual (GMRES) method and the flexible GMRES (FGMRES) method. We present theoretical and practical justifications for using this approach. Numerical experiments show that the multistep generalized shifted splitting (GSS) and Hermitian and skew-Hermitian splitting (HSS) iteration preconditioning are more robust and efficient compared to standard preconditioners for some test problems of large sparse singular linear systems.

math.NA↗

arXiv2015

Inner-iteration preconditioning with a symmetric splitting matrix for rank-deficient least squares problems

Stationary iterative methods with a symmetric splitting matrix are performed as inner-iteration preconditioning for Krylov subspace methods. We give conditions such that the inner-iteration preconditioning matrix is definite, and show that conjugate gradient (CG) method preconditioned by the inner iterations determines a solution of symmetric and positive semidefinite linear systems, and the minimal residual (MINRES) method preconditioned by the inner iterations determines a solution of symmetric linear systems including the singular case. These results are applied to the CG and MINRES-type methods such as the CGLS, LSMR, and CGNE methods preconditioned by inner iterations, and thus justify using these methods for solving least squares and minimum-norm solution problems whose coefficient matrices are not necessarily of full rank. Thus, we complement the convergence theories of these methods presented in [K. Morikuni and K. Hayami, SIAM J. Matrix Appl. Anal., 34 (2013), pp.1-22], [K. Morikuni and K. Hayami, SIAM J. Matrix Appl. Anal., 36 (2015), pp. 225-250], and give bounds for these methods.

math.NA↗