arXiv subjects

James G. Nagy

Publications and source records attributed to James G. Nagy.

11 recordsLinked to original sources

arXiv2026

Nonlinear RMM-GKS for Large-Scale Dynamic and Streaming Inverse Problems with Uncertain Forward Operators

Many practical imaging systems suffer from uncertainty in acquisition geometry -- such as projection angles in computed tomography or sensor positions in photoacoustic tomography -- leading to nonlinear inverse problems that require joint estimation of both the image and the forward model parameters. Standard approaches that assume a known linear forward operator fail to account for these uncertainties, resulting in significant reconstruction artifacts. We propose a nonlinear recycled majorization-minimization generalized Krylov subspace (NL-RMM-GKS) framework for large-scale inverse problems with uncertain forward operators. The method extends MM-GKS to nonlinear settings by combining majorization-minimization for nonsmooth regularization with Krylov subspace projection and subspace recycling, ensuring bounded memory usage. Two complementary formulations are developed: an alternating minimization approach that alternates between image updates and Gauss-Newton parameter estimation, and a variable projection approach that eliminates the image variable and optimizes directly over the parameters using inexact inner solves. We further introduce streaming variants that process data sequentially, enabling reconstruction from large or dynamically acquired datasets without storing the full operator. For dynamic problems, we incorporate two temporal regularization strategies -- optical flow and anisotropic total variation -- as plug-in choices within the framework. We carry out rigorous numerical experiments in fan-beam computed tomography and photoacoustic tomography to demonstrate that our proposed framework achieves high-quality reconstructions with bounded memory requirements, making it suitable for large-scale dynamic imaging problems.

math.NA↗

arXiv2025

Mixed precision iterative refinement for linear inverse problems

This study investigates the iterative refinement method applied to the solution of linear discrete inverse problems by considering its application to the Tikhonov problem in mixed precision. Previous works on mixed precision iterative refinement methods for the solution of symmetric positive definite linear systems and least-squares problems have shown regularization to be a key requirement when computing low precision factorizations. For problems that are naturally severely ill-posed, we formulate the iterates of iterative refinement in mixed precision as a filtered solution using the preconditioned Landweber method with a Tikhonov-type preconditioner. Through numerical examples simulating various mixed precision choices, we showcase the filtering properties of the method and the achievement of comparable working accuracy of discrete inverse problems (i.e., to within a few decimal places in relative error) compared to results computed in double precision as well as another approximate iterative refinement method which we use as a benchmark.

math.NA↗

arXiv2025

Randomized and Inner-product Free Krylov Methods for Large-scale Inverse Problems

Iterative Krylov projection methods have become widely used for solving large-scale linear inverse problems. However, methods based on orthogonality include the computation of inner-products, which become costly when the number of iterations is high; are a bottleneck for parallelization; and can cause the algorithms to break down in low precision due to information loss in the projections. Recent works on inner-product free Krylov iterative algorithms alleviate these concerns, but they are quasi-minimal residual rather than minimal residual methods. This is a potential concern for inverse problems where the residual norm provides critical information from the observations via the likelihood function, and we do not have any way of controlling how close the quasi-norm is from the norm we want to minimize. In this work, we introduce a new Krylov method that is both inner-product-free and minimizes a functional that is theoretically closer to the residual norm. The proposed scheme combines an inner-product free Hessenberg projection approach for generating a solution subspace with a randomized sketch-and-solve approach for solving the resulting strongly overdetermined projected least-squares problem. Numerical results show that the proposed algorithm can solve large-scale inverse problems efficiently and without requiring inner-products.

math.NA↗

arXiv2024

H-CMRH: a novel inner product free hybrid Krylov method for large-scale inverse problems

This study investigates the iterative regularization properties of two Krylov methods for solving large-scale ill-posed problems: the changing minimal residual Hessenberg method (CMRH) and a novel hybrid variant called the hybrid changing minimal residual Hessenberg method (H-CMRH). Both methods share the advantages of avoiding inner products, making them efficient and highly parallelizable, and particularly suited for implementations that exploit randomization and mixed precision arithmetic. Theoretical results and extensive numerical experiments suggest that H-CMRH exhibits comparable performance to the established hybrid GMRES method in terms of stabilizing semiconvergence, but H-CMRH has does not require any inner products, and requires less work and storage per iteration.

math.NA↗

arXiv2024

Inner Product Free Krylov Methods for Large-Scale Inverse Problems

In this study, we introduce two new Krylov subspace methods for solving rectangular large-scale linear inverse problems. The first approach is a modification of the Hessenberg iterative algorithm that is based off an LU factorization and is therefore referred to as the least squares LU (LSLU) method. The second approach incorporates Tikhonov regularization in an efficient manner; we call this the Hybrid LSLU method. Both methods are inner-product free, making them advantageous for high performance computing and mixed precision arithmetic. Theoretical findings and numerical results show that Hybrid LSLU can be effective in solving large-scale inverse problems and has comparable performance with existing iterative projection methods.

math.NA↗

arXiv2023

LSEMINK: A Modified Newton-Krylov Method for Log-Sum-Exp Minimization

This paper introduces LSEMINK, an effective modified Newton-Krylov algorithm geared toward minimizing the log-sum-exp function for a linear model. Problems of this kind arise commonly, for example, in geometric programming and multinomial logistic regression. Although the log-sum-exp function is smooth and convex, standard line search Newton-type methods can become inefficient because the quadratic approximation of the objective function can be unbounded from below. To circumvent this, LSEMINK modifies the Hessian by adding a shift in the row space of the linear model. We show that the shift renders the quadratic approximation to be bounded from below and that the overall scheme converges to a global minimizer under mild assumptions. Our convergence proof also shows that all iterates are in the row space of the linear model, which can be attractive when the model parameters do not have an intuitive meaning, as is common in machine learning. Since LSEMINK uses a Krylov subspace method to compute the search direction, it only requires matrix-vector products with the linear model, which is critical for large-scale problems. Our numerical experiments on image classification and geometric programming illustrate that LSEMINK considerably reduces the time-to-solution and increases the scalability compared to geometric programming and natural gradient descent approaches. It has significantly faster initial convergence than standard Newton-Krylov methods, which is particularly attractive in applications like machine learning. In addition, LSEMINK is more robust to ill-conditioning arising from the nonsmoothness of the problem. We share our MATLAB implementation at https://github.com/KelvinKan/LSEMINK.

math.OC↗

arXiv2020

An ADMM-LAP method for total variation myopic deconvolution of adaptive optics retinal images

Adaptive optics (AO) corrected ood imaging of the retina is a popular technique for studying the retinal structure and function in the living eye. However, the raw retinal images are usually of poor contrast and the interpretation of such images requires image deconvolution. Different from standard deconvolution problems where the point spread function (PSF) is completely known, the PSF in these retinal imaging problems is only partially known which leads to the more complicated myopic (mildly blind) deconvolution problem. In this paper, we propose an efficient numerical scheme for solving this myopic deconvolution problem with total variational (TV) regularization. First, we apply the alternating direction method of multipliers (ADMM) to tackle the TV regularizer. Specifically, we reformulate the TV problem as an equivalent equality constrained problem where the objective function is separable, and then minimize the augmented Lagrangian function by alternating between two (separated) blocks of unknowns to obtain the solution. Due to the structure of the retinal images, the subproblems with respect to the fidelity term appearing within each ADMM iteration are tightly coupled and a variation of the Linearize And Project (LAP) method is designed to solve these subproblems efficiently. The proposed method is called the ADMM-LAP method. Theoretically, we establish the subsequence convergence of the ADMM-LAP method to a stationary point. Both the theoretical complexity analysis and numerical results are provided to demonstrate the efficiency of the ADMM-LAP method.

math.OC↗

arXiv2019

An Inner-Outer Iterative Method for Edge Preservation in Image Restoration and Reconstruction

We present a new inner-outer iterative algorithm for edge enhancement in imaging problems. At each outer iteration, we formulate a Tikhonov-regularized problem where the penalization is expressed in the 2-norm and involves a regularization operator designed to improve edge resolution as the outer iterations progress, through an adaptive process. An efficient hybrid regularization method is used to project the Tikhonov-regularized problem onto approximation subspaces of increasing dimensions (inner iterations), while conveniently choosing the regularization parameter (by applying well-known techniques, such as the discrepancy principle or the ${\mathcal L}$-curve criterion, to the projected problem). This procedure results in an automated algorithm for edge recovery that does not involve regularization parameter tuning by the user, nor repeated calls to sophisticated optimization algorithms, and is therefore particularly attractive from a computational point of view. A key to the success of the new algorithm is the design of the regularization operator through the use of an adaptive diagonal weighting matrix that effectively enforces smoothness only where needed. We demonstrate the value of our approach on applications in X-ray CT image reconstruction and in image deblurring, and show that it can be computationally much more attractive than other well-known strategies for edge preservation, while providing solutions of greater or equal quality.

math.NA↗

arXiv2018

IR Tools: A MATLAB Package of Iterative Regularization Methods and Large-Scale Test Problems

This paper describes a new MATLAB software package of iterative regularization methods and test problems for large-scale linear inverse problems. The software package, called IR Tools, serves two related purposes: we provide implementations of a range of iterative solvers, including several recently proposed methods that are not available elsewhere, and we provide a set of large-scale test problems in the form of discretizations of 2D linear inverse problems. The solvers include iterative regularization methods where the regularization is due to the semi-convergence of the iterations, Tikhonov-type formulations where the regularization is explicitly formulated in the form of a regularization term, and methods that can impose bound constraints on the computed solutions. All the iterative methods are implemented in a very flexible fashion that allows the problem's coefficient matrix to be available as a (sparse) matrix, a function handle, or an object. The most basic call to all of the various iterative methods requires only this matrix and the right hand side vector; if the method uses any special stopping criteria, regularization parameters, etc., then default values are set automatically by the code. Moreover, through the use of an optional input structure, the user can also have full control of any of the algorithm parameters. The test problems represent realistic large-scale problems found in image reconstruction and several other applications. Numerical examples illustrate the various algorithms and test problems available in this package.

math.NA↗

arXiv2018

Singular Value Decomposition Approximation via Kronecker Summations for Imaging Applications

In this paper we propose an approach to approximate a truncated singular value decomposition of a large structured matrix. By first decomposing the matrix into a sum of Kronecker products, our approach can be used to approximate a large number of singular values and vectors more efficiently than other well known schemes, such as randomized matrix algorithms or iterative algorithms based on Golub-Kahan bidiagonalization. We provide theoretical results and numerical experiments to demonstrate the accuracy of our approximation and show how the approximation can be used to solve large scale ill-posed inverse problems, either as an approximate filtering method, or as a preconditioner to accelerate iterative algorithms.

math.NA↗

arXiv2017

Robust regression for mixed Poisson-Gaussian model

This paper focuses on efficient computational approaches to compute approximate solutions of a linear inverse problem that is contaminated with mixed Poisson--Gaussian noise, and when there are additional outliers in the measured data. The Poisson--Gaussian noise leads to a weighted minimization problem, with solution-dependent weights. To address outliers, the standard least squares fit-to-data metric is replaced by the Talwar robust regression function. Convexity, regularization parameter selection schemes, and incorporation of non-negative constraints are investigated. A projected Newton algorithm is used to solve the resulting constrained optimization problem, and a preconditioner is proposed to accelerate conjugate gradient Hessian solves. Numerical experiments on problems from image deblurring illustrate the effectiveness of the methods.

math.NA↗