arXiv subjects

Eric de Sturler

Publications and source records attributed to Eric de Sturler.

At least 19 recordsLinked to original sources

arXiv2026

Nonlinear RMM-GKS for Large-Scale Dynamic and Streaming Inverse Problems with Uncertain Forward Operators

Many practical imaging systems suffer from uncertainty in acquisition geometry -- such as projection angles in computed tomography or sensor positions in photoacoustic tomography -- leading to nonlinear inverse problems that require joint estimation of both the image and the forward model parameters. Standard approaches that assume a known linear forward operator fail to account for these uncertainties, resulting in significant reconstruction artifacts. We propose a nonlinear recycled majorization-minimization generalized Krylov subspace (NL-RMM-GKS) framework for large-scale inverse problems with uncertain forward operators. The method extends MM-GKS to nonlinear settings by combining majorization-minimization for nonsmooth regularization with Krylov subspace projection and subspace recycling, ensuring bounded memory usage. Two complementary formulations are developed: an alternating minimization approach that alternates between image updates and Gauss-Newton parameter estimation, and a variable projection approach that eliminates the image variable and optimizes directly over the parameters using inexact inner solves. We further introduce streaming variants that process data sequentially, enabling reconstruction from large or dynamically acquired datasets without storing the full operator. For dynamic problems, we incorporate two temporal regularization strategies -- optical flow and anisotropic total variation -- as plug-in choices within the framework. We carry out rigorous numerical experiments in fan-beam computed tomography and photoacoustic tomography to demonstrate that our proposed framework achieves high-quality reconstructions with bounded memory requirements, making it suitable for large-scale dynamic imaging problems.

math.NA↗

arXiv2026

ResGene-T: A Tensor-Based Residual Network Approach for Genomic Prediction

In this work, we propose a new deep learning model for Genomic Prediction (GP), which involves correlating genotypic data with phenotypic. The genotypes are typically fed as a sequence of characters to the 1D-Convolution Neural Network layer of the underlying deep learning model. Inspired by earlier work that represented genotype as a 2D-image for genotype-phenotype classification, we extend this idea to GP, which is a regression task. We use a ResNet-18 as the underlying architecture, and term this model as ResGene-2D. Although the 2D-image representation captures biological interactions well, it requires all the layers of the model to do so. This limits training efficiency. Thus, as seen in the earlier work that proposed a 2D-image representation, our ResGene-2D performs almost the same as other models (3% improvement). To overcome this, we propose a novel idea of converting the 2D-image into a 3D/ tensor and feed this to the ResNet-18 architecture, and term this model as ResGene-T. We evaluate our proposed models on three crop species having ten phenotypic traits and compare it with seven most popular models (two statistical, two machine learning, and three deep learning). ResGene-T performs the best among all these seven methods (gains from 14.51% to 41.51%).

cs.LG↗

arXiv2025

An Adaptive Mixed Precision and Dynamically Scaled Preconditioned Conjugate Gradient Algorithm

We propose an adaptive mixed precision and dynamically scaled preconditioned conjugate gradient algorithm (AMP-PCG). It dynamically adjusts the precision for storing vectors and computing, exploiting low precision when appropriate, while maintaining a convergence rate and accuracy comparable to that of double precision PCG. Our mixed precision strategy consists of three main components: (1) The residual and matrix-vector product are initially computed in double precision, and the algorithm switches these to single precision based on the chosen convergence tolerance and an estimate of the residual gap. (2) Depending on the eigenvalue distribution, the preconditioned residual and search direction are either in half precision throughout the iterations or initially in double precision and then stepwise reduced to single and half precision. (3) A dynamically scaled residual is used at every iteration to mitigate underflow in half precision. We provide theoretical support for our estimates and we demonstrate the effectiveness of AMP-PCG through numerical experiments, highlighting both its robustness and the significant performance gains (1.63x speedup) achieved compared to double precision PCG on a GPU.

math.NA↗

arXiv2024

Stopping Criteria for the Conjugate Gradient Algorithm in High-Order Finite Element Methods

We consider stopping criteria that balance algebraic and discretization errors for the conjugate gradient algorithm applied to high-order finite element discretizations of Poisson problems. Firstly, we introduce a new stopping criterion that suggests stopping when the norm of the linear system residual is less than a small fraction of an error indicator derived directly from the residual. This indicator shares the same mesh size and polynomial degree scaling as the norm of the residual, resulting in a robust criterion regardless of the mesh size, the polynomial degree, and the shape regularity of the mesh. Secondly, for solving Poisson problems with highly variable piecewise constant coefficients, we introduce a subdomain-based criterion that recommends stopping when the norm of the linear system residual restricted to each subdomain is smaller than the corresponding indicator also restricted to that subdomain. Reliability and efficiency theorems for the first criterion are established. Numerical experiments, including tests with highly variable piecewise constant coefficients and a GPU-accelerated three-dimensional elliptic solver, demonstrate that the proposed criteria efficiently avoid both premature termination and over-solving.

math.NA↗

arXiv2024

Parametric Level-sets Enhanced To Improve Reconstruction (PaLEnTIR)

We introduce PaLEnTIR, a significantly enhanced parametric level-set (PaLS) method addressing the restoration and reconstruction of piecewise constant objects. Our key contribution involves a unique PaLS formulation utilizing a single level-set function to restore scenes containing multi-contrast piecewise-constant objects without requiring knowledge of the number of objects or their contrasts. Unlike standard PaLS methods employing radial basis functions (RBFs), our model integrates anisotropic basis functions (ABFs), thereby expanding its capacity to represent a wider class of shapes. Furthermore, PaLEnTIR improves the conditioning of the Jacobian matrix, required as part of the parameter identification process, and consequently accelerates optimization methods. We validate PaLEnTIR's efficacy through diverse experiments encompassing sparse and limited angle of view X-ray computed tomography (2D and 3D), nonlinear diffuse optical tomography (DOT), denoising, and deconvolution tasks using both real and simulated data sets.

math.NA↗

arXiv2023

Recycling MMGKS for large-scale dynamic and streaming data

Reconstructing high-quality images with sharp edges requires the use of edge-preserving constraints in the regularized form of the inverse problem. The use of the $\ell_q$-norm on the gradient of the image is a common such constraint. For implementation purposes, the $\ell_q$-norm term is typically replaced with a sequence of $\ell_2$-norm weighted gradient terms with the weights determined from the current solution estimate. While (hybrid) Krylov subspace methods can be employed on this sequence, it would require generating a new Krylov subspace for every new two-norm regularized problem. The majorization-minimization Krylov subspace method (MM-GKS) addresses this disadvantage by combining norm reweighting with generalized Krylov subspaces (GKS). After projecting the problem using a small dimensional subspace - one that expands each iteration - the regularization parameter is selected. Basis expansion repeats until a sufficiently accurate solution is found. Unfortunately, for large-scale problems that require many expansion steps to converge, storage and the cost of repeated orthogonalizations presents overwhelming memory and computational requirements. In this paper we present a new method, recycled MM-GKS (RMM-GKS), that keeps the memory requirements bounded through recycling the solution subspace. Specifically, our method alternates between enlarging and compressing the GKS subspace, recycling directions that are deemed most important via one of our tailored compression routines. We further generalize the RMM-GKS approach to handle experiments where the data is either not all available simultaneously, or needs to be treated as such because of the extreme memory requirements. Numerical examples from dynamic photoacoustic tomography and streaming X-ray computerized tomography (CT) imaging are used to illustrate the effectiveness of the described methods.

math.NA↗

arXiv2023

Subspace Recycling for Sequences of Shifted Systems with Applications in Image Recovery

For many applications involving a sequence of linear systems with slowly changing system matrices, subspace recycling, which exploits relationships among systems and reuses search space information, can achieve huge gains in iterations across the total number of linear system solves in the sequence. However, for general (i.e., non-identity) shifted systems with the shift value varying over a wide range, the properties of the linear systems vary widely as well, which makes recycling less effective. If such a sequence of systems is embedded in a nonlinear iteration, the problem is compounded, and special approaches are needed to use recycling effectively. In this paper, we develop new, more efficient, Krylov subspace recycling approaches for large-scale image reconstruction and restoration techniques that employ a nonlinear iteration to compute a suitable regularization matrix. For each new regularization matrix, we need to solve regularized linear systems, ${\bf A} + γ_\ell {\bf E}_k$, for a sequence of regularization parameters, $γ_\ell$, to find the optimally regularized solution that, in turn, will be used to update the regularization matrix. In this paper, we analyze system and solution characteristics to choose appropriate techniques to solve each system rapidly. Specifically, we use an inner-outer recycling approach with a larger, principal recycle space for each nonlinear step and smaller recycle spaces for each shift. We propose an efficient way to obtain good initial guesses from the principle recycle space and smaller shift-specific recycle spaces that lead to fast convergence. Our method is substantially reduces the total number of matrix-vector products that would arise in a naive approach. Our approach is more generally applicable to sequences of shifted systems where the matrices in the sum are positive semi-definite.

math.NA↗

arXiv2022

Analysis of GMRES for Low-Rank and Small-Norm Perturbations of the Identity Matrix

In many applications, linear systems arise where the coefficient matrix takes the special form ${\bf I} + {\bf K} + {\bf E}$, where ${\bf I}$ is the identity matrix of dimension $n$, ${\rm rank}({\bf K}) = p \ll n$, and $\|{\bf E}\| \leq ε< 1$. GMRES convergence rates for linear systems with coefficient matrices of the forms ${\bf I} + {\bf K}$ and ${\bf I} + {\bf E}$ are guaranteed by well-known theory, but only relatively weak convergence bounds specific to matrices of the form ${\bf I} + {\bf K} + {\bf E}$ currently exist. In this paper, we explore the convergence properties of linear systems with such coefficient matrices by considering the pseudospectrum of ${\bf I} + {\bf K}$. We derive a bound for the GMRES residual in terms of $ε$ when approximately solving the linear system $({\bf I} + {\bf K} + {\bf E}){\bf x} = {\bf b}$ and identify the eigenvalues of ${\bf I} + {\bf K}$ that are sensitive to perturbation. In particular, while a clustered spectrum away from the origin is often a good indicator of fast GMRES convergence, that convergence may be slow when some of those eigenvalues are ill-conditioned. We show there can be at most $2p$ eigenvalues of ${\bf I} + {\bf K}$ that are sensitive to small perturbations. We present numerical results when using GMRES to solve a sequence of linear systems of the form $({\bf I} + {\bf K}_j + {\bf E}_j){\bf x}_j = {\bf b}_j$ that arise from the application of Broyden's method to solve a nonlinear partial differential equation.

math.NA↗

arXiv2021

Efficient edge-preserving methods for dynamic inverse problems

We consider efficient methods for computing solutions to dynamic inverse problems, where both the quantities of interest and the forward operator (measurement process) may change at different time instances but we want to solve for all the images simultaneously. We are interested in large-scale ill-posed problems that are made more challenging by their dynamic nature and, possibly, by the limited amount of available data per measurement step. To remedy these difficulties, we apply regularization methods that enforce simultaneous regularization in space and time (such as edge enhancement at each time instant and proximity at consecutive time instants) and achieve this with low computational cost and enhanced accuracy. More precisely, we develop iterative methods based on a majorization-minimization (MM) strategy with quadratic tangent majorant, which allows the resulting least squares problem to be solved with a generalized Krylov subspace (GKS) method; the regularization parameter can be defined automatically and efficiently at each iteration. Numerical examples from a wide range of applications, such as limited-angle computerized tomography (CT), space-time image deblurring, and photoacoustic tomography (PAT), illustrate the effectiveness of the described approaches.

math.NA↗

arXiv2020

Preconditioning Parametrized Linear Systems

Preconditioners are generally essential for fast convergence in the iterative solution of linear systems of equations. However, the computation of a good preconditioner can be expensive. So, while solving a sequence of many linear systems, it is advantageous to recycle preconditioners, that is, update a previous preconditioner and reuse the updated version. In this paper, we introduce a simple and effective method for doing this. Although our approach can be used for matrices changing slowly in any way, we focus on the important case of sequences of the type $(s_k\textbf{E}(\textbf{p}) + \textbf{A}(\textbf{p}))\textbf{x}_k = \textbf{b}_k$, where the right hand side may or may not change. More general changes in matrices will be discussed in a future paper. We update preconditioners by defining a map from a new matrix to a previous matrix, for example the first matrix in the sequence, and combine the preconditioner for this previous matrix with the map to define the new preconditioner. This approach has several advantages. The update is entirely independent from the original preconditioner, so it can be applied to any preconditioner. The possibly high cost of an initial preconditioner can be amortized over many linear solves. The cost of updating the preconditioner is more or less constant and independent of the original preconditioner. There is flexibility in balancing the quality of the map with the computational cost. In the numerical experiments section we demonstrate good results for several applications, in particular when using an algebraic multigrid preconditioner.

math.NA↗

arXiv2020

Krylov Subspace Recycling for Evolving Structures

Krylov subspace recycling is a powerful tool for solving long series of large, sparse linear systems that change slowly. In PDE constrained shape optimization, these appear naturally, as hundreds or more optimization steps are needed with only small changes in the geometry. In this setting, however, applying Krylov subspace recycling can be difficult. As the geometry evolves, so does the finite element mesh, especially if re-meshing is needed. As a result, the number of algebraic degrees of freedom in the system may change from one optimization step to the next, and with it the size of the finite element system matrix. Changes in the mesh also lead to structural changes in the matrices. In the case of remeshing, even if the geometry changes only a little, the corresponding mesh might differ substantially from the previous one. This prevents any straightforward mapping of the approximate invariant subspace of the linear system matrix (the focus of recycling in this paper) from one step to the next; similar problems arise for other selected subspaces. We present an algorithm for general meshes to map an approximate invariant subspace of the system matrix for the previous optimization step to an approximate invariant subspace of the system matrix for the current optimization step. We exploit the map from coefficient vectors to finite element functions on the mesh combined with function approximation on the finite element mesh. In addition, we develop a straightforward warm-start adaptation of the Krylov-Schur algorithm [G.W. Stewart, SIAM J. Matrix Anal. Appl. 23, 2001] to improve the approximate invariant subspace at the start of a new optimization step if needed. We demonstrate the effectiveness of our approach numerically with several proof of concept studies for a specific meshing technique.

math.NA↗

arXiv2020

A survey of subspace recycling iterative methods

This survey concerns subspace recycling methods, a popular class of iterative methods that enable effective reuse of subspace information in order to speed up convergence and find good initial guesses over a sequence of linear systems with slowly changing coefficient matrices, multiple right-hand sides, or both. The subspace information that is recycled is usually generated during the run of an iterative method (usually a Krylov subspace method) on one or more of the systems. Following introduction of definitions and notation, we examine the history of early augmentation schemes along with deflation preconditioning schemes and their influence on the development of recycling methods. We then discuss a general residual constraint framework through which many augmented Krylov and recycling methods can both be viewed. We review several augmented and recycling methods within this framework. We then discuss some known effective strategies for choosing subspaces to recycle before taking the reader through more recent developments that have generalized recycling for (sequences of) shifted linear systems, some of them with multiple right-hand sides in mind. We round out our survey with a brief review of application areas that have seen benefit from subspace recycling methods.

math.NA↗

arXiv2020

Randomization for the Efficient Computation of Parametric Reduced Order Models for Inversion

Nonlinear parametric inverse problems appear in many applications. Here, we focus on diffuse optical tomography (DOT) in medical imaging to recover unknown images of interest, such as cancerous tissue in a given medium, using a mathematical (forward) model. The forward model in DOT is a diffusion-absorption model for the photon flux. The main bottleneck in these problems is the repeated evaluation of the large-scale forward model. For DOT, this corresponds to solving large linear systems for each source and frequency at each optimization step. Moreover, Newton-type methods, often the method of choice, require additional linear solves with the adjoint to compute derivative information. Emerging technology allows for large numbers of sources and detectors, making these problems prohibitively expensive. Reduced order models (ROM) have been used to drastically reduce the system size in each optimization step, while solving the inverse problem accurately. However, for large numbers of sources and detectors, just the construction of the candidate basis for the ROM projection space incurs a substantial cost, as matching the full parameter gradient matrix in interpolatory model reduction requires large linear solves for all sources and frequencies and all detectors and frequencies for each parameter interpolation point. As this candidate basis numerically has low rank, this construction is followed by a rank-revealing factorization that typically reduces the number of vectors in the candidate basis substantially. We propose to use randomization to approximate this basis with a drastically reduced number of large linear solves. We also provide a detailed analysis for the low-rank structure of the candidate basis for our problem of interest. Even though we focus on the DOT problem, the ideas presented are relevant to many other large scale inverse problems and optimization problems.

math.NA↗

arXiv2020

Hybrid Projection Methods with Recycling for Inverse Problems

Iterative hybrid projection methods have proven to be very effective for solving large linear inverse problems due to their inherent regularizing properties as well as the added flexibility to select regularization parameters adaptively. In this work, we develop Golub-Kahan-based hybrid projection methods that can exploit compression and recycling techniques in order to solve a broad class of inverse problems where memory requirements or high computational cost may otherwise be prohibitive. For problems that have many unknown parameters and require many iterations, hybrid projection methods with recycling can be used to compress and recycle the solution basis vectors to reduce the number of solution basis vectors that must be stored, while obtaining a solution accuracy that is comparable to that of standard methods. If reorthogonalization is required, this may also reduce computational cost substantially. In other scenarios, such as streaming data problems or inverse problems with multiple datasets, hybrid projection methods with recycling can be used to efficiently integrate previously computed information for faster and better reconstruction. Additional benefits of the proposed methods are that various subspace selection and compression techniques can be incorporated, standard techniques for automatic regularization parameter selection can be used, and the methods can be applied multiple times in an iterative fashion. Theoretical results show that, under reasonable conditions, regularized solutions for our proposed recycling hybrid method remain close to regularized solutions for standard hybrid methods and reveal important connections among the resulting projection matrices. Numerical examples from image processing show the potential benefits of combining recycling with hybrid projection methods.

math.NA↗

arXiv2020

Robust Parameter Inversion using Adaptive Reduced Order Models

Nonlinear parametric inverse problems appear in many applications and are typically very expensive to solve, especially if they involve many measurements. These problems pose huge computational challenges as evaluating the objective function or misfit requires the solution of a large number of parameterized partial differential equations, typically one per source term. Newton-type algorithms, which may be required for fast convergence, typically require the additional solution of a large number of adjoint problems. The use of parametric model reduction may substantially alleviate this problem. In [de Sturler, E., Gugercin, S., Kilmer, M. E., Chaturantabut, S., Beattie, C., and O'Connell, M. (2015). Nonlinear Parametric Inversion Using Interpolatory Model Reduction. SIAM Journal on Scientific Computing, 37(3)], interpolatory model reduction was successfully used to drastically speed up inversion for Diffuse Optical Tomography (DOT). However, when using model reduction in high dimensional parameter spaces, obtaining error bounds in parameter space is typically intractable. In this paper, we propose to use stochastic estimates to remedy this problem. At the cost of one (randomized) full-scale linear solve per optimization step we obtain a robust algorithm. Moreover, since we can now update the model when needed, this robustness allows us to further reduce the order of the reduced order model and hence the cost of computing and using it, further decreasing the cost of inversion. We also propose a method to update the model reduction basis that reduces the number of large linear solves required by 46%-98% compared with the fixed reduced-order model. We demonstrate that this leads to a highly efficient and robust inversion method.

math.NA↗

arXiv2017

Randomized Approach to Nonlinear Inversion Combining Simultaneous Random and Optimized Sources and Detectors

In partial differential equations-based (PDE-based) inverse problems with many measurements, many large-scale discretized PDEs must be solved for each evaluation of the misfit or objective function. In the nonlinear case, evaluating the Jacobian requires solving an additional set of systems. This leads to a tremendous computational cost, and this is by far the dominant cost for these problems. Several authors have proposed randomization and stochastic programming techniques to drastically reduce the number of system solves by estimating the objective function using only a few appropriately chosen random linear combinations of the sources. While some have reported good solution quality at a greatly reduced cost, for our problem of interest, diffuse optical tomography, the approach often does not lead to sufficiently accurate solutions. We propose two improvements. First, to efficiently exploit Newton-type methods, we modify the stochastic estimates to include random linear combinations of detectors, drastically reducing the number of adjoint solves. Second, after solving to a modest tolerance, we compute a few simultaneous sources and detectors that maximize the Frobenius norm of the sampled Jacobian to improve the rate of convergence and obtain more accurate solutions. We complement these optimized simultaneous sources and detectors by random simultaneous sources and detectors constrained to a complementary subspace. Our approach leads to solutions of the same quality as obtained using all sources and detectors but at a greatly reduced computational cost, as the number of large-scale linear systems to be solved is significantly reduced.

math.NA↗

arXiv2016

Stochastic Sampling for Structural Topology Optimization with Many Load Cases: Density-Based and Ground Structure Approaches

We propose an efficient probabilistic method to solve a deterministic problem -- we present a randomized optimization approach that drastically reduces the enormous computational cost of optimizing designs under many load cases for both continuum and truss topology optimization. Practical structural designs by topology optimization typically involve many load cases, possibly hundreds or more. The optimal design minimizes a, possibly weighted, average of the compliance under each load case (or some other objective). This means that in each optimization step a large finite element problem must be solved for each load case, leading to an enormous computational effort. On the contrary, the proposed randomized optimization method with stochastic sampling requires the solution of only a few (e.g., 5 or 6) finite element problems (large linear systems) per optimization step. Based on simulated annealing, we introduce a damping scheme for the randomized approach. Through numerical examples in two and three dimensions, we demonstrate that the stochastic algorithm drastically reduces computational cost to obtain similar final topologies and results (e.g., compliance) compared with the standard algorithms. The results indicate that the damping scheme is effective and leads to rapid convergence of the proposed algorithm.

math.OC↗

arXiv2016

Computing Reduced Order Models via Inner-Outer Krylov Recycling in Diffuse Optical Tomography

In nonlinear imaging problems whose forward model is described by a partial differential equation (PDE), the main computational bottleneck in solving the inverse problem is the need to solve many large-scale discretized PDEs at each step of the optimization process. In the context of absorption imaging in diffuse optical tomography, one approach to addressing this bottleneck proposed recently (de Sturler, et al, 2015) reformulates the viewing of the forward problem as a differential algebraic system, and then employs model order reduction (MOR). However, the construction of the reduced model requires the solution of several full order problems (i.e. the full discretized PDE for multiple right-hand sides) to generate a candidate global basis. This step is then followed by a rank-revealing factorization of the matrix containing the candidate basis in order to compress the basis to a size suitable for constructing the reduced transfer function. The present paper addresses the costs associated with the global basis approximation in two ways. First, we use the structure of the matrix to rewrite the full order transfer function, and corresponding derivatives, such that the full order systems to be solved are symmetric (positive definite in the zero frequency case). Then we apply MOR to the new formulation of the problem. Second, we give an approach to computing the global basis approximation dynamically as the full order systems are solved. In this phase, only the incrementally new, relevant information is added to the existing global basis, and redundant information is not computed. This new approach is achieved by an inner-outer Krylov recycling approach which has potential use in other applications as well. We show the value of the new approach to approximate global basis computation on two DOT absorption image reconstruction problems.

math.NA↗