arXiv subjects

Silvia Gazzola

Publications and source records attributed to Silvia Gazzola.

At least 19 recordsLinked to original sources

arXiv2026

Randomized Flexible LSQR and LSMR with applications to inverse problems

LSQR and LSMR are iterative methods, based on the Golub-Kahan bidiagonalization algorithm, widely used for large-scale linear least squares problems. FLSQR and FLSMR are flexible variants of LSQR and LSMR, respectively, based on a flexible Golub-Kahan (Arnoldi-like) factorization algorithm, which naturally allow modifications of the solution approximation subspace and/or handling inexact matrix-vector multiplications with the (transpose of the) coefficient matrix, thereby enabling to enforce prior information into the computed solution. The goal of this paper is to introduce sFLSQR and sFLSMR, i.e., sketched variants of FLSQR and FLSMR, respectively, where randomization becomes particularly effective, as it allows to recover short recurrences for the solution approximation. In particular, this paper explores applications to large-scale inverse problems, showing the ability of the new randomized solvers to alleviate computational bottlenecks while preserving reconstruction quality. A theoretical analysis of sFLSQR and sFLSMR is provided, and their performance is validated through numerical experiments.

math.NA↗

arXiv2025

New flexible and inexact Golub-Kahan algorithms for inverse problems

This paper introduces a new class of algorithms for solving large-scale linear inverse problems based on new flexible and inexact Golub-Kahan factorizations. The proposed methods iteratively compute regularized solutions by approximating a solution to (re)weighted least squares problems via projection onto adaptively generated subspaces, where the constraint subspaces for the residuals are (formally) equipped with iteration-dependent preconditioners or inexactness. The new solvers offer a flexible and inexact Krylov subspace alternative to other existing Krylov-based approaches for handling general data fidelity functionals, e.g., those expressed in the $p$-norm. Numerical experiments in imaging applications, such as image deblurring and computed tomography, highlight the effectiveness and competitiveness of the proposed methods with respect to other popular methods.

math.NA↗

arXiv2025

Efficient gradient-based methods for bilevel learning via recycling Krylov subspaces

Many optimization problems require hyperparameters, i.e., parameters that must be pre-specified in advance, such as regularization parameters and parametric regularizers in variational regularization methods for inverse problems, and dictionaries in compressed sensing. A data-driven approach to determine appropriate hyperparameter values is via a nested optimization framework known as bilevel learning. Even when it is possible to employ a gradient-based solver to the bilevel optimization problem, construction of the gradients, known as hypergradients, is computationally challenging, each one requiring both a solution of a minimization problem and a linear system solve. These systems do not change much during the iterations, which motivates us to apply recycling Krylov subspace methods, wherein information from one linear system solve is re-used to solve the next linear system. Existing recycling strategies often employ eigenvector approximations called Ritz vectors. In this work we propose a novel recycling strategy based on a new concept, Ritz generalized singular vectors, which acknowledge the bilevel setting. Additionally, while existing iterative methods primarily terminate according to the residual norm, this new concept allows us to define a new stopping criterion that directly approximates the error of the associated hypergradient. The proposed approach is validated through extensive numerical testing in the context of inverse problems in imaging.

math.OC↗

arXiv2025

Randomized Krylov methods for inverse problems

In this paper we develop randomized Krylov subspace methods for efficiently computing regularized solutions to large-scale linear inverse problems. Building on the recently developed randomized Gram-Schmidt process, where sketched inner products are used to estimate inner products of high-dimensional vectors, we propose a randomized Golub-Kahan approach that works for general rectangular matrices. We describe new iterative solvers based on the randomized Golub-Kahan approach and show how they can be used for solving inverse problems with rectangular matrices, thus extending the capabilities of the recently proposed randomized GMRES method. We also consider hybrid projection methods that combine iterative projection methods, based on both the randomized Arnoldi and randomized Golub-Kahan factorizations, with Tikhonov regularization, where regularization parameters can be selected automatically during the iterative process. Numerical results from image deblurring and seismic tomography show the potential benefits of these approaches.

math.NA↗

arXiv2025

Alternating steepest descent methods for tensor completion with applications to spectromicroscopy

In this paper we develop two new Tensor Alternating Steepest Descent algorithms for tensor completion in the low-rank $\star_{M}$-product format, whereby we aim to reconstruct an entire low-rank tensor from a small number of measurements thereof. Both algorithms are rooted in the Alternating Steepest Descent (ASD) method for matrix completion, first proposed in [J. Tanner and K. Wei, Appl. Comput. Harmon. Anal., 40 (2016), pp. 417-429]. In deriving the new methods we target the X-ray spectromicroscopy undersampling problem, whereby data are collected by scanning a specimen on a rectangular viewpoint with X-ray beams of different energies. The recorded absorptions coefficients of the mixed specimen materials are naturally stored in a third-order tensor, with spatial horizontal and vertical axes, and an energy axis. To speed the X-ray spectromicroscopy measurement process up, only a fraction of tubes from (a reshaped version of) this tensor are fully scanned, leading to a tensor completion problem. In this framework we can apply any transform (such as the Fourier transform) to the tensor tube by tube, providing a natural way to work with the $\star_{M}$-tensor algebra, and propose: (1) a tensor completion algorithm that is essentially ASD reformulated in the $\star_{M}$-induced metric space and (2) a tensor completion algorithm that solves a set of (readily parallelizable) independent matrix completion problems for the frontal slices of the transformed tensor. The two new methods are tested on real X-ray spectromicroscopy data, demonstrating that they achieve the same reconstruction error with fewer samples from the tensor compared to the matrix completion algorithms applied to a flattened tensor.

math.NA↗

arXiv2025

Optimal Space-Variant Anisotropic Tikhonov Regularization for Full Waveform Inversion of Sparse Data

Full waveform inversion (FWI) is a challenging, ill-posed nonlinear inverse problem that requires robust regularization techniques to stabilize the solution and yield geologically meaningful results, especially when dealing with sparse data. Standard Tikhonov regularization, though commonly employed in FWI, applies uniform smoothing that often leads to oversmoothing of key geological features, as it fails to account for the underlying structural complexity of the subsurface. To overcome this limitation, we propose an FWI algorithm enhanced by a novel Tikhonov regularization technique involving a parametric regularizer, which is automatically optimized to apply directional space-variant smoothing. Specifically, the parameters defining the regularizer (orientation and anisotropy) are treated as additional unknowns in the objective function, allowing the algorithm to estimate them simultaneously with the model. We introduce an efficient numerical implementation for FWI with the proposed space-variant regularization. Numerical tests on sparse data demonstrate the proposed method's effectiveness and robustness in reconstructing models with complex structures, significantly improving the inversion results compared to the standard Tikhonov regularization.

math.NA↗

arXiv2024

Robust Estimation of Structural Orientation Parameters and 2D/3D Local Anisotropic Tikhonov Regularization

Understanding the orientation of geological structures is crucial for analyzing the complexity of the Earths' subsurface. For instance, information about geological structure orientation can be incorporated into local anisotropic regularization methods as a valuable tool to stabilize the solution of inverse problems and produce geologically plausible solutions. We introduce a new variational method that employs the alternating direction method of multipliers within an alternating minimization scheme to jointly estimate orientation and model parameters in both 2D and 3D inverse problems. Specifically, the proposed approach adaptively integrates recovered information about structural orientation, enhancing the effectiveness of anisotropic Tikhonov regularization in recovering geophysical parameters. The paper also discusses the automatic tuning of algorithmic parameters to maximize the new method's performance. The proposed algorithm is tested across diverse 2D and 3D examples, including structure-oriented denoising and trace interpolation. The results show that the algorithm is robust in solving the considered large and challenging problems, alongside efficiently estimating the associated tilt field in 2D cases and the dip, strike, and tilt fields in 3D cases. Synthetic and field examples show that the proposed anisotropic regularization method produces a model with enhanced resolution and provides a more accurate representation of the true structures.

physics.geo-ph↗

arXiv2024

Optimising seismic imaging design parameters via bilevel learning

Full Waveform Inversion (FWI) is a standard algorithm in seismic imaging. Its implementation requires the a priori choice of a number of "design parameters", such as the positions of sensors for the actual measurements and one (or more) regularisation weights. In this paper we describe a novel algorithm for determining these design parameters automatically from a set of training images, using a (supervised) bilevel learning approach. In our algorithm, the upper level objective function measures the quality of the reconstructions of the training images, where the reconstructions are obtained by solving the lower level optimisation problem -- in this case FWI. Our algorithm employs (variants of) the BFGS quasi-Newton method to perform the optimisation at each level, and thus requires the repeated solution of the forward problem -- here taken to be the Helmholtz equation. This paper focuses on the implementation of the algorithm. The novel contributions are: (i) an adjoint-state method for the efficient computation of the upper-level gradient; (ii) a complexity analysis for the bilevel algorithm, which counts the number of Helmholtz solves needed and shows this number is independent of the number of design parameters optimised; (iii) an effective preconditioning strategy for iteratively solving the linear systems required at each step of the bilevel algorithm; (iv) a smoothed extraction process for point values of the discretised wavefield, necessary for ensuring a smooth upper level objective function. The algorithm also uses an extension to the bilevel setting of classical frequency-continuation strategies, helping avoid convergence to spurious stationary points. The advantage of our algorithm is demonstrated on a problem derived from the standard Marmousi test problem.

math.NA↗

arXiv2024

Automatic nonstationary anisotropic Tikhonov regularization through bilevel optimization

Regularization techniques are necessary to compute meaningful solutions to discrete ill-posed inverse problems. The well-known 2-norm Tikhonov regularization method equipped with a discretization of the gradient operator as regularization operator penalizes large gradient components of the solution to overcome instabilities. However, this method is homogeneous, i.e., it does not take into account the orientation of the regularized solution and therefore tends to smooth the desired structures, textures and discontinuities, which often contain important information. If the local orientation field of the solution is known, a possible way to overcome this issue is to implement local anisotropic regularization by penalizing weighted directional derivatives. In this paper, considering problems that are inherently two-dimensional, we propose to automatically and simultaneously recover the regularized solution and the local orientation parameters (used to define the anisotropic regularization term) by solving a bilevel optimization problem. Specifically, the lower level problem is Tikhonov regularization equipped with local anisotropic regularization, while the objective function of the upper level problem encodes some natural assumptions about the local orientation parameters and the Tikhonov regularization parameter. Application of the proposed algorithm to a variety of inverse problems in imaging (such as denoising, deblurring, tomography and Dix inversion), with both real and synthetic data, shows its effectiveness and robustness.

math.NA↗

arXiv2024

TRIPs-Py: Techniques for Regularization of Inverse Problems in Python

In this paper, we describe TRIPs-Py, a new Python package of linear discrete inverse problems solvers and test problems. The goal of the package is two-fold: 1) to provide tools for solving small and large-scale inverse problems, and 2) to introduce test problems arising from a wide range of applications. The solvers available in TRIPs-Py include direct regularization methods (such as truncated singular value decomposition and Tikhonov) and iterative regularization techniques (such as Krylov subspace methods and recent solvers for $\ell_p$-$\ell_q$ formulations, which enforce sparse or edge-preserving solutions and handle different noise types). All our solvers have built-in strategies to define the regularization parameter(s). Some of the test problems in TRIPs-Py arise from simulated image deblurring and computerized tomography, while other test problems model realistic problems in dynamic computerized tomography. Numerical examples are included to illustrate the usage as well as the performance of the described methods on the provided test problems. To the best of our knowledge, TRIPs-Py is the first Python software package of this kind, which may serve both research and didactical purposes.

math.NA↗

arXiv2024

On Optimal Regularization Parameters via Bilevel Learning

Variational regularization is commonly used to solve linear inverse problems, and involves augmenting a data fidelity by a regularizer. The regularizer is used to promote a priori information and is weighted by a regularization parameter. Selection of an appropriate regularization parameter is critical, with various choices leading to very different reconstructions. Classical strategies used to determine a suitable parameter value include the discrepancy principle and the L-curve criterion, and in recent years a supervised machine learning approach called bilevel learning has been employed. Bilevel learning is a powerful framework to determine optimal parameters and involves solving a nested optimization problem. While previous strategies enjoy various theoretical results, the well-posedness of bilevel learning in this setting is still an open question. In particular, a necessary property is positivity of the determined regularization parameter. In this work, we provide a new condition that better characterizes positivity of optimal regularization parameters than the existing theory. Numerical results verify and explore this new condition for both small and high-dimensional problems.

math.OC↗

arXiv2023

Undersampling Raster Scans in Spectromicroscopy for reduced dose and faster measurements

Combinations of spectroscopic analysis and microscopic techniques are used across many disciplines of scientific research, including material science, chemistry and biology. X-ray spectromicroscopy, in particular, is a powerful tool used for studying chemical state distributions at the micro and nano scales. With the beam fixed, a specimen is typically rastered through the probe with continuous motion and a range of multimodal data is collected at fixed time intervals. The application of this technique is limited in some areas due to: long scanning times to collect the data, either because of the area/volume under study or the compositional properties of the specimen; and material degradation due to the dose absorbed during the measurement. In this work, we propose a novel approach for reducing the dose and scanning times by undersampling the raster data. This is achieved by skipping rows within scans and reconstructing the x-ray spectromicroscopic measurements using low-rank matrix completion. The new method is robust and allows for x 5-6 reduction in sampling. Experimental results obtained on real data are illustrated.

physics.med-ph↗

arXiv2022

Symmetrization Techniques in Image Deblurring

This paper presents a couple of preconditioning techniques that can be used to enhance the performance of iterative regularization methods applied to image deblurring problems with a variety of point spread functions (PSFs) and boundary conditions. More precisely, we first consider the anti-identity preconditioner, which symmetrizes the coefficient matrix associated to problems with zero boundary conditions, allowing the use of MINRES as a regularization method. When considering more sophisticated boundary conditions and strongly nonsymmetric PSFs, the anti-identity preconditioner improves the performance of GMRES. We then consider both stationary and iteration-dependent regularizing circulant preconditioners that, applied in connection with the anti-identity matrix and both standard and flexible Krylov subspaces, speed up the iterations. A theoretical result about the clustering of the eigenvalues of the preconditioned matrices is proved in a special case. The results of many numerical experiments are reported to show the effectiveness of the new preconditioning techniques, including when considering the deblurring of sparse images.

math.NA↗

arXiv2022

Automatic balancing parameter selection for Tikhonov-TV regularization

This paper considers large-scale linear ill-posed inverse problems whose solutions can be represented as sums of smooth and piecewise constant components. To solve such problems we consider regularizers consisting of two terms that must be balanced. Namely, a Tikhonov term guarantees the smoothness of the smooth solution component, while a total-variation (TV) regularizer promotes blockiness of the non-smooth solution component. A scalar parameter allows to balance between these two terms and, hence, to appropriately separate and regularize the smooth and non-smooth components of the solution. This paper proposes an efficient algorithm to solve this regularization problem by the alternating direction method of multipliers (ADMM). Furthermore, a novel algorithm for automatic choice of the balancing parameter is introduced, using robust statistics. The proposed approach is supported by some theoretical analysis, and numerical experiments concerned with different inverse problems are presented to validate the choice of the balancing parameter.

math.NA↗

arXiv2021

Efficient learning methods for large-scale optimal inversion design

In this work, we investigate various approaches that use learning from training data to solve inverse problems, following a bi-level learning approach. We consider a general framework for optimal inversion design, where training data can be used to learn optimal regularization parameters, data fidelity terms, and regularizers, thereby resulting in superior variational regularization methods. In particular, we describe methods to learn optimal $p$ and $q$ norms for ${\rm L}^p-{\rm L}^q$ regularization and methods to learn optimal parameters for regularization matrices defined by covariance kernels. We exploit efficient algorithms based on Krylov projection methods for solving the regularized problems, both at training and validation stages, making these methods well-suited for large-scale problems. Our experiments show that the learned regularization methods perform well even when there is some inexactness in the forward operator, resulting in a mixture of model and measurement error.

math.NA↗

arXiv2021

Computational methods for large-scale inverse problems: a survey on hybrid projection methods

This paper surveys an important class of methods that combine iterative projection methods and variational regularization methods for large-scale inverse problems. Iterative methods such as Krylov subspace methods are invaluable in the numerical linear algebra community and have proved important in solving inverse problems due to their inherent regularizing properties and their ability to handle large-scale problems. Variational regularization describes a broad and important class of methods that are used to obtain reliable solutions to inverse problems, whereby one solves a modified problem that incorporates prior knowledge. Hybrid projection methods combine iterative projection methods with variational regularization techniques in a synergistic way, providing researchers with a powerful computational framework for solving very large inverse problems. Although the idea of a hybrid Krylov method for linear inverse problems goes back to the 1980s, several recent advances on new regularization frameworks and methodologies have made this field ripe for extensions, further analyses, and new applications. In this paper, we provide a practical and accessible introduction to hybrid projection methods in the context of solving large (linear) inverse problems.

math.NA↗

arXiv2021

Efficient edge-preserving methods for dynamic inverse problems

We consider efficient methods for computing solutions to dynamic inverse problems, where both the quantities of interest and the forward operator (measurement process) may change at different time instances but we want to solve for all the images simultaneously. We are interested in large-scale ill-posed problems that are made more challenging by their dynamic nature and, possibly, by the limited amount of available data per measurement step. To remedy these difficulties, we apply regularization methods that enforce simultaneous regularization in space and time (such as edge enhancement at each time instant and proximity at consecutive time instants) and achieve this with low computational cost and enhanced accuracy. More precisely, we develop iterative methods based on a majorization-minimization (MM) strategy with quadratic tangent majorant, which allows the resulting least squares problem to be solved with a generalized Krylov subspace (GKS) method; the regularization parameter can be defined automatically and efficiently at each iteration. Numerical examples from a wide range of applications, such as limited-angle computerized tomography (CT), space-time image deblurring, and photoacoustic tomography (PAT), illustrate the effectiveness of the described approaches.

math.NA↗

arXiv2021

Regularization by inexact Krylov methods with applications to blind deblurring

This paper is concerned with the regularization of large-scale discrete inverse problems by means of inexact Krylov methods. Specifically, we derive two new inexact Krylov methods that can be efficiently applied to unregularized or Tikhonov-regularized least squares problems, and we study their theoretical properties, including links with their exact counterparts and strategies to monitor the amount of inexactness. We then apply the new methods to separable nonlinear inverse problems arising in blind deblurring. In this setting inexactness stems from the uncertainty in the parameters defining the blur, which may be recovered using a variable projection method leading to an inner-outer iteration scheme (i.e., one cycle of inner iterations is performed to solve one linear deblurring subproblem for any intermediate values of the blurring parameters computed by a nonlinear least squares solver). The new inexact solvers can naturally handle varying inexact blurring parameters while solving the linear deblurring subproblems, allowing for a much reduced number of total iterations and substantial computational savings with respect to their exact counterparts.

math.NA↗