arXiv subjects

Ali Gholami

Publications and source records attributed to Ali Gholami.

At least 37 records · Page 2Linked to original sources

arXiv2024

An extended Gauss-Newton method for full waveform inversion

Full waveform inversion (FWI) is a large-scale nonlinear ill-posed problem for which computationally expensive Newton-type methods can become trapped in undesirable local minima, particularly when the initial model lacks a low-wavenumber component and the recorded data lacks low-frequency content. A modification to the Gauss-Newton (GN) method is proposed to address these issues. The standard GN system for multisource multireceiver FWI is reformulated into an equivalent matrix equation form, with the solution becoming a diagonal matrix rather than a vector as in the standard system. The search direction is transformed from a vector to a matrix by relaxing the diagonality constraint, effectively adding a degree of freedom to the subsurface offset axis. The relaxed system can be explicitly solved with only the inversion of two small matrices that deblur the data residual matrix along the source and receiver dimensions, which simplifies the inversion of the Hessian matrix. When used to solve the extended source FWI objective function, the Extended GN (EGN) method integrates the benefits of both model and source extension. The EGN method effectively combines the computational effectiveness of the reduced FWI method with the robustness characteristics of extended formulations and offers a promising solution for addressing the challenges of FWI. It bridges the gap between these extended formulations and the reduced FWI method, enhancing inversion robustness while maintaining computational efficiency. The robustness and stability of the EGN algorithm for waveform inversion are demonstrated numerically.

physics.geo-ph↗

arXiv2023

Full Waveform Inversion and Lagrange Multipliers

Full-waveform inversion (FWI) is an effective method for imaging subsurface properties using sparsely recorded data. It involves solving a wave propagation problem to estimate model parameters that accurately reproduce the data. Recent trends in FWI have led to the development of extended methodologies, among which source extension methods leveraging reconstructed wavefields to solve penalty or augmented Lagrangian (AL) formulations have emerged as robust algorithms, even for inaccurate initial models. Despite their demonstrated robustness, challenges remain, such as the lack of a clear physical interpretation, difficulty in comparison, and reliance on difficult-to-compute least squares (LS) wavefields. This paper is divided into two critical parts. In the first, a novel formulation of these methods is explored within a unified Lagrangian framework. This novel perspective permits the introduction of alternative algorithms that employ LS multipliers instead of wavefields. These multiplier-oriented variants appear as regularizations of the standard FWI, are adaptable to the time domain, offer tangible physical interpretations, and foster enhanced convergence efficiency. The second part of the paper delves into understanding the underlying mechanisms of these techniques. This is achieved by solving the FWI equations using iterative linearization and inverse scattering methods. The paper provides insight into the role and significance of Lagrange multipliers in enhancing the linearization of FWI equations. It explains how different methods estimate multipliers or make approximations to increase computing efficiency. Additionally, it presents a new physical understanding of the Lagrange multiplier used in the AL method, highlighting how important it is for improving algorithm performance when compared to penalty methods.

math.OC↗

arXiv2023

Coded Caching with Private Demands and Caches

Recently it was shown that the seminal Maddah-Ali and Niesen (MAN) coded caching scheme leaks the demand information of each user to the others. Many works have considered coded caching with demand privacy, while each non-trivial existing coded caching scheme with private demands was built on the fact that the cache information of each user is private to the others. However, most of these schemes leak the users' cache information. Consequently, in most realistic settings (e.g., video streaming), where the system is used over time with multiple sequential transmission rounds, these schemes leak demand privacy beyond the first round. This observation motivates our new formulation of coded caching with simultaneously private demands and caches. The main contribution of this paper is a new construction that generates private coded caching schemes by leveraging two-server private information retrieval (PIR) schemes. We show that if in the PIR scheme the demand is uniform over all files and the queries are independent, the resulting caching scheme is private on both the demands and on the caches. Interestingly, we propose a new construction of two-server PIR schemes in this class by leveraging coded caching schemes. By applying the seminal MAN coded caching scheme into our construction, the resulting two-server PIR scheme is proved to be order optimal. This is a second new structural result, somehow closing the loop in the relation between coded caching and PIR. Finally, to explore a broader tradeoff between cache privacy and transmission load, we relax the cache privacy constraint and introduce the definition of leakage on cache information. Then, again as a by-product of our new construction, we propose new schemes with perfect demand privacy and imperfect cache privacy that achieve an order-gain in load with respect to the scheme with perfect privacy on both demands and caches.

cs.IT↗

arXiv2023

A practical implementation of data-space Hessian in the time-domain extended-source full-waveform inversion

Full-waveform inversion (FWI) with extended sources first computes wavefields with data-driven source extensions, such that the simulated data in inaccurate velocity models match the observed counterpart well enough to prevent cycle skipping. Then, the source extensions are minimized to update the model parameters. This two-step workflow is iterated until both data and sources are matched. It was recently shown that the source extensions are the least-squares solutions of the recorded scattered data fitting problem. As a result, they are computed by propagating backward in time the deblurred FWI data residuals, where the deblurring operator is the inverse of the damped data-domain Hessian of the scattering-source estimation problem. Estimating the deblurred data residuals is the main computational bottleneck of time-domain extended-source FWI (ES-FWI). To mitigate this issue, we first estimate them when the inverse of the data-domain Hessians is approximated by matching filters in Fourier and short-time Fourier domains. Second, we refine them with conjugate-gradient iterations when necessary. Computing the matching filters and performing one conjugate-gradient iteration each require two simulations per source. Therefore, it is critical to design some workflows that minimize this computational burden. We implement time-domain ES-FWI with the augmented Lagrangian method. Moreover, we further extend its linear regime with a multiscale frequency continuation approach, which is combined with grid coarsening to mitigate the computational burden and regularize the inversion. Finally, we use total-variation regularization to deal with large-contrast reconstruction. We present synthetic cases where different inversion workflows carried out with data-domain Hessians of variable accuracy were assessed with the aim at converging toward accurate solutions while minimizing computational cost.

physics.geo-ph↗

arXiv2023

Full waveform inversion beyond the Born approximation: A tutorial review

Full Waveform Inversion can be made immune to cycle skipping by matching the recorded data arbitrarily well from inaccurate subsurface models. To achieve this goal, the simulated wavefields can be computed in an extended search space as the solution of an overdetermined problem aiming at jointly satisfying the wave equation and fitting the data in a least-squares sense. Simply put, the wavefields are computed by solving the wave equation in the inaccurate background model with a feedback term to the data added to the physical source in the right-hand side. Then, the subsurface parameters are updated by canceling out these additional source terms, sometimes called unwisely wave-equation errors, to push the background model toward the true model in the left-hand side wave-equation operator. Although many studies were devoted to these approaches with promising numerical results, their governing physical principles and their relationships with classical FWI don't seem to be understood well yet. The goal of this tutorial is to review these principles in the theoretical framework of inverse scattering theory whose governing forward equation is the Lippmann-Schwinger equation. From this equation, we show how the data-assimilated wavefields embed an approximation of the scattered field generated by the sought model perturbation and how they modify the sensitivity kernel of classical FWI beyond the Born approximation. We also clarify how the approximation with which these wavefields approximate the unknown true wavefields is accounted for in the adjoint source of the parameter estimation problem. The theory is finally illustrated with numerical examples. Understanding the physical principles governing these methods is a necessary prerequisite to assessing their potential and limits and designing relevant heuristics to manage the latter.

physics.geo-ph↗

arXiv2022

AdaMAE: Adaptive Masking for Efficient Spatiotemporal Learning with Masked Autoencoders

Masked Autoencoders (MAEs) learn generalizable representations for image, text, audio, video, etc., by reconstructing masked input data from tokens of the visible data. Current MAE approaches for videos rely on random patch, tube, or frame-based masking strategies to select these tokens. This paper proposes AdaMAE, an adaptive masking strategy for MAEs that is end-to-end trainable. Our adaptive masking strategy samples visible tokens based on the semantic context using an auxiliary sampling network. This network estimates a categorical distribution over spacetime-patch tokens. The tokens that increase the expected reconstruction error are rewarded and selected as visible tokens, motivated by the policy gradient algorithm in reinforcement learning. We show that AdaMAE samples more tokens from the high spatiotemporal information regions, thereby allowing us to mask 95% of tokens, resulting in lower memory requirements and faster pre-training. We conduct ablation studies on the Something-Something v2 (SSv2) dataset to demonstrate the efficacy of our adaptive sampling approach and report state-of-the-art results of 70.0% and 81.7% in top-1 accuracy on SSv2 and Kinetics-400 action classification datasets with a ViT-Base backbone and 800 pre-training epochs.

cs.CV↗

arXiv2022

MaskTune: Mitigating Spurious Correlations by Forcing to Explore

A fundamental challenge of over-parameterized deep learning models is learning meaningful data representations that yield good performance on a downstream task without over-fitting spurious input features. This work proposes MaskTune, a masking strategy that prevents over-reliance on spurious (or a limited number of) features. MaskTune forces the trained model to explore new features during a single epoch finetuning by masking previously discovered features. MaskTune, unlike earlier approaches for mitigating shortcut learning, does not require any supervision, such as annotating spurious features or labels for subgroup samples in a dataset. Our empirical results on biased MNIST, CelebA, Waterbirds, and ImagenNet-9L datasets show that MaskTune is effective on tasks that often suffer from the existence of spurious correlations. Finally, we show that MaskTune outperforms or achieves similar performance to the competing methods when applied to the selective classification (classification with rejection option) task. Code for MaskTune is available at https://github.com/aliasgharkhani/Masktune.

cs.LG↗

arXiv2022

Counterbalancing Teacher: Regularizing Batch Normalized Models for Robustness

Batch normalization (BN) is a ubiquitous technique for training deep neural networks that accelerates their convergence to reach higher accuracy. However, we demonstrate that BN comes with a fundamental drawback: it incentivizes the model to rely on low-variance features that are highly specific to the training (in-domain) data, hurting generalization performance on out-of-domain examples. In this work, we investigate this phenomenon by first showing that removing BN layers across a wide range of architectures leads to lower out-of-domain and corruption errors at the cost of higher in-domain errors. We then propose Counterbalancing Teacher (CT), a method which leverages a frozen copy of the same model without BN as a teacher to enforce the student network's learning of robust representations by substantially adapting its weights through a consistency loss function. This regularization signal helps CT perform well in unforeseen data shifts, even without information from the target domain as in prior works. We theoretically show in an overparameterized linear regression setting why normalization leads to a model's reliance on such in-domain features, and empirically demonstrate the efficacy of CT by outperforming several baselines on robustness benchmarks such as CIFAR-10-C, CIFAR-100-C, and VLCS.

cs.LG↗

arXiv2022

Automatic balancing parameter selection for Tikhonov-TV regularization

This paper considers large-scale linear ill-posed inverse problems whose solutions can be represented as sums of smooth and piecewise constant components. To solve such problems we consider regularizers consisting of two terms that must be balanced. Namely, a Tikhonov term guarantees the smoothness of the smooth solution component, while a total-variation (TV) regularizer promotes blockiness of the non-smooth solution component. A scalar parameter allows to balance between these two terms and, hence, to appropriately separate and regularize the smooth and non-smooth components of the solution. This paper proposes an efficient algorithm to solve this regularization problem by the alternating direction method of multipliers (ADMM). Furthermore, a novel algorithm for automatic choice of the balancing parameter is introduced, using robust statistics. The proposed approach is supported by some theoretical analysis, and numerical experiments concerned with different inverse problems are presented to validate the choice of the balancing parameter.

math.NA↗

arXiv2022

On the connection between WRI and FWI: Analysis of the nonlinear term in the Hessian matrix

Implementation of the standard full waveform inversion (FWI) poses difficulties as the initial model offsets from the true model. The wavefield reconstruction inversion (WRI) was proposed to mitigate these difficulties by relaxing the wave-equation constraint. In this abstract, working on the nonlinear term in the Hessian matrix of FWI, we develop a new approximate Hessian as an Augmented Gauss-Newton (AGN) Hessian including second-order derivative information. Moreover, we establish an intimate connection between an updating formula which results from approximate solve of the Newton's method with the AGN Hessian on the FWI problem and the WRI method. Our analysis opens new perspectives for developing efficient algorithms for FWI based on the Newton's method and highlights the importance of the nonlinear term in the Hessian matrix, which is ignored in most cases.

math.OC↗

arXiv2022

Localized Wavefield Inversion (LWI): an Adaptation of Multi-Block ADMM for Localized FWI

Full-waveform inversion (FWI) is a high-resolution and computationally intensive imaging technique to reconstruct unknown parameters in the computational model in which the waves propagate; however, an accurate model of only part of this medium is required for some applications. To decrease the computational burden of such problems, target-oriented FWI was proposed where the redatumed data on the part of the medium or localized solvers for the wave equation are used. On the other hand, the classical formulation of FWI suffers from non-linearity and ill-posedness, which makes FWI sensitive to the initial model, the low-frequency content of the data, and limited illumination. In this study, we propose a localized version of the alternating direction method of multipliers (ADMM)-based FWI method, which was proposed to solve these problems in classical FWI. In our localized FWI or LWI, the medium is decomposed into a few subdomains, where some of them are updated, and the others are kept fixed based on an adaptation of multi-block ADMM, which is a powerful algorithm for solving inverse problems with decomposition and block separability. Numerical tests on the Marmousi model for a time-lapse application confirm the computational efficiency and robustness against background velocity model errors.

math.OC↗

arXiv2022

Large-scale highly-accurate extended full waveform inversion using convergent Born series

Full-waveform inversion (FWI) is a seismic imaging method that provides quantitative inference about subsurface properties with a wavelength-scale resolution. Its frequency-domain formulation is computationally efficient when processing only a few discrete frequencies. However, classical FWI, which is formulated on the reduced-parameter space, requires starting the inversion with a sufficiently-accurate initial model and low frequency to prevent being stuck in local minima due to cycle skipping. FWI with extended search space has been proposed to mitigate this issue. It contains two main steps: first, data-assimilated (DA) wavefields are computed by allowing for wave-equation errors to match the data at receivers closely. Then, subsurface parameters are estimated from these wavefields by minimizing the wave-equation errors. The DA wavefields are the least-squares solution of an overdetermined system gathering the wave and observation equations. The numerical bandwidth of the resulting normal-equation system is two times that of the wave-equation system, which can be a limiting factor for 3D large-scale applications. Therefore, computing highly accurate DA wavefields at a reasonable computational cost is an issue in extended FWI. This issue is addressed here by rewriting the normal system such that its solution can be computed by solving the time-harmonic wave equation several times in sequence. Moreover, the computational burden of multi-right-hand side (RHS) simulations is mitigated with a sketching method. Finally, we solve the time-harmonic wave equation with the convergent Born series method, which conciliates accuracy and computational efficiency. Application of the new extended FWI algorithm on the salt benchmark shows that it reconstructs at a reasonable cost subsurface models that are similar to those obtained with the classical extended FWI.

math.OC↗

arXiv2021

Multipliers waveform inversion

The full-waveform inversion (FWI) addresses the computation and characterization of subsurface model parameters by matching predicted data to observed seismograms in the frame of nonlinear optimization. We formulate FWI as a nonlinearly constrained optimization problem, for which a regularization term is minimized subject to the nonlinear data matching constraint. Unlike FWI which is based on the penalty function, the method of multipliers solves the resulting optimization problems by using the augmented Lagrangian function; and leads to a two-step recursive algorithm. The primal step requires solving an unconstrained minimization problem like the traditional FWI with a difference that the data are replaced by the Lagrange multipliers. The dual step involves an update of the Lagrange multipliers. The overall performance of the algorithm is improved considering that this multiplier method does not require an exact solution of these primal-dual subproblems. In fact, convergence is attained when only one step of a gradient-based method is taken on both subproblems. The proposed algorithm greatly improves the overall performance of FWI such as convergence from inaccurate starting models and robustness with respect to the determination of the step length. Furthermore, it can be performed by the existing FWI engines with minimal change. We only have to replace the observed data at each iteration with the multipliers, thus all the nice properties of the traditional FWI algorithms are kept. Numerical experiments confirm that the multipliers waveform inversion can converge to a solution of the inverse problem in the absence of low-frequency data from an inaccurate initial model even with a constant step size.

math.OC↗

arXiv2021

Accurate 3D frequency-domain seismic wave modeling with the wavelength-adaptive 27-point finite-difference stencil: a tool for full waveform inversion

Efficient frequency-domain Full Waveform Inversion (FWI) of long-offset/wide-azimuth node data can be designed with a few discrete frequencies. However, 3D frequency-domain seismic modeling remains challenging since it requires solving a large and sparse linear indefinite system per frequency. When such systems are solved with direct methods or hybrid direct/iterative solvers, based upon domain decomposition preconditioner, finite-difference stencils on regular Cartesian grids should be designed to conciliate compactness and accuracy, the former being necessary to mitigate the fill-in induced by the Lower-Upper (LU) factorization. Compactness is classically implemented by combining several second-order accurate stencils covering the eight cells surrounding the collocation point, leading to the so-called 27-point stencil. Accuracy is obtained by applying optimal weights on the different stiffness and consistent mass matrices such that numerical dispersion is jointly minimized for several number of grid points per wavelength ($G$). However, with this approach, the same weights are used at each collocation point, leading to suboptimal accuracy in heterogeneous media. In this study, we propose a straightforward recipe to improve the accuracy of the 27-point stencil. First, we finely tabulate the values of $G$ covering the range of wavelengths spanned by the subsurface model and the frequency. Then, we estimate with a classical dispersion analysis in homogeneous media the corresponding table of optimal weights that minimize dispersion for each $G$ treated separately. We however apply a Tikhonov regularization to guarantee smooth variation of the weights with $G$. Finally, we build the impedance matrix by selecting the optimal weights at each collocation point according to the local wavelength, hence leading to a wavelength-adaptive stencil.

cs.CE↗

arXiv2021

Randomized source sketching for full waveform inversion

Partial differential equation (PDE) constrained optimization problems such as seismic full waveform inversion (FWI) frequently arise in the geoscience and related fields. For such problems, many observations are usually gathered by multiple sources, which form the right-hand-sides of the PDE constraint. Solving the inverse problem with such massive data sets is computationally demanding, in particular when dealing with large number of model parameters. This paper proposes a novel randomized source sketching method for the efficient resolution of multisource PDE constrained optimization problems. We first formulate the different source-encoding strategies used in seismic imaging into a unified framework based on a randomized sketching. To this end, the source dimension of the problem is projected in a smaller domain by a suitably defined projection matrix that gathers the physical sources in super-sources through a weighted summation. This reduction in the number of physical sources decreases significantly the number of PDE solves while suitable sparsity-promoting regularization can efficiently mitigate the footprint of the cross-talk noise to maintain the convergence speed of the algorithm. We implement the randomized sketching method in an extended search-space formulation of frequency-domain FWI, which relies on the alternating direction method of multipliers (ADMM). Numerical examples carried out with a series of well documented 2D benchmarks demonstrate that the randomized sketching algorithm reduces the cost of large-scale problems by at least one order of magnitude compared to the original deterministic algorithm.

physics.geo-ph↗

arXiv2021

Anderson accelerated augmented Lagrangian for extended waveform inversion

The augmented Lagrangian (AL) method provides a flexible and efficient framework for solving extended-space full-waveform inversion (FWI), a constrained nonlinear optimization problem whereby we seek model parameters and wavefields that minimize the data residuals and satisfy the wave equation constraint. The AL-based wavefield reconstruction inversion, also known as iteratively refined wavefield reconstruction inversion, extends the search space of FWI in the source dimension and decreases sensitivity of the inversion to the initial model accuracy. Furthermore, it benefits from the advantages of the alternating direction method of multipliers (ADMM), such as generality and decomposability for dealing with non-differentiable regularizers, e.g., total variation regularization, and large scale problems, respectively. In practice any extension of the method aiming at improving its convergence and decreasing the number of wave-equation solves would have a great importance. To achieve this goal, we recast the method as a general fixed-point iteration problem, which enables us to apply sophisticated acceleration strategies like Anderson acceleration. The accelerated algorithm stores a predefined number of previous iterates and uses their linear combination together with the current iteration to predict the next iteration. We investigate the performance of the proposed accelerated algorithm on a simple checkerboard model and the benchmark Marmousi II and 2004 BP salt models through numerical examples. These numerical results confirm the effectiveness of the proposed algorithm in terms of convergence rate and the quality of the final estimated model.

physics.geo-ph↗

arXiv2021

Efficient extended-search space full-waveform inversion with unknown source signatures

Full waveform inversion (FWI) requires an accurate estimation of source signatures. Due to the coupling between the source signatures and the subsurface model, small errors in the former can translate into large errors in the latter. When direct methods are used to solve the forward problem, classical frequency-domain FWI efficiently processes multiple sources for source signature and wavefield estimations once a single Lower-Upper (LU) decomposition of the wave-equation operator has been performed. However, this efficient FWI formulation is based on the exact solution of the wave equation and hence is highly sensitive to the inaccuracy of the velocity model due to the cycle skipping pathology. Recent extended-space FWI variants tackle this sensitivity issue through a relaxation of the wave equation combined with data assimilation, allowing the wavefields to closely match the data from the first inversion iteration. Then, the subsurface parameters are updated by minimizing the wave-equation violations. When the wavefields and the source signatures are jointly estimated with this approach, the extended wave equation operator becomes source dependent, hence making direct methods ineffective. In this paper, we propose a simple method to bypass this issue and estimate source signatures efficiently during extended FWI. The proposed method replaces each source with a blended source during each data-assimilated wavefield reconstruction to make the extended wave equation operator source independent. Besides computational efficiency, the additional degrees of freedom introduced by spatially distributing the sources allows for a better signature estimation at the physical location when the velocity model is rough. Numerical tests on the Marmousi II and 2004 BP salt synthetic models confirm the efficiency and the robustness of the proposed method.

math.OC↗

arXiv2021

ADMM-based full-waveform inversion for microseismic imaging

Full waveform inversion (FWI) is beginning to be used to characterize weak seismic events at different scales, an example of which is microseismic event (MSE) characterization. However, FWI with unknown sources is a severely underdetermined optimization problem, and hence requires strong prior information about the sources and/or the velocity model. The frequency-domain wavefield reconstruction inversion method (WRI) has shown promising results to mitigate the nonlinearity of the FWI objective function that is generated by cycle-skipping. WRI relies on the reconstruction of data-assimilated wavefields, which approach the true wavefields near the receivers, a helpful feature when the source is added as an additional optimization variable. We present an adaptation of a recently proposed version of WRI based on the alternating direction method of multipliers (ADMM) that first finds the location of the MSEs and then reconstructs the wavefields and the source signatures jointly. Finally, the subsurface model is updated to focus the MSEs at their true location. The method does not require prior knowledge of the number of MSEs. The inversion is stabilized by sparsifying regularizations separately tailored to the source location and velocity model subproblems. The method is tested on the Marmousi model using one MSE and two clusters of MSEs with two different initial velocity models, an accurate one and a rough one, as well as with added noise. In all cases, the method accurately locates the MSEs and recovers their source signatures.

math.OC↗