arXiv subjects

David Pardo

Publications and source records attributed to David Pardo.

At least 37 records · Page 2Linked to original sources

arXiv2024

Ensemble Deep Learning for enhanced seismic data reconstruction

Seismic data often contain gaps due to various obstacles in the investigated area and recording instrument failures. Deep learning techniques offer promising solutions for reconstructing missing data parts by leveraging existing information. However, self-supervised methods frequently struggle with capturing under-represented features such as weaker events, crossing dips, and higher frequencies. To address these challenges, we propose a novel ensemble deep model along with a tailored self-supervised training approach for reconstructing seismic data with consecutive missing traces. Our model comprises two branches of U-nets, each fed from distinct data transformation modules aimed at amplifying under-represented features and promoting diversity among learners. Our loss function minimizes relative errors at the outputs of individual branches and the entire model, ensuring accurate reconstruction of various features while maintaining overall data integrity. Additionally, we employ masking while training to enhance sample diversity and memory efficiency. Application on two benchmark synthetic datasets and two real datasets demonstrates improved accuracy compared to a conventional U-net, successfully reconstructing weak events, diffractions, higher frequencies, and reflections obscured by groundroll. However, our method requires a threefold of training time compared to a simple U-net. An implementation of our method with TensorFlow is also made available.

physics.geo-ph↗

arXiv2024

Robust Variational Physics-Informed Neural Networks

We introduce a Robust version of the Variational Physics-Informed Neural Networks method (RVPINNs). As in VPINNs, we define the quadratic loss functional in terms of a Petrov-Galerkin-type variational formulation of the PDE problem: the trial space is a (Deep) Neural Network (DNN) manifold, while the test space is a finite-dimensional vector space. Whereas the VPINN's loss depends upon the selected basis functions of a given test space, herein, we minimize a loss based on the discrete dual norm of the residual. The main advantage of such a loss definition is that it provides a reliable and efficient estimator of the true error in the energy norm under the assumption of the existence of a local Fortin operator. We test the performance and robustness of our algorithm in several advection-diffusion problems. These numerical results perfectly align with our theoretical findings, showing that our estimates are sharp.

math.NA↗

arXiv2024

Learning quantities of interest from parametric PDEs: An efficient neural-weighted Minimal Residual approach

The efficient approximation of parametric PDEs is of tremendous importance in science and engineering. In this paper, we show how one can train Galerkin discretizations to efficiently learn quantities of interest of solutions to a parametric PDE. The central component in our approach is an efficient neural-network-weighted Minimal-Residual formulation, which, after training, provides Galerkin-based approximations in standard discrete spaces that have accurate quantities of interest, regardless of the coarseness of the discrete space.

math.NA↗

arXiv2024

Adaptive Deep Fourier Residual method via overlapping domain decomposition

The Deep Fourier Residual (DFR) method is a specific type of variational physics-informed neural networks (VPINNs). It provides a robust neural network-based solution to partial differential equations (PDEs). The DFR strategy is based on approximating the dual norm of the weak residual of a PDE. This is equivalent to minimizing the energy norm of the error. To compute the dual of the weak residual norm, the DFR method employs an orthonormal spectral basis of the test space, which is known for rectangles or cuboids for multiple function spaces. In this work, we extend the DFR method with ideas of traditional domain decomposition (DD). This enables two improvements: (a) to solve problems in more general polygonal domains, and (b) to develop an adaptive refinement technique in the test space using a Dofler marking algorithm. In the former case, we show that under non-restrictive assumptions we retain the desirable equivalence between the employed loss function and the H1-error, numerically demonstrating adherence to explicit bounds in the case of the L-shaped domain problem. In the latter, we show how refinement strategies lead to potentially significant improvements against a reference, classical DFR implementation with a test function space of significantly lower dimensionality, allowing us to better approximate singular solutions at a more reasonable computational cost.

math.NA↗

arXiv2023

Semi-blind-trace algorithm for self-supervised attenuation of trace-wise coherent noise

Trace-wise noise is a type of noise often seen in seismic data, which is characterized by vertical coherency and horizontal incoherency. Using self-supervised deep learning to attenuate this type of noise, the conventional blind-trace deep learning trains a network to blindly reconstruct each trace in the data from its surrounding traces; it attenuates isolated trace-wise noise but causes signal leakage in clean and noisy traces and reconstruction errors next to each noisy trace. To reduce signal leakage and improve denoising, we propose a new loss function and masking procedure in semi-blind-trace deep learning. Our hybrid loss function has weighted active zones that cover masked and non-masked traces. Therefore, the network is not blinded to clean traces during their reconstruction. During training, we dynamically change the masks' characteristics. The goal is to train the network to learn the characteristics of the signal instead of noise. The proposed algorithm enables the designed U-net to detect and attenuate trace-wise noise without having prior information about the noise. A new hyperparameter of our method is the relative weight between the masked and non-masked traces' contribution to the loss function. Numerical experiments show that selecting a small value for this parameter is enough to significantly decrease signal leakage. The proposed algorithm is tested on synthetic and real off-shore and land datasets with different noises. The results show the superb ability of the method to attenuate trace-wise noise while preserving other events. An implementation of the proposed algorithm as a Python code is also made available.

physics.geo-ph↗

arXiv2023

Deep Fourier Residual method for solving time-harmonic Maxwell's equations

Solving PDEs with machine learning techniques has become a popular alternative to conventional methods. In this context, Neural networks (NNs) are among the most commonly used machine learning tools, and in those models, the choice of an appropriate loss function is critical. In general, the main goal is to guarantee that minimizing the loss during training translates to minimizing the error in the solution at the same rate. In this work, we focus on the time-harmonic Maxwell's equations, whose weak formulation takes H(curl) as the space of test functions. We propose a NN in which the loss function is a computable approximation of the dual norm of the weak-form PDE residual. To that end, we employ the Helmholtz decomposition of the space H(curl) and construct an orthonormal basis for this space in two and three spatial dimensions. Here, we use the Discrete Sine/Cosine Transform to accurately and efficiently compute the discrete version of our proposed loss function. Moreover, in the numerical examples we show a high correlation between the proposed loss function and the H(curl)-norm of the error, even in problems with low-regularity solutions.

math.NA↗

arXiv2023

Machine Learning Discovery of Optimal Quadrature Rules for Isogeometric Analysis

We propose the use of machine learning techniques to find optimal quadrature rules for the construction of stiffness and mass matrices in isogeometric analysis (IGA). We initially consider 1D spline spaces of arbitrary degree spanned over uniform and non-uniform knot sequences, and then the generated optimal rules are used for integration over higher-dimensional spaces using tensor product sense. The quadrature rule search is posed as an optimization problem and solved by a machine learning strategy based on gradient-descent. However, since the optimization space is highly non-convex, the success of the search strongly depends on the number of quadrature points and the parameter initialization. Thus, we use a dynamic programming strategy that initializes the parameters from the optimal solution over the spline space with a lower number of knots. With this method, we found optimal quadrature rules for spline spaces when using IGA discretizations with up to 50 uniform elements and polynomial degrees up to 8, showing the generality of the approach in this scenario. For non-uniform partitions, the method also finds an optimal rule in a reasonable number of test cases. We also assess the generated optimal rules in two practical case studies, namely, the eigenvalue problem of the Laplace operator and the eigenfrequency analysis of freeform curved beams, where the latter problem shows the applicability of the method to curved geometries. In particular, the proposed method results in savings with respect to traditional Gaussian integration of up to 44% in 1D, 68% in 2D, and 82% in 3D spaces.

math.NA↗

arXiv2023

A Deep Double Ritz Method (D$^2$RM) for solving Partial Differential Equations using Neural Networks

Residual minimization is a widely used technique for solving Partial Differential Equations in variational form. It minimizes the dual norm of the residual, which naturally yields a saddle-point (min-max) problem over the so-called trial and test spaces. In the context of neural networks, we can address this min-max approach by employing one network to seek the trial minimum, while another network seeks the test maximizers. However, the resulting method is numerically unstable as we approach the trial solution. To overcome this, we reformulate the residual minimization as an equivalent minimization of a Ritz functional fed by optimal test functions computed from another Ritz functional minimization. We call the resulting scheme the Deep Double Ritz Method (D$^2$RM), which combines two neural networks for approximating trial functions and optimal test functions along a nested double Ritz minimization strategy. Numerical results on different diffusion and convection problems support the robustness of our method, up to the approximation properties of the networks and the training capacity of the optimizers.

math.NA↗

arXiv2022

A Deep Fourier Residual Method for solving PDEs using Neural Networks

When using Neural Networks as trial functions to numerically solve PDEs, a key choice to be made is the loss function to be minimised, which should ideally correspond to a norm of the error. In multiple problems, this error norm coincides with--or is equivalent to--the $H^{-1}$-norm of the residual; however, it is often difficult to accurately compute it. This work assumes rectangular domains and proposes the use of a Discrete Sine/Cosine Transform to accurately and efficiently compute the $H^{-1}$ norm. The resulting Deep Fourier-based Residual (DFR) method efficiently and accurately approximate solutions to PDEs. This is particularly useful when solutions lack $H^{2}$ regularity and methods involving strong formulations of the PDE fail. We observe that the $H^1$-error is highly correlated with the discretised loss during training, which permits accurate error estimation via the loss.

math.NA↗

arXiv2022

$r-$Adaptive Deep Learning Method for Solving Partial Differential Equations

We introduce an $r-$adaptive algorithm to solve Partial Differential Equations using a Deep Neural Network. The proposed method restricts to tensor product meshes and optimizes the boundary node locations in one dimension, from which we build two- or three-dimensional meshes. The method allows the definition of fixed interfaces to design conforming meshes, and enables changes in the topology, i.e., some nodes can jump across fixed interfaces. The method simultaneously optimizes the node locations and the PDE solution values over the resulting mesh. To numerically illustrate the performance of our proposed $r-$adaptive method, we apply it in combination with a collocation method, a Least Squares Method, and a Deep Ritz Method. We focus on the latter to solve one- and two-dimensional problems whose solutions are smooth, singular, and/or exhibit strong gradients.

math.NA↗

arXiv2021

Performance of Refined Isogeometric Analysis in Solving Quadratic Eigenvalue Problems

Certain applications that analyze damping effects require the solution of quadratic eigenvalue problems (QEPs). We use refined isogeometric analysis (rIGA) to solve quadratic eigenproblems. rIGA discretization, while conserving desirable properties of maximum-continuity isogeometric analysis (IGA), reduces the interconnection between degrees of freedom by adding low-continuity basis functions. This connectivity reduction in rIGA's algebraic system results in faster matrix LU factorizations when using multifrontal direct solvers. We compare computational costs of rIGA versus those of IGA when employing Krylov eigensolvers to solve quadratic eigenproblems arising in 2D vector-valued multifield problems. For large problem sizes, the eigencomputation cost is governed by the cost of LU factorization, followed by costs of several matrix-vector and vector-vector multiplications, which correspond to Krylov projections. We minimize the computational cost by introducing C^0 and C^1 separators at specific element interfaces for our rIGA generalizations of the curl-conforming Nedelec and divergence-conforming Raviart-Thomas finite elements. Let p be the polynomial degree of basis functions; the LU factorization is up to O((p-1)^2) times faster when using rIGA compared to IGA in the asymptotic regime. Thus, rIGA theoretically improves the total eigencomputation cost by O((p-1)^2) for sufficiently large problem sizes. Yet, in practical cases of moderate-size eigenproblems, the improvement rate deteriorates as the number of computed eigenvalues increases because of multiple matrix-vector and vector-vector operations. Our numerical tests show that rIGA accelerates the solution of quadratic eigensystems by O(p-1) for moderately sized problems when we seek to compute a reasonable number of eigenvalues.

math.NA↗

arXiv2021

Deep-Learning Inversion Method for the Interpretation of Noisy Logging-While-Drilling Resistivity Measurements

Deep Learning (DL) inversion is a promising method for real time interpretation of logging while drilling (LWD) resistivity measurements for well navigation applications. In this context, measurement noise may significantly affect inversion results. Existing publications examining the effects of measurement noise on DL inversion results are scarce. We develop a method to generate training data sets and construct DL architectures that enhance the robustness of DL inversion methods in the presence of noisy LWD resistivity measurements. We use two synthetic resistivity models to test three approaches that explicitly consider the presence of noise: (1) adding noise to the measurements in the training set, (2) augmenting the training set by replicating it and adding varying noise realizations, and (3) adding a noise layer in the DL architecture. Numerical results confirm that the three approaches produce a denoising effect, yielding better inversion results in both predicted earth model and measurements compared not only to the basic DL inversion but also to traditional gradient based inversion results. A combination of the second and third approaches delivers the best results. The proposed methods can be readily generalized to multi dimensional DL inversion.

physics.geo-ph↗

arXiv2021

On quadrature rules for solving Partial Differential Equations using Neural Networks

Neural Networks have been widely used to solve Partial Differential Equations. These methods require to approximate definite integrals using quadrature rules. Here, we illustrate via 1D numerical examples the quadrature problems that may arise in these applications and propose different alternatives to overcome them, namely: Monte Carlo methods, adaptive integration, polynomial approximations of the Neural Network output, and the inclusion of regularization terms in the loss. We also discuss the advantages and limitations of each proposed alternative. We advocate the use of Monte Carlo methods for high dimensions (above 3 or 4), and adaptive integration or polynomial approximations for low dimensions (3 or below). The use of regularization terms is a mathematically elegant alternative that is valid for any spacial dimension, however, it requires certain regularity assumptions on the solution and complex mathematical analysis when dealing with sophisticated Neural Networks.

math.NA↗

arXiv2021

Modeling extra-deep electromagnetic logs using a deep neural network

Modern geosteering is heavily dependent on real-time interpretation of deep electromagnetic (EM) measurements. We present a methodology to construct a deep neural network (DNN) model trained to reproduce a full set of extra-deep EM logs consisting of 22 measurements per logging position. The model is trained in a 1D layered environment consisting of up to seven layers with different resistivity values. A commercial simulator provided by a tool vendor is used to generate a training dataset. The dataset size is limited because the simulator provided by the vendor is optimized for sequential execution. Therefore, we design a training dataset that embraces the geological rules and geosteering specifics supported by the forward model. We use this dataset to produce an EM simulator based on a DNN without access to the proprietary information about the EM tool configuration or the original simulator source code. Despite employing a relatively small training set size, the resulting DNN forward model is quite accurate for the considered examples: a multi-layer synthetic case and a section of a published historical operation from the Goliat Field. The observed average evaluation time of 0.15 ms per logging position makes it also suitable for future use as part of evaluation-hungry statistical and/or Monte-Carlo inversion algorithms within geosteering workflows.

eess.SP↗

arXiv2020

Goal-oriented adaptivity for a conforming residual minimization method in a dual discontinuous Galerkin norm

We propose a goal-oriented mesh-adaptive algorithm for a finite element method stabilized via residual minimization on dual discontinuous-Galerkin norms. By solving a saddle-point problem, this residual minimization delivers a stable continuous approximation to the solution on each mesh instance and a residual projection onto a broken polynomial space, which is a robust error estimator to minimize the discrete energy norm via automatic mesh refinement. In this work, we propose and analyze a goal-oriented adaptive algorithm for this stable residual minimization. We solve the primal and adjoint problems considering the same saddle-point formulation and different right-hand sides. By solving a third stable problem, we obtain two efficient error estimates to guide goal-oriented adaptivity. We illustrate the performance of this goal-oriented adaptive strategy on advection-diffusion-reaction problems.

math.NA↗

arXiv2020

Database Generation for Deep Learning Inversion of 2.5D Borehole Electromagnetic Measurements using Refined Isogeometric Analysis

Borehole resistivity measurements are routinely inverted in real-time during geosteering operations. The inversion process can be efficiently performed with the help of advanced artificial intelligence algorithms such as deep learning. These methods require a large dataset that relates multiple earth models with the corresponding borehole resistivity measurements. In here, we propose to use an advanced numerical method --refined isogeometric analysis (rIGA)-- to perform rapid and accurate 2.5D simulations and generate databases when considering arbitrary 2D earth models. Numerical results show that we can generate a meaningful synthetic database composed of 100,000 earth models with the corresponding measurements in 56 hours using a workstation equipped with two CPUs.

math.NA↗

arXiv2020

Refined isogeometric analysis for generalized Hermitian eigenproblems

We use the refined isogeometric analysis (rIGA) to solve generalized Hermitian eigenproblems $({Ku=λMu})$. The rIGA framework conserves the desirable properties of maximum-continuity isogeometric analysis (IGA) discretizations while reducing the computation cost of the solution through partitioning the computational domain by adding zero-continuity basis functions. As a result, rIGA enriches the approximation space and decreases the interconnection between degrees of freedom. We compare computational costs of rIGA versus those of IGA when employing a Lanczos eigensolver with a shift-and-invert spectral transformation. When all eigenpairs within a given interval ${[λ_s,λ_e]}$ are of interest, we select several shifts ${σ_k\in[λ_s,λ_e]}$ using a spectrum slicing technique. For each shift $σ_k$, the cost of factorization of the spectral transformation matrix ${K-σ_k M}$ drives the total computational cost of the eigensolution. Several multiplications of the operator matrices ${(K-σ_k M)^{-1} M}$ by vectors follow this factorization. Let $p$ be the polynomial degree of basis functions and assume that IGA has maximum continuity of ${p-1}$, while rIGA introduces $C^0$ separators to minimize the factorization cost. For this setup, our theoretical estimates predict computational savings to compute a fixed number of eigenpairs of up to ${O(p^2)}$ in the asymptotic regime, that is, large problem sizes. Yet, our numerical tests show that for moderately-sized eigenproblems, the total computational cost reduction is $O(p)$. Nevertheless, rIGA improves the accuracy of every eigenpair of the first $N_0$ eigenvalues and eigenfunctions. Here, we allow $N_0$ to be as large as the total number of eigenmodes of the original maximum-continuity IGA discretization.

math.NA↗

arXiv2018

Variational Formulations for Explicit Runge-Kutta Methods

Variational space-time formulations for Partial Differential Equations have been of great interest in the last decades. While it is known that implicit time marching schemes have variational structure, the Galerkin formulation of explicit methods in time remains elusive. In this work, we prove that the explicit Runge-Kutta methods can be expressed as discontinuous Petrov-Galerkin methods both in space and time. We build trial and test spaces for the linear diffusion equation that lead to one, two, and general stage explicit Runge-Kutta methods. This approach enables us to design explicit time-domain (goal-oriented) adaptive algorithms

math.NA↗