arXiv subjects

Youzuo Lin

Publications and source records attributed to Youzuo Lin.

At least 55 records · Page 3Linked to original sources

arXiv2022

An Intriguing Property of Geophysics Inversion

Inversion techniques are widely used to reconstruct subsurface physical properties (e.g., velocity, conductivity) from surface-based geophysical measurements (e.g., seismic, electric/magnetic (EM) data). The problems are governed by partial differential equations (PDEs) like the wave or Maxwell's equations. Solving geophysical inversion problems is challenging due to the ill-posedness and high computational cost. To alleviate those issues, recent studies leverage deep neural networks to learn the inversion mappings from measurements to the property directly. In this paper, we show that such a mapping can be well modeled by a very shallow (but not wide) network with only five layers. This is achieved based on our new finding of an intriguing property: a near-linear relationship between the input and output, after applying integral transform in high dimensional space. In particular, when dealing with the inversion from seismic data to subsurface velocity governed by a wave equation, the integral results of velocity with Gaussian kernels are linearly correlated to the integral of seismic data with sine kernels. Furthermore, this property can be easily turned into a light-weight encoder-decoder network for inversion. The encoder contains the integration of seismic data and the linear transformation without need for fine-tuning. The decoder only consists of a single transformer block to reverse the integral of velocity. Experiments show that this interesting property holds for two geophysics inversion problems over four different datasets. Compared to much deeper InversionNet, our method achieves comparable accuracy, but consumes significantly fewer parameters.

cs.LG↗

arXiv2022

Extremely Weak Supervision Inversion of Multi-physical Properties

Multi-physical inversion plays a critical role in geophysics. It has been widely used to infer various physical properties~(such as velocity and conductivity). Among those inversion problems, some are explicitly governed by partial differential equations~(PDEs), while others are not. Without explicit governing equations, conventional multi-physical inversion techniques will not be feasible and data-driven inversion requires expensive full labels. To overcome this issue, we develop a new data-driven multi-physics inversion technique with extremely weak supervision. Our key finding is that the pseudo labels can be constructed by learning the local relationship among geophysical properties at very sparse well-logging locations. We explore a multi-physics inversion problem from two distinct measurements~(seismic and EM data) to three geophysical properties~(velocity, conductivity, and CO$_2$ saturation). Our results show that we are able to invert for properties without explicit governing equations. Moreover, the label data on three geophysical properties can be significantly reduced by 50 times~(from 100 down to only 2 locations).

physics.geo-ph↗

arXiv2022

Unsupervised Learning of Full-Waveform Inversion: Connecting CNN and Partial Differential Equation in a Loop

This paper investigates unsupervised learning of Full-Waveform Inversion (FWI), which has been widely used in geophysics to estimate subsurface velocity maps from seismic data. This problem is mathematically formulated by a second order partial differential equation (PDE), but is hard to solve. Moreover, acquiring velocity map is extremely expensive, making it impractical to scale up a supervised approach to train the mapping from seismic data to velocity maps with convolutional neural networks (CNN). We address these difficulties by integrating PDE and CNN in a loop, thus shifting the paradigm to unsupervised learning that only requires seismic data. In particular, we use finite difference to approximate the forward modeling of PDE as a differentiable operator (from velocity map to seismic data) and model its inversion by CNN (from seismic data to velocity map). Hence, we transform the supervised inversion task into an unsupervised seismic data reconstruction task. We also introduce a new large-scale dataset OpenFWI, to establish a more challenging benchmark for the community. Experiment results show that our model (using seismic data alone) yields comparable accuracy to the supervised counterpart (using both seismic data and velocity map). Furthermore, it outperforms the supervised model when involving more seismic data.

cs.LG↗

arXiv2022

Making Invisible Visible: Data-Driven Seismic Inversion with Spatio-temporally Constrained Data Augmentation

Deep learning and data-driven approaches have shown great potential in scientific domains. The promise of data-driven techniques relies on the availability of a large volume of high-quality training datasets. Due to the high cost of obtaining data through expensive physical experiments, instruments, and simulations, data augmentation techniques for scientific applications have emerged as a new direction for obtaining scientific data recently. However, existing data augmentation techniques originating from computer vision, yield physically unacceptable data samples that are not helpful for the domain problems that we are interested in. In this paper, we develop new data augmentation techniques based on convolutional neural networks. Specifically, our generative models leverage different physics knowledge (such as governing equations, observable perception, and physics phenomena) to improve the quality of the synthetic data. To validate the effectiveness of our data augmentation techniques, we apply them to solve a subsurface seismic full-waveform inversion using simulated CO$_2$ leakage data. Our interest is to invert for subsurface velocity models associated with very small CO$_2$ leakage. We validate the performance of our methods using comprehensive numerical tests. Via comparison and analysis, we show that data-driven seismic imaging can be significantly enhanced by using our data augmentation techniques. Particularly, the imaging quality has been improved by 15% in test scenarios of general-sized leakage and 17% in small-sized leakage when using an augmented training set obtained with our techniques.

cs.LG↗

arXiv2021

On the Robustness and Generalization of Deep Learning Driven Full Waveform Inversion

The data-driven approach has been demonstrated as a promising technique to solve complicated scientific problems. Full Waveform Inversion (FWI) is commonly epitomized as an image-to-image translation task, which motivates the use of deep neural networks as an end-to-end solution. Despite being trained with synthetic data, the deep learning-driven FWI is expected to perform well when evaluated with sufficient real-world data. In this paper, we study such properties by asking: how robust are these deep neural networks and how do they generalize? For robustness, we prove the upper bounds of the deviation between the predictions from clean and noisy data. Moreover, we demonstrate an interplay between the noise level and the additional gain of loss. For generalization, we prove a norm-based generalization error upper bound via a stability-generalization framework. Experimental results on seismic FWI datasets corroborate with the theoretical results, shedding light on a better understanding of utilizing Deep Learning for complicated scientific applications.

cs.LG↗

arXiv2021

InversionNet3D: Efficient and Scalable Learning for 3D Full Waveform Inversion

Seismic full-waveform inversion (FWI) techniques aim to find a high-resolution subsurface geophysical model provided with waveform data. Some recent effort in data-driven FWI has shown some encouraging results in obtaining 2D velocity maps. However, due to high computational complexity and large memory consumption, the reconstruction of 3D high-resolution velocity maps via deep networks is still a great challenge. In this paper, we present InversionNet3D, an efficient and scalable encoder-decoder network for 3D FWI. The proposed method employs group convolution in the encoder to establish an effective hierarchy for learning information from multiple sources while cutting down unnecessary parameters and operations at the same time. The introduction of invertible layers further reduces the memory consumption of intermediate features during training and thus enables the development of deeper networks with more layers and higher capacity as required by different application scenarios. Experiments on the 3D Kimberlina dataset demonstrate that InversionNet3D achieves state-of-the-art reconstruction performance with lower computational cost and lower memory footprint compared to the baseline.

cs.LG↗

arXiv2021

Connect the Dots: In Situ 4D Seismic Monitoring of CO2 Storage with Spatio-temporal CNNs

4D seismic imaging has been widely used in CO$_2$ sequestration projects to monitor the fluid flow in the volumetric subsurface region that is not sampled by wells. Ideally, real-time monitoring and near-future forecasting would provide site operators with great insights to understand the dynamics of the subsurface reservoir and assess any potential risks. However, due to obstacles such as high deployment cost, availability of acquisition equipment, exclusion zones around surface structures, only very sparse seismic imaging data can be obtained during monitoring. That leads to an unavoidable and growing knowledge gap over time. The operator needs to understand the fluid flow throughout the project lifetime and the seismic data are only available at a limited number of times. This is insufficient for understanding the reservoir behavior. To overcome those challenges, we have developed spatio-temporal neural-network-based models that can produce high-fidelity interpolated or extrapolated images effectively and efficiently. Specifically, our models are built on an autoencoder, and incorporate the long short-term memory (LSTM) structure with a new loss function regularized by optical flow. We validate the performance of our models using real 4D post-stack seismic imaging data acquired at the Sleipner CO$_2$ sequestration field. We employ two different strategies in evaluating our models. Numerically, we compare our models with different baseline approaches using classic pixel-based metrics. We also conduct a blind survey and collect a total of 20 responses from domain experts to evaluate the quality of data generated by our models. Via both numerical and expert evaluation, we conclude that our models can produce high-quality 2D/3D seismic imaging data at a reasonable cost, offering the possibility of real-time monitoring or even near-future forecasting of the CO$_2$ storage reservoir.

physics.geo-ph↗

arXiv2021

PreMevE Update: Forecasting Ultra-relativistic Electrons inside Earth's Outer Radiation Belt

Energetic electrons inside Earth's outer Van Allen belt pose a major radiation threat to space-borne electronics that often play vital roles in our modern society. Ultra-relativistic electrons with energies greater than or equal to two Megaelectron-volt (MeV) are of particular interest due to their high penetrating ability, and thus forecasting these >=2 MeV electron levels has significant meaning to all space sectors. Here we update the latest development of the predictive model for MeV electrons inside the Earth's outer radiation belt. The new version, called PreMevE-2E, focuses on forecasting ultra-relativistic electron flux distributions across the outer radiation belt, with no need of in-situ measurements except for at the geosynchronous (GEO) orbit. Model inputs include precipitating electrons observed in low-Earth-orbits by NOAA satellites, upstream solar wind conditions (speeds and densities) from solar wind monitors, as well as ultra-relativistic electrons measured by one Los Alamos GEO satellite. We evaluated a total of 32 supervised machine learning models that fall into four different classes of linear and neural network architectures, and also successfully tested ensemble forecasting by using groups of top-performing models. All models are individually trained, validated, and tested by in-situ electron data from NASA's Van Allen Probes mission. It is shown that the final ensemble model generally outperforms individual models overs L-shells, and this PreMevE-2E model provides reliable and high-fidelity 25-hr (~1-day) and 50-hr (~2-day) forecasts with high mean performance efficiency values. Our results also suggest this new model is dominated by non-linear components at low L-shells (< ~4) for ultra-relativistic electrons, which is different from the dominance of linear components at all L-shells for 1 MeV electrons as previously discovered.

physics.space-ph↗

arXiv2020

Physics-Consistent Data-driven Waveform Inversion with Adaptive Data Augmentation

Seismic full-waveform inversion (FWI) is a nonlinear computational imaging technique that can provide detailed estimates of subsurface geophysical properties. Solving the FWI problem can be challenging due to its ill-posedness and high computational cost. In this work, we develop a new hybrid computational approach to solve FWI that combines physics-based models with data-driven methodologies. In particular, we develop a data augmentation strategy that can not only improve the representativity of the training set but also incorporate important governing physics into the training process and therefore improve the inversion accuracy. To validate the performance, we apply our method to synthetic elastic seismic waveform data generated from a subsurface geologic model built on a carbon sequestration site at Kimberlina, California. We compare our physics-consistent data-driven inversion method to both purely physics-based and purely data-driven approaches and observe that our method yields higher accuracy and greater generalization ability.

cs.LG↗

arXiv2020

SeismoGen: Seismic Waveform Synthesis Using Generative Adversarial Networks

Detecting earthquake events from seismic time series has proved itself a challenging task. Manual detection can be expensive and tedious due to the intensive labor and large scale data set. In recent years, automatic detection methods based on machine learning have been developed to improve accuracy and efficiency. However, the accuracy of those methods relies on a sufficient amount of high-quality training data, which itself can be expensive to obtain due to the requirement of domain knowledge and subject matter expertise. This paper is to resolve this dilemma by answering two questions: (1) provided with a limited number of reliable labels, can we use them to generate more synthetic labels; (2) Can we use those synthetic labels to improve the detectability? Among all the existing generative models, the generative adversarial network (GAN) shows its supreme capability in generating high-quality synthetic samples in multiple domains. We designed our model based on GAN. In particular, we studied several different network structures. By comparing the generated results, our GAN-based generative model yields the highest quality. We further combine the dataset with synthetic samples generated by our generative model and show that the detectability of our earthquake classification model is significantly improved than the one trained without augmenting the training set.

cs.LG↗

arXiv2019

Forecasting Megaelectron-Volt Electrons inside Earth's Outer Radiation Belt: PreMevE 2.0 Based on Supervised Machine Learning Algorithms

Here we present the recent progress in upgrading a predictive model for Megaelectron-Volt (MeV) electrons inside the Earth's outer Van Allen belt. This updated model, called PreMevE 2.0, is demonstrated to make much improved forecasts, particularly at outer Lshells, by including upstream solar wind speeds to the model's input parameter list. Furthermore, based on several kinds of linear and artificial machine learning algorithms, a list of models were constructed, trained, validated and tested with 42-month MeV electron observations from Van Allen Probes. Out-of-sample test results from these models show that, with optimized model hyperparameters and input parameter combinations, the top performer from each category of models has the similar capability of making reliable 1-day (2-day) forecasts with Lshell-averaged performance efficiency values ~ 0.87 (~0.82). Interestingly, the linear regression model is often the most successful one when compared to other models, which indicates the relationship between 1 MeV electron dynamics and precipitating electrons is dominated by linear components. It is also shown that PreMevE 2.0 can reasonably predict the onsets of MeV electron events in 2-day forecasts. This improved PreMevE model is driven by observations from longstanding space infrastructure (a NOAA satellite in low-Earth-orbit, the solar wind monitor at the L1 point, and one LANL satellite in geosynchronous orbit) to make high-fidelity forecasts for MeV electrons, and thus can be an invaluable space weather forecasting tool for the future.

physics.space-ph↗

arXiv2019

Data-driven Seismic Waveform Inversion: A Study on the Robustness and Generalization

Acoustic- and elastic-waveform inversion is an important and widely used method to reconstruct subsurface velocity image. Waveform inversion is a typical non-linear and ill-posed inverse problem. Existing physics-driven computational methods for solving waveform inversion suffer from the cycle skipping and local minima issues, and not to mention solving waveform inversion is computationally expensive. In recent years, data-driven methods become a promising way to solve the waveform inversion problem. However, most deep learning frameworks suffer from generalization and over-fitting issue. In this paper, we developed a real-time data-driven technique and we call it VelocityGAN, to accurately reconstruct subsurface velocities. Our VelocityGAN is built on a generative adversarial network (GAN) and trained end-to-end to learn a mapping function from the raw seismic waveform data to the velocity image. Different from other encoder-decoder based data-driven seismic waveform inversion approaches, our VelocityGAN learns regularization from data and further impose the regularization to the generator so that inversion accuracy is improved. We further develop a transfer learning strategy based on VelocityGAN to alleviate the generalization issue. A series of experiments are conducted on the synthetic seismic reflection data to evaluate the effectiveness, efficiency, and generalization of VelocityGAN. We not only compare it with existing physics-driven approaches and data-driven frameworks but also conduct several transfer learning experiments. The experiment results show that VelocityGAN achieves state-of-the-art performance among the baselines and can improve the generalization results to some extent.

eess.SP↗

arXiv2019

Earthquake Detection in 1-D Time Series Data with Feature Selection and Dictionary Learning

Earthquakes can be detected by matching spatial patterns or phase properties from 1-D seismic waves. Current earthquake detection methods, such as waveform correlation and template matching, have difficulty detecting anomalous earthquakes that are not similar to other earthquakes. In recent years, machine-learning techniques for earthquake detection have been emerging as a new active research direction. In this paper, we develop a novel earthquake detection method based on dictionary learning. Our detection method first generates rich features via signal processing and statistical methods and further employs feature selection techniques to choose features that carry the most significant information. Based on these selected features, we build a dictionary for classifying earthquake events from non-earthquake events. To evaluate the performance of our dictionary-based detection methods, we test our method on a labquake dataset from Penn State University, which contains 3,357,566 time series data points with a 400 MHz sampling rate. 1,000 earthquake events are manually labeled in total, and the length of these earthquake events varies from 74 to 7151 data points. Through comparison to other detection methods, we show that our feature selection and dictionary learning incorporated earthquake detection method achieves an 80.1% prediction accuracy and outperforms the baseline methods in earthquake detection, including Template Matching (TM) and Support Vector Machine (SVM).

physics.geo-ph↗

arXiv2019

A Data-Driven CO2 Leakage Detection Using Seismic Data and Spatial-Temporal Densely Connected Convolutional Neural Networks

In carbon capture and sequestration, developing effective monitoring methods is needed to detect and respond to CO2 leakage. CO2 leakage detection methods rely on geophysical observations and monitoring sensor network. However, traditional methods usually require the development of site-specific physical models and expert interpretation, and the effectiveness of these methods can be limited to different application locations, operational scenarios, and conditions. In this paper, we developed a novel data-driven leakage detection method based on densely connected convolutional neural networks. Our method differs from conventional leakage monitoring methods by directly learning a mapping relationship between seismic data and the CO2 leakage mass. To account for the spatial and temporal characteristics of seismic data, our novel networks architecture combines 1D and 2D convolutional neural networks. To overcome the computational expense of solving optimization problems, we apply a densely-connecting strategy in our network architecture that reduces the number of network parameters. Based on the features generated by our convolutional neural networks, we further incorporate a long short-term memory network to utilize time-sequential information, which further improves the detection accuracy. Finally, we employ our detection method to synthetic seismic datasets generated based on flow simulations of a hypothetical CO2 storage scenario with injection into a partially compartmentalized sandstone storage reservoir. To evaluate method performance, we conducted multiple experiments including a random leakage test, a sequential test, and a robustness test. Numerical results show that our CO2 leakage detection method successfully detects the leakage and accurately predicts the leakage mass, suggesting that it has the potential for application in monitoring of real CO2 storage sites.

physics.geo-ph↗

arXiv2019

InversionNet: A Real-Time and Accurate Full Waveform Inversion with CNNs and continuous CRFs

Full-waveform inversion problems are usually formulated as optimization problems, where the forward-wave propagation operator $f$ maps the subsurface velocity structures to seismic signals. The existing computational methods for solving full-waveform inversion are not only computationally expensive, but also yields low-resolution results because of the ill-posedness and cycle skipping issues of full-waveform inversion. To resolve those issues, we employ machine-learning techniques to solve the full-waveform inversion. Specifically, we focus on applying the convolutional neural network~(CNN) to directly derive the inversion operator $f^{-1}$ so that the velocity structure can be obtained without knowing the forward operator $f$. We build a convolutional neural network with an encoder-decoder structure to model the correspondence from seismic data to subsurface velocity structures. Furthermore, we employ the conditional random field~(CRF) on top of the CNN to generate structural predictions by modeling the interactions between different locations on the velocity model. Our numerical examples using synthetic seismic reflection data show that the propose CNN-CRF model significantly improve the accuracy of the velocity inversion while the computational time is reduced.

eess.SP↗

arXiv2018

Deep Learning Approach in Automatic Iceberg - Ship Detection with SAR Remote Sensing Data

Deep Learning is gaining traction with geophysics community to understand subsurface structures, such as fault detection or salt body in seismic data. This study describes using deep learning method for iceberg or ship recognition with synthetic aperture radar (SAR) data. Drifting icebergs pose a potential threat to activities offshore around the Arctic, including for both ship navigation and oil rigs. Advancement of satellite imagery using weather-independent cross-polarized radar has enabled us to monitor and delineate icebergs and ships, however a human component is needed to classify the images. Here we present Transfer Learning, a convolutional neural network (CNN) designed to work with a limited training data and features, while demonstrating its effectiveness in this problem. Key aspect of the approach is data augmentation and stacking of multiple outputs, resulted in a significant boost in accuracy (logarithmic score of 0.1463). This algorithm has been tested through participation at the Statoil/C-Core Kaggle competition.

cs.LG↗

arXiv2018

Spatial-Temporal Densely Connected Convolutional Networks: An Application to CO2 Leakage Detection

In carbon capture and sequestration, building an effective monitoring method is a crucial step to detect and respond to CO2 leakage. CO2 leakage detection methods rely on geophysical observations and monitoring sensor network. However, traditional methods usually require physical models to be interpreted by experts, and the accuracy of these methods will be restricted by different application conditions. In this paper, we develop a novel data-driven detection method based on densely connected convolutional networks. Our detection method learns a mapping relation between seismic data and the CO2 leakage mass. To account for the spatial and temporal characteristics of seismic data, we design a novel network architecture by combining 1-D and 2-D convolutional neural networks together. To overcome the expensive computational cost, we further apply a densely-connecting policy to our network architecture to reduce the network parameters. We employ our detection method to synthetic seismic datasets using Kimberlina model. The numerical results show that our leakage detection method accurately detects the leakage mass. Therefore, our novel CO2 leakage detection method has great potential for monitoring CO2 storage.

physics.geo-ph↗

arXiv2018

Proceedings of the Workshop on Data Mining for Geophysics and Geology

Modern geosciences have to deal with large quantities and a wide variety of data, including 2-D, 3-D and 4-D seismic surveys, well logs generated by sensors, detailed lithological records, satellite images and meteorological records. These data serve important industries, such as the exploration of mineral deposits and the production of energy (Oil and Gas, Geothermal, Wind, Hydroelectric), are important in the study of the earth crust to reduce the impact of earthquakes, in land use planning, and have a fundamental role in sustainability. The volume of raw data being stored by different earth science archives today makes it impossible to rely on manual examination by scientists. The data volumes resultant of different sources, from terrestrial or aerial to satellite surveys, will reach a terabyte per day by the time all the planned satellites are flown. In particular, the oil industry has been using large quantities of data for quite a long time. Although there are published works in this area since the 70s, these days, the ubiquity of computing and sensor devices enables the collection of higher resolution data in real time, giving a new life to a mature industrial field. Understanding and finding value in this data has an impact on the efficiency of the operations in the oil and gas production chain. Efficiency gains are particularly important since the steep fall in oil prices in 2014, and represent an important opportunity for data mining and data science.

physics.geo-ph↗