arXiv subjects

Maarten V. de Hoop

Publications and source records attributed to Maarten V. de Hoop.

At least 55 records · Page 3Linked to original sources

arXiv2024

SeisLM: a Foundation Model for Seismic Waveforms

We introduce the Seismic Language Model (SeisLM), a foundational model designed to analyze seismic waveforms -- signals generated by Earth's vibrations such as the ones originating from earthquakes. SeisLM is pretrained on a large collection of open-source seismic datasets using a self-supervised contrastive loss, akin to BERT in language modeling. This approach allows the model to learn general seismic waveform patterns from unlabeled data without being tied to specific downstream tasks. When fine-tuned, SeisLM excels in seismological tasks like event detection, phase-picking, onset time regression, and foreshock-aftershock classification. The code has been made publicly available on https://github.com/liutianlin0121/seisLM.

physics.geo-ph↗

arXiv2024

Transformers are Universal In-context Learners

Transformers are deep architectures that define "in-context mappings" which enable predicting new tokens based on a given set of tokens (such as a prompt in NLP applications or a set of patches for a vision transformer). In this work, we study in particular the ability of these architectures to handle an arbitrarily large number of context tokens. To mathematically, uniformly address their expressivity, we consider the case that the mappings are conditioned on a context represented by a probability distribution of tokens which becomes discrete for a finite number of these. The relevant notion of smoothness then corresponds to continuity in terms of the Wasserstein distance between these contexts. We demonstrate that deep transformers are universal and can approximate continuous in-context mappings to arbitrary precision, uniformly over compact token domains. A key aspect of our results, compared to existing findings, is that for a fixed precision, a single transformer can operate on an arbitrary (even infinite) number of tokens. Additionally, it operates with a fixed embedding dimension of tokens (this dimension does not increase with precision) and a fixed number of heads (proportional to the dimension). The use of MLPs between multi-head attention layers is also explicitly controlled. We consider both unmasked attentions (as used for the vision transformer) and masked causal attentions (as used for NLP and time series applications). We tackle the causal setting leveraging a space-time lifting to analyze causal attention as a mapping over probability distributions of tokens.

cs.CL↗

arXiv2024

Taming Score-Based Diffusion Priors for Infinite-Dimensional Nonlinear Inverse Problems

This work introduces a sampling method capable of solving Bayesian inverse problems in function space. It does not assume the log-concavity of the likelihood, meaning that it is compatible with nonlinear inverse problems. The method leverages the recently defined infinite-dimensional score-based diffusion models as a learning-based prior, while enabling provable posterior sampling through a Langevin-type MCMC algorithm defined on function spaces. A novel convergence analysis is conducted, inspired by the fixed-point methods established for traditional regularization-by-denoising algorithms and compatible with weighted annealing. The obtained convergence bound explicitly depends on the approximation error of the score; a well-approximated score is essential to obtain a well-approximated posterior. Stylized and PDE-based examples are provided, demonstrating the validity of our convergence analysis. We conclude by presenting a discussion of the method's challenges related to learning the score and computational complexity.

stat.ML↗

arXiv2024

Stable recovery of coefficients in an inverse fault friction problem

We consider the inverse fault friction problem of determining the friction coefficient in the Tresca friction model, which can be formulated as an inverse problem for differential inequalities. We show that the measurements of elastic waves during a rupture uniquely determine the friction coefficient at the rupture surface with explicit stability estimates.

math.AP↗

arXiv2024

Geometric inverse problems on gas giants

On gas giant planets the speed of sound is isotropic and goes to zero at the surface. Geometrically, this corresponds to a Riemannian manifold whose metric tensor has a conformal blow-up near the boundary. The blow-up is tamer than in asymptotically hyperbolic geometry: the boundary is at a finite distance. We study the differential geometry of such manifolds, especially the asymptotic behavior of geodesics near the boundary. We relate the geometry to the propagation of singularities of a hydrodynamic PDE and we give the basic properties of the Laplace--Beltrami operator. We solve two inverse problems, showing that the interior structure of a gas giant is uniquely determined by different types of boundary data.

math.DG↗

arXiv2023

Spherically symmetric terrestrial planets with discontinuities are spectrally rigid

We establish spectral rigidity for spherically symmetric manifolds with boundary and interior interfaces determined by discontinuities in the metric under certain conditions. Rather than a single metric, we allow two distinct metrics in between the interfaces enabling the consideration of two wave types, like P- and S-polarized waves in isotropic elastic solids. Terrestrial planets in our solar system are approximately spherically symmetric and support toroidal and spheroidal modes. Discontinuities typically correspond with phase transitions in their interiors. Our rigidity result applies to such planets as we ensure that our conditions are satisfied in generally accepted models in the presence of a fluid outer core. The proof is based on a novel trace formula. We also prove that the length spectrum of the Euclidean ball is simple.

math.AP↗

arXiv2023

Conditional score-based diffusion models for Bayesian inference in infinite dimensions

Since their initial introduction, score-based diffusion models (SDMs) have been successfully applied to solve a variety of linear inverse problems in finite-dimensional vector spaces due to their ability to efficiently approximate the posterior distribution. However, using SDMs for inverse problems in infinite-dimensional function spaces has only been addressed recently, primarily through methods that learn the unconditional score. While this approach is advantageous for some inverse problems, it is mostly heuristic and involves numerous computationally costly forward operator evaluations during posterior sampling. To address these limitations, we propose a theoretically grounded method for sampling from the posterior of infinite-dimensional Bayesian linear inverse problems based on amortized conditional SDMs. In particular, we prove that one of the most successful approaches for estimating the conditional score in finite dimensions - the conditional denoising estimator - can also be applied in infinite dimensions. A significant part of our analysis is dedicated to demonstrating that extending infinite-dimensional SDMs to the conditional setting requires careful consideration, as the conditional score typically blows up for small times, contrarily to the unconditional score. We conclude by presenting stylized and large-scale numerical examples that validate our approach, offer additional insights, and demonstrate that our method enables large-scale, discretization-invariant Bayesian inference.

stat.ML↗

arXiv2023

Implicit Neural Representations and the Algebra of Complex Wavelets

Implicit neural representations (INRs) have arisen as useful methods for representing signals on Euclidean domains. By parameterizing an image as a multilayer perceptron (MLP) on Euclidean space, INRs effectively represent signals in a way that couples spatial and spectral features of the signal that is not obvious in the usual discrete representation, paving the way for continuous signal processing and machine learning approaches that were not previously possible. Although INRs using sinusoidal activation functions have been studied in terms of Fourier theory, recent works have shown the advantage of using wavelets instead of sinusoids as activation functions, due to their ability to simultaneously localize in both frequency and space. In this work, we approach such INRs and demonstrate how they resolve high-frequency features of signals from coarse approximations done in the first layer of the MLP. This leads to multiple prescriptions for the design of INR architectures, including the use of complex wavelets, decoupling of low and band-pass approximations, and initialization schemes based on the singularities of the desired signal.

eess.SP↗

arXiv2023

Inverse problem for Love waves in a layered, elastic half-space

In this paper we study Love waves in a layered, elastic half-space. We first address the direct problem and we characterize the existence of Love waves through the dispersion relation. We then address the inverse problem and we show how to recover the parameters of the elastic medium from the empirical knowledge of the frequency--wavenumber couples of the Love waves.

math.AP↗

arXiv2023

Resolvent Estimates for Viscoelastic Systems of Extended Maxwell Type and their Applications

In the theory of viscoelasticity, an important class of models admits a representation in terms of springs and dashpots. Widely used members of this class are the Maxwell model and its extended version. This paper concerns resolvent estimates for the system of equations for the anisotropic, extended Maxwell model, abbreviated as the EMM, and its marginal realization which includes an inertia term; special attention is paid to the introduction of augmented variables. This leads to the augmented system that will also be referred to as the "original" system. A reduced system is then formed which encodes essentially the EMM; it is a closed system with respect to the particle velocity and the difference between the elastic and viscous strains. Based on resolvent estimates, it is shown that the original and reduced systems generate $C_0$-groups and the reduced system generates a $C_0$-semigroup of contraction. Naturally, the EMM can be written in integrodifferential form leading explicitly to relaxation and a viscoelastic integro-differential system. However, there is a difference between the original and integrodifferential systems, in general, with consequences for whether their solutions generate semigroups or not. Finally, an energy estimate is obtained for the reduced system, and it is proven that its solutions decay exponentially as time tends to infinity. The limiting amplitude principle follows readily from these two results.

math.AP↗

arXiv2023

Coupling of flow, contact mechanics and friction, generating waves in a fractured porous medium

We present a mixed dimensional model for a fractured poro-elasic medium including contact mechanics. The fracture is a lower dimensional surface embedded in a bulk poro-elastic matrix. The flow equation on the fracture is a Darcy type model that follows the cubic law for permeability. The bulk poro-elasticity is governed by fully dynamic Biot equations. The resulting model is a mixed dimensional type where the fracture flow on a surface is coupled to a bulk flow and geomechanics model. The particularity of the work here is in considering fully dynamic Biot equation, that is, including an inertia term, and the contact mechanics including friction for the fracture surface. We prove the well-posedness of the continuous model.

math.AP↗

arXiv2023

An Approximation Theory for Metric Space-Valued Functions With A View Towards Deep Learning

Motivated by the developing mathematics of deep learning, we build universal functions approximators of continuous maps between arbitrary Polish metric spaces $\mathcal{X}$ and $\mathcal{Y}$ using elementary functions between Euclidean spaces as building blocks. Earlier results assume that the target space $\mathcal{Y}$ is a topological vector space. We overcome this limitation by ``randomization'': our approximators output discrete probability measures over $\mathcal{Y}$. When $\mathcal{X}$ and $\mathcal{Y}$ are Polish without additional structure, we prove very general qualitative guarantees; when they have suitable combinatorial structure, we prove quantitative guarantees for Hölder-like maps, including maps between finite graphs, solution operators to rough differential equations between certain Carnot groups, and continuous non-linear operators between Banach spaces arising in inverse problems. In particular, we show that the required number of Dirac measures is determined by the combinatorial structure of $\mathcal{X}$ and $\mathcal{Y}$. For barycentric $\mathcal{Y}$, including Banach spaces, $\mathbb{R}$-trees, Hadamard manifolds, or Wasserstein spaces on Polish metric spaces, our approximators reduce to $\mathcal{Y}$-valued functions. When the Euclidean approximators are neural networks, our constructions generalize transformer networks, providing a new probabilistic viewpoint of geometric deep learning.

cs.LG↗

arXiv2023

Out-of-distributional risk bounds for neural operators with applications to the Helmholtz equation

Despite their remarkable success in approximating a wide range of operators defined by PDEs, existing neural operators (NOs) do not necessarily perform well for all physics problems. We focus here on high-frequency waves to highlight possible shortcomings. To resolve these, we propose a subfamily of NOs enabling an enhanced empirical approximation of the nonlinear operator mapping wave speed to solution, or boundary values for the Helmholtz equation on a bounded domain. The latter operator is commonly referred to as the ''forward'' operator in the study of inverse problems. Our methodology draws inspiration from transformers and techniques such as stochastic depth. Our experiments reveal certain surprises in the generalization and the relevance of introducing stochastic depth. Our NOs show superior performance as compared with standard NOs, not only for testing within the training distribution but also for out-of-distribution scenarios. To delve into this observation, we offer an in-depth analysis of the Rademacher complexity associated with our modified models and prove an upper bound tied to their stochastic depth that existing NOs do not satisfy. Furthermore, we obtain a novel out-of-distribution risk bound tailored to Gaussian measures on Banach spaces, again relating stochastic depth with the bound. We conclude by proposing a hypernetwork version of the subfamily of NOs as a surrogate model for the mentioned forward operator.

cs.LG↗

arXiv2023

Globally injective and bijective neural operators

Recently there has been great interest in operator learning, where networks learn operators between function spaces from an essentially infinite-dimensional perspective. In this work we present results for when the operators learned by these networks are injective and surjective. As a warmup, we combine prior work in both the finite-dimensional ReLU and operator learning setting by giving sharp conditions under which ReLU layers with linear neural operators are injective. We then consider the case the case when the activation function is pointwise bijective and obtain sufficient conditions for the layer to be injective. We remark that this question, while trivial in the finite-rank case, is subtler in the infinite-rank case and is proved using tools from Fredholm theory. Next, we prove that our supplied injective neural operators are universal approximators and that their implementation, with finite-rank neural networks, are still injective. This ensures that injectivity is not `lost' in the transcription from analytical operators to their finite-rank implementation with networks. Finally, we conclude with an increase in abstraction and consider general conditions when subnetworks, which may be many layers deep, are injective and surjective and provide an exact inversion from a `linearization.' This section uses general arguments from Fredholm theory and Leray-Schauder degree theory for non-linear integral equations to analyze the mapping properties of neural operators in function spaces. These results apply to subnetworks formed from the layers considered in this work, under natural conditions. We believe that our work has applications in Bayesian UQ where injectivity enables likelihood estimation and in inverse problems where surjectivity and injectivity corresponds to existence and uniqueness, respectively.

cs.LG↗

arXiv2023

Unearthing InSights into Mars: Unsupervised Source Separation with Limited Data

Source separation involves the ill-posed problem of retrieving a set of source signals that have been observed through a mixing operator. Solving this problem requires prior knowledge, which is commonly incorporated by imposing regularity conditions on the source signals, or implicitly learned through supervised or unsupervised methods from existing data. While data-driven methods have shown great promise in source separation, they often require large amounts of data, which rarely exists in planetary space missions. To address this challenge, we propose an unsupervised source separation scheme for domains with limited data access that involves solving an optimization problem in the wavelet scattering covariance representation space$\unicode{x2014}$an interpretable, low-dimensional representation of stationary processes. We present a real-data example in which we remove transient, thermally-induced microtilts$\unicode{x2014}$known as glitches$\unicode{x2014}$from data recorded by a seismometer during NASA's InSight mission on Mars. Thanks to the wavelet scattering covariances' ability to capture non-Gaussian properties of stochastic processes, we are able to separate glitches using only a few glitch-free data snippets.

cs.LG↗

arXiv2023

Early-warning inverse source problem for the elasto-gravitational equations

Through coupled physics, we study an early-warning inverse source problem for the elasto-gravitational equations. It consists of a mixed hyperbolic-elliptic system of partial differential equations describing elastic wave displacement and gravity perturbations produced by a source in a homogeneous bounded medium. Within the Cowling approximation, we prove uniqueness and Lipschitz stability for the inverse problem of recovering the moment tensor and the location of the source from early-time measurements of the changes of the gravitational field. The setup studied in this paper is motivated by gravity-based earthquake early warning systems, which are gaining much attention recently.

math.AP↗

arXiv2023

Quantitative unique continuation for the elasticity system with application to the kinematic inverse rupture problem

We obtain explicit estimates on the stability of the unique continuation for a linear system of hyperbolic equations. In particular our result applies to the elasticity system and also the Maxwell system. As an application, we study the kinematic inverse rupture problem of determining the jump in displacement and the friction force at the rupture surface, and we obtain new features on the stable unique continuation up to the rupture surface.

math.AP↗

arXiv2022

Analysis of leaky modes or wavenumber resonances for the Rayleigh system in a half space

We present a comprehensive analysis of wavenumber resonances or leaky modes associated with the Rayleigh operator in a half space containing a heterogeneous slab, being motivated by seismology. To this end, we introduce Jost solutions on an appropriate Riemann surface, a boundary matrix and a reflection matrix in analogy to the studies of scattering resonances associated with the Schrödinger operator. We analyze their analytic properties and characterize the distribution of these wavenumber resonances. Furthermore, we show that the resonances appear as poles of the meromorphic continuation of the resolvent to the nonphysical sheets of the mentioned Riemann surface as expected.

math.SP↗