arXiv subjects

Kody J. H. Law

Publications and source records attributed to Kody J. H. Law.

At least 37 records · Page 2Linked to original sources

arXiv2020

Reconstruction of effective potential from statistical analysis of dynamic trajectories

The broad incorporation of microscopic methods is yielding a wealth of information on atomic and mesoscale dynamics of individual atoms, molecules, and particles on surfaces and in open volumes. Analysis of such data necessitates statistical frameworks to convert observed dynamic behaviors to effective properties of materials. Here we develop a method for stochastic reconstruction of effective acting potentials from observed trajectories. Using the Silicon vacancy defect in graphene as a model, we develop a statistical framework to reconstruct the free energy landscape from calculated atomic displacements.

cond-mat.mes-hall↗

arXiv2020

A practical and efficient approach for Bayesian quantum state estimation

Bayesian inference is a powerful paradigm for quantum state tomography, treating uncertainty in meaningful and informative ways. Yet the numerical challenges associated with sampling from complex probability distributions hampers Bayesian tomography in practical settings. In this Article, we introduce an improved, self-contained approach for Bayesian quantum state estimation. Leveraging advances in machine learning and statistics, our formulation relies on highly efficient preconditioned Crank--Nicolson sampling and a pseudo-likelihood. We theoretically analyze the computational cost, and provide explicit examples of inference for both actual and simulated datasets, illustrating improved performance with respect to existing approaches.

quant-ph↗

arXiv2019

Cluster, Classify, Regress: A General Method For Learning Discountinous Functions

This paper presents a method for solving the supervised learning problem in which the output is highly nonlinear and discontinuous. It is proposed to solve this problem in three stages: (i) cluster the pairs of input-output data points, resulting in a label for each point; (ii) classify the data, where the corresponding label is the output; and finally (iii) perform one separate regression for each class, where the training data corresponds to the subset of the original input-output pairs which have that label according to the classifier. It has not yet been proposed to combine these 3 fundamental building blocks of machine learning in this simple and powerful fashion. This can be viewed as a form of deep learning, where any of the intermediate layers can itself be deep. The utility and robustness of the methodology is illustrated on some toy problems, including one example problem arising from simulation of plasma fusion in a tokamak.

cs.LG↗

arXiv2019

Estimation and uncertainty quantification for the output from quantum simulators

The problem of estimating certain distributions over $\{0,1\}^d$ is considered here. The distribution represents a quantum system of $d$ qubits, where there are non-trivial dependencies between the qubits. A maximum entropy approach is adopted to reconstruct the distribution from exact moments or observed empirical moments. The Robbins Monro algorithm is used to solve the intractable maximum entropy problem, by constructing an unbiased estimator of the un-normalized target with a sequential Monte Carlo sampler at each iteration. In the case of empirical moments, this coincides with a maximum likelihood estimator. A Bayesian formulation is also considered in order to quantify posterior uncertainty. Several approaches are proposed in order to tackle this challenging problem, based on recently developed methodologies. In particular, unbiased estimators of the gradient of the log posterior are constructed and used within a provably convergent Langevin-based Markov chain Monte Carlo method. The methods are illustrated on classically simulated output from quantum simulators.

stat.CO↗

arXiv2018

Bayesian Point Set Registration

Point set registration involves identifying a smooth invertible transformation between corresponding points in two point sets, one of which may be smaller than the other and possibly corrupted by observation noise. This problem is traditionally decomposed into two separate optimization problems: (i) assignment or correspondence, and (ii) identification of the optimal transformation between the ordered point sets. In this work, we propose an approach solving both problems simultaneously. In particular, a coherent Bayesian formulation of the problem results in a marginal posterior distribution on the transformation, which is explored within a Markov chain Monte Carlo scheme. Motivated by Atomic Probe Tomography (APT), in the context of structure inference for high entropy alloys (HEA), we focus on the registration of noisy sparse observations of rigid transformations of a known reference configuration.Lastly, we test our method on synthetic data sets.

stat.AP↗

arXiv2018

Multilevel Particle Filters for Lévy-driven stochastic differential equations

We develop algorithms for computing expectations of the laws of models associated to stochastic differential equations (SDEs) driven by pure Lévy processes. We consider filtering such processes and well as pricing of path dependent options. We propose a multilevel particle filter (MLPF) to address the computational issues involved in solving these continuum problems. We show via numerical simulations and theoretical results that under suitable assumptions of the discretization of the underlying driving Lévy proccess, our proposed method achieves optimal convergence rates. The cost to obtain MSE $O(ε^2)$ scales like $O(ε^{-2})$ for our method, as compared with the standard particle filter $O(ε^{-3})$.

stat.CO↗

arXiv2017

Bayesian Static Parameter Estimation for Partially Observed Diffusions via Multilevel Monte Carlo

In this article we consider static Bayesian parameter estimation for partially observed diffusions that are discretely observed. We work under the assumption that one must resort to discretizing the underlying diffusion process, for instance using the Euler-Maruyama method. Given this assumption, we show how one can use Markov chain Monte Carlo (MCMC) and particularly particle MCMC [Andrieu, C., Doucet, A. and Holenstein, R. (2010). Particle Markov chain Monte Carlo methods (with discussion). J. R. Statist. Soc. Ser. B, 72, 269--342] to implement a new approximation of the multilevel (ML) Monte Carlo (MC) collapsing sum identity. Our approach comprises constructing an approximate coupling of the posterior density of the joint distribution over parameter and hidden variables at two different discretization levels and then correcting by an importance sampling method. The variance of the weights are independent of the length of the observed data set. The utility of such a method is that, for a prescribed level of mean square error, the cost of this MLMC method is provably less than i.i.d. sampling from the posterior associated to the most precise discretization. However the method here comprises using only known and efficient simulation methodologies. The theoretical results are illustrated by inference of the parameters of two prototypical processes given noisy partial observations of the process: the first is an Ornstein Uhlenbeck process and the second is a more general Langevin equation.

stat.CO↗

arXiv2016

Deterministic Mean-field Ensemble Kalman Filtering

The proof of convergence of the standard ensemble Kalman filter (EnKF) from Legland etal. (2011) is extended to non-Gaussian state space models. A density-based deterministic approximation of the mean-field limit EnKF (DMFEnKF) is proposed, consisting of a PDE solver and a quadrature rule. Given a certain minimal order of convergence $κ$ between the two, this extends to the deterministic filter approximation, which is therefore asymptotically superior to standard EnKF when the dimension $d<2κ$. The fidelity of approximation of the true distribution is also established using an extension of total variation metric to random measures. This is limited by a Gaussian bias term arising from non-linearity/non-Gaussianity of the model, which exists for both DMFEnKF and standard EnKF. Numerical results support and extend the theory.

math.PR↗

arXiv2016

Multilevel ensemble Kalman filtering

This work embeds a multilevel Monte Carlo sampling strategy into the Monte Carlo step of the ensemble Kalman filter (EnKF) in the setting of finite dimensional signal evolution and noisy discrete-time observations. The signal dynamics is assumed to be governed by a stochastic differential equation (SDE), and a hierarchy of time grids is introduced for multilevel numerical integration of that SDE. The resulting multilevel EnKF is proved to asymptotically outperform EnKF in terms of computational cost versus approximation accuracy. The theoretical results are illustrated numerically.

math.NA↗

arXiv2015

Dimension-independent likelihood-informed MCMC

Many Bayesian inference problems require exploring the posterior distribution of high-dimensional parameters that represent the discretization of an underlying function. This work introduces a family of Markov chain Monte Carlo (MCMC) samplers that can adapt to the particular structure of a posterior distribution over functions. Two distinct lines of research intersect in the methods developed here. First, we introduce a general class of operator-weighted proposal distributions that are well defined on function space, such that the performance of the resulting MCMC samplers is independent of the discretization of the function. Second, by exploiting local Hessian information and any associated low-dimensional structure in the change from prior to posterior distributions, we develop an inhomogeneous discretization scheme for the Langevin stochastic differential equation that yields operator-weighted proposals adapted to the non-Gaussian structure of the posterior. The resulting dimension-independent, likelihood-informed (DILI) MCMC samplers may be useful for a large class of high-dimensional problems where the target probability measure has a density with respect to a Gaussian reference measure. Two nonlinear inverse problems are used to demonstrate the efficiency of these DILI samplers: an elliptic PDE coefficient inverse problem and path reconstruction in a conditioned diffusion.

stat.CO↗

arXiv2015

Multilevel particle filter

In this paper the filtering of partially observed diffusions, with discrete-time observations, is considered. It is assumed that only biased approximations of the diffusion can be obtained, for choice of an accuracy parameter indexed by $l$. A multilevel estimator is proposed, consisting of a telescopic sum of increment estimators associated to the successive levels. The work associated to $\mathcal{O}(\varepsilon^2)$ mean-square error between the multilevel estimator and average with respect to the filtering distribution is shown to scale optimally, for example as $\mathcal{O}(\varepsilon^{-2})$ for optimal rates of convergence of the underlying diffusion approximation. The method is illustrated on some toy examples as well as estimation of interest rate based on real S&P 500 stock price data.

stat.CO↗

arXiv2015

Accelerated dimension-independent adaptive Metropolis

This work considers black-box Bayesian inference over high-dimensional parameter spaces. The well-known adaptive Metropolis (AM) algorithm of (Haario etal. 2001) is extended herein to scale asymptotically uniformly with respect to the underlying parameter dimension for Gaussian targets, by respecting the variance of the target. The resulting algorithm, referred to as the dimension-independent adaptive Metropolis (DIAM) algorithm, also shows improved performance with respect to adaptive Metropolis on non-Gaussian targets. This algorithm is further improved, and the possibility of probing high-dimensional targets is enabled, via GPU-accelerated numerical libraries and periodically synchronized concurrent chains (justified a posteriori). Asymptotically in dimension, this GPU implementation exhibits a factor of four improvement versus a competitive CPU-based Intel MKL parallel version alone. Strong scaling to concurrent chains is exhibited, through a combination of longer time per sample batch (weak scaling) and yet fewer necessary samples to convergence. The algorithm performance is illustrated on several Gaussian and non-Gaussian target examples, in which the dimension may be in excess of one thousand.

stat.CO↗

arXiv2014

Determining White Noise Forcing From Eulerian Observations in the Navier Stokes Equation

The Bayesian approach to inverse problems is of paramount importance in quantifying uncertainty about the input to and the state of a system of interest given noisy observations. Herein we consider the forward problem of the forced 2D Navier Stokes equation. The inverse problem is inference of the forcing, and possibly the initial condition, given noisy observations of the velocity field. We place a prior on the forcing which is in the form of a spatially correlated temporally white Gaussian process, and formulate the inverse problem for the posterior distribution. Given appropriate spatial regularity conditions, we show that the solution is a continuous function of the forcing. Hence, for appropriately chosen spatial regularity in the prior, the posterior distribution on the forcing is absolutely continuous with respect to the prior and is hence well-defined. Furthermore, the posterior distribution is a continuous function of the data. We complement this theoretical result with numerical simulation of the posterior distribution.

math.PR↗

arXiv2013

MAP Estimators and Their Consistency in Bayesian Nonparametric Inverse Problems

We consider the inverse problem of estimating an unknown function $u$ from noisy measurements $y$ of a known, possibly nonlinear, map $\mathcal{G}$ applied to $u$. We adopt a Bayesian approach to the problem and work in a setting where the prior measure is specified as a Gaussian random field $μ_0$. We work under a natural set of conditions on the likelihood which imply the existence of a well-posed posterior measure, $μ^y$. Under these conditions we show that the {\em maximum a posteriori} (MAP) estimator is well-defined as the minimiser of an Onsager-Machlup functional defined on the Cameron-Martin space of the prior; thus we link a problem in probability with a problem in the calculus of variations. We then consider the case where the observational noise vanishes and establish a form of Bayesian posterior consistency. We also prove a similar result for the case where the observation of $\mathcal{G}(u)$ can be repeated as many times as desired with independent identically distributed noise. The theory is illustrated with examples from an inverse problem for the Navier-Stokes equation, motivated by problems arising in weather forecasting, and from the theory of conditioned diffusions, motivated by problems arising in molecular dynamics.

math.PR↗

arXiv2013

Proposals which speed-up function-space MCMC

Inverse problems lend themselves naturally to a Bayesian formulation, in which the quantity of interest is a posterior distribution of state and/or parameters given some uncertain observations. For the common case in which the forward operator is smoothing, then the inverse problem is ill-posed. Well-posedness is imposed via regularisation in the form of a prior, which is often Gaussian. Under quite general conditions, it can be shown that the posterior is absolutely continuous with respect to the prior and it may be well-defined on function space in terms of its density with respect to the prior. In this case, by constructing a proposal for which the prior is invariant, one can define Metropolis-Hastings schemes for MCMC which are well-defined on function space, and hence do not degenerate as the dimension of the underlying quantity of interest increases to infinity, e.g. under mesh refinement when approximating PDE in finite dimensions. However, in practice, despite the attractive theoretical properties of the currently available schemes, they may still suffer from long correlation times, particularly if the data is very informative about some of the unknown parameters. In fact, in this case it may be the directions of the posterior which coincide with the (already known) prior which decorrelate the slowest. The information incorporated into the posterior through the data is often contained within some finite-dimensional subspace, in an appropriate basis, perhaps even one defined by eigenfunctions of the prior. We aim to exploit this fact and improve the mixing time of function-space MCMC by careful rescaling of the proposal. To this end, we introduce two new basic methods of increasing complexity, involving (i) characteristic function truncation of high frequencies and (ii) hessian information to interpolate between low and high frequencies.

stat.ME↗

arXiv2013

The Ensemble Kalman Filter for Inverse Problems

The Ensemble Kalman filter (EnKF) was introduced by Evensen in 1994 [10] as a novel method for data assimilation: state estimation for noisily observed time-dependent problems. Since that time it has had enormous impact in many application domains because of its robustness and ease of implementation, and numerical evidence of its accuracy. In this paper we propose the application of an iterative ensemble Kalman method for the solution of a wide class of inverse problems. In this context we show that the estimate of the unknown function that we obtain with the ensemble Kalman method lies in a subspace A spanned by the initial ensemble. Hence the resulting error may be bounded above by the error found from the best approximation in this subspace. We provide numerical experiments which compare the error incurred by the ensemble Kalman method for inverse problems with the error of the best approximation in A, and with variants on traditional least-squares approaches, restricted to the subspace A. In so doing we demonstrate that the ensemble Kalman method for inverse problems provides a derivative-free optimization method with comparable accuracy to that achieved by traditional least-squares approaches. Furthermore, we also demonstrate that the accuracy is of the same order of magnitude as that achieved by the best approximation. Three examples are used to demonstrate these assertions: inversion of a compact linear operator; inversion of piezometric head to determine hydraulic conductivity in a Darcy model of groundwater flow; and inversion of Eulerian velocity measurements at positive times to determine the initial condition in an incompressible fluid.

math.OC↗

arXiv2013

Dark Solitary Waves in a Class of Collisionally Inhomogeneous Bose-Einstein Condensates

We study the structure, stability, and dynamics of dark solitary waves in parabolically trapped, collisionally inhomogeneous Bose-Einstein condensates (BECs) with spatially periodic variations of the scattering length. This collisional inhomogeneity yields a nonlinear lattice, which we tune from a small-amplitude, approximately sinusoidal structure to a periodic sequence of densely spaced spikes. We start by investigating time-independent inhomogeneities, and we subsequently examine the dynamical response when one starts with a collisionally homogeneous BEC and then switches on an inhomogeneity either adiabatically or nonadiabatically. Using Bogoliubov-de Gennes linearization as well as direct numerical simulations of the Gross-Pitaevskii equation, we observe dark solitary waves, which can become unstable through oscillatory or exponential instabilities. We find a critical wavelength of the nonlinear lattice that is comparable to the healing length. Near this value, the fundamental eigenmode responsible for the stability of the dark solitary wave changes its direction of movement as a function of the strength of the nonlinearity. When it increases, it collides with other eigenmodes, leading to oscillatory instabilities; when it decreases, it collides with the origin and becomes imaginary, illustrating that the instability mechanism is fundamentally different in wide-well versus narrow-well lattices. When starting from a collisionally homogeneous setup and switching on inhomogeneities, we find that dark solitary waves are preserved generically for aligned lattices. We briefly examine the time scales for the onset of solitary-wave oscillations in this scenario.

nlin.PS↗

arXiv2012

Evaluation of Gaussian approximations for data assimilation in reservoir models

In this paper we propose to numerically assess the performance of standard Gaussian approximations to probe the posterior distribution that arises from Bayesian data assimilation in petroleum reservoirs. In particular we assess the performance of (i) the linearization around the maximum a posterior estimate, (ii) the randomized maximum likelihood and (iii) standard ensemble Kalman filter-type methods. In order to fully resolve the posterior distribution we implement a state-of-the art MCMC method that scales well with respect to the dimension of the parameter space. Our implementation of the MCMC method provides the gold standard against which to assess the aforementioned Gaussian approximations. We present numerical synthetic experiments where we quantify the capability of each of the {\em ad hoc} Gaussian approximation in reproducing the mean and the variance of the posterior distribution (characterized via MCMC) associated to a data assimilation problem. The main objective of our controlled experiments is to exhibit the substantial discrepancies of the approximation properties of standard {\em ad hoc} Gaussian approximations. Numerical investigations of the type we present here will lead to greater understanding of the cost-efficient, but {\em ad hoc}, Bayesian techniques used for data assimilation in petroleum reservoirs, and hence ultimately to improved techniques with more accurate uncertainty quantification.

stat.AP↗