arXiv subjects

Lukasz Szpruch

Publications and source records attributed to Lukasz Szpruch.

At least 55 records · Page 3Linked to original sources

arXiv2019

Non-asymptotic bounds for sampling algorithms without log-concavity

Discrete time analogues of ergodic stochastic differential equations (SDEs) are one of the most popular and flexible tools for sampling high-dimensional probability measures. Non-asymptotic analysis in the $L^2$ Wasserstein distance of sampling algorithms based on Euler discretisations of SDEs has been recently developed by several authors for log-concave probability distributions. In this work we replace the log-concavity assumption with a log-concavity at infinity condition. We provide novel $L^2$ convergence rates for Euler schemes, expressed explicitly in terms of problem parameters. From there we derive non-asymptotic bounds on the distance between the laws induced by Euler schemes and the invariant laws of SDEs, both for schemes with standard and with randomised (inaccurate) drifts. We also obtain bounds for the hierarchy of discretisation, which enables us to deploy a multi-level Monte Carlo estimator. Our proof relies on a novel construction of a coupling for the Markov chains that can be used to control both the $L^1$ and $L^2$ Wasserstein distances simultaneously. Finally, we provide a weak convergence analysis that covers both the standard and the randomised (inaccurate) drift case. In particular, we reveal that the variance of the randomised drift does not influence the rate of weak convergence of the Euler scheme to the SDE.

math.PR↗

arXiv2019

Iterative Multilevel density estimation for McKean-Vlasov SDEs via projections

In this paper, we present a generic methodology for the efficient numerical approximation of the density function of the McKean-Vlasov SDEs. The weak error analysis for the projected process motivates us to combine the iterative Multilevel Monte Carlo method for McKean-Vlasov SDEs \cite{szpruch2019} with non-interacting kernels and projection estimation of particle densities \cite{belomestny2018projected}. By exploiting smoothness of the coefficients for McKean-Vlasov SDEs, in the best case scenario (i.e $C^{\infty}$ for the coefficients), we obtain the complexity of order $O(ε^{-2}|\logε|^4)$ for the approximation of expectations and $O(ε^{-2}|\logε|^5)$ for density estimation.

math.NA↗

arXiv2019

Multilevel Monte Carlo methods for the approximation of invariant measures of stochastic differential equations

We develop a framework that allows the use of the multi-level Monte Carlo (MLMC) methodology (Giles2015) to calculate expectations with respect to the invariant measure of an ergodic SDE. In that context, we study the (over-damped) Langevin equations with a strongly concave potential. We show that, when appropriate contracting couplings for the numerical integrators are available, one can obtain a uniform in time estimate of the MLMC variance in contrast to the majority of the results in the MLMC literature. As a consequence, a root mean square error of $\mathcal{O}(\varepsilon)$ is achieved with $\mathcal{O}(\varepsilon^{-2})$ complexity on par with Markov Chain Monte Carlo (MCMC) methods, which however can be computationally intensive when applied to large data sets. Finally, we present a multi-level version of the recently introduced Stochastic Gradient Langevin Dynamics (SGLD) method (Welling and Teh, 2011) built for large datasets applications. We show that this is the first stochastic gradient MCMC method with complexity $\mathcal{O}(\varepsilon^{-2}|\log {\varepsilon}|^{3})$, in contrast to the complexity $\mathcal{O}(\varepsilon^{-3})$ of currently available methods. Numerical experiments confirm our theoretical findings.

math.NA↗

arXiv2019

An adaptive Euler-Maruyama scheme for stochastic differential equations with discontinuous drift and its convergence analysis

We study the strong approximation of stochastic differential equations with discontinuous drift coefficients and (possibly) degenerate diffusion coefficients. To account for the discontinuity of the drift coefficient we construct an adaptive step sizing strategy for the explicit Euler-Maruyama scheme. As a result, we obtain a numerical method which has -- up to logarithmic terms -- strong convergence order $1/2$ with respect to the average computational cost. We support our theoretical findings with several numerical examples.

math.NA↗

arXiv2019

New particle representations for ergodic McKean-Vlasov SDEs

The aim of this paper is to introduce several new particle representations for \textit{ergodic} McKean-Vlasov SDEs. We construct new algorithms by leveraging recent progress in weak convergence analysis of interacting particle system. We present detailed analysis of errors and associated costs of various estimators, highlighting key differences between long-time simulations of linear (classical SDEs) versus non-linear (Mckean-Vlasov SDEs) process.

math.PR↗

arXiv2019

Iterative Particle Approximation for McKean-Vlasov SDEs with application to Multilevel Monte Carlo estimation

The mean field limits of systems of interacting diffusions (also called stochastic interacting particle systems (SIPS)) have been intensively studied since McKean \cite{mckean1966class}. The interacting diffusions pave a way to probabilistic representations for many important nonlinear/nonlocal PDEs, but provide a great challenge for Monte Carlo simulations. This is due to the nonlinear dependence of the bias on the statistical error arising through the approximation of the law of the process. This and the fact that particles/diffusions are not independent render classical variance reduction techniques not directly applicable and consequently make simulations of interacting diffusions prohibitive. In this article, we provide an alternative iterative particle representation, inspired by the fixed point argument by Sznitman \cite{sznitman1991topics}. This new representation has the same mean field limit as the classical SIPS. However, unlike classical SIPS, it also allows decomposing the statistical error and the approximation bias. We develop a general framework to study integrability and regularity properties of the iterated particle system. Moreover, we establish its weak convergence to the McKean-Vlasov SDEs (MVSDEs). One of the immediate advantages of iterative particle system is that it can be combined with the Multilevel Monte Carlo (MLMC) approach for the simulation of MVSDEs. We proved that the MLMC approach reduces the computational complexity of calculating expectations by an order of magnitude. Another perspective on this work is that we analyse the error of nested Multilevel Monte Carlo estimators, which is of independent interest. Furthermore, we work with state dependent functionals, unlike scalar outputs which are common in literature on MLMC. The error analysis is carried out in uniform, and what seems to be new, weighted norms.

math.PR↗

arXiv2019

Weak quantitative propagation of chaos via differential calculus on the space of measures

Consider the metric space $(\mathcal{P}_2(\mathbb{R}^d),W_2)$ of square integrable laws on $\mathbb{R}^d$ with the topology induced by the 2-Wasserstein distance $W_2$. Let $Φ: \mathcal{P}_2( \mathbb{R}^d) \to \mathbb{R}$ be a function and $μ_N$ be the empirical measure of a sample of $N$ random variables distributed as $μ$. The main result of this paper is to show that under suitable regularity conditions, we have \[ |Φ(μ) - \mathbb{E}Φ(μ_N)|= \sum_{j=1}^{k-1}\frac{C_j}{N^j} + O(\frac{1}{N^k}), \] for some positive constants $C_1, \ldots, C_{k-1}$ that do not depend on $N$, where $k$ corresponds to the degree of smoothness. We distinguish two cases: a) $μ_N$ is the empirical measure of $N$-samples from $μ$; b) $μ$ is a marginal law of McKean-Vlasov stochastic differential equation in which case $μ_N$ is an empirical law of marginal laws of the corresponding particle system. The first case is studied using functional derivatives on the space of measures. The second case relies on an Itô-type formula for the flow of probability measures and is intimately connected to PDEs on the space of measures, called the master equation in the literature of mean-field games. We state the general regularity conditions required for each case and analyse the regularity in the case of functionals of the laws of McKean-Vlasov SDEs. Ultimately, this work reveals quantitative estimates of propagation of chaos for interacting particle systems. Furthermore, we are able to provide weak propagation of chaos estimates for ensembles of interacting particles and show that these may have some remarkable properties.

math.PR↗

arXiv2017

Full-Projection explicit FBSDE scheme for parabolic PDEs with superlinear nonlinearities

Developing efficient and stable approximations for high dimensional PDEs is of key importance for numerous applications. The language of Forward-Backward Stochastic Differential Equations (FBSDE), with its nonlinear Feynman-Kac formula, allows for purely probabilistic representations of the solution and its gradient for parabolic nonlinear PDEs. In this work we build on the recent results of [Lionnet, dos Reis and Szpruch 2015] by introducing and studying a Full-Projection explicit time-discretization scheme for the approximation of FBSDEs with non-globally Lipschitz drivers of polynomial growth. We establish convergence rates and we show that, unlike classical explicit schemes, it preserves stability properties present in the continuous-time dynamics, in particular, the scheme is able to preserve the possible coercivity/contraction property of the PDE's coefficients. The scheme is then coupled with a quantization-type approximation of the conditional expectations on a space-time grid in order to provide a complete approximation scheme for these FBSDEs/nonlinear PDEs and a full analysis is also carried out. We illustrate our findings with numerical examples.

math.NA↗

arXiv2017

The True Cost of Stochastic Gradient Langevin Dynamics

The problem of posterior inference is central to Bayesian statistics and a wealth of Markov Chain Monte Carlo (MCMC) methods have been proposed to obtain asymptotically correct samples from the posterior. As datasets in applications grow larger and larger, scalability has emerged as a central problem for MCMC methods. Stochastic Gradient Langevin Dynamics (SGLD) and related stochastic gradient Markov Chain Monte Carlo methods offer scalability by using stochastic gradients in each step of the simulated dynamics. While these methods are asymptotically unbiased if the stepsizes are reduced in an appropriate fashion, in practice constant stepsizes are used. This introduces a bias that is often ignored. In this paper we study the mean squared error of Lipschitz functionals in strongly log- concave models with i.i.d. data of growing data set size and show that, given a batchsize, to control the bias of SGLD the stepsize has to be chosen so small that the computational cost of reaching a target accuracy is roughly the same for all batchsizes. Using a control variate approach, the cost can be reduced dramatically. The analysis is performed by considering the algorithms as noisy discretisations of the Langevin SDE which correspond to the Euler method if the full data set is used. An important observation is that the 1scale of the step size is determined by the stability criterion if the accuracy is required for consistent credible intervals. Experimental results confirm our theoretical findings.

stat.ME↗

arXiv2016

Multilevel Monte Carlo for Scalable Bayesian Computations

Markov chain Monte Carlo (MCMC) algorithms are ubiquitous in Bayesian computations. However, they need to access the full data set in order to evaluate the posterior density at every step of the algorithm. This results in a great computational burden in big data applications. In contrast to MCMC methods, Stochastic Gradient MCMC (SGMCMC) algorithms such as the Stochastic Gradient Langevin Dynamics (SGLD) only require access to a batch of the data set at every step. This drastically improves the computational performance and scales well to large data sets. However, the difficulty with SGMCMC algorithms comes from the sensitivity to its parameters which are notoriously difficult to tune. Moreover, the Root Mean Square Error (RMSE) scales as $\mathcal{O}(c^{-\frac{1}{3}})$ as opposed to standard MCMC $\mathcal{O}(c^{-\frac{1}{2}})$ where $c$ is the computational cost. We introduce a new class of Multilevel Stochastic Gradient Markov chain Monte Carlo algorithms that are able to mitigate the problem of tuning the step size and more importantly of recovering the $\mathcal{O}(c^{-\frac{1}{2}})$ convergence of standard Markov Chain Monte Carlo methods without the need to introduce Metropolis-Hasting steps. A further advantage of this new class of algorithms is that it can easily be parallelised over a heterogeneous computer architecture. We illustrate our methodology using Bayesian logistic regression and provide numerical evidence that for a prescribed relative RMSE the computational cost is sublinear in the number of data items.

stat.ML↗

arXiv2016

Convergence and qualitative properties of modified explicit schemes for BSDEs with polynomial growth

The theory of Forward-Backward Stochastic Differential Equations (FBSDEs) paves a way to probabilistic numerical methods for nonlinear parabolic PDEs. The majority of the results on the numerical methods for FBSDEs relies on the global Lipschitz assumption, which is not satisfied for a number of important cases such as the Fisher--KPP or the FitzHugh--Nagumo equations. Furthermore, it has been shown in \cite{LionnetReisSzpruch2015} that for BSDEs with monotone drivers having polynomial growth in the primary variable $y$, only the (sufficiently) implicit schemes converge. But these require an additional computational effort compared to explicit schemes. This article develops a general framework that allows the analysis, in a systematic fashion, of the integrability properties, convergence and qualitative properties (e.g.~comparison theorem) for whole families of modified explicit schemes. The framework yields the convergence of some modified explicit scheme with the same rate as implicit schemes and with the computational cost of the standard explicit scheme. To illustrate our theory, we present several classes of easily implementable modified explicit schemes that can computationally outperform the implicit one and preserve the qualitative properties of the solution to the BSDE. These classes fit into our developed framework and are tested in computational experiments.

math.PR↗

arXiv2015

Time discretization of FBSDE with polynomial growth drivers and reaction-diffusion PDEs

In this paper, we undertake the error analysis of the time discretization of systems of Forward-Backward Stochastic Differential Equations (FBSDEs) with drivers having polynomial growth and that are also monotone in the state variable. We show with a counter-example that the natural explicit Euler scheme may diverge, unlike in the canonical Lipschitz driver case. This is due to the lack of a certain stability property of the Euler scheme which is essential to obtain convergence. However, a thorough analysis of the family of $θ$-schemes reveals that this required stability property can be recovered if the scheme is sufficiently implicit. As a by-product of our analysis, we shed some light on higher order approximation schemes for FBSDEs under non-Lipschitz condition. We then return to fully explicit schemes and show that an appropriately tamed version of the explicit Euler scheme enjoys the required stability property and as a consequence converges. In order to establish convergence of the several discretizations, we extend the canonical path- and first-order variational regularity results to FBSDEs with polynomial growth drivers which are also monotone. These results are of independent interest for the theory of FBSDEs.

math.PR↗

arXiv2015

$V$-Integrability, Asymptotic Stability And Comparison Theorem of Explicit Numerical Schemes for SDEs

Khasminski's \cite{chas1980stochastic} showed that many of the asymptotic stability and the integrability properties of the solutions to the Stochastic Differential Equations (SDEs) can be obtained using Lyapunov functions techniques. These properties are rarely inherited by standard numerical integrators. In this article we introduce a family of explicit numerical approximations for the SDEs and derive conditions that allow to use Khasminski's techniques in the context of numerical approximations, particularly in the case where SDEs have non globally Lipschitz coefficients. Consequently, we show that it is possible to construct a numerical scheme, that is bounded in expectation with respect to a Lyapunov function, and/or inherit the asymptotic stability property from the SDEs. Finally we show that using suitable schemes it is possible to recover comparison theorem for scalar SDEs.

math.NA↗

arXiv2014

Antithetic multilevel Monte Carlo estimation for multi-dimensional SDEs without Lévy area simulation

In this paper we introduce a new multilevel Monte Carlo (MLMC) estimator for multi-dimensional SDEs driven by Brownian motions. Giles has previously shown that if we combine a numerical approximation with strong order of convergence $O(Δt)$ with MLMC we can reduce the computational complexity to estimate expected values of functionals of SDE solutions with a root-mean-square error of $ε$ from $O(ε^{-3})$ to $O(ε^{-2})$. However, in general, to obtain a rate of strong convergence higher than $O(Δt^{1/2})$ requires simulation, or approximation, of Lévy areas. In this paper, through the construction of a suitable antithetic multilevel correction estimator, we are able to avoid the simulation of Lévy areas and still achieve an $O(Δt^2)$ multilevel correction variance for smooth payoffs, and almost an $O(Δt^{3/2})$ variance for piecewise smooth payoffs, even though there is only $O(Δt^{1/2})$ strong convergence. This results in an $O(ε^{-2})$ complexity for estimating the value of European and Asian put and call options.

q-fin.CP↗

arXiv2012

Multilevel Monte Carlo methods for applications in finance

Since Giles introduced the multilevel Monte Carlo path simulation method [18], there has been rapid development of the technique for a variety of applications in computational finance. This paper surveys the progress so far, highlights the key features in achieving a high rate of multilevel variance convergence, and suggests directions for future research.

q-fin.CP↗

arXiv2012

First order strong approximations of scalar SDEs with values in a domain

We are interested in strong approximations of one-dimensional SDEs which have non-Lipschitz coefficients and which take values in a domain. Under a set of general assumptions we derive an implicit scheme that preserves the domain of the SDEs and is strongly convergent with rate one. Moreover, we show that this general result can be applied to many SDEs we encounter in mathematical finance and bio-mathematics. We will demonstrate flexibility of our approach by analysing classical examples of SDEs with sublinear coefficients (CIR, CEV models and Wright-Fisher diffusion) and also with superlinear coefficients (3/2-volatility, Ait-Sahalia model). Our goal is to justify an efficient Multi-Level Monte Carlo (MLMC) method for a rich family of SDEs, which relies on good strong convergence properties.

q-fin.CP↗

arXiv2012

Strong convergence and stability of implicit numerical methods for stochastic differential equations with non-globally Lipschitz continuous coefficients

We are interested in the strong convergence and almost sure stability of Euler-Maruyama (EM) type approximations to the solutions of stochastic differential equations (SDEs) with non-linear and non-Lipschitzian coefficients. Motivation comes from finance and biology where many widely applied models do not satisfy the standard assumptions required for the strong convergence. In addition we examine the globally almost surely asymptotic stability in this non-linear setting for EM type schemes. In particular, we present a stochastic counterpart of the discrete LaSalle principle from which we deduce stability properties for numerical methods.

math.NA↗

arXiv2012

Convergence, Non-negativity and Stability of a New Milstein Scheme with Applications to Finance

We propose and analyse a new Milstein type scheme for simulating stochastic differential equations (SDEs) with highly nonlinear coefficients. Our work is motivated by the need to justify multi-level Monte Carlo simulations for mean-reverting financial models with polynomial growth in the diffusion term. We introduce a double implicit Milstein scheme and show that it possesses desirable properties. It converges strongly and preserves non-negativity for a rich family of financial models and can reproduce linear and nonlinear stability behaviour of the underlying SDE without severe restriction on the time step. Although the scheme is implicit, we point out examples of financial models where an explicit formula for the solution to the scheme can be found.

math.NA↗