arXiv subjects

Tim Johnston

Publications and source records attributed to Tim Johnston.

9 recordsLinked to original sources

arXiv2026

A Lyapunov-tamed Euler method for singular SDEs

Many applications, such as systems of interacting particles in physics, require the simulation of diffusion processes with singular coefficients. Standard Euler schemes are then not convergent, and theoretical guarantees in this situation are scarce. In this work we introduce a Lyapunov-tamed Euler scheme, for drift coefficients for which the weak derivative is dominated by a function that obeys a certain generic Lyapunov-type condition. This allows for a range of coefficients that explode to infinity on a bounded set. We establish that, in terms of Lp-strong error, the Lyapunov-tamed scheme is consistent and moreover achieves the same order of convergence as the standard Euler scheme for Lipschitz coefficients. The general result is applied to systems of mean-field particles with singular repulsive interaction in 1D, yielding an error bound with polynomial dependency in the number of particles.

math.PR↗

arXiv2025

Differential privacy guarantees of Markov chain Monte Carlo algorithms

This paper aims to provide differential privacy (DP) guarantees for Markov chain Monte Carlo (MCMC) algorithms. In a first part, we establish DP guarantees on samples output by MCMC algorithms as well as Monte Carlo estimators associated with these methods under assumptions on the convergence properties of the underlying Markov chain. In particular, our results highlight the critical condition of ensuring the target distribution is differentially private itself. In a second part, we specialise our analysis to the unadjusted Langevin algorithm and stochastic gradient Langevin dynamics and establish guarantees on their (R\'enyi) DP. To this end, we develop a novel methodology based on Girsanov's theorem combined with a perturbation trick to obtain bounds for an unbounded domain and in a non-convex setting. We establish: (i) uniform in $n$ privacy guarantees when the state of the chain after $n$ iterations is released, (ii) bounds on the privacy of the entire chain trajectory. These findings provide concrete guidelines for privacy-preserving MCMC.

stat.ML↗

arXiv2025

The Performance Of The Unadjusted Langevin Algorithm Without Smoothness Assumptions

In this article, we study the problem of sampling from distributions whose densities are not necessarily smooth nor logconcave. We propose a simple Langevin-based algorithm that does not rely on popular but computationally challenging techniques, such as the Moreau-Yosida envelope or Gaussian smoothing, and show consequently that the performance of samplers like ULA does not necessarily degenerate arbitrarily with low regularity. In particular, we show that the Lipschitz or H\"older continuity assumption can be replaced by a geometric one-sided Lipschitz condition that allows even for discontinuous log-gradients. We derive non-asymptotic guarantees for the convergence of the algorithm to the target distribution in Wasserstein distances. Non-asymptotic bounds are also provided for the performance of the algorithm as an optimizer, specifically for the solution of associated excess risk optimization problems.

stat.ML↗

arXiv2024

A Tamed Euler Scheme for SDEs with Non-Locally Integrable Drift Coefficient

In this article we show that for SDEs with a drift coefficient that is non-locally integrable, one may define a tamed Euler scheme that converges in $L^p$ at rate $1/2$ to the true solution. The taming is required in this case since one cannot expect the regular Euler scheme to have finite moments in $L^p$. We additionally show that our setting applies to the case of two scalar valued particles with singular interaction kernel. To the best of the author's knowledge, this is the first work we are aware of to prove strong convergence of an Euler-type scheme in the case of non-locally integrable drift.

math.PR↗

arXiv2024

Taming the Interacting Particle Langevin Algorithm: The Superlinear case

Recent advances in stochastic optimization have yielded the interacting particle Langevin algorithm (IPLA), which leverages the notion of interacting particle systems (IPS) to efficiently sample from approximate posterior densities. This becomes particularly crucial in relation to the framework of Expectation-Maximization (EM), where the E-step is computationally challenging or even intractable. Although prior research has focused on scenarios involving convex cases with gradients of log densities that grow at most linearly, our work extends this framework to include polynomial growth. Taming techniques are employed to produce an explicit discretization scheme that yields a new class of stable, under such non-linearities, algorithms which are called tamed interacting particle Langevin algorithms (tIPLA). We obtain non-asymptotic convergence error estimates in Wasserstein-2 distance for the new class under the best known rate.

math.PR↗

arXiv2023

Convergence Of The Unadjusted Langevin Algorithm For Discontinuous Gradients

We demonstrate that for strongly log-convex densities whose potentials are discontinuous on manifolds, the ULA algorithm converges with stepsize bias of order $1/2$ in Wasserstein-p distance. Our resulting bound is then of the same order as the convergence of ULA for gradient Lipschitz potential. Additionally, we show that so long as the gradient of the potential obeys a growth bound (therefore imposing no regularity condition), the algorithm has stepsize bias of order $1/4$. We therefore unite two active areas of research: i) the study of numerical methods for SDEs with discontinuous coefficients and ii) the study of the non-asymptotic bias of the ULA algorithm (and variants). In particular this is the first result of the former kind we are aware of on an unbounded time interval.

math.PR↗

arXiv2023

Interacting Particle Langevin Algorithm for Maximum Marginal Likelihood Estimation

We develop a class of interacting particle systems for implementing a maximum marginal likelihood estimation (MMLE) procedure to estimate the parameters of a latent variable model. We achieve this by formulating a continuous-time interacting particle system which can be seen as a Langevin diffusion over an extended state space of parameters and latent variables. In particular, we prove that the parameter marginal of the stationary measure of this diffusion has the form of a Gibbs measure where number of particles acts as the inverse temperature parameter in classical settings for global optimisation. Using a particular rescaling, we then prove geometric ergodicity of this system and bound the discretisation error in a manner that is uniform in time and does not increase with the number of particles. The discretisation results in an algorithm, termed Interacting Particle Langevin Algorithm (IPLA) which can be used for MMLE. We further prove nonasymptotic bounds for the optimisation error of our estimator in terms of key parameters of the problem, and also extend this result to the case of stochastic gradients covering practical scenarios. We provide numerical experiments to illustrate the empirical behaviour of our algorithm in the context of logistic regression with verifiable assumptions. Our setting provides a straightforward way to implement a diffusion-based optimisation routine compared to more classical approaches such as the Expectation Maximisation (EM) algorithm, and allows for especially explicit nonasymptotic bounds.

stat.CO↗

arXiv2023

Kinetic Langevin MCMC Sampling Without Gradient Lipschitz Continuity -- the Strongly Convex Case

In this article we consider sampling from log concave distributions in Hamiltonian setting, without assuming that the objective gradient is globally Lipschitz. We propose two algorithms based on monotone polygonal (tamed) Euler schemes, to sample from a target measure, and provide non-asymptotic 2-Wasserstein distance bounds between the law of the process of each algorithm and the target measure. Finally, we apply these results to bound the excess risk optimization error of the associated optimization problem.

math.PR↗

arXiv2021

A Strongly Monotonic Polygonal Euler Scheme

In recent years tamed schemes have become an important technique for simulating SDEs and SPDEs whose continuous coefficients display superlinear growth. The taming method, which involves curbing the growth of the coefficients as a function of stepsize, has so far however not been adapted to preserve the monotonicity of the coefficients. This has arisen as an issue particularly in \cite{articletam}, where the lack of a strongly monotonic tamed scheme forces strong conditions on the setting. In the present work we give a novel and explicit method for truncating monotonic functions in separable Hilbert spaces, and show how this can be used to define a polygonal (tamed) Euler scheme on finite dimensional space, preserving the monotonicity of the drift coefficient. This new method of truncation is well-defined with almost no assumptions and, unlike the well-known Moreau-Yosida regularisation, does not require an optimisation problem to be solved at each evaluation. Our construction is the first infinite dimensional method for truncating monotone functions that we are aware of, as well as the first explicit method in any number of dimensions.

math.PR↗