Stochstic Sampling for Generative Diffusion Models: From Euler-Maruyama to Higher-Order Schemes

Abstract

We develop a convergence analysis for generative diffusion models that simultaneously accounts for the three principal sources of error in stochastic sampling: initialization error, score-matching error, and discretization of the reverse-time SDE. Our central tool is the notion of a general strong scheme, a broad class of discretization methods for the reverse dynamics defined via explicit, index-wise tolerances on their It\^o-Taylor coefficients. This notion extends the classical strong-scheme framework of Kloeden and Platen to an iterate-wise formulation, which is strictly stronger and recovers their bound as a corollary. We prove a convergence theorem in the 2-Wasserstein distance that applies to this entire class of schemes at once, reducing the analysis of any concrete sampler to a finite verification checklist, and covers general forward processes with time-dependent, spatially linear drift and spatially independent diffusion coefficient, rather than a fixed variance-preserving, variance-exploding, or Ornstein--Uhlenbeck schedule. We instantiate this theorem for the Euler--Maruyama scheme, the exponential integrator, and, as our main application, a derivative-free stochastic Runge-Kutta scheme of strong order 1.5, yielding the first stochastic sampler for generative diffusion models with a provably higher convergence order than Euler--Maruyama. We further derive the resulting iteration complexity and an accompanying parameter-selection rule for the terminal time, score accuracy, and step size, and discuss the dissipative setting, in which the discretization and score-matching errors decouple from the terminal time. Numerical experiments on Gaussian toy models and the CIFAR-10 benchmark confirm the predicted convergence orders. Code available at: https://github.com/emanuelpfarr/SSGDM.