Searcharxiv⌕ Search

arXiv subjects

J. M. Sanz-Serna

Publications and source records attributed to J. M. Sanz-Serna.

At least 19 recordsLinked to original sources
arXiv2026

Optimal scaling of MCMC algorithms: the Hamiltonian approach

We present a simple, yet general approach to study the scaling properties as the dimensionality of Metropolised MCMC sampling algorithms increases. The study relies on the symmetries of the Hamiltonian formalism and ultimately on the symmetry of the Metropolis-Hastings formula. Our findings contain, as particular cases, many known results for the Random Walk Metropolis, MALA and other algorithms. In addition, they provide, in an easy way, new optimal scaling results for a variety of proposal mechanisms, including implicit proposals and proposals generated with the help of differential equation integrators. The analysis applies to targets that are products of a given, not necessarily univariate distribution, and also to cases where the different terms in the product are scaled differently. We show how to construct gradient-based MALA-like proposals where the variance of the proposal as the dimension $d$ increases may be taken as $O(1/d^μ)$, with $μ>0$ arbitrarily small, to be compared with the values $μ= 1$ for Random Walk Metropolis and $μ=1/3$ for MALA.

stat.CO↗
arXiv2026

On the unconventional Hug integrator

Hug is a recently proposed iterative mapping used to design efficient updates in Markov chain Monte Carlo (MCMC) methods. Hug generates proposals that remain very close to hypersurfaces (level sets) of constant probabilty density. We analyse a generalization of Hug from hypersurfaces to manifolds of arbitrary dimensions, not necessarily arising in a sampling context. The analysis is based on interpreting, in a nonstandard way, Hug as a consistent discretization of a system of differential equations with a rather complicated structure. The proof of convergence of this discretization includes a number of unusual features we explore fully, in particular a supraconvergence property is established, whereby second order of convergence is attained with consistency of the first order. We uncover and discuss an unexpected property of the solutions of the underlying dynamical system that manifest itself by the existence of Hug trajectories that fail to cover the manifold of interest.

math.NA↗
arXiv2024

Stroboscopic averaging methods to study autoresonance and other problems with slowly varying forcing frequencies

Autoresonance is a phenomenon of physical interest that may take place when a nonlinear oscillator is forced at a frequency that varies slowly. The stroboscopic averaging method (SAM), which provides an efficient numerical technique for the integration of highly oscillatory systems, cannot be used directly to study autoresonance due to the slow changes of the forcing frequency. We study how to modify SAM to cater for such slow variations. Numerical experiments show the computational advantages of using SAM.

math.NA↗
arXiv2021

Wasserstein distance estimates for the distributions of numerical approximations to ergodic stochastic differential equations

We present a framework that allows for the non-asymptotic study of the $2$-Wasserstein distance between the invariant distribution of an ergodic stochastic differential equation and the distribution of its numerical approximation in the strongly log-concave case. This allows us to study in a unified way a number of different integrators proposed in the literature for the overdamped and underdamped Langevin dynamics. In addition, we analyse a novel splitting method for the underdamped Langevin dynamics which only requires one gradient evaluation per time step. Under an additional smoothness assumption on a $d$--dimensional strongly log-concave distribution with condition number $κ$, the algorithm is shown to produce with an $\mathcal{O}\big(κ^{5/4} d^{1/4}ε^{-1/2} \big)$ complexity samples from a distribution that, in Wasserstein distance, is at most $ε>0$ away from the target distribution.

stat.ML↗
arXiv2021

Symmetrically processed splitting integrators for enhanced Hamiltonian Monte Carlo sampling

We construct integrators to be used in Hamiltonian (or Hybrid) Monte Carlo sampling. The new integrators are easily implementable and, for a given computational budget, may deliver five times as many accepted proposals as standard leapfrog/Verlet without impairing in any way the quality of the samples. They are based on a suitable modification of the processing technique first introduced by J.C. Butcher. The idea of modified processing may also be useful for other purposes, like the construction of high-order splitting integrators with positive coefficients.

math.NA↗
arXiv2021

HMC: avoiding rejections by not using leapfrog and some results on the acceptance rate

The leapfrog integrator is routinely used within the Hamiltonian Monte Carlo method and its variants. We give strong numerical evidence that alternative, easy to implement algorithms yield fewer rejections with a given computational effort. When the dimensionality of the target distribution is high, the number of accepted proposals may be multiplied by a factor of three or more. This increase in the number of accepted proposals is not achieved by impairing any positive features of the sampling. We also establish new non-asymptotic and asymptotic results on the monotonic relationship between the expected acceptance rate and the expected energy error. These results further validate the derivation of one of the integrators we consider and are of independent interest.

stat.CO↗
arXiv2021

Contractivity of Runge-Kutta methods for convex gradient systems

We consider the application of Runge-Kutta (RK) methods to gradient systems $(d/dt)x = -\nabla V(x)$, where, as in many optimization problems, $V$ is convex and $\nabla V$ (globally) Lipschitz-continuous with Lipschitz constant $L$. Solutions of this system behave contractively, i.e. the Euclidean distance between two solutions $x(t)$ and $\widetilde{x}(t)$ is a nonincreasing function of $t$. It is then of interest to investigate whether a similar contraction takes place, at least for suitably small step sizes $h$, for the discrete solution. Dahlquist and Jeltsch results' imply that (1) there are explicit RK schemes that behave contractively whenever $Lh$ is below a scheme-dependent constant and (2) Euler's rule is optimal in this regard. We prove however, by explicit construction of a convex potential using ideas from robust control theory, that there exists RK schemes that fail to behave contractively for any choice of the time-step $h$.

math.NA↗
arXiv2021

The connections between Lyapunov functions for some optimization algorithms and differential equations

In this manuscript, we study the properties of a family of second-order differential equations with damping, its discretizations and their connections with accelerated optimization algorithms for $m$-strongly convex and $L$-smooth functions. In particular, using the Linear Matrix Inequality LMI framework developed by \emph{Fazlyab et. al. $(2018)$}, we derive analytically a (discrete) Lyapunov function for a two-parameter family of Nesterov optimization methods, which allows for the complete characterization of their convergence rate. In the appropriate limit, this family of methods may be seen as a discretization of a family of second-order ordinary differential equations for which we construct(continuous) Lyapunov functions by means of the LMI framework. The continuous Lyapunov functions may alternatively, be obtained by studying the limiting behaviour of their discrete counterparts. Finally, we show that the majority of typical discretizations of the family of ODEs, such as the Heavy ball method, do not possess Lyapunov functions with properties similar to those of the Lyapunov function constructed here for the Nesterov method.

math.NA↗
arXiv2020

Is the NUTS algorithm correct?

This paper is devoted to investigate whether the popular No U-turn (NUTS) sampling algorithm is correct, i.e.\ whether the target probability distribution is \emph{exactly} conserved by the algorithm. It turns out that one of the Gibbs substeps used in the algorithm cannot always be guaranteed to be correct.

stat.CO↗
arXiv2019

Word-series high-order averaging of highly oscillatory differential equations with delay

We show that, for appropriate combinations of the values of the delay and the forcing frequency, it is possible to obtain easily high-order averaged versions of periodically forced systems of delay differential equations with constant delay. Our approach is based on the use of word-series techniques to obtain high-order averaged equations for differential equations without delay.

math.DS↗
arXiv2018

La cuadratura gaussiana según Gauss

This article is an abridged and commented translation into Spanish of the 1815 memoir where Gauss introduced the quadrature rules now associated with his name. Gauss' work does not resemble at all the stardard text-book treatment of Gaussian quadrature. The original memoir is an example of mathematical virtuosity, based on a superb use of series, where the problem is reformulated as a problem in functional approximation that is solved by means of continued fractions.

math.HO↗
arXiv2018

Word combinatorics for stochastic differential equations: splitting integrators

We present an analysis based on word combinatorics of splitting integrators for Ito or Stratonovich systems of stochastic differential equations. In particular we present a technique to write down systematically the expansion of the local error; this makes it possible to easily formulate the conditions that guarantee that a given integrator achieves a prescribed strong or weak order. This approach bypasses the need to use the Baker-Campbell-Hausdorff (BCH) formula and shows the existence of an order barrier of two for the attainable weak order. The paper also provides a succinct introduction to the combinatorics of words.

math.NA↗
arXiv2018

A stroboscopic averaging algorithm for highly oscillatory delay problems

We propose and analyze a heterogenous multiscale method for the efficient integration of constant-delay differential equations subject to fast periodic forcing. The stroboscopic averaging method (SAM) suggested here may provide approximations with $\(\mathcal{O}(H^2+1/Ω^2)\)$ errors with a computational effort that grows like $\(H^{-1}\)$ (the inverse of the stepsize), uniformly in the forcing frequency Omega.

math.NA↗
arXiv2017

Hopf algebra techniques to handle dynamical systems and numerical integrators

In a series of papers the present authors and their coworkers have developed a family of algebraic techniques to solve a number of problems in the theory of discrete or continuous dynamical systems and to analyze numerical integrators. Given a specific problem, those techniques construct an abstract, {\em universal} version of it which is solved algebraically; then, the results are tranferred to the original problem with the help of a suitable morphism. In earlier contributions, the abstract problem is formulated either in the dual of the shuffle Hopf algebra or in the dual of the Connes-Kreimer Hopf algebra. In the present contribution we extend these techniques to more general Hopf algebras, which in some cases lead to more efficient computations.

math.DS↗
arXiv2017

Palindromic 3-stage splitting integrators, a roadmap

The implementation of multi-stage splitting integrators is essentially the same as the implementation of the familiar Strang/Verlet method. Therefore multi-stage formulas may be easily incorporated into software that now uses the Strang/Verlet integrator. We study in detail the two-parameter family of palindromic, three-stage splitting formulas and identify choices of parameters that may outperform the Strang/Verlet method. One of these choices leads to a method of effective order four suitable to integrate in time some partial differential equations. Other choices may be seen as perturbations of the Strang method that increase efficiency in molecular dynamics simulations and in Hybrid Monte Carlo sampling.

math.NA↗
arXiv2017

Averaging and computing normal forms with word series algorithms

In the first part of the present work we consider periodically or quasiperiodically forced systems of the form $(d/dt)x = εf(x,t ω)$, where $ε\ll 1$, $ω\in\mathbb{R}^d$ is a nonresonant vector of frequencies and $f(x,θ)$ is $2π$-periodic in each of the $d$ components of $θ$ (i.e.\ $θ\in\mathbb{T}^d$). We describe in detail a technique for explicitly finding a change of variables $x = u(X,θ;ε)$ and an (autonomous) averaged system $(d/dt) X = εF(X;ε)$ so that, formally, the solutions of the given system may be expressed in terms of the solutions of the averaged system by means of the relation $x(t) = u(X(t),tω;ε)$. Here $u$ and $F$ are found as series whose terms consist of vector-valued maps weighted by suitable scalar coefficients. The maps are easily written down by combining the Fourier coefficients of $f$ and the coefficients are found with the help of simple recursions. Furthermore these coefficients are {\em universal} in the sense that they do not depend on the particular $f$ under consideration. In the second part of the contribution, we study problems of the form $(d/dt) x = g(x)+f(x)$, where one knows how to integrate the "unperturbed" problem $(d/dt)x = g(x)$ and $f$ is a perturbation satisfying appropriate hypotheses. It is shown how to explicitly rewrite the system in the "normal form" $(d/dt) x = \bar g(x)+\bar f(x)$, where $\bar g$ and $\bar f$ are {\em commuting} vector fields and the flow of $(d/dt) x = \bar g(x)$ is conjugate to that of the unperturbed $(d/dt)x = g(x)$. In Hamiltonian problems the normal form directly leads to the explicit construction of formal invariants of motion. Again, $\bar g$, $\bar f$ and the invariants are written as series consisting of known vector-valued maps and universal scalar coefficients that may be found recursively.

math.DS↗
arXiv2016

A technique for studying strong and weak local errors of splitting stochastic integrators

We present a technique, based on so-called word series, to write down in a systematic way expansions of the strong and weak local errors of splitting algorithms for the integration of Stratonovich stochastic differential equations. Those expansions immediately lead to the corresponding order conditions. Word series are similar to, but simpler than, the B-series used to analyze Runge-Kutta and other one-step integrators. The suggested approach makes it unnecessary to use the Baker-Campbell-Hausdorff formula. As an application, we compare two splitting algorithms recently considered by Leimkuhler and Matthews to integrate the Langevin equations. The word series method bears out clearly reasons for the advantages of one algorithm over the other.

math.NA↗