arXiv subjects

Daniel Rudolf

Publications and source records attributed to Daniel Rudolf.

At least 19 recordsLinked to original sources

arXiv2026

Multi-modal Laplace-based sampling

We study the multi-modal Laplace approximation, given by a weighted Gaussian mixture centred at the local minima of the negative log-density, for sampling from probability measures that concentrate around finitely many modes as an inverse-temperature parameter $n$ grows. Such distributions occur in Bayesian inference in the small noise or large data regime. We derive explicit, non-asymptotic bounds on the Hellinger distance between the multi-modal Laplace approximation and the concentrating posterior. The bound decays like $n^{-1/2}$ with $n$ and the constants depend only polynomially on the dimension $d$. Moreover, we analyse the Laplace mixture as a proposal distribution for self-normalised importance sampling and independent Metropolis--Hastings (IMH) targeting concentration multimodal distributions. We derive asymptotic results, as $n\to\infty$, on the effective sample size in case of importance sampling as well as on the average acceptance probability, lag-one autocorrelation, and spectral gap in case of IMH. Numerical experiments illustrate and verify our theoretical findings.

math.ST↗

arXiv2026

Delayed Acceptance Slice Sampling

Slice sampling is a well-established Markov chain Monte Carlo method for approximate sampling of target distributions which are only known up to a normalizing constant. The method is based on choosing a new state on a slice, i.e., a superlevel set of the given unnormalized target density (with respect to a reference measure). However, slice sampling algorithms usually require per step multiple evaluations of the target density, and thus can become computationally expensive. This is particularly the case for Bayesian inference with costly likelihoods. In this paper, we exploit deterministic approximations of the target density, which are relatively cheap to evaluate, and propose delayed acceptance versions of several common (hybrid) slice samplers. We show ergodicity of the resulting slice sampling methods, discuss the superiority of delayed acceptance (ideal) slice sampling over delayed acceptance Metropolis-Hastings algorithms, and illustrate the benefits of our novel approach in terms of improved computational efficiency in numerical experiments.

stat.CO↗

arXiv2026

Error bounds for simultaneous Wasserstein contractive adaptive increasingly rare MCMC

We investigate adaptive increasingly rare Markov chain Monte Carlo algorithms and the associated time-average estimator for approximating expectations. Under a simultaneous Wasserstein contraction assumption on the underlying family of Markov kernels we derive explicit bounds for the mean squared error. We illustrate the applicability of our estimate through adaptive stereographic algorithms and Metropolis-Hastings schemes that employ normalizing flows for adaptation. We also consider a generic adaptive algorithm for doubly intractable problems and provide a corresponding cost analysis to achieve a desired precision.

math.ST↗

arXiv2026

Large SVARs

We develop a new algorithm for inference in structural vector autoregressions (SVARs) identified with sign restrictions that can accommodate big data and modern identification schemes. The key innovation of our approach is to move beyond the traditional accept-reject framework commonly used in sign-identified SVARs. We show that an elliptical slice within Gibbs sampler can deliver dramatic gains in computational speed and render previously infeasible applications tractable. We also prove that the algorithm is well-defined, in the sense that its stationary distribution coincides with the posterior distribution of interest. To illustrate the approach in the context of sign-identified SVARs, we use a tractable example. We further assess the performance of our algorithm through two applications: a well-known small-SVAR model of the oil market featuring a tight identified set, and a large SVAR model with more than ten shocks and 100 sign restrictions.

econ.EM↗

arXiv2026

Geodesic slice sampling on the sphere

Probability measures on the sphere form an important class of statistical models and are used, for example, in modeling directional data or shapes. Due to their widespread use, but also as an algorithmic building block, efficient sampling of distributions on the sphere is highly desirable. We propose a shrinkage based and an idealized geodesic slice sampling Markov chain, designed to generate approximate samples from distributions on the sphere. In particular, the shrinkage-based version of the algorithm can be implemented such that it runs efficiently and has no tuning parameters. We verify reversibility and prove that under weak regularity conditions geodesic slice sampling is uniformly ergodic. Numerical experiments show that the proposed slice samplers achieve excellent mixing on challenging targets including distributions arising in rigid-registration problems and mixtures of von Mises-Fisher distributions. In these settings our approach outperforms standard samplers such as random-walk Metropolis-Hastings and Hamiltonian Monte Carlo.

stat.ME↗

arXiv2025

Almost sure convergence rates of adaptive increasingly rare Markov chain Monte Carlo

We consider adaptive increasingly rare Markov chain Monte Carlo (MCMC) algorithms, which are adaptive MCMC methods, where the adaptation concerning the "past'' happens less and less frequently over time. Under a contraction assumption with respect to a Wasserstein-like function we deduce upper bounds of the convergence rate of Monte Carlo sums taking a renormalisation factor into account that is "almost'' the one that appears in a law of the iterated logarithm. We demonstrate the applicability of our results by considering different settings, among which are those of simultaneous geometric and uniform ergodicity. All proofs are carried out on an augmented state space, including the classical non-augmented setting as a special case. In contrast to other adaptive MCMC limit theory, some technical assumptions, like diminishing adaptation, are not needed.

math.NA↗

arXiv2025

Geodesic slice sampling on Riemannian manifolds

We propose a theoretically justified and practically applicable slice sampling based Markov chain Monte Carlo (MCMC) method for approximate sampling from probability measures on Riemannian manifolds. The latter naturally arise as posterior distributions in Bayesian inference of matrix-valued parameters, for example belonging to either the Stiefel or the Grassmann manifold. Our method, called geodesic slice sampling, is reversible with respect to the distribution of interest, and generalizes Hit-and-run slice sampling on $\mathbb{R}^{d}$ to Riemannian manifolds by using geodesics instead of straight lines. We demonstrate the robustness of our sampler's performance compared to other MCMC methods dealing with manifold valued distributions through extensive numerical experiments, on both synthetic and real data. In particular, we illustrate its remarkable ability to cope with anisotropic target densities, without using gradient information and preconditioning.

stat.CO↗

arXiv2025

Weak Poincaré inequality comparisons for ideal and hybrid slice sampling

Using the framework of weak Poincaré inequalities, we provide a general comparison between Hybrid and Ideal Slice Sampling in terms of their corresponding Dirichlet forms. In particular, under suitable assumptions Hybrid Slice Sampling inherits fast convergence from Ideal Slice Sampling and conversely. We apply our results to analyse the convergence of the Independent Metropolis-Hastings, stepping-out and shrinkage, as well as Hit-and-Run within slice sampling algorithms.

stat.CO↗

arXiv2024

Perturbation theory for killed Markov processes and quasi-stationary distributions

Motivated by recent developments of quasi-stationary Monte Carlo methods, we investigate the stability of quasi-stationary distributions of killed Markov processes under perturbations of the generator. We first consider a general bounded self-adjoint perturbation operator, and after that, study a particular unbounded perturbation corresponding to truncation of the killing rate. In both scenarios, we quantify the difference between eigenfunctions of the smallest eigenvalue of the perturbed and unperturbed generators in a Hilbert space norm. As a consequence, L1 norm estimates of the difference of the resulting quasi-stationary distributions in terms of the perturbation are provided.

math.PR↗

arXiv2024

Parallel Affine Transformation Tuning of Markov Chain Monte Carlo

The performance of Markov chain Monte Carlo samplers strongly depends on the properties of the target distribution such as its covariance structure, the location of its probability mass and its tail behavior. We explore the use of bijective affine transformations of the sample space to improve the properties of the target distribution and thereby the performance of samplers running in the transformed space. In particular, we propose a flexible and user-friendly scheme for adaptively learning the affine transformation during sampling. Moreover, the combination of our scheme with Gibbsian polar slice sampling is shown to produce samples of high quality at comparatively low computational cost in several settings based on real-world data.

stat.ME↗

arXiv2024

Reversibility of elliptical slice sampling revisited

We extend elliptical slice sampling, a Markov chain transition kernel suggested in Murray, Adams and MacKay 2010, to infinite-dimensional separable Hilbert spaces and discuss its well-definedness. We point to a regularity requirement, provide an alternative proof of the desirable reversibility property and show that it induces a positive semi-definite Markov operator. Crucial within the proof of the formerly mentioned results is the analysis of a shrinkage Markov chain that may be interesting on its own.

math.ST↗

arXiv2024

Perturbations of Markov Chains

This chapter surveys progress on three related topics in perturbations of Markov chains: the motivating question of when and how "perturbed" MCMC chains are developed, the theoretical problem of how perturbation theory can be used to analyze such chains, and finally the question of how the theoretical analyses can lead to practical advice.

stat.ME↗

arXiv2024

Convergence of hybrid slice sampling via spectral gap

It is known that the simple slice sampler has robust convergence properties, however the class of problems where it can be implemented is limited. In contrast, we consider hybrid slice samplers which are easily implementable and where another Markov chain approximately samples the uniform distribution on each slice. Under appropriate assumptions on the Markov chain on the slice we show a lower bound and an upper bound of the spectral gap of the hybrid slice sampler in terms of the spectral gap of the simple slice sampler. An immediate consequence of this is that spectral gap and geometric ergodicity of the hybrid slice sampler can be concluded from spectral gap and geometric ergodicity of its simple version which is very well understood. These results indicate that robustness properties of the simple slice sampler are inherited by (appropriately designed) easily implementable hybrid versions. We apply the developed theory and analyse a number of specific algorithms such as the stepping-out shrinkage slice sampling, hit-and-run slice sampling on a class of multivariate targets and an easily implementable combination of both procedures on multidimensional bimodal densities.

stat.ME↗

arXiv2023

Wasserstein convergence rates of increasingly concentrating probability measures

For $\ell\colon \mathbb{R}^d \to [0,\infty)$ we consider the sequence of probability measures $\left(μ_n\right)_{n \in \mathbb{N}}$, where $μ_n$ is determined by a density that is proportional to $\exp(-n\ell)$. We allow for infinitely many global minimal points of $\ell$, as long as they form a finite union of compact manifolds. In this scenario, we show estimates for the $p$-Wasserstein convergence of $\left(μ_n\right)_{n \in \mathbb{N}}$ to its limit measure. Imposing regularity conditions we obtain a speed of convergence of $n^{-1/(2p)}$ and adding a further technical assumption, we can improve this to a $p$-independent rate of $1/2$ for all orders $p\in\mathbb{N}$ of the Wasserstein distance.

math.PR↗

arXiv2023

Dimension-independent spectral gap of polar slice sampling

Polar slice sampling, a Markov chain construction for approximate sampling, performs, under suitable assumptions on the target and initial distribution, provably independent of the state space dimension. We extend the aforementioned result of Roberts & Rosenthal (2002) by developing a theory which identifies conditions, in terms of a generalized level set function, that imply an explicit lower bound on the spectral gap even in a general slice sampling context. Verifying the identified conditions for polar slice sampling yields a lower bound of 1/2 on the spectral gap for arbitrary dimension if the target density is rotationally invariant, log-concave along rays emanating from the origin and sufficiently smooth. The general theoretical result is potentially applicable beyond the polar slice sampling framework.

math.ST↗

arXiv2023

Analyzing cross-talk between superimposed signals: Vector norm dependent hidden Markov models and applications to ion channels

We propose and investigate a hidden Markov model (HMM) for the analysis of dependent, aggregated, superimposed two-state signal recordings. A major motivation for this work is that often these signals cannot be observed individually but only their superposition. Among others, such models are in high demand for the understanding of cross-talk between ion channels, where each single channel cannot be measured separately. As an essential building block, we introduce a parameterized vector norm dependent Markov chain model and characterize it in terms of permutation invariance as well as conditional independence. This building block leads to a hidden Markov chain sum process which can be used for analyzing the dependence structure of superimposed two-state signal observations within an HMM. Notably, the model parameters of the vector norm dependent Markov chain are uniquely determined by the parameters of the sum process and are therefore identifiable. We provide algorithms to estimate the parameters, discuss model selection and apply our methodology to real-world ion channel data from the heart muscle, where we show competitive gating.

stat.ME↗

arXiv2023

Gibbsian polar slice sampling

Polar slice sampling (Roberts & Rosenthal, 2002) is a Markov chain approach for approximate sampling of distributions that is difficult, if not impossible, to implement efficiently, but behaves provably well with respect to the dimension. By updating the directional and radial components of chain iterates separately, we obtain a family of samplers that mimic polar slice sampling, and yet can be implemented efficiently. Numerical experiments in a variety of settings indicate that our proposed algorithm outperforms the two most closely related approaches, elliptical slice sampling (Murray et al., 2010) and hit-and-run uniform slice sampling (MacKay, 2003). We prove the well-definedness and convergence of our methods under suitable assumptions on the target distribution.

stat.ME↗

arXiv2023

Consistency of randomized integration methods

We prove that a class of randomized integration methods, including averages based on $(t,d)$-sequences, Latin hypercube sampling, Frolov points as well as Cranley-Patterson rotations, consistently estimates expectations of integrable functions. Consistency here refers to convergence in mean and/or convergence in probability of the estimator to the integral of interest. Moreover, we suggest median modified methods and show for integrands in $L^p$ with $p>1$ consistency in terms of almost sure convergence

math.NA↗