arXiv subjects

Simo Särkkä

Publications and source records attributed to Simo Särkkä.

At least 37 records · Page 2Linked to original sources

arXiv2025

Modelling pathwise uncertainty of Stochastic Differential Equations samplers via Probabilistic Numerics

Probabilistic ordinary differential equation (ODE) solvers have been introduced over the past decade as uncertainty-aware numerical integrators. They typically proceed by assuming a functional prior to the ODE solution, which is then queried on a grid to form a posterior distribution over the ODE solution. As the queries span the integration interval, the approximate posterior solution then converges to the true deterministic one. Gaussian ODE filters, in particular, have enjoyed a lot of attention due to their computational efficiency, the simplicity of their implementation, as well as their provable fast convergence rates. In this article, we extend the methodology to stochastic differential equations (SDEs) and propose a probabilistic simulator for SDEs. Our approach involves transforming the SDE into a sequence of random ODEs using piecewise differentiable approximations of the Brownian motion. We then apply probabilistic ODE solvers to the individual ODEs, resulting in a pathwise probabilistic solution to the SDE\@. We establish worst-case strong $1.5$ local and $1.0$ global convergence orders for a specific instance of our method. We further show how we can marginalise the Brownian approximations, by incorporating its coefficients as part of the prior ODE model, allowing for computing exact transition densities under our model. Finally, we numerically validate the theoretical findings, showcasing reasonable weak convergence properties in the marginalised version.

math.NA↗

arXiv2025

Auxiliary MCMC and particle Gibbs samplers for parallelisable inference in latent dynamical systems

Sampling from the full posterior distribution of high-dimensional non-linear, non-Gaussian latent dynamical models presents significant computational challenges. While Particle Gibbs (also known as conditional sequential Monte Carlo) is considered the gold standard for this task, it quickly degrades in performance as the latent space dimensionality increases. Conversely, globally Gaussian-approximated methods like extended Kalman filtering, though more robust, are seldom used for posterior sampling due to their inherent bias. We introduce novel auxiliary sampling approaches that address these limitations. By incorporating artificial observations of the system as auxiliary variables in our MCMC kernels, we develop both efficient exact Kalman-based samplers and enhanced Particle Gibbs algorithms that maintain performance in high-dimensional latent spaces. Some of our methods support parallelisation along the time dimension, achieving logarithmic scaling when implemented on GPUs. Empirical evaluations demonstrate superior statistical and computational performance compared to existing approaches for high-dimensional latent dynamical systems.

stat.CO↗

arXiv2025

Conditioning diffusion models by explicit forward-backward bridging

Given an unconditional diffusion model targeting a joint model $π(x, y)$, using it to perform conditional simulation $π(x \mid y)$ is still largely an open question and is typically achieved by learning conditional drifts to the denoising SDE after the fact. In this work, we express \emph{exact} conditional simulation within the \emph{approximate} diffusion model as an inference problem on an augmented space corresponding to a partial SDE bridge. This perspective allows us to implement efficient and principled particle Gibbs and pseudo-marginal samplers marginally targeting the conditional distribution $π(x \mid y)$. Contrary to existing methodology, our methods do not introduce any additional approximation to the unconditional diffusion model aside from the Monte Carlo error. We showcase the benefits and drawbacks of our approach on a series of synthetic and real data examples.

stat.ML↗

arXiv2025

Provable Quantum Algorithm Advantage for Gaussian Process Quadrature

The aim of this paper is to develop novel quantum algorithms for Gaussian process quadrature methods. Gaussian process quadratures are numerical integration methods where Gaussian processes are used as functional priors for the integrands to capture the uncertainty arising from the sparse function evaluations. Quantum computers have emerged as potential replacements for classical computers, offering exponential reductions in the computational complexity of machine learning tasks. In this paper, we combine Gaussian process quadratures and quantum computing by proposing a quantum low-rank Gaussian process quadrature method based on a Hilbert space approximation of the Gaussian process kernel and enhancing the quadrature using a quantum circuit. The method combines the quantum phase estimation algorithm with the quantum principal component analysis technique to extract information up to a desired rank. Then, Hadamard and SWAP tests are implemented to find the expected value and variance that determines the quadrature. We use numerical simulations of a quantum computer to demonstrate the effectiveness of the method. Furthermore, we provide a theoretical complexity analysis that shows a polynomial advantage over classical Gaussian process quadrature methods. The code is available at https://github.com/cagalvisf/Quantum_HSGPQ.

stat.CO↗

arXiv2025

Gaussian multi-target filtering with target dynamics driven by a stochastic differential equation

This paper proposes multi-target filtering algorithms in which target dynamics are given in continuous time and measurements are obtained at discrete time instants. In particular, targets appear according to a Poisson point process (PPP) in time with a given Gaussian spatial distribution, targets move according to a general time-invariant linear stochastic differential equation, and the life span of each target is modelled with an exponential distribution. For this multi-target dynamic model, we derive the distribution of the set of new born targets and calculate closed-form expressions for the best fitting mean and covariance of each target at its time of birth by minimising the Kullback-Leibler divergence via moment matching. This yields a novel Gaussian continuous-discrete Poisson multi-Bernoulli mixture (PMBM) filter, and its approximations based on Poisson multi-Bernoulli and probability hypothesis density filtering. These continuous-discrete multi-target filters are also extended to target dynamics driven by nonlinear stochastic differential equations.

cs.CV↗

arXiv2025

Posterior linearisation smoothing with robust iterations

This paper considers the problem of iterative Bayesian smoothing in nonlinear state-space models with additive noise using Gaussian approximations. Iterative methods are known to improve smoothed estimates but are not guaranteed to converge, motivating the development of methods with better convergence properties. The aim of this article is to extend Levenberg-Marquardt (LM) and line-search versions of the classical iterated extended Kalman smoother (IEKS) to the iterated posterior linearisation smoother (IPLS). The IEKS has previously been shown to be equivalent to the Gauss-Newton (GN) method. We derive a similar GN interpretation for the IPLS and use this to develop extensions to the IPLS, with improved convergence properties. We show that an LM extension for the IPLS can be achieved with a simple modification of the smoothing iterations, enabling algorithms with efficient implementations. We also derive the Armijo--Wolfe step length conditions for the IPLS enabling an efficient inexact line-search method. Our numerical experiments show the benefits of these extensions in highly nonlinear scenarios.

math.OC↗

arXiv2025

Communication-Efficient Distributed Kalman Filtering using ADMM

This paper addresses the problem of optimal linear filtering in a network of local estimators, commonly referred to as distributed Kalman filtering (DKF). The DKF problem is formulated within a distributed optimization framework, where coupling constraints require the exchange of local state and covariance updates between neighboring nodes to achieve consensus. To address these constraints, the problem is transformed into an unconstrained optimization form using the augmented Lagrangian method. The distributed alternating direction method of multipliers (ADMM) is then applied to derive update steps that achieve the desired performance while exchanging only the primal variables. Notably, the proposed method enhances communication efficiency by eliminating the need for dual variable exchange. We show that the design parameters depend on the maximum eigenvalue of the network's Laplacian matrix, yielding a significantly tighter bound compared to existing results. A rigorous convergence analysis is provided, proving that the state estimates converge to the true state and that the covariance matrices across all local estimators converge to a globally optimal solution. Numerical results are presented to validate the efficacy of the proposed approach.

eess.SY↗

arXiv2024

Temporal Parallelisation of the HJB Equation and Continuous-Time Linear Quadratic Control

This paper presents a mathematical formulation to perform temporal parallelisation of continuous-time optimal control problems, which can be solved via the Hamilton--Jacobi--Bellman (HJB) equation. We divide the time interval of the control problem into sub-intervals, and define a control problem in each sub-interval, conditioned on the start and end states, leading to conditional value functions for the sub-intervals. By defining an associative operator as the minimisation of the sum of conditional value functions, we obtain the elements and associative operators for a parallel associative scan operation. This allows for solving the optimal control problem on the whole time interval in parallel in logarithmic time complexity in the number of sub-intervals. We derive the HJB-type of backward and forward equations for the conditional value functions and solve them in closed form for linear quadratic problems. We also discuss numerical methods for computing the conditional value functions. The computational advantages of the proposed parallel methods are demonstrated via simulations run on a multi-core central processing unit and a graphics processing unit.

math.OC↗

arXiv2024

Recursive Nested Filtering for Efficient Amortized Bayesian Experimental Design

This paper introduces the Inside-Out Nested Particle Filter (IO-NPF), a novel, fully recursive, algorithm for amortized sequential Bayesian experimental design in the non-exchangeable setting. We frame policy optimization as maximum likelihood estimation in a non-Markovian state-space model, achieving (at most) $\mathcal{O}(T^2)$ computational complexity in the number of experiments. We provide theoretical convergence guarantees and introduce a backward sampling algorithm to reduce trajectory degeneracy. IO-NPF offers a practical, extensible, and provably consistent approach to sequential Bayesian experimental design, demonstrating improved efficiency over existing methods.

stat.ML↗

arXiv2024

Parallel state estimation for systems with integrated measurements

This paper presents parallel-in-time state estimation methods for systems with Slow-Rate inTegrated Measurements (SRTM). Integrated measurements are common in various applications, and they appear in analysis of data resulting from processes that require material collection or integration over the sampling period. Current state estimation methods for SRTM are inherently sequential, preventing temporal parallelization in their standard form. This paper proposes parallel Bayesian filters and smoothers for linear Gaussian SRTM models. For that purpose, we develop a novel smoother for SRTM models and develop parallel-in-time filters and smoother for them using an associative scan-based parallel formulation. Empirical experiments ran on a GPU demonstrate the superior time complexity of the proposed methods over traditional sequential approaches.

stat.CO↗

arXiv2024

Parallel-in-Time Probabilistic Numerical ODE Solvers

Probabilistic numerical solvers for ordinary differential equations (ODEs) treat the numerical simulation of dynamical systems as problems of Bayesian state estimation. Aside from producing posterior distributions over ODE solutions and thereby quantifying the numerical approximation error of the method itself, one less-often noted advantage of this formalism is the algorithmic flexibility gained by formulating numerical simulation in the framework of Bayesian filtering and smoothing. In this paper, we leverage this flexibility and build on the time-parallel formulation of iterated extended Kalman smoothers to formulate a parallel-in-time probabilistic numerical ODE solver. Instead of simulating the dynamical system sequentially in time, as done by current probabilistic solvers, the proposed method processes all time steps in parallel and thereby reduces the span cost from linear to logarithmic in the number of time steps. We demonstrate the effectiveness of our approach on a variety of ODEs and compare it to a range of both classic and probabilistic numerical ODE solvers.

math.NA↗

arXiv2024

Physics-Informed Machine Learning for Grade Prediction in Froth Flotation

In this paper, physics-informed neural network models are developed to predict the concentrate gold grade in froth flotation cells. Accurate prediction of concentrate grades is important for the automatic control and optimization of mineral processing. Both first-principles and data-driven machine learning methods have been used to model the flotation process. The complexity of models based on first-principles restricts their direct use, while purely data-driven models often fail in dynamic industrial environments, leading to poor generalization. To address these limitations, this study integrates classical mathematical models of froth flotation processes with conventional deep learning methods to construct physics-informed neural networks. These models demonstrated superior generalization and predictive performance compared to purely data-driven models, on simulated data from two flotation cells, in terms of mean squared error and mean relative error.

cs.CE↗

arXiv2024

Nesting Particle Filters for Experimental Design in Dynamical Systems

In this paper, we propose a novel approach to Bayesian experimental design for non-exchangeable data that formulates it as risk-sensitive policy optimization. We develop the Inside-Out SMC$^2$ algorithm, a nested sequential Monte Carlo technique to infer optimal designs, and embed it into a particle Markov chain Monte Carlo framework to perform gradient-based policy amortization. Our approach is distinct from other amortized experimental design techniques, as it does not rely on contrastive estimators. Numerical validation on a set of dynamical systems showcases the efficacy of our method in comparison to other state-of-the-art strategies.

stat.ML↗

arXiv2024

Utilizing U-Net Architectures with Auxiliary Information for Scatter Correction in CBCT Across Different Field-of-View Settings

Cone-beam computed tomography (CBCT) has become a vital imaging technique in various medical fields but scatter artifacts are a major limitation in CBCT scanning. This challenge is exacerbated by the use of large flat panel 2D detectors. The scatter-to-primary ratio increases significantly with the increase in the size of FOV being scanned. Several deep learning methods, particularly U-Net architectures, have shown promising capabilities in estimating the scatter directly from the CBCT projections. However, the influence of varying FOV sizes on these deep learning models remains unexplored. Having a single neural network for the scatter estimation of varying FOV projections can be of significant importance towards real clinical applications. This study aims to train and evaluate the performance of a U-Net network on a simulated dataset with varying FOV sizes. We further propose a new method (Aux-Net) by providing auxiliary information, such as FOV size, to the U-Net encoder. We validate our method on 30 different FOV sizes and compare it with the U-Net. Our study demonstrates that providing auxiliary information to the network enhances the generalization capability of the U-Net. Our findings suggest that this novel approach outperforms the baseline U-Net, offering a significant step towards practical application in real clinical settings where CBCT systems are employed to scan a wide range of FOVs.

eess.SP↗

arXiv2024

Quantum-Assisted Hilbert-Space Gaussian Process Regression

Gaussian processes are probabilistic models that are commonly used as functional priors in machine learning. Due to their probabilistic nature, they can be used to capture the prior information on the statistics of noise, smoothness of the functions, and training data uncertainty. However, their computational complexity quickly becomes intractable as the size of the data set grows. We propose a Hilbert space approximation-based quantum algorithm for Gaussian process regression to overcome this limitation. Our method consists of a combination of classical basis function expansion with quantum computing techniques of quantum principal component analysis, conditional rotations, and Hadamard and Swap tests. The quantum principal component analysis is used to estimate the eigenvalues while the conditional rotations and the Hadamard and Swap tests are employed to evaluate the posterior mean and variance of the Gaussian process. Our method provides polynomial computational complexity reduction over the classical method.

stat.CO↗

arXiv2023

Risk-Sensitive Stochastic Optimal Control as Rao-Blackwellized Markovian Score Climbing

Stochastic optimal control of dynamical systems is a crucial challenge in sequential decision-making. Recently, control-as-inference approaches have had considerable success, providing a viable risk-sensitive framework to address the exploration-exploitation dilemma. Nonetheless, a majority of these techniques only invoke the inference-control duality to derive a modified risk objective that is then addressed within a reinforcement learning framework. This paper introduces a novel perspective by framing risk-sensitive stochastic control as Markovian score climbing under samples drawn from a conditional particle filter. Our approach, while purely inference-centric, provides asymptotically unbiased estimates for gradient-based policy optimization with optimal importance weighting and no explicit value function learning. To validate our methodology, we apply it to the task of learning neural non-Gaussian feedback policies, showcasing its efficacy on numerical benchmarks of stochastic dynamical systems.

cs.LG↗

arXiv2023

Nonparametric modeling of the composite effect of multiple nutrients on blood glucose dynamics

In biomedical applications it is often necessary to estimate a physiological response to a treatment consisting of multiple components, and learn the separate effects of the components in addition to the joint effect. Here, we extend existing probabilistic nonparametric approaches to explicitly address this problem. We also develop a new convolution-based model for composite treatment-response curves that is more biologically interpretable. We validate our models by estimating the impact of carbohydrate and fat in meals on blood glucose. By differentiating treatment components, incorporating their dosages, and sharing statistical information across patients via a hierarchical multi-output Gaussian process, our method improves prediction accuracy over existing approaches, and allows us to interpret the different effects of carbohydrates and fat on the overall glucose response.

stat.ML↗

arXiv2023

Deep learning based projection domain metal segmentation for metal artifact reduction in cone beam computed tomography

Metal artifact correction is a challenging problem in cone beam computed tomography (CBCT) scanning. Metal implants inserted into the anatomy cause severe artifacts in reconstructed images. Widely used inpainting-based metal artifact reduction (MAR) methods require segmentation of metal traces in the projections as a first step, which is a challenging task. One approach is to use a deep learning method to segment metals in the projections. However, the success of deep learning methods is limited by the availability of realistic training data. It is laborious and time consuming to get reliable ground truth annotations due to unclear implant boundaries and large numbers of projections. We propose to use X-ray simulations to generate synthetic metal segmentation training dataset from clinical CBCT scans. We compare the effect of simulations with different numbers of photons and also compare several training strategies to augment the available data. We compare our model's performance on real clinical scans with conventional region growing threshold-based MAR, moving metal artifact reduction method, and a recent deep learning method. We show that simulations with relatively small number of photons are suitable for the metal segmentation task and that training the deep learning model with full size and cropped projections together improves the robustness of the model. We show substantial improvement in the image quality affected by severe motion, voxel size under-sampling, and out-of-FOV metals. Our method can be easily integrated into the existing projection-based MAR pipeline to get improved image quality. This method can provide a novel paradigm to accurately segment metals in CBCT projections.

eess.IV↗