arXiv subjects

Joaquín Míguez

Publications and source records attributed to Joaquín Míguez.

At least 19 recordsLinked to original sources

arXiv2026

An Operator-Theoretic Analysis of Nonlinear Filtering under Model Misspecification

We study nonlinear filtering with misspecified initial distributions and dynamics, but a correctly specified observation model. Observations are treated as a random process generated by the true model, rather than as fixed realizations, and both the correct filter and the misspecified filter are analyzed under this common observation law. Our main result is a Bayesian total variation balance identity for the expected change in total variation distance under a Bayesian update: a covariance term minus a non-negative remainder. The identity characterizes conditional and averaged update non-expansivity. Combined with prediction contraction, it yields finite-time bounds separating the initialization error from accumulated dynamical discrepancies, while allowing expansive updates. When the averaged updates are non-expansive and a uniform joint prediction--update coefficient is less than one, a persistent dynamical misspecification yields an asymptotic error bound proportional to the one-step discrepancy. When the conditional updates are non-expansive, we characterize an accumulated joint contraction that implies almost-sure and $L^1$ coalescence of the true and misspecified filters. Finally, we verify update non-expansivity for scalar Gaussian predictive laws under a variance-ordering condition and, separately, derive explicit prediction contraction bounds for truncated kernels and bounded transition functions with elliptically symmetric noise.

math.ST↗

arXiv2026

Importance sampling for Bayesian inference: polynomial-dimension dependent error bounds

Many Bayesian inference problems involve high-dimensional models where the performance of standard importance sampling (IS) methods often degrades rapidly as the dimensionality increases. Classical analyses of IS typically rely on the assumption that observations are arbitrary but fixed (i.e., deterministic), thereby neglecting the probabilistic structure that the Bayesian model induces on the data. In this paper, we adopt the perspective that observations are themselves random variables whose distribution is governed by the underlying model. Within this probabilistic framework, we identify a model-dependent function, referred to as the link function, which connects the fixed- and random-observation formulations. We provide a characterization of the $L^2$ Monte Carlo estimation error: specifically, we show that the $L^2$ error bounds are finite and converge at the standard Monte Carlo rate $O(N^{-1/2})$, for arbitrarily large dimension, if and only if the link function is Bochner integrable. This result reveals the fundamental quantity controlling the approximation error and establishes a mechanism to manage the dependence on the model state dimension. Consequently, our approach provides a principled way to alleviate the challenges of high dimensionality, offering insights that transcend worst-case analyses dominant in the existing literature. Finally, we derive explicit analytical examples of the dimensional scaling of the associated errors for several model classes, including linear-Gaussian systems and models with bounded observation functions.

math.ST↗

arXiv2025

Explicit Density Approximation for Neural Implicit Samplers Using a Bernstein-Based Convex Divergence

Rank-based statistical metrics, such as the invariant statistical loss (ISL), have recently emerged as robust and practically effective tools for training implicit generative models. In this work, we introduce dual-ISL, a novel likelihood-free objective for training implicit generative models that interchanges the roles of the target and model distributions in the ISL framework, yielding a convex optimization problem in the space of model densities. We prove that the resulting rank-based discrepancy $d_K$ is i) continuous under weak convergence and with respect to the $L^1$ norm, and ii) convex in its first argument-properties not shared by classical divergences such as KL or Wasserstein distances. Building on this, we develop a theoretical framework that interprets $d_K$ as an $L^2$-projection of the density ratio $q = p/\tilde p$ onto a Bernstein polynomial basis, from which we derive exact bounds on the truncation error, precise convergence rates, and a closed-form expression for the truncated density approximation. We further extend our analysis to the multivariate setting via random one-dimensional projections, defining a sliced dual-ISL divergence that retains both convexity and continuity. We empirically show that these theoretical advantages translate into practical ones. Specifically, across several benchmarks dual-ISL converges more rapidly, delivers markedly smoother and more stable training, and more effectively prevents mode collapse than classical ISL and other leading implicit generative methods-while also providing an explicit density approximation.

cs.LG↗

arXiv2025

Robust training of implicit generative models for multivariate and heavy-tailed distributions with an invariant statistical loss

Traditional implicit generative models are capable of learning highly complex data distributions. However, their training involves distinguishing real data from synthetically generated data using adversarial discriminators, which can lead to unstable training dynamics and mode dropping issues. In this work, we build on the \textit{invariant statistical loss} (ISL) method introduced in \cite{de2024training}, and extend it to handle heavy-tailed and multivariate data distributions. The data generated by many real-world phenomena can only be properly characterised using heavy-tailed probability distributions, and traditional implicit methods struggle to effectively capture their asymptotic behavior. To address this problem, we introduce a generator trained with ISL, that uses input noise from a generalised Pareto distribution (GPD). We refer to this generative scheme as Pareto-ISL for conciseness. Our experiments demonstrate that Pareto-ISL accurately models the tails of the distributions while still effectively capturing their central characteristics. The original ISL function was conceived for 1D data sets. When the actual data is $n$-dimensional, a straightforward extension of the method was obtained by targeting the $n$ marginal distributions of the data. This approach is computationally infeasible and ineffective in high-dimensional spaces. To overcome this, we extend the 1D approach using random projections and define a new loss function suited for multivariate data, keeping problems tractable by adjusting the number of projections. We assess its performance in multidimensional generative modeling and explore its potential as a pretraining technique for generative adversarial networks (GANs) to prevent mode collapse, reporting promising results and highlighting its robustness across various hyperparameter settings.

cs.LG↗

arXiv2024

Training Implicit Generative Models via an Invariant Statistical Loss

Implicit generative models have the capability to learn arbitrary complex data distributions. On the downside, training requires telling apart real data from artificially-generated ones using adversarial discriminators, leading to unstable training and mode-dropping issues. As reported by Zahee et al. (2017), even in the one-dimensional (1D) case, training a generative adversarial network (GAN) is challenging and often suboptimal. In this work, we develop a discriminator-free method for training one-dimensional (1D) generative implicit models and subsequently expand this method to accommodate multivariate cases. Our loss function is a discrepancy measure between a suitably chosen transformation of the model samples and a uniform distribution; hence, it is invariant with respect to the true distribution of the data. We first formulate our method for 1D random variables, providing an effective solution for approximate reparameterization of arbitrary complex distributions. Then, we consider the temporal setting (both univariate and multivariate), in which we model the conditional distribution of each sample given the history of the process. We demonstrate through numerical simulations that this new method yields promising results, successfully learning true distributions in a variety of scenarios and mitigating some of the well-known problems that state-of-the-art implicit methods present.

cs.LG↗

arXiv2022

Parallel sequential Monte Carlo for stochastic gradient-free nonconvex optimization

We introduce and analyze a parallel sequential Monte Carlo methodology for the numerical solution of optimization problems that involve the minimization of a cost function that consists of the sum of many individual components. The proposed scheme is a stochastic zeroth order optimization algorithm which demands only the capability to evaluate small subsets of components of the cost function. It can be depicted as a bank of samplers that generate particle approximations of several sequences of probability measures. These measures are constructed in such a way that they have associated probability density functions whose global maxima coincide with the global minima of the original cost function. The algorithm selects the best performing sampler and uses it to approximate a global minimum of the cost function. We prove analytically that the resulting estimator converges to a global minimum of the cost function almost surely and provide explicit convergence rates in terms of the number of generated Monte Carlo samples and the dimension of the search space. We show, by way of numerical examples, that the algorithm can tackle cost functions with multiple minima or with broad "flat" regions which are hard to minimize using gradient-based techniques.

stat.CO↗

arXiv2021

On the performance of particle filters with adaptive number of particles

We investigate the performance of a class of particle filters (PFs) that can automatically tune their computational complexity by evaluating online certain predictive statistics which are invariant for a broad class of state-space models. To be specific, we propose a family of block-adaptive PFs based on the methodology of Elvira et al (2017). In this class of algorithms, the number of Monte Carlo samples (known as particles) is adjusted periodically, and we prove that the theoretical error bounds of the PF actually adapt to the updates in the number of particles. The evaluation of the predictive statistics that lies at the core of the methodology is done by generating fictitious observations, i.e., particles in the observation space. We study, both analytically and numerically, the impact of the number $K$ of these particles on the performance of the algorithm. In particular, we prove that if the predictive statistics with $K$ fictitious observations converged exactly, then the particle approximation of the filtering distribution would match the first $K$ elements in a series of moments of the true filter. This result can be understood as a converse to some convergence theorems for PFs. From this analysis, we deduce an alternative predictive statistic that can be computed (for some models) without sampling any fictitious observations at all. Finally, we conduct an extensive simulation study that illustrates the theoretical results and provides further insights into the complexity, performance and behavior of the new class of algorithms.

stat.CO↗

arXiv2021

Nested Gaussian filters for recursive Bayesian inference and nonlinear tracking in state space models

We introduce a new sequential methodology to calibrate the fixed parameters and track the stochastic dynamical variables of a state-space system. The proposed method is based on the nested hybrid filtering (NHF) framework of [1], that combines two layers of filters, one inside the other, to compute the joint posterior probability distribution of the static parameters and the state variables. In particular, we explore the use of deterministic sampling techniques for Gaussian approximation in the first layer of the algorithm, instead of the Monte Carlo methods employed in the original procedure. The resulting scheme reduces the computational cost and so makes the algorithms potentially better-suited for high-dimensional state and parameter spaces. We describe a specific instance of the new method and then study its performance and efficiency of the resulting algorithms for a stochastic Lorenz 63 model with uncertain parameters.

stat.CO↗

arXiv2020

Convergence rates for optimised adaptive importance samplers

Adaptive importance samplers are adaptive Monte Carlo algorithms to estimate expectations with respect to some target distribution which \textit{adapt} themselves to obtain better estimators over a sequence of iterations. Although it is straightforward to show that they have the same $\mathcal{O}(1/\sqrt{N})$ convergence rate as standard importance samplers, where $N$ is the number of Monte Carlo samples, the behaviour of adaptive importance samplers over the number of iterations has been left relatively unexplored. In this work, we investigate an adaptation strategy based on convex optimisation which leads to a class of adaptive importance samplers termed \textit{optimised adaptive importance samplers} (OAIS). These samplers rely on the iterative minimisation of the $χ^2$-divergence between an exponential-family proposal and the target. The analysed algorithms are closely related to the class of adaptive importance samplers which minimise the variance of the weight function. We first prove non-asymptotic error bounds for the mean squared errors (MSEs) of these algorithms, which explicitly depend on the number of iterations and the number of samples together. The non-asymptotic bounds derived in this paper imply that when the target belongs to the exponential family, the $L_2$ errors of the optimised samplers converge to the optimal rate of $\mathcal{O}(1/\sqrt{N})$ and the rate of convergence in the number of iterations are explicitly provided. When the target does not belong to the exponential family, the rate of convergence is the same but the asymptotic $L_2$ error increases by a factor $\sqrt{ρ^\star} > 1$, where $ρ^\star - 1$ is the minimum $χ^2$-divergence between the target and an exponential-family proposal.

stat.CO↗

arXiv2019

A probabilistic incremental proximal gradient method

In this paper, we propose a probabilistic optimization method, named probabilistic incremental proximal gradient (PIPG) method, by developing a probabilistic interpretation of the incremental proximal gradient algorithm. We explicitly model the update rules of the incremental proximal gradient method and develop a systematic approach to propagate the uncertainty of the solution estimate over iterations. The PIPG algorithm takes the form of Bayesian filtering updates for a state-space model constructed by using the cost function. Our framework makes it possible to utilize well-known exact or approximate Bayesian filters, such as Kalman or extended Kalman filters, to solve large-scale regularized optimization problems.

math.OC↗

arXiv2019

Nudging the particle filter

We investigate a new sampling scheme aimed at improving the performance of particle filters whenever (a) there is a significant mismatch between the assumed model dynamics and the actual system, or (b) the posterior probability tends to concentrate in relatively small regions of the state space. The proposed scheme pushes some particles towards specific regions where the likelihood is expected to be high, an operation known as nudging in the geophysics literature. We re-interpret nudging in a form applicable to any particle filtering scheme, as it does not involve any changes in the rest of the algorithm. Since the particles are modified, but the importance weights do not account for this modification, the use of nudging leads to additional bias in the resulting estimators. However, we prove analytically that nudged particle filters can still attain asymptotic convergence with the same error rates as conventional particle methods. Simple analysis also yields an alternative interpretation of the nudging operation that explains its robustness to model errors. Finally, we show numerical results that illustrate the improvements that can be attained using the proposed scheme. In particular, we present nonlinear tracking examples with synthetic data and a model inference example using real-world financial data.

stat.CO↗

arXiv2018

A probabilistic scheme for joint parameter estimation and state prediction in complex dynamical systems

Many problems in the geophysical sciences demand the ability to calibrate the parameters and predict the time evolution of complex dynamical models using sequentially-collected data. Here we introduce a general methodology for the joint estimation of the static parameters and the forecasting of the state variables of nonlinear, and possibly chaotic, dynamical models. The proposed scheme is essentially probabilistic. It aims at recursively computing the sequence of joint posterior probability distributions of the unknown model parameters and its (time varying) state variables conditional on the available observations. The latter are possibly partial and contaminated by noise. The new framework combines a Monte Carlo scheme to approximate the posterior distribution of the fixed parameters with filtering (or {\em data assimilation}) techniques to track and predict the distribution of the state variables. For this reason, we refer to the proposed methodology as {\em nested filtering}. In this paper we specifically explore the use of Gaussian filtering methods, but other approaches fit naturally within the new framework. As an illustrative example, we apply three different implementations of the methodology to the tracking of the state, and the estimation of the fixed parameters, of a stochastic two-scale Lorenz 96 system. This model is commonly used to assess data assimilation procedures in meteorology. For this example, we compare different nested filters and show estimation and forecasting results for a 4,000-dimensional system.

stat.CO↗

arXiv2017

Adapting the Number of Particles in Sequential Monte Carlo Methods through an Online Scheme for Convergence Assessment

Particle filters are broadly used to approximate posterior distributions of hidden states in state-space models by means of sets of weighted particles. While the convergence of the filter is guaranteed when the number of particles tends to infinity, the quality of the approximation is usually unknown but strongly dependent on the number of particles. In this paper, we propose a novel method for assessing the convergence of particle filters online manner, as well as a simple scheme for the online adaptation of the number of particles based on the convergence assessment. The method is based on a sequential comparison between the actual observations and their predictive probability distributions approximated by the filter. We provide a rigorous theoretical analysis of the proposed methodology and, as an example of its practical use, we present simulations of a simple algorithm for the dynamic and online adaption of the number of particles during the operation of a particle filter on a stochastic version of the Lorenz system.

stat.CO↗

arXiv2017

Importance sampling with transformed weights

The importance sampling (IS) method lies at the core of many Monte Carlo-based techniques. IS allows the approximation of a target probability distribution by drawing samples from a proposal (or importance) distribution, different from the target, and computing importance weights (IWs) that account for the discrepancy between these two distributions. The main drawback of IS schemes is the degeneracy of the IWs, which significantly reduces the efficiency of the method. It has been recently proposed to use transformed IWs (TIWs) to alleviate the degeneracy problem in the context of Population Monte Carlo, which is an iterative version of IS. However, the effectiveness of this technique for standard IS is yet to be investigated. In this letter we numerically assess the performance of IS when using TIWs, and show that the method can attain robustness to weight degeneracy thanks to a bias/variance trade-off.

stat.AP↗

arXiv2016

Particle-kernel estimation of the filter density in state-space models

Sequential Monte Carlo (SMC) methods, also known as particle filters, are simulation-based recursive algorithms for the approximation of the a posteriori probability measures generated by state-space dynamical models. At any given time $t$, a SMC method produces a set of samples over the state space of the system of interest (often termed "particles") that is used to build a discrete and random approximation of the posterior probability distribution of the state variables, conditional on a sequence of available observations. One potential application of the methodology is the estimation of the densities associated to the sequence of a posteriori distributions. While practitioners have rather freely applied such density approximations in the past, the issue has received less attention from a theoretical perspective. In this paper, we address the problem of constructing kernel-based estimates of the posterior probability density function and its derivatives, and obtain asymptotic convergence results for the estimation errors. In particular, we find convergence rates for the approximation errors that hold uniformly on the state space and guarantee that the error vanishes almost surely as the number of particles in the filter grows. Based on this uniform convergence result, we first show how to build continuous measures that converge almost surely (with known rate) toward the posterior measure and then address a few applications. The latter include maximum a posteriori estimation of the system state using the approximate derivatives of the posterior density and the approximation of functionals of it, for example, Shannon's entropy. This manuscript is identical to the published paper, including a gap in the proof of Theorem 4.2. The Theorem itself is correct. We provide an {\em erratum} at the end of this document with a complete proof and a brief discussion.

stat.CO↗

arXiv2014

A comparison of nonlinear population Monte Carlo and particle Markov chain Monte Carlo algorithms for Bayesian inference in stochastic kinetic models

In this paper we address the problem of Monte Carlo approximation of posterior probability distributions in stochastic kinetic models (SKMs). SKMs are multivariate Markov jump processes that model the interactions among species in biochemical systems according to a set of uncertain parameters. Markov chain Monte Carlo (MCMC) methods have been typically preferred for this Bayesian inference problem. Specifically, the particle MCMC (pMCMC) method has been recently shown to be an effective, while computationally demanding, method applicable to this problem. Within the pMCMC framework, importance sampling (IS) has been used only as the basis of the sequential Monte Carlo (SMC) approximation of the acceptance ratio in the Metropolis-Hastings kernel. However, the recently proposed nonlinear population Monte Carlo (NPMC) algorithm, based on an iterative IS scheme, has also been shown to be effective as a Bayesian inference tool for low dimensional (predator-prey) SKMs. In this paper, we provide an extensive performance comparison of pMCMC versus NPMC, when applied to the challenging prokaryotic autoregulatory network. We show how the NPMC method can greatly outperform the pMCMC algorithm in this scenario, with an overall moderate computational effort. We complement the numerical comparison of the two techniques with an asymptotic convergence analysis of the nonlinear IS scheme at the core of the proposed method when the importance weights can only be computed approximately.

stat.ME↗

arXiv2013

On the Generalized Ratio of Uniforms as a Combination of Transformed Rejection and Extended Inverse of Density Sampling

In this work we investigate the relationship among three classical sampling techniques: the inverse of density (Khintchine's theorem), the transformed rejection (TR) and the generalized ratio of uniforms (GRoU). Given a monotonic probability density function (PDF), we show that the transformed area obtained using the generalized ratio of uniforms method can be found equivalently by applying the transformed rejection sampling approach to the inverse function of the target density. Then we provide an extension of the classical inverse of density idea, showing that it is completely equivalent to the GRoU method for monotonic densities. Although we concentrate on monotonic probability density functions (PDFs), we also discuss how the results presented here can be extended to any non-monotonic PDF that can be decomposed into a collection of intervals where it is monotonically increasing or decreasing. In this general case, we show the connections with transformations of certain random variables and the generalized inverse PDF with the GRoU technique. Finally, we also introduce a GRoU technique to handle unbounded target densities.

stat.CO↗

arXiv2012

A population Monte Carlo scheme with transformed weights and its application to stochastic kinetic models

This paper addresses the problem of Monte Carlo approximation of posterior probability distributions. In particular, we have considered a recently proposed technique known as population Monte Carlo (PMC), which is based on an iterative importance sampling approach. An important drawback of this methodology is the degeneracy of the importance weights when the dimension of either the observations or the variables of interest is high. To alleviate this difficulty, we propose a novel method that performs a nonlinear transformation on the importance weights. This operation reduces the weight variation, hence it avoids their degeneracy and increases the efficiency of the importance sampling scheme, specially when drawing from a proposal functions which are poorly adapted to the true posterior. For the sake of illustration, we have applied the proposed algorithm to the estimation of the parameters of a Gaussian mixture model. This is a very simple problem that enables us to clearly show and discuss the main features of the proposed technique. As a practical application, we have also considered the popular (and challenging) problem of estimating the rate parameters of stochastic kinetic models (SKM). SKMs are highly multivariate systems that model molecular interactions in biological and chemical problems. We introduce a particularization of the proposed algorithm to SKMs and present numerical results.

stat.CO↗