arXiv subjects

Mikołaj Kasprzak

Publications and source records attributed to Mikołaj Kasprzak.

8 recordsLinked to original sources

arXiv2026

Minimax estimation of the pseudo-spectral gap of continuous Markov chains, with applications to empirical concentration bounds

We study the minimax estimation of the pseudo-spectral gap $γ_{ps}$ of a Markov chain from the observation of a single trajectory. Extending earlier works on finite-state space chains by Wolfer and Kontorovich (2024), we provide an estimator in the case of a compact, continuous state-space. Our estimator appears to be minimax-optimal on the class of chains with a transition operator with Lipschitz density, up to logarithmic terms. Our estimator, combined with Paulin's version of Bernstein inequality, allows to derive empirical concentration inequalities for Markov chains. This unlocks many applications, from confidence intervals, to active learning and generalization bounds. In particular, for rested bandits with Markov rewards, we provide a version of UCB with a regret bound similar to the one in the iid case, without assuming any knowledge on the pseudo-spectral gap of the chains.

stat.ML↗

arXiv2023

A Fourier representation of kernel Stein discrepancy with application to Goodness-of-Fit tests for measures on infinite dimensional Hilbert spaces

Kernel Stein discrepancy (KSD) is a widely used kernel-based measure of discrepancy between probability measures. It is often employed in the scenario where a user has a collection of samples from a candidate probability measure and wishes to compare them against a specified target probability measure. KSD has been employed in a range of settings including goodness-of-fit testing, parametric inference, MCMC output assessment and generative modelling. However, so far the method has been restricted to finite-dimensional data. We provide the first analysis of KSD in the generality of data lying in a separable Hilbert space, for example functional data. The main result is a novel Fourier representation of KSD obtained by combining the theory of measure equations with kernel methods. This allows us to prove that KSD can separate measures and thus is valid to use in practice. Additionally, our results improve the interpretability of KSD by decoupling the effect of the kernel and Stein operator. We demonstrate the efficacy of the proposed methodology by performing goodness-of-fit tests for various Gaussian and non-Gaussian functional models in a number of synthetic data experiments.

math.ST↗

arXiv2023

A Targeted Accuracy Diagnostic for Variational Approximations

Variational Inference (VI) is an attractive alternative to Markov Chain Monte Carlo (MCMC) due to its computational efficiency in the case of large datasets and/or complex models with high-dimensional parameters. However, evaluating the accuracy of variational approximations remains a challenge. Existing methods characterize the quality of the whole variational distribution, which is almost always poor in realistic applications, even if specific posterior functionals such as the component-wise means or variances are accurate. Hence, these diagnostics are of practical value only in limited circumstances. To address this issue, we propose the TArgeted Diagnostic for Distribution Approximation Accuracy (TADDAA), which uses many short parallel MCMC chains to obtain lower bounds on the error of each posterior functional of interest. We also develop a reliability check for TADDAA to determine when the lower bounds should not be trusted. Numerical experiments validate the practical utility and computational efficiency of our approach on a range of synthetic distributions and real-data examples, including sparse logistic regression and Bayesian neural network models.

stat.ML↗

arXiv2022

The multivariate functional de Jong CLT

We prove a multivariate functional version of de Jong's CLT (1990) yielding that, given a sequence of vectors of Hoeffding-degenerate U-statistics, the corresponding empirical processes on $[0,1]$ weakly converge in the Skorohod space as soon as their fourth cumulants in $t=1$ vanish asymptotically and a certain strengthening of the Lindeberg-type condition is verified. As an application, we lift to the functional level the `universality of Wiener chaos' phenomenon first observed in Nourdin, Peccati and Reinert (2010).

math.PR↗

arXiv2022

Functional Convergence of Sequential U-processes with Size-Dependent Kernels

We consider sequences of $U$-processes based on symmetric kernels of a fixed order, that possibly depend on the sample size. Our main contribution is the derivation of a set of analytic sufficient conditions, under which the aforementioned $U$-processes weakly converge to a linear combination of time-changed independent Brownian motions. In view of the underlying symmetric structure, the involved time-changes and weights remarkably depend only on the order of the U-statistic, and have consequently a universal nature. Checking these sufficient conditions requires calculations that have roughly the same complexity as those involved in the computation of fourth moments and cumulants. As such, when applied to the degenerate case, our findings are infinite-dimensional extensions of the central limit theorems (CLTs) proved in de Jong (1990) and Döbler and Peccati (2017). As important tools in our analysis, we exploit the multidimensional central limit theorems established in Döbler and Peccati (2019) together with upper bounds on absolute moments of degenerate $U$-statistics by Ibragimov and Sharakhmetov (2002), and also prove some novel multiplication formulae for degenerate symmetric $U$-statistics -- allowing for different sample sizes -- that are of independent interest. We provide applications to random geometric graphs and to a class of $U$-statistics of order two, whose Gaussian fluctuations have been recently studied by Robins et al. (2016), in connection with quadratic estimators in non-parametric models. In particular, our application to random graphs yields a class of new functional central limit theorems for subgraph counting statistics, extending previous findings in the literature. Finally, some connections with invariance principles in changepoint analysis are established.

math.PR↗

arXiv2020

Validated Variational Inference via Practical Posterior Error Bounds

Variational inference has become an increasingly attractive fast alternative to Markov chain Monte Carlo methods for approximate Bayesian inference. However, a major obstacle to the widespread use of variational methods is the lack of post-hoc accuracy measures that are both theoretically justified and computationally efficient. In this paper, we provide rigorous bounds on the error of posterior mean and uncertainty estimates that arise from full-distribution approximations, as in variational inference. Our bounds are widely applicable, as they require only that the approximating and exact posteriors have polynomial moments. Our bounds are also computationally efficient for variational inference because they require only standard values from variational objectives, straightforward analytic calculations, and simple Monte Carlo estimates. We show that our analysis naturally leads to a new and improved workflow for validated variational inference. Finally, we demonstrate the utility of our proposed workflow and error bounds on a robust regression problem and on a real-data example with a widely used multilevel hierarchical model.

stat.ML↗

arXiv2019

Scalable Gaussian Process Inference with Finite-data Mean and Variance Guarantees

Gaussian processes (GPs) offer a flexible class of priors for nonparametric Bayesian regression, but popular GP posterior inference methods are typically prohibitively slow or lack desirable finite-data guarantees on quality. We develop an approach to scalable approximate GP regression with finite-data guarantees on the accuracy of pointwise posterior mean and variance estimates. Our main contribution is a novel objective for approximate inference in the nonparametric setting: the preconditioned Fisher (pF) divergence. We show that unlike the Kullback--Leibler divergence (used in variational inference), the pF divergence bounds the 2-Wasserstein distance, which in turn provides tight bounds the pointwise difference of the mean and variance functions. We demonstrate that, for sparse GP likelihood approximations, we can minimize the pF divergence efficiently. Our experiments show that optimizing the pF divergence has the same computational requirements as variational sparse GPs while providing comparable empirical performance--in addition to our novel finite-data quality guarantees.

stat.ML↗

arXiv2018

Practical bounds on the error of Bayesian posterior approximations: A nonasymptotic approach

Bayesian inference typically requires the computation of an approximation to the posterior distribution. An important requirement for an approximate Bayesian inference algorithm is to output high-accuracy posterior mean and uncertainty estimates. Classical Monte Carlo methods, particularly Markov Chain Monte Carlo, remain the gold standard for approximate Bayesian inference because they have a robust finite-sample theory and reliable convergence diagnostics. However, alternative methods, which are more scalable or apply to problems where Markov Chain Monte Carlo cannot be used, lack the same finite-data approximation theory and tools for evaluating their accuracy. In this work, we develop a flexible new approach to bounding the error of mean and uncertainty estimates of scalable inference algorithms. Our strategy is to control the estimation errors in terms of Wasserstein distance, then bound the Wasserstein distance via a generalized notion of Fisher distance. Unlike computing the Wasserstein distance, which requires access to the normalized posterior distribution, the Fisher distance is tractable to compute because it requires access only to the gradient of the log posterior density. We demonstrate the usefulness of our Fisher distance approach by deriving bounds on the Wasserstein error of the Laplace approximation and Hilbert coresets. We anticipate that our approach will be applicable to many other approximate inference methods such as the integrated Laplace approximation, variational inference, and approximate Bayesian computation

math.ST↗