arXiv subjects

Raúl Tempone

Publications and source records attributed to Raúl Tempone.

At least 19 recordsLinked to original sources

arXiv2026

On the sample complexity of the active subspace method

Active subspaces identify low-dimensional linear structure in high-dimensional parameter-to-output maps by estimating the dominant eigenspace of a gradient covariance operator. In practice this covariance is replaced by a Monte Carlo estimator built from a limited number of gradient evaluations. Classical analyses based on controlling the covariance error in operator norm lead to sample-complexity estimates that can be substantially more pessimistic than the sampling rules commonly used in computations. This paper studies the empirical active subspace method directly in the projection-error metric relevant for ridge approximation. We derive non-asymptotic quasi-optimality bounds governed by a regularized inverse Christoffel function associated with the gradient field. Under a bounded-gradient assumption, the resulting estimates already improve the sample-complexity estimates obtained from operator-norm covariance bounds. We then show that additional smoothness of the gradient map, expressed through membership in a reproducing kernel Hilbert space, yields sharper coherence estimates and motivates tractable importance sampling from kernel diagonal measures. Furthermore, the same smoothness assumption yields a priori decay bounds for the population active subspace tail energy, which can be combined with our finite-sample estimate to prescribe rank, regularization scale, and sample size, allowing to fully characterize the a priori sample complexity. The abstract assumptions are verified for lognormal Gaussian and affine uniform parametric elliptic PDEs using weighted summability of Hermite and Legendre series expansions.

math.NA↗

arXiv2026

Multi-index importance sampling for McKean--Vlasov stochastic differential equations

This work addresses the estimation of rare-event quantities expressed as expectations of smooth observables of solutions to a broad class of McKean--Vlasov stochastic differential equations (MV-SDEs). Building on the double loop Monte Carlo (DLMC) method with stochastic optimal control-based importance sampling (IS) introduced by Ben Rached et al. (2024a), this work extends this framework to the multi-index Monte Carlo (MIMC) setting. The resulting multi-index DLMC estimator mitigates the explosion of the coefficient of variation for rare event quantities. Moreover, it exploits the sampling efficiency of MIMC by leveraging the propagation of chaos to ensure mixed-difference variances vanish in the mean-field limit. The complexity analysis relies on assumptions on mixed-difference bias and variance decay, similar to standard MIMC assumptions. Although not rigorously proved, this work presents strong numerical evidence in support of these assumptions. The primary contribution of this work is the novel numerical integration of the MIMC method with IS for MV-SDEs. This approach reduces the computational complexity from $\mathcal{O}(\mathrm{TOL}_{\mathrm{r}}^{-4})$ for the DLMC estimator to $\mathcal{O}(\mathrm{TOL}_{\mathrm{r}}^{-2} (\log \mathrm{TOL}_{\mathrm{r}}^{-1})^2)$, enabling an accurate estimation of rare-event quantities within a prescribed relative error tolerance $\mathrm{TOL}_{\mathrm{r}}$. Numerical experiments on the Kuramoto model from statistical physics demonstrate computational savings of several orders of magnitude for the multi-index DLMC estimator with IS, compared with the standard Monte Carlo (MC) method.

math.NA↗

arXiv2026

Goal-Oriented Adaptive Finite Element Multilevel Quasi-Monte Carlo

The efficient approximation of quantities of interest derived from PDEs with lognormal diffusivity is a central challenge in uncertainty quantification. This paper targets a problem class that combines four analytical difficulties: a geometric boundary singularity, a lognormal coefficient field without a deterministic positive lower bound, sample-dependent mesh selection that introduces parameter-space discontinuities, and infinitely many discontinuity locations that preclude classical pre-integration smoothing. In this study, we propose a multilevel quasi-Monte Carlo framework to approximate deterministic, real-valued, bounded linear functionals that depend on the solution of a linear elliptic PDE with a lognormal diffusivity coefficient parameterized by a multi-dimensional Gaussian random vector and deterministic geometric singularities in bounded domains of $\mathbb{R}^d$. We analyze the parametric regularity and develop the multilevel implementation based on a sequence of adaptive meshes, developed in our earlier work "Goal-oriented adaptive finite element multilevel Monte Carlo with convergence rates", CMAME, 402 (2022), p. 115582. For further variance reduction, we incorporate importance sampling and introduce a level-0 control variate within the multilevel hierarchy. Introducing such a control variate can alter the optimal choice for the initial mesh, further highlighting the advantages of adaptive meshes. On a 2-D slit benchmark discretized with bilinear, quadrilateral Q1-FEM, numerical experiments show that, in the parameter range explored, the proposed adaptive MLQMC algorithm achieves a prescribed accuracy at markedly lower computational cost than a standard multilevel Monte Carlo estimator on the same mesh hierarchy.

math.NA↗

arXiv2026

Pontryagin-Based Solver with Smoothed Hamiltonian, Adaptive $Δt$, and PA-Bundle Refinement

We present a Pontryagin-based numerical solver for deterministic optimal control problems in Bolza form. The solver regularizes the generally nonsmooth Hamiltonian using a log-sum-exp smoothing of a concave piecewise-affine bundle surrogate, yielding a smoothed Hamiltonian $H_δ$ that is $C^\infty$ and concave in the costate and $C^{1,1}$ in the state. The resulting two-point boundary value problem is discretized by a symplectic Euler scheme and solved by damped Newton iteration on the full-space nonlinear system in the discrete state and costate variables. A unified adaptive outer loop jointly controls the time-step distribution $Δt$, the number of planes in the bundle surrogate, and the smoothing parameter $δ$. The time-adaptation strategy follows the error-density framework of Karlsson, Larsson, Sandberg, and Tempone (2015). Related approaches include Pontryagin shooting, Hamilton--Jacobi--Bellman methods, occupation-measure relaxations, max-plus approximations, and direct collocation. Numerical examples, including nonsmooth, singular, hypersensitive, and quadratic-programming-oracle benchmarks, demonstrate the roles of time refinement, bundle enrichment, and smoothing continuation. A linear--quadratic regulator with a closed-form Riccati solution is used as a calibration check.

math.OC↗

arXiv2026

Multilevel randomized quasi-Monte Carlo estimator for nested integration

Nested integration problems arise in various scientific and engineering applications, including Bayesian experimental design, financial risk assessment, and uncertainty quantification. These nested integrals take the form $\int f\left(\int g(\boldsymbol{y},\boldsymbol{x})\mathrm{d}\boldsymbol{x}\right)\mathrm{d}\boldsymbol{y}$, for nonlinear $f$, making them computationally challenging, particularly in high-dimensional settings. Although widely used for single integrals, traditional Monte Carlo (MC) methods can be inefficient when encountering complexities of nested integration. This work introduces a novel multilevel estimator, combining deterministic and randomized quasi-MC (rQMC) methods to handle nested integration problems efficiently. In this context, the inner number of samples and the discretization accuracy of the inner integrand evaluation constitute the level. We provide a comprehensive theoretical analysis of the estimator, deriving error bounds demonstrating significant reductions in bias and variance compared with standard methods. The proposed estimator is particularly effective in scenarios where the integrand is evaluated approximately, as it adapts to different levels of resolution without compromising precision. We verify the performance of our method via numerical experiments, focusing on estimating the expected information gain of experiments. When applied to Gaussian noise in the experiment, a truncation scheme ensures finite error bounds. The results reveal that the proposed multilevel rQMC estimator outperforms existing MC and rQMC approaches, offering a substantial reduction in computational costs and offering a powerful tool for practitioners dealing with complex, nested integration problems across various domains.

math.NA↗

arXiv2026

Double-loop randomized quasi-Monte Carlo estimator for nested integration

Nested integration of the form $\int f\left(\int g(\bs{y},\bs{x})\di{}\bs{x}\right)\di{}\bs{y}$, characterized by an outer integral connected to an inner integral through a nonlinear function $f$, is a challenging problem in various fields, such as engineering and mathematical finance. The available numerical methods for nested integration based on Monte Carlo (MC) methods can be prohibitively expensive owing to the error propagating from the inner to the outer integral. Attempts to enhance the efficiency of these approximations using the quasi-MC (QMC) or randomized QMC (rQMC) method have focused on either the inner or outer integral approximation. This work introduces a novel nested rQMC method that simultaneously addresses the approximation of the inner and outer integrals. The method leverages the unique nested integral structure to offer a more efficient approximation mechanism. As the primary contribution, we derive asymptotic error bounds for the bias and variance of our estimator, along with the regularity conditions under which these bounds can be attained. Incorporating Owen's scrambling techniques, we address integrands exhibiting infinite variation in the Hardy--Krause sense, enabling theoretically sound error estimates. Moreover, we derive a truncation scheme for applications in the context of expected information gain estimation. We verify the estimator quality through numerical experiments by comparing the computational efficiency of the nested rQMC method against standard nested MC estimation to highlight the computational savings and enhanced applicability of the proposed approach.

math.NA↗

arXiv2026

Convergence for adaptive resampling of random Fourier features

The machine learning random Fourier feature method for data in high dimension is computationally and theoretically attractive since the optimization is based on a convex standard least squares problem and independent sampling of Fourier frequencies. The challenge is to sample the Fourier frequencies well. This work proves convergence of a data adaptive method based on resampling the frequencies asymptotically optimally, as the number of nodes and amount of data tend to infinity. Numerical results based on resampling and adaptive random walk steps together with approximations of the least squares problem by conjugate gradient iterations confirm the analysis for regression and classification problems.

math.NA↗

arXiv2026

Quasi-Monte Carlo with a Hankel random digital net

This paper proposes a new randomized design of digital nets in which the generating matrices are chosen to be random Hankel matrices. Compared with previous randomized designs of digital nets, this approach simplifies the construction process and reduces the number of random variables required, while still achieving desirable convergence rates when combined with appropriate estimators. We analyze the properties of the proposed design, derive bounds for Walsh coefficients, and provide error analysis for both the median-of-means estimator and a newly proposed greedy selection estimator, i.e. the selection of the best design from a batch in terms of a worst-case error bound. Numerical experiments validate our theoretical findings and demonstrate the practical performance of the proposed methods.

math.NA↗

arXiv2026

PurSAMERE: Reliable Adversarial Purification via Sharpness-Aware Minimization of Expected Reconstruction Error

We propose a novel deterministic purification method to improve adversarial robustness by mapping a potentially adversarial sample toward a nearby sample that lies close to a mode of the data distribution, where classifiers are more reliable. We design the method to be deterministic to ensure reliable test accuracy and to prevent the degradation of effective robustness observed in stochastic purification approaches when the adversary has full knowledge of the system and its randomness. We employ a score model trained by minimizing the expected reconstruction error of noise-corrupted data, thereby learning the structural characteristics of the input data distribution. Given a potentially adversarial input, the method searches within its local neighborhood for a purified sample that minimizes the expected reconstruction error under noise corruption and then feeds this purified sample to the classifier. During purification, sharpness-aware minimization is used to guide the purified samples toward flat regions of the expected reconstruction error landscape, thereby enhancing robustness. We further show that, as the noise level decreases, minimizing the expected reconstruction error biases the purified sample toward local maximizers of the Gaussian-smoothed density; under additional local assumptions on the score model, we prove recovery of a local maximizer in the small-noise limit. Experimental results demonstrate significant gains in adversarial robustness over state-of-the-art methods under strong deterministic white-box attacks.

cs.LG↗

arXiv2025

Filtered Markovian Projection: Dimensionality Reduction in Filtering for Stochastic Reaction Networks

Stochastic reaction networks (SRNs) model stochastic effects for various applications, including intracellular chemical or biological processes and epidemiology. A typical challenge in practical problems modeled by SRNs is that only a few state variables can be dynamically observed. Given the measurement trajectories, one can estimate the conditional probability distribution of unobserved (hidden) state variables by solving a stochastic filtering problem. In this setting, the conditional distribution evolves over time according to an extensive or potentially infinite-dimensional system of coupled ordinary differential equations with jumps, known as the filtering equation. The current numerical filtering techniques, such as the filtered finite state projection (D'Ambrosio et al., 2022), are hindered by the curse of dimensionality, significantly affecting their computational performance. To address these limitations, we propose to use a dimensionality reduction technique based on the Markovian projection (MP), initially introduced for forward problems (Ben Hammouda et al., 2024). In this work, we explore how to adapt the existing MP approach to the filtering problem and introduce a novel version of the MP, the Filtered MP, that guarantees the consistency of the resulting estimator. The novel consistent MP filter employs a reduced-variance particle filter for estimating the jump intensities of the projected model and solves the filtering equations in a low-dimensional space. The analysis and empirical results highlight the superior computational efficiency of projection methods compared to the existing filtered finite state projection in the large dimensional setting.

math.NA↗

arXiv2025

Efficient Stochastic BFGS methods Inspired by Bayesian Principles

Quasi-Newton methods are ubiquitous in deterministic local search due to their efficiency and low computational cost. This class of methods uses the history of gradient evaluations to approximate second-order derivatives. However, only noisy gradient observations are accessible in stochastic optimization; thus, deriving quasi-Newton methods in this setting is challenging. Although most existing quasi-Newton methods for stochastic optimization rely on deterministic equations that are modified to circumvent noise, we propose a new approach inspired by Bayesian inference to assimilate noisy gradient information and derive the stochastic counterparts to standard quasi-Newton methods. We focus on the derivations of stochastic BFGS and L-BFGS, but our methodology can also be employed to derive stochastic analogs of other quasi-Newton methods. The resulting stochastic BFGS (S-BFGS) and stochastic L-BFGS (L-S-BFGS) can effectively learn an inverse Hessian approximation even with small batch sizes. For a problem of dimension $d$, the iteration cost of S-BFGS is $\mathcal{O}(d^2)$, and the cost of L-S-BFGS is $\mathcal{O}(d)$. Numerical experiments with a dimensionality of up to $30,720$ demonstrate the efficiency and robustness of the proposed method.

math.OC↗

arXiv2025

An Adaptive Random Fourier Features approach Applied to Learning Stochastic Differential Equations

This work proposes a training algorithm based on adaptive random Fourier features (ARFF) with Metropolis sampling and resampling \cite{kammonen2024adaptiverandomfourierfeatures} for learning drift and diffusion components of stochastic differential equations from snapshot data. Specifically, this study considers Itô diffusion processes and a likelihood-based loss function derived from the Euler-Maruyama integration introduced in \cite{Dietrich2023} and \cite{dridi2021learningstochasticdynamicalsystems}. This work evaluates the proposed method against benchmark problems presented in \cite{Dietrich2023}, including polynomial examples, underdamped Langevin dynamics, a stochastic susceptible-infected-recovered model, and a stochastic wave equation. Across all cases, the ARFF-based approach matches or surpasses the performance of conventional Adam-based optimization in both loss minimization and convergence speed. These results highlight the potential of ARFF as a compelling alternative for data-driven modeling of stochastic dynamics.

cs.LG↗

arXiv2025

Adaptive Random Fourier Features Training Stabilized By Resampling With Applications in Image Regression

This paper presents an enhanced adaptive random Fourier features (ARFF) training algorithm for shallow neural networks, building upon the work introduced in "Adaptive Random Fourier Features with Metropolis Sampling", Kammonen et al., \emph{Foundations of Data Science}, 2(3):309--332, 2020. This improved method uses a particle filter-type resampling technique to stabilize the training process and reduce the sensitivity to parameter choices. The Metropolis test can also be omitted when resampling is used, reducing the number of hyperparameters by one and reducing the computational cost per iteration compared to the ARFF method. We present comprehensive numerical experiments demonstrating the efficacy of the proposed algorithm in function regression tasks as a stand-alone method and as a pretraining step before gradient-based optimization, using the Adam optimizer. Furthermore, we apply the proposed algorithm to a simple image regression problem, illustrating its utility in sampling frequencies for the random Fourier features (RFF) layer of coordinate-based multilayer perceptrons. In this context, we use the proposed algorithm to sample the parameters of the RFF layer in an automated manner.

cs.LG↗

arXiv2025

Sparse Bayesian Learning for Label Efficiency in Cardiac Real-Time MRI

Cardiac real-time magnetic resonance imaging (MRI) is an emerging technology that images the heart at up to 50 frames per second, offering insight into the respiratory effects on the heartbeat. However, this method significantly increases the number of images that must be segmented to derive critical health indicators. Although neural networks perform well on inner slices, predictions on outer slices are often unreliable. This work proposes sparse Bayesian learning (SBL) to predict the ventricular volume on outer slices with minimal manual labeling to address this challenge. The ventricular volume over time is assumed to be dominated by sparse frequencies corresponding to the heart and respiratory rates. Moreover, SBL identifies these sparse frequencies on well-segmented inner slices by optimizing hyperparameters via type -II likelihood, automatically pruning irrelevant components. The identified sparse frequencies guide the selection of outer slice images for labeling, minimizing posterior variance. This work provides performance guarantees for the greedy algorithm. Testing on patient data demonstrates that only a few labeled images are necessary for accurate volume prediction. The labeling procedure effectively avoids selecting inefficient images. Furthermore, the Bayesian approach provides uncertainty estimates, highlighting unreliable predictions (e.g., when choosing suboptimal labels).

stat.ME↗

arXiv2025

Optimal power procurement for green cellular wireless networks under uncertainty and chance constraints

Given the increasing global emphasis on sustainable energy usage and the rising energy demands of cellular wireless networks, this work seeks an optimal short-term, continuous-time power procurement schedule to minimize operating expenditure and the carbon footprint of cellular wireless networks equipped with energy storage capacity, and hybrid energy systems comprising uncertain renewable energy sources. Despite the stochastic nature of wireless fading channels, the network operator must ensure a certain quality-of-service (QoS) constraint with high probability. This probabilistic constraint prevents using the dynamic programming principle to solve the stochastic optimal control problem. This work introduces a novel time-continuous Lagrangian relaxation approach tailored for real-time, near-optimal energy procurement in cellular networks, overcoming tractability problems associated with the probabilistic QoS constraint. The numerical solution procedure includes an efficient upwind finite-difference solver for the Hamilton--Jacobi--Bellman equation corresponding to the relaxed problem, and an effective combination of the limited memory bundle method (LMBM) for handling nonsmooth optimization and the stochastic subgradient method (SSM) to navigate the stochasticity of the dual problem. Numerical results, based on the German power system and daily cellular traffic data, demonstrate the computational efficiency of the proposed numerical approach, providing a near-optimal policy in a practical timeframe.

math.OC↗

arXiv2024

Laplace-based strategies for Bayesian optimal experimental design with nuisance uncertainty

Finding the optimal design of experiments in the Bayesian setting typically requires estimation and optimization of the expected information gain functional. This functional consists of one outer and one inner integral, separated by the logarithm function applied to the inner integral. When the mathematical model of the experiment contains uncertainty about the parameters of interest and nuisance uncertainty, (i.e., uncertainty about parameters that affect the model but are not themselves of interest to the experimenter), two inner integrals must be estimated. Thus, the already considerable computational effort required to determine good approximations of the expected information gain is increased further. The Laplace approximation has been applied successfully in the context of experimental design in various ways, and we propose two novel estimators featuring the Laplace approximation to alleviate the computational burden of both inner integrals considerably. The first estimator applies Laplace's method followed by a Laplace approximation, introducing a bias. The second estimator uses two Laplace approximations as importance sampling measures for Monte Carlo approximations of the inner integrals. Both estimators use Monte Carlo approximation for the remaining outer integral estimation. We provide four numerical examples demonstrating the applicability and effectiveness of our proposed estimators.

math.NA↗

arXiv2024

Forward Propagation of Low Discrepancy Through McKean-Vlasov Dynamics: From QMC to MLQMC

This work develops a particle system addressing the approximation of McKean-Vlasov stochastic differential equations (SDEs). The novelty of the approach lies in involving low discrepancy sequences nontrivially in the construction of a particle system with coupled noise and initial conditions. Weak convergence for SDEs with additive noise is proven. A numerical study demonstrates that the novel approach presented here doubles the respective convergence rates for weak and strong approximation of the mean-field limit, compared with the standard particle system. These rates are proven in the simplified setting of a mean-field ordinary differential equation in terms of appropriate bounds involving the star discrepancy for low discrepancy sequences with a group structure, such as Rank-1 lattice points. This construction nontrivially provides an antithetic multilevel quasi-Monte Carlo estimator. An asymptotic error analysis reveals that the proposed approach outperforms methods based on the classic particle system with independent initial conditions and noise.

math.NA↗

arXiv2024

Nonasymptotic Convergence Rate of Quasi-Monte Carlo: Applications to Linear Elliptic PDEs with Lognormal Coefficients and Importance Samplings

This study analyzes the nonasymptotic convergence behavior of the quasi-Monte Carlo (QMC) method with applications to linear elliptic partial differential equations (PDEs) with lognormal coefficients. Building upon the error analysis presented in (Owen, 2006), we derive a nonasymptotic convergence estimate depending on the specific integrands, the input dimensionality, and the finite number of samples used in the QMC quadrature. We discuss the effects of the variance and dimensionality of the input random variable. Then, we apply the QMC method with importance sampling (IS) to approximate deterministic, real-valued, bounded linear functionals that depend on the solution of a linear elliptic PDE with a lognormal diffusivity coefficient in bounded domains of $\mathbb{R}^d$, where the random coefficient is modeled as a stationary Gaussian random field parameterized by the trigonometric and wavelet-type basis. We propose two types of IS distributions, analyze their effects on the QMC convergence rate, and observe the improvements.

math.NA↗