arXiv subjects

Raúl Tempone

Publications and source records attributed to Raúl Tempone.

At least 37 records · Page 2Linked to original sources

arXiv2024

Nonasymptotic Convergence Rate of Quasi-Monte Carlo: Applications to Linear Elliptic PDEs with Lognormal Coefficients and Importance Samplings

This study analyzes the nonasymptotic convergence behavior of the quasi-Monte Carlo (QMC) method with applications to linear elliptic partial differential equations (PDEs) with lognormal coefficients. Building upon the error analysis presented in (Owen, 2006), we derive a nonasymptotic convergence estimate depending on the specific integrands, the input dimensionality, and the finite number of samples used in the QMC quadrature. We discuss the effects of the variance and dimensionality of the input random variable. Then, we apply the QMC method with importance sampling (IS) to approximate deterministic, real-valued, bounded linear functionals that depend on the solution of a linear elliptic PDE with a lognormal diffusivity coefficient in bounded domains of $\mathbb{R}^d$, where the random coefficient is modeled as a stationary Gaussian random field parameterized by the trigonometric and wavelet-type basis. We propose two types of IS distributions, analyze their effects on the QMC convergence rate, and observe the improvements.

math.NA↗

arXiv2024

Weak convergence analysis in the particle limit of the McKean--Vlasov equations using stochastic flows of particle systems

We present a proof showing that the weak error of a system of $n$ interacting stochastic particles approximating the solution of the McKean-Vlasov equation is $\mathcal O(n^{-1})$. Our proof is based on the Kolmogorov backward equation for the particle system and bounds on the derivatives of its solution, which we derive more generally using the variations of the stochastic particle system. The convergence rate is verified by numerical experiments, which also indicate that the assumptions made here and in the literature can be relaxed.

math.PR↗

arXiv2024

Data-driven uncertainty quantification for constrained stochastic differential equations and application to solar photovoltaic power forecast data

In this work, we extend the data-driven Itô stochastic differential equation (SDE) framework for the pathwise assessment of short-term forecast errors to account for the time-dependent upper bound that naturally constrains the observable historical data and forecast. We propose a new nonlinear and time-inhomogeneous SDE model with a Jacobi-type diffusion term for the phenomenon of interest, simultaneously driven by the forecast and the constraining upper bound. We rigorously demonstrate the existence and uniqueness of a strong solution to the SDE model by imposing a condition for the time-varying mean-reversion parameter appearing in the drift term. The normalized forecast function is thresholded to keep such mean-reversion parameters bounded. The SDE model parameter calibration is applied to user-selected approximations of the likelihood function. Another novel contribution is estimating the unknown transition density of the forecast error process with a tailored kernel smoothing technique with the control variate method, coupling an adequate SDE to the original one. We provide a theoretical study about how to choose the optimal bandwidth. We fit the model to the 2019 photovoltaic (PV) solar power daily production and forecast data in Uruguay, computing the daily maximum solar PV production estimation. Two statistical versions of the constrained SDE model are fit, with the beta and truncated normal distributions as proxies for the transition density. Empirical results include simulations of the normalized solar PV power production and pathwise confidence bands generated through an indirect inference method. An objective comparison of optimal parametric points associated with the two selected statistical approximations is provided by applying our innovative kernel smoothing estimation technique of the transition function of the forecast error process.

stat.ME↗

arXiv2024

Lagrangian Relaxation for Continuous-Time Optimal Control of Coupled Hydrothermal Power Systems Including Storage Capacity and a Cascade of Hydropower Systems with Time Delays

This work considers a short-term, continuous time setting characterized by a coupled power supply system controlled exclusively by a single provider and comprising a cascade of hydropower systems (dams), fossil fuel power stations, and a storage capacity modeled by a single large battery. Cascaded hydropower generators introduce time-delay effects in the state dynamics, which are modeled with differential equations, making it impossible to use classical dynamic programming. We address this issue by introducing a novel Lagrangian relaxation technique over continuous-time constraints, constructing a nearly optimal policy efficiently. This approach yields a convex, nonsmooth optimization dual problem to recover the optimal Lagrangian multipliers, which is numerically solved using a limited memory bundle method. At each step of the dual optimization, we need to solve an optimization subproblem. Given the current values of the Lagrangian multipliers, the time delays are no longer active, and we can solve a corresponding nonlinear Hamilton--Jacobi--Bellman (HJB) Partial Differential Equation (PDE) for the optimization subproblem. The HJB PDE solver provides both the current value of the dual function and its subgradient, and is trivially parallelizable over the state space for each time step. To handle the infinite-dimensional nature of the Lagrange multipliers, we design an adaptive refinement strategy to control the duality gap. Furthermore, we use a penalization technique for the constructed admissible primal solution to smooth the controls while achieving a sufficiently small duality gap. Numerical results based on the Uruguayan power system demonstrate the efficiency of the proposed mathematical models and numerical approach.

math.OC↗

arXiv2024

Multilevel Importance Sampling for Rare Events Associated With the McKean--Vlasov Equation

This work combines multilevel Monte Carlo (MLMC) with importance sampling to estimate rare-event quantities that can be expressed as the expectation of a Lipschitz observable of the solution to a broad class of McKean--Vlasov stochastic differential equations. We extend the double loop Monte Carlo (DLMC) estimator introduced in this context in (Ben Rached et al., 2023) to the multilevel setting. We formulate a novel multilevel DLMC estimator and perform a comprehensive cost-error analysis yielding new and improved complexity results. Crucially, we devise an antithetic sampler to estimate level differences guaranteeing reduced computational complexity for the multilevel DLMC estimator compared with the single-level DLMC estimator. To address rare events, we apply the importance sampling scheme, obtained via stochastic optimal control in (Ben Rached et al., 2023), over all levels of the multilevel DLMC estimator. Combining importance sampling and multilevel DLMC reduces computational complexity by one order and drastically reduces the associated constant compared to the single-level DLMC estimator without importance sampling. We illustrate the effectiveness of the proposed multilevel DLMC estimator on the Kuramoto model from statistical physics with Lipschitz observables, confirming the reduced complexity from $\mathcal{O}(\mathrm{TOL}_{\mathrm{r}}^{-4})$ for the single-level DLMC estimator to $\mathcal{O}(\mathrm{TOL}_{\mathrm{r}}^{-3})$ while providing a feasible estimate of rare-event quantities up to prescribed relative error tolerance $\mathrm{TOL}_{\mathrm{r}}$.

math.NA↗

arXiv2024

Automated Importance Sampling via Optimal Control for Stochastic Reaction Networks: A Markovian Projection-based Approach

We propose a novel alternative approach to our previous work (Ben Hammouda et al., 2023) to improve the efficiency of Monte Carlo (MC) estimators for rare event probabilities for stochastic reaction networks (SRNs). In the same spirit of (Ben Hammouda et al., 2023), an efficient path-dependent measure change is derived based on a connection between determining optimal importance sampling (IS) parameters within a class of probability measures and a stochastic optimal control formulation, corresponding to solving a variance minimization problem. In this work, we propose a novel approach to address the encountered curse of dimensionality by mapping the problem to a significantly lower-dimensional space via a Markovian projection (MP) idea. The output of this model reduction technique is a low-dimensional SRN (potentially even one dimensional) that preserves the marginal distribution of the original high-dimensional SRN system. The dynamics of the projected process are obtained by solving a related optimization problem via a discrete $L^2$ regression. By solving the resulting projected Hamilton-Jacobi-Bellman (HJB) equations for the reduced-dimensional SRN, we obtain projected IS parameters, which are then mapped back to the original full-dimensional SRN system, resulting in an efficient IS-MC estimator for rare events probabilities of the full-dimensional SRN. Our analysis and numerical experiments reveal that the proposed MP-HJB-IS approach substantially reduces the MC estimator variance, resulting in a lower computational complexity in the rare event regime than standard MC estimators.

math.NA↗

arXiv2024

Comparing Spectral Bias and Robustness For Two-Layer Neural Networks: SGD vs Adaptive Random Fourier Features

We present experimental results highlighting two key differences resulting from the choice of training algorithm for two-layer neural networks. The spectral bias of neural networks is well known, while the spectral bias dependence on the choice of training algorithm is less studied. Our experiments demonstrate that an adaptive random Fourier features algorithm (ARFF) can yield a spectral bias closer to zero compared to the stochastic gradient descent optimizer (SGD). Additionally, we train two identically structured classifiers, employing SGD and ARFF, to the same accuracy levels and empirically assess their robustness against adversarial noise attacks.

cs.LG↗

arXiv2023

Double-Loop Importance Sampling for McKean--Vlasov Stochastic Differential Equation

This paper investigates Monte Carlo (MC) methods to estimate probabilities of rare events associated with solutions to the $d$-dimensional McKean-Vlasov stochastic differential equation (MV-SDE). MV-SDEs are usually approximated using a stochastic interacting $P$-particle system, which is a set of $P$ coupled $d$-dimensional stochastic differential equations (SDEs). Importance sampling (IS) is a common technique for reducing high relative variance of MC estimators of rare-event probabilities. We first derive a zero-variance IS change of measure for the quantity of interest by using stochastic optimal control theory. However, when this change of measure is applied to stochastic particle systems, it yields a $P \times d$-dimensional partial differential control equation (PDE), which is computationally expensive to solve. To address this issue, we use the decoupling approach introduced in [dos Reis et al., 2023], generating a $d$-dimensional control PDE for a zero-variance estimator of the decoupled SDE. Based on this approach, we develop a computationally efficient double loop MC (DLMC) estimator. We conduct a comprehensive numerical error and work analysis of the DLMC estimator. As a result, we show optimal complexity of $\mathcal{O}(\mathrm{TOL}_{\mathrm{r}}^{-4})$ with a significantly reduced constant to achieve a prescribed relative error tolerance $\mathrm{TOL}_{\mathrm{r}}$. Subsequently, we propose an adaptive DLMC method combined with IS to numerically estimate rare-event probabilities, substantially reducing relative variance and computational runtimes required to achieve a given $\mathrm{TOL}_{\mathrm{r}}$ compared with standard MC estimators in the absence of IS. Numerical experiments are performed on the Kuramoto model from statistical physics.

math.NA↗

arXiv2023

Optimal Damping with Hierarchical Adaptive Quadrature for Efficient Fourier Pricing of Multi-Asset Options in Lévy Models

Efficiently pricing multi-asset options is a challenging problem in quantitative finance. When the characteristic function is available, Fourier-based methods are competitive compared to alternative techniques because the integrand in the frequency space often has a higher regularity than that in the physical space. However, when designing a numerical quadrature method for most Fourier pricing approaches, two key aspects affecting the numerical complexity should be carefully considered: (i) the choice of damping parameters that ensure integrability and control the regularity class of the integrand and (ii) the effective treatment of high dimensionality. We propose an efficient numerical method for pricing European multi-asset options based on two complementary ideas to address these challenges. First, we smooth the Fourier integrand via an optimized choice of the damping parameters based on a proposed optimization rule. Second, we employ sparsification and dimension-adaptivity techniques to accelerate the convergence of the quadrature in high dimensions. The extensive numerical study on basket and rainbow options under the multivariate geometric Brownian motion and some Lévy models demonstrates the advantages of adaptivity and the damping rule on the numerical complexity of quadrature methods. Moreover, for the tested two-asset examples, the proposed approach outperforms the COS method in terms of computational time. Finally, we show significant speed-up compared to the Monte Carlo method for up to six dimensions.

q-fin.CP↗

arXiv2023

Scalable method for Bayesian experimental design without integrating over posterior distribution

We address the computational efficiency in solving the A-optimal Bayesian design of experiments problems for which the observational map is based on partial differential equations and, consequently, is computationally expensive to evaluate. A-optimality is a widely used and easy-to-interpret criterion for Bayesian experimental design. This criterion seeks the optimal experimental design by minimizing the expected conditional variance, which is also known as the expected posterior variance. This study presents a novel likelihood-free approach to the A-optimal experimental design that does not require sampling or integrating the Bayesian posterior distribution. The expected conditional variance is obtained via the variance of the conditional expectation using the law of total variance, and we take advantage of the orthogonal projection property to approximate the conditional expectation. We derive an asymptotic error estimation for the proposed estimator of the expected conditional variance and show that the intractability of the posterior distribution does not affect the performance of our approach. We use an artificial neural network (ANN) to approximate the nonlinear conditional expectation in the implementation of our method. We then extend our approach for dealing with the case that the domain of experimental design parameters is continuous by integrating the training process of the ANN into minimizing the expected conditional variance. Through numerical experiments, we demonstrate that our method greatly reduces the number of observation model evaluations compared with widely used importance sampling-based approaches. This reduction is crucial, considering the high computational cost of the observational models. Code is available at https://github.com/vinh-tr-hoang/DOEviaPACE.

math.NA↗

arXiv2023

On the equivalence of different adaptive batch size selection strategies for stochastic gradient descent methods

In this study, we demonstrate that the norm test and inner product/orthogonality test presented in \cite{Bol18} are equivalent in terms of the convergence rates associated with Stochastic Gradient Descent (SGD) methods if $ε^2=θ^2+ν^2$ with specific choices of $θ$ and $ν$. Here, $ε$ controls the relative statistical error of the norm of the gradient while $θ$ and $ν$ control the relative statistical error of the gradient in the direction of the gradient and in the direction orthogonal to the gradient, respectively. Furthermore, we demonstrate that the inner product/orthogonality test can be as inexpensive as the norm test in the best case scenario if $θ$ and $ν$ are optimally selected, but the inner product/orthogonality test will never be more computationally affordable than the norm test if $ε^2=θ^2+ν^2$. Finally, we present two stochastic optimization problems to illustrate our results.

math.OC↗

arXiv2023

Nonlinear Isometric Manifold Learning for Injective Normalizing Flows

To model manifold data using normalizing flows, we employ isometric autoencoders to design embeddings with explicit inverses that do not distort the probability distribution. Using isometries separates manifold learning and density estimation and enables training of both parts to high accuracy. Thus, model selection and tuning are simplified compared to existing injective normalizing flows. Applied to data sets on (approximately) flat manifolds, the combined approach generates high-quality data.

cs.LG↗

arXiv2023

Small-noise approximation for Bayesian optimal experimental design with nuisance uncertainty

Calculating the expected information gain in optimal Bayesian experimental design typically relies on nested Monte Carlo sampling. When the model also contains nuisance parameters, which are parameters that contribute to the overall uncertainty of the system but are of no interest in the Bayesian design framework, this introduces a second inner loop. We propose and derive a small-noise approximation for this additional inner loop. The computational cost of our method can be further reduced by applying a Laplace approximation to the remaining inner loop. Thus, we present two methods, the small-noise Double-loop Monte Carlo and small-noise Monte Carlo Laplace methods. Moreover, we demonstrate that the total complexity of these two approaches remains comparable to the case without nuisance uncertainty. To assess the efficiency of these methods, we present three examples, and the last example includes the partial differential equation for the electrical impedance tomography experiment for composite laminate materials.

math.NA↗

arXiv2023

Learning-Based Importance Sampling via Stochastic Optimal Control for Stochastic Reaction Networks

We explore efficient estimation of statistical quantities, particularly rare event probabilities, for stochastic reaction networks. Consequently, we propose an importance sampling (IS) approach to improve the Monte Carlo (MC) estimator efficiency based on an approximate tau-leap scheme. The crucial step in the IS framework is choosing an appropriate change of probability measure to achieve substantial variance reduction. This task is typically challenging and often requires insights into the underlying problem. Therefore, we propose an automated approach to obtain a highly efficient path-dependent measure change based on an original connection in the stochastic reaction network context between finding optimal IS parameters within a class of probability measures and a stochastic optimal control formulation. Optimal IS parameters are obtained by solving a variance minimization problem. First, we derive an associated dynamic programming equation. Analytically solving this backward equation is challenging, hence we propose an approximate dynamic programming formulation to find near-optimal control parameters. To mitigate the curse of dimensionality, we propose a learning-based method to approximate the value function using a neural network, where the parameters are determined via a stochastic optimization algorithm. Our analysis and numerical experiments verify that the proposed learning-based IS approach substantially reduces MC estimator variance, resulting in a lower computational complexity in the rare event regime, compared with standard tau-leap MC estimators.

math.NA↗

arXiv2022

State-dependent Importance Sampling for Estimating Expectations of Functionals of Sums of Independent Random Variables

Estimating the expectations of functionals applied to sums of random variables (RVs) is a well-known problem encountered in many challenging applications. Generally, closed-form expressions of these quantities are out of reach. A naive Monte Carlo simulation is an alternative approach. However, this method requires numerous samples for rare event problems. Therefore, it is paramount to use variance reduction techniques to develop fast and efficient estimation methods. In this work, we use importance sampling (IS), known for its efficiency in requiring fewer computations to achieve the same accuracy requirements. We propose a state-dependent IS scheme based on a stochastic optimal control formulation, where the control is dependent on state and time. We aim to calculate rare event quantities that could be written as an expectation of a functional of the sums of independent RVs. The proposed algorithm is generic and can be applied without restrictions on the univariate distributions of RVs or the functional applied to the sum. We apply this approach to the log-normal distribution to compute the left tail and cumulative distribution of the ratio of independent RVs. For each case, we numerically demonstrate that the proposed state-dependent IS algorithm compares favorably to most well-known estimators dealing with similar problems.

cs.IT↗

arXiv2022

Multi-index ensemble Kalman filtering

In this work we combine ideas from multi-index Monte Carlo and ensemble Kalman filtering (EnKF) to produce a highly efficient filtering method called multi-index EnKF (MIEnKF). MIEnKF is based on independent samples of four-coupled EnKF estimators on a multi-index hierarchy of resolution levels, and it may be viewed as an extension of the multilevel EnKF (MLEnKF) method developed by the same authors in 2020. Multi-index here refers to a two-index method, consisting of a hierarchy of EnKF estimators that are coupled in two degrees of freedom: time discretization and ensemble size. Under certain assumptions, when strong coupling between solutions on neighboring numerical resolutions is attainable, the MIEnKF method is proven to be more tractable than EnKF and MLEnKF. Said efficiency gains are also verified numerically in a series of test problems.

math.NA↗

arXiv2022

Goal-Oriented Adaptive Finite Element Multilevel Monte Carlo with Convergence Rates

We present an adaptive multilevel Monte Carlo (AMLMC) algorithm for approximating deterministic, real-valued, bounded linear functionals that depend on the solution of a linear elliptic PDE with a lognormal diffusivity coefficient and geometric singularities in bounded domains of $\mathbb{R}^d$. Our AMLMC algorithm is built on the results of the weak convergence rates in the work [Moon et al., BIT Numer. Math., 46 (2006), 367-407] for an adaptive algorithm using isoparametric d-linear quadrilateral finite element approximations and the dual weighted residual error representation in a deterministic setting. Designed to suit the geometric nature of the singularities in the solution, our AMLMC algorithm uses a sequence of deterministic, non-uniform auxiliary meshes as a building block. The deterministic adaptive algorithm generates these meshes, corresponding to a geometrically decreasing sequence of tolerances. For a given realization of the diffusivity coefficient and accuracy level, AMLMC constructs its approximate sample using the first mesh in the hierarchy that satisfies the corresponding bias accuracy constraint. This adaptive approach is particularly useful for the lognormal case treated here, which lacks uniform coercivity and thus produces functional outputs that vary over orders of magnitude when sampled. We discuss iterative solvers and compare their efficiency with direct ones. To reduce computational work, we propose a stopping criterion for the iterative solver with respect to the quantity of interest, the realization of the diffusivity coefficient, and the desired level of AMLMC approximation. From the numerical experiments, based on a Fourier expansion of the coefficient field, we observe improvements in efficiency compared with both standard Monte Carlo and standard MLMC for a problem with a singularity similar to that at the tip of a slit modeling a crack.

math.NA↗

arXiv2022

Machine learning-based conditional mean filter: a generalization of the ensemble Kalman filter for nonlinear data assimilation

This paper presents the machine learning-based ensemble conditional mean filter (ML-EnCMF) -- a filtering method based on the conditional mean filter (CMF) previously introduced in the literature. The updated mean of the CMF matches that of the posterior, obtained by applying Bayes' rule on the filter's forecast distribution. Moreover, we show that the CMF's updated covariance coincides with the expected conditional covariance. Implementing the EnCMF requires computing the conditional mean (CM). A likelihood-based estimator is prone to significant errors for small ensemble sizes, causing the filter divergence. We develop a systematical methodology for integrating machine learning into the EnCMF based on the CM's orthogonal projection property. First, we use a combination of an artificial neural network (ANN) and a linear function, obtained based on the ensemble Kalman filter (EnKF), to approximate the CM, enabling the ML-EnCMF to inherit EnKF's advantages. Secondly, we apply a suitable variance reduction technique to reduce statistical errors when estimating loss function. Lastly, we propose a model selection procedure for element-wisely selecting the applied filter, i.e., either the EnKF or ML-EnCMF, at each updating step. We demonstrate the ML-EnCMF performance using the Lorenz-63 and Lorenz-96 systems and show that the ML-EnCMF outperforms the EnKF and the likelihood-based EnCMF.

cs.LG↗