SearcharxivSearch

arXiv subjects

Tommaso Vanzan

Publications and source records attributed to Tommaso Vanzan.

At least 19 recordsLinked to original sources

Multilevel Stochastic Gradient Descent for Risk-Averse PDE-Constrained Optimization

We present recent advances in applying and analyzing multilevel stochastic gradient descent algorithms to risk-averse, three-dimensional PDE-constrained optimization problems. The algorithm uses adaptive multilevel Monte Carlo gradient estimates, provides parallel scalability as well as improved convergence rates and computational complexity compared to standard batched stochastic gradient descent methods. We study the method in computationally demanding settings using three-dimensional elliptic diffusion problems and large risk-aversion parameters.

math.OC

LAGO: A Local-Global Optimization Framework Combining Trust Region Methods and Bayesian Optimization

We introduce LAGO, a LocAl-Global Optimization framework coupling Bayesian Optimization (BO) and gradient-based trust region local refinement through an adaptive competition mechanism for smooth expensive-to-evaluate objective functions with available gradients. At each iteration, global and local optimization strategies independently propose candidate points, and the next evaluation is selected based on predicted improvement. LAGO separates global exploration from local refinement at the proposal level: the BO acquisition function is optimized outside the active trust region, while local candidates are proposed within the trust region. Points in the vicinity of the accepted local step are incorporated in the global GP dataset only when satisfying a lengthscale-based minimum-distance criterion, hence reducing the risk of numerical instability during local exploitation. LAGO enhances BO with efficient local refinement when reaching promising regions, and reverts to exploratory behavior when local steps are not competitive.

cs.LG

Optimized multilevel Monte Carlo methods in Banach spaces

We present a theoretical and numerical analysis of Monte Carlo methods for the estimation of statistical moments of random variables $X:Ω\rightarrow E$ taking values in a Banach space $E$. For practical computation, we consider finite-dimensional approximation subspaces ${(E_\ell)_{\ell\in\mathbb{N}}\subset E}$ of increasing dimension. We develop a refined error analysis that explicitly accounts for a dependence of the Rademacher type constants on the dimension of $E_\ell$, leading to novel complexity results for single- and multilevel Monte Carlo methods to estimate the mean and injective moments of arbitrary order, which are, in certain cases, sharper than those derived in [Kirchner, Schwab, J. Funct. Anal, 2024]. Moreover, we show that, in favorable cases, the resulting error-vs.-work bounds are independent of the Rademacher type of $E$. We then focus on $L^p(S)$-valued random variables for a $σ$-finite measure space satisfying certain approximation properties, and prove that for a random variable $X\in L^q(Ω;L^p(S))\cap L^p(S;L^q(Ω))$, with $q\in (1,\infty)$ and $p\in [1,\infty)$, the $L^q$-convergence rate of a Monte Carlo estimator is determined exclusively by the integrability parameter $\min\{q,2\}$, with no dependence on the Rademacher type $\min\{p,2\}$ of $L^p(S)$. We further investigate the impact of measuring the (multilevel) Monte Carlo error in the $L^q(Ω;L^p(S))$-norm while $X$ possesses additional regularity, $X\in L^{\tilde{q}}(Ω;L^p(S))\cap L^p(S;L^{\tilde{q}}(Ω))$ with $\tilde{q}\in [q,\infty)$. This analysis reveals an interplay between the sampling error and the strong approximation error, and leads to optimized error-vs.-work bounds for both single- and multilevel Monte Carlo methods. Numerical experiments confirm the sharpness of the analyses presented.

math.NA

Low-rank solutions to a class of parametrized systems using Riemannian optimization

We propose a computational framework for computing low-rank approximations to the ensemble of solutions of a parametrized system of the form $A(ξ)x(ξ)+g(x(ξ))=b(ξ)$ for multiple parameter values. The central idea is to reinterpret the parametrized system as the first-order optimality condition of an optimization problem set over the space of real matrices, which is then minimized over the manifold of fixed-rank matrices. This formulation enables the use of Riemannian optimization techniques, including conjugate gradient and trust-region methods, and covers both linear and nonlinear instances under mild assumptions on the structure of the parametrized system. We further provide a theoretical analysis establishing conditions under which the solution matrix admits accurate low-rank approximations, extending existing results from linear to nonlinear problems. To enhance computational efficiency and robustness, we discuss tailored preconditioning strategies and a rank-compression mechanism to control the rank growth induced by nonlinearities. Numerical experiments demonstrate that the proposed approach achieves significant computational savings compared to solving each system independently, as well as highlight the potential of Riemannian optimization methods for low-rank approximations in large-scale parametrized nonlinear problems.

math.NA

Weak Scalability of time parallel Schwarz methods for parabolic optimal control problems

Parabolic optimal control problems arise in numerous scientific and engineering applications. They typically lead to large-scale coupled forward-backward systems that cannot be treated with classical time-stepping schemes and are computationally expensive to solve. Therefore, parallel methods are essential to reduce the computational time required. In this work, we investigate a time domain decomposition approach, namely the time parallel Schwarz method, applied to parabolic optimal control problems. We analyze the convergence behavior and focus on the weak scalability property of this method as the number of time intervals increases. To characterize the spectral radius of the iteration matrix, we present two analysis techniques: the construction of a tailored matrix norm and the application of block Toeplitz matrix theory. Our analyses yield both nonasymptotic bounds on the spectral radius and an asymptotic characterization of the eigenvalues as the number of time intervals tends to infinity. Numerical experiments further confirm our theoretical findings and demonstrate the weak scalability of the time parallel Schwarz method. This work introduces the first theoretical tool for analyzing the weak scalability of time domain decomposition methods, and our results shed light on the suitability of our algorithm for large-scale simulations on modern high-performance computing architectures.

math.NA

Multilevel quadrature formulae for the optimal control of random PDEs

This manuscript presents a framework for using multilevel quadrature formulae to compute the solution of optimal control problems constrained by random partial differential equations. Our approach consists in solving a sequence of optimal control problems discretized with different levels of accuracy of the physical and probability discretizations. The final approximation of the control is then obtained in a postprocessing step, by suitably combining the adjoint variables computed on the different levels. We present a general convergence and complexity analysis for an unconstrained linear quadratic problem under abstract assumptions on the spatial discretization and on the quadrature formulae. We detail our framework for the specific case of a MultiLevel Monte Carlo (MLMC) quadrature formula, and numerical experiments confirm the better computational complexity of our MLMC approach compared to a standard Monte Carlo sample average approximation, even beyond the theoretical assumptions.

math.NA

An adaptive importance sampling algorithm for risk-averse optimization

Adaptive sampling algorithms are modern and efficient methods that dynamically adjust the sample size throughout the optimization process. However, they may encounter difficulties in risk-averse settings, particularly due to the challenge of accurately sampling from the tails of the underlying distribution of random inputs. This often leads to a much faster growth of the sample size compared to risk-neutral problems. In this work, we propose a novel adaptive sampling algorithm that adapts both the sample size and the sampling distribution at each iteration. The biasing distributions are constructed on the fly, leveraging a reduced-order model of the objective function to be minimized, and are designed to oversample a so-called risk region. As a result, a reduction of the variance of the gradients is achieved, which permits to use fewer samples per iteration compared to a standard algorithm, while still preserving the asymptotic convergence rate. Our focus is on the minimization of the Conditional Value-at-Risk (CVaR), and we establish the convergence of the proposed computational framework. Numerical experiments confirm the substantial computational savings achieved by our approach.

math.OC

Nonlinear Schwarz methods to compute geodesics on manifolds

We consider the leapfrog algorithm by Noakes for computing geodesics on Riemannian manifolds. The main idea behind this algorithm is to subdivide the original endpoint geodesic problem into several local problems, for which the endpoint geodesic problem can be solved more easily by any local method (e.g., the single shooting method). The algorithm then iteratively updates a piecewise geodesic to obtain a global geodesic between the original endpoints. From a domain decomposition perspective, we show that the leapfrog algorithm can be viewed as a classical Schwarz alternating method. Thanks to this analogy, we use techniques from nonlinear preconditioning to improve the convergence properties of the method. Preliminary numerical experiments suggest that this is a promising approach.

math.NA

Variable reduction as a nonlinear preconditioning approach for optimization problems

When considering an unconstrained minimization problem, a standard approach is to solve the optimality system with a Newton method possibly preconditioned by, e.g., nonlinear elimination. In this contribution, we argue that nonlinear elimination could be used to reduce the number of optimization variables by artificially constraining them to satisfy a subset of the optimality conditions. Consequently, a reduced objective function is derived which can now be minimized with any optimization algorithm. By choosing suitable variables to eliminate, the conditioning of the reduced optimization problem is largely improved. We here focus in particular on a right preconditioned gradient descent and show theoretical and numerical results supporting the validity of the presented approach.

math.NA

A multigrid solver for PDE-constrained optimization with uncertain inputs

In this manuscript, we present a collective multigrid algorithm to solve efficiently the large saddle-point systems of equations that typically arise in PDE-constrained optimization under uncertainty, and develop a novel convergence analysis of collective smoothers and collective two-level methods. The multigrid algorithm is based on a collective smoother that at each iteration sweeps over the nodes of the computational mesh, and solves a reduced saddle-point system whose size is proportional to the number $N$ of samples used to discretized the probability space. We show that this reduced system can be solved with optimal $O(N)$ complexity. The multigrid method is tested both as a stationary method and as a preconditioner for GMRES on three problems: a linear-quadratic problem, possibly with a local or a boundary control, for which the multigrid method is used to solve directly the linear optimality system; a nonsmooth problem with box constraints and $L^1$-norm penalization on the control, in which the multigrid scheme is used as an inner solver within a semismooth Newton iteration; a risk-averse problem with the smoothed CVaR risk measure where the multigrid method is called within a preconditioned Newton iteration. In all cases, the multigrid algorithm exhibits excellent performances and robustness with respect to the parameters of interest.

math.OC

A combination technique for optimal control problems constrained by random PDEs

We present a combination technique based on mixed differences of both spatial approximations and quadrature formulae for the stochastic variables to solve efficiently a class of Optimal Control Problems (OCPs) constrained by random partial differential equations. The method requires to solve the OCP for several low-fidelity spatial grids and quadrature formulae for the objective functional. All the computed solutions are then linearly combined to get a final approximation which, under suitable regularity assumptions, preserves the same accuracy of fine tensor product approximations, while drastically reducing the computational cost. The combination technique involves only tensor product quadrature formulae, thus the discretized OCPs preserve the (possible) convexity of the continuous OCP. Hence, the combination technique avoids the inconveniences of Multilevel Monte Carlo and/or sparse grids approaches, but remains suitable for high dimensional problems. The manuscript presents an a-priori procedure to choose the most important mixed differences and an asymptotic complexity analysis, which states that the asymptotic complexity is exclusively determined by the spatial solver. Numerical experiments validate the results.

math.NA

Robust optimization of control parameters for WEC arrays using stochastic methods

This work presents a new computational optimization framework for the robust control of parks of Wave Energy Converters (WEC) in irregular waves. The power of WEC parks is maximized with respect to the individual control damping and stiffness coefficients of each device. The results are robust with respect to the incident wave direction, which is treated as a random variable. Hydrodynamic properties are computed using the linear potential model, and the dynamics of the system is computed in the frequency domain. A slamming constraint is enforced to ensure that the results are physically realistic. We show that the stochastic optimization problem is well posed. Two optimization approaches for dealing with stochasticity are then considered: stochastic approximation and sample average approximation. The outcomes of the above mentioned methods in terms of accuracy and computational time are presented. The results of the optimization for complex and realistic array configurations of possible engineering interest are then discussed. Results of extensive numerical experiments demonstrate the efficiency of the proposed computational framework.

math.OC

Optimized Schwarz methods for the time-dependent Stokes-Darcy coupling

This paper derives optimal coefficients for optimized Schwarz iterations for the time-dependent Stokes-Darcy problem using an innovative strategy to solve a nonstandard min-max problem. The coefficients take into account both physical and discretization parameters that characterize the coupled problem, and they guarantee the robustness of the associated domain decomposition method. Numerical results validate the proposed approach in several test cases with physically relevant parameters.

math.NA

Weak scalability of domain decomposition methods for discrete fracture networks

Discrete Fracture Networks (DFNs) are complex three-dimensional structures characterized by the intersections of planar polygonal fractures, and are used to model flows in fractured media. Despite being suitable for Domain Decomposition (DD) techniques, there are relatively few works on the application of DD methods to DFNs. In this manuscript, we present a theoretical study of Optimized Schwarz Methods (OSMs) applied to DFNs. Interestingly, we prove that the OSMs can be weakly scalable (that is, they converge to a given tolerance in a number of iterations independent of the number of fractures) under suitable assumptions on the domain decomposition. This contribution fits in the renewed interest on the weak scalability of DD methods after recent works showed weak scalability of DD methods for specific geometric configurations, even without coarse spaces. Despite simplifying assumptions which may be violated in practice, our analysis provides heuristics to minimize the computational efforts in realistic settings. Finally, we emphasize that the methodology proposed can be straightforwardly generalized to study other classical DD methods applied to DFNs.

math.NA

On the nonlinear Dirichlet-Neumann method and preconditioner for Newton's method

The Dirichlet-Neumann (DN) method has been extensively studied for linear partial differential equations, while little attention has been devoted to the nonlinear case. In this paper, we analyze the DN method both as a nonlinear iterative method and as a preconditioner for Newton's method. We discuss the nilpotent property and prove that under special conditions, there exists a relaxation parameter such that the DN method converges quadratically. We further prove that the convergence of Newton's method preconditioned by the DN method is independent of the relaxation parameter. Our numerical experiments further illustrate the mesh independent convergence of the DN method and compare it with other standard nonlinear preconditioners.

math.NA

Preconditioners for robust optimal control problems under uncertainty

The discretization of robust quadratic optimal control problems under uncertainty using the finite element method and the stochastic collocation method leads to large saddle-point systems, which are fully coupled across the random realizations. Despite its relevance for numerous engineering problems, the solution of such systems is notoriusly challenging. In this manuscript, we study efficient preconditioners for all-at-once approaches using both an algebraic and an operator preconditioning framework. We show in particular that for values of the regularization parameter not too small, the saddle-point system can be efficiently solved by preconditioning in parallel all the state and adjoint equations. For small values of the regularization parameter, robustness can be recovered by the additional solution of a small linear system, which however couples all realizations. A mean approximation and a Chebyshev semi-iterative method are investigated to solve this reduced system. Our analysis considers a random elliptic partial differential equation whose diffusion coefficient $κ(x,ω)$ is modeled as an almost surely continuous and positive random field, though not necessarily uniformly bounded and coercive. We further provide estimates on the dependence of the preconditioned system on the variance of the random field. Such estimates involve either the first or second moment of the random variables $1/\min_{x\in \overline{D}} κ(x,ω)$ and $\max_{x\in \overline{D}}κ(x,ω)$, where $D$ is the spatial domain. The theoretical results are confirmed by numerical experiments, and implementation details are further addressed.

math.NA

A numerical algorithm based on probing to find optimized transmission conditions

Optimized Schwarz Methods (OSMs) are based on optimized transmission conditions along the interfaces between the subdomains. Optimized transmission conditions are derived at the theoretical level, using techniques developed in the last decades. The hypothesis behind these analyses are quite strong, so that the applicability of OSMs is still limited. In this manuscript, we present a numerical algorithm to obtain optimized transmission conditions for any given problem at hand. This algorithm requires few subdomain solves to be performed in an offline phase. This additional cost is usually negligible due to the resulting faster convergence, even in a single-query context.

math.NA

Spectral substructured two-level domain decomposition methods

Two-level domain decomposition (DD) methods are very powerful techniques for the efficient numerical solution of partial differential equations (PDEs). A two-level domain decomposition method requires two main components: a one-level preconditioner (or its corresponding smoothing iterative method), which is based on domain decomposition techniques, and a coarse correction step, which relies on a coarse space. The coarse space must properly represent the error components that the chosen one-level method is not capable to deal with. In the literature most of the works introduced efficient coarse spaces obtained as the span of functions defined on the entire space domain of the considered PDE. Therefore, the corresponding two-level preconditioners and iterative methods are defined in volume. In this paper, a new class of substructured two-level methods is introduced,for which both domain decomposition smoothers and coarse correction steps are defined on the interfaces (or skeletons). This approach has several advantages. On the one hand, the required computational effort is cheaper than the one required by classical volumetric two-level methods. On the other hand, it allows one to use some of the well-known efficient coarse spaces proposed in the literature. While analyzing in detail the new substructured methods, we present a new convergence analysis for two-level iterative methods, which covers the proposed substructured framework. Further, we study the asymptotic optimality of coarse spaces both theoretically and numerically using deep neural networks. Numerical experiments demonstrate the effectiveness of the proposed new numerical framework.

math.NA