arXiv subjects

Anthony Nouy

Publications and source records attributed to Anthony Nouy.

At least 37 records · Page 2Linked to original sources

arXiv2022

Moment-SoS Methods for Optimal Transport Problems

Most common Optimal Transport (OT) solvers are currently based on an approximation of underlying measures by discrete measures. However, it is sometimes relevant to work only with moments of measures instead of the measure itself, and many common OT problems can be formulated as moment problems (the most relevant examples being $L^p$-Wasserstein distances, barycenters, and Gromov-Wasserstein discrepancies on Euclidean spaces). We leverage this fact to develop a generalized moment formulation that covers these classes of OT problems. The transport plan is represented through its moments on a given basis, and the marginal constraints are expressed in terms of moment constraints. A practical computation then consists in considering a truncation of the involved moment sequences up to a certain order, and using the polynomial sums-of-squares hierarchy for measures supported on semi-algebraic sets. We prove that the strategy converges to the solution of the OT problem as the order increases. We also show how to approximate linear quantities of interest, and how to estimate the support of the optimal transport map from the computed moments using Christoffel-Darboux kernels. Numerical experiments illustrate the good behavior of the approach.

math.NA↗

arXiv2022

A PAC algorithm in relative precision for bandit problem with costly sampling

This paper considers the problem of maximizing an expectation function over a finite set, or finite-arm bandit problem. We first propose a naive stochastic bandit algorithm for obtaining a probably approximately correct (PAC) solution to this discrete optimization problem in relative precision, that is a solution which solves the optimization problem up to a relative error smaller than a prescribed tolerance, with high probability. We also propose an adaptive stochastic bandit algorithm which provides a PAC-solution with the same guarantees. The adaptive algorithm outperforms the mean complexity of the naive algorithm in terms of number of generated samples and is particularly well suited for applications with high sampling cost.

math.OC↗

arXiv2021

Approximation by tree tensor networks in high dimensions: Sobolev and compositional functions

This paper is concerned with convergence estimates for fully discrete tree tensor network approximations of high-dimensional functions from several model classes. For functions having standard or mixed Sobolev regularity, new estimates generalizing and refining known results are obtained, based on notions of linear widths of multivariate functions. In the main results of this paper, such techniques are applied to classes of functions with compositional structure, which are known to be particularly suitable for approximation by deep neural networks. As shown here, such functions can also be approximated by tree tensor networks without a curse of dimensionality -- however, subject to certain conditions, in particular on the depth of the underlying tree. In addition, a constructive encoding of compositional functions in tree tensor networks is given.

math.NA↗

arXiv2021

Approximation of Smoothness Classes by Deep Rectifier Networks

We consider approximation rates of sparsely connected deep rectified linear unit (ReLU) and rectified power unit (RePU) neural networks for functions in Besov spaces $B^α_{q}(L^p)$ in arbitrary dimension $d$, on general domains. We show that \alert{deep rectifier} networks with a fixed activation function attain optimal or near to optimal approximation rates for functions in the Besov space $B^α_τ(L^τ)$ on the critical embedding line $1/τ=α/d+1/p$ for \emph{arbitrary} smoothness order $α>0$. Using interpolation theory, this implies that the entire range of smoothness classes at or above the critical line is (near to) optimally approximated by deep ReLU/RePU networks.

math.FA↗

arXiv2021

Learning high-dimensional probability distributions using tree tensor networks

We consider the problem of the estimation of a high-dimensional probability distribution from i.i.d. samples of the distribution using model classes of functions in tree-based tensor formats, a particular case of tensor networks associated with a dimension partition tree. The distribution is assumed to admit a density with respect to a product measure, possibly discrete for handling the case of discrete random variables. After discussing the representation of classical model classes in tree-based tensor formats, we present learning algorithms based on empirical risk minimization using a $L^2$ contrast. These algorithms exploit the multilinear parametrization of the formats to recast the nonlinear minimization problem into a sequence of empirical risk minimization problems with linear models. A suitable parametrization of the tensor in tree-based tensor format allows to obtain a linear model with orthogonal bases, so that each problem admits an explicit expression of the solution and cross-validation risk estimates. These estimations of the risk enable the model selection, for instance when exploiting sparsity in the coefficients of the representation. A strategy for the adaptation of the tensor format (dimension tree and tree-based ranks) is provided, which allows to discover and exploit some specific structures of high-dimensional probability distributions such as independence or conditional independence. We illustrate the performances of the proposed algorithms for the approximation of classical probabilistic models (such as Gaussian distribution, graphical models, Markov chain).

stat.ML↗

arXiv2021

Learning with tree tensor networks: complexity estimates and model selection

Tree tensor networks, or tree-based tensor formats, are prominent model classes for the approximation of high-dimensional functions in computational and data science. They correspond to sum-product neural networks with a sparse connectivity associated with a dimension tree and widths given by a tuple of tensor ranks. The approximation power of these models has been proved to be (near to) optimal for classical smoothness classes. However, in an empirical risk minimization framework with a limited number of observations, the dimension tree and ranks should be selected carefully to balance estimation and approximation errors. We propose and analyze a complexity-based model selection method for tree tensor networks in an empirical risk minimization framework and we analyze its performance over a wide range of smoothness classes. Given a family of model classes associated with different trees, ranks, tensor product feature spaces and sparsity patterns for sparse tensor networks, a model is selected (à la Barron, Birgé, Massart) by minimizing a penalized empirical risk, with a penalty depending on the complexity of the model class and derived from estimates of the metric entropy of tree tensor networks. This choice of penalty yields a risk bound for the selected predictor. In a least-squares setting, after deriving fast rates of convergence of the risk, we show that our strategy is (near to) minimax adaptive to a wide range of smoothness classes including Sobolev or Besov spaces (with isotropic, anisotropic or mixed dominating smoothness) and analytic functions. We discuss the role of sparsity of the tensor network for obtaining optimal performance in several regimes. In practice, the amplitude of the penalty is calibrated with a slope heuristics method. Numerical experiments in a least-squares regression setting illustrate the performance of the strategy.

math.ST↗

arXiv2021

Active learning of tree tensor networks using optimal least-squares

In this paper, we propose new learning algorithms for approximating high-dimensional functions using tree tensor networks in a least-squares setting. Given a dimension tree or architecture of the tensor network, we provide an algorithm that generates a sequence of nested tensor subspaces based on a generalization of principal component analysis for multivariate functions. An optimal least-squares method is used for computing projections onto the generated tensor subspaces, using samples generated from a distribution depending on the previously generated subspaces. We provide an error bound in expectation for the obtained approximation. Practical strategies are proposed for adapting the feature spaces and ranks to achieve a prescribed error. Also, we propose an algorithm that progressively constructs the dimension tree by suitable pairings of variables, that allows to further reduce the number of samples necessary to reach that error. Numerical examples illustrate the performance of the proposed algorithms and show that stable approximations are obtained with a number of samples close to the number of free parameters of the estimated tensor networks.

math.NA↗

arXiv2021

A new splitting algorithm for dynamical low-rank approximation motivated by the fibre bundle structure of matrix manifolds

In this paper, we propose a new splitting algorithm for dynamical low-rank approximation motivated by the fibre bundle structure of the set of fixed rank matrices. We first introduce a geometric description of the set of fixed rank matrices which relies on a natural parametrization of matrices. More precisely, it is endowed with the structure of analytic principal bundle, with an explicit description of local charts. For matrix differential equations, we introduce a first order numerical integrator working in local coordinates. The resulting algorithm can be interpreted as a particular splitting of the projection operator onto the tangent space of the low-rank matrix manifold. It is proven to be exact in some particular case. Numerical experiments confirm this result and illustrate the robustness of the proposed algorithm.

math.NA↗

arXiv2020

Randomized linear algebra for model reduction. Part II: minimal residual methods and dictionary-based approximation

A methodology for using random sketching in the context of model order reduction for high-dimensional parameter-dependent systems of equations was introduced in [Balabanov and Nouy 2019, Part I]. Following this framework, we here construct a reduced model from a small, efficiently computable random object called a sketch of a reduced model, using minimal residual methods. We introduce a sketched version of the minimal residual based projection as well as a novel nonlinear approximation method, where for each parameter value, the solution is approximated by minimal residual projection onto a subspace spanned by several vectors picked (online) from a dictionary of candidate basis vectors. It is shown that random sketching technique can improve not only efficiency but also numerical stability. A rigorous analysis of the conditions on the random sketch required to obtain a given accuracy is presented. These conditions may be ensured a priori with high probability by considering for the sketching matrix an oblivious embedding of sufficiently large size. Furthermore, a simple and reliable procedure for a posteriori verification of the quality of the sketch is provided. This approach can be used for certification of the approximation as well as for adaptive selection of the size of the random sketching matrix.

math.NA↗

arXiv2019

Singular Value Decomposition in Sobolev Spaces: Part I

A well known result from functional analysis states that any compact operator between Hilbert spaces admits a singular value decomposition (SVD). This decomposition is a powerful tool that is the workhorse of many methods both in mathematics and applied fields. A prominent application in recent years is the approximation of high-dimensional functions in a low-rank format. This is based on the fact that, under certain conditions, a tensor can be identified with a compact operator and SVD applies to the latter. One key assumption for this application is that the tensor product norm is not weaker than the injective norm. This assumption is not fulfilled in Sobolev spaces, which are widely used in the theory and numerics of partial differential equations. Our goal is the analysis of the SVD in Sobolev spaces. This work consists of two parts. In this manuscript (part I), we address low-rank approximations and minimal subspaces in H1. We analyze the H1-error of the SVD performed in the ambient L2-space. In part II, we will address variants of the SVD in norms stronger than the L2-norm. We will provide a few numerical examples that support our theoretical findings.

math.FA↗

arXiv2019

Singular Value Decomposition in Sobolev Spaces: Part II

Under certain conditions, an element of a tensor product space can be identified with a compact operator and the singular value decomposition (SVD) applies to the latter. These conditions are not fulfilled in Sobolev spaces. In the previous part of this work (part I), we introduced some preliminary notions in the theory of tensor product spaces. We analyzed low-rank approximations in H1 and the error of the SVD performed in the ambient L2 space. In this work (part II), we continue by considering variants of the SVD in norms stronger than the L2-norm. Overall and, perhaps surprisingly, this leads to a more difficult control of the H1-error. We briefly consider an isometric embedding of H1 that allows direct application of the SVD to H1-functions. Finally, we provide a few numerical examples that support our theoretical findings.

math.FA↗

arXiv2019

Randomized linear algebra for model reduction. Part I: Galerkin methods and error estimation

We propose a probabilistic way for reducing the cost of classical projection-based model order reduction methods for parameter-dependent linear equations. A reduced order model is here approximated from its random sketch, which is a set of low-dimensional random projections of the reduced approximation space and the spaces of associated residuals. This approach exploits the fact that the residuals associated with approximations in low-dimensional spaces are also contained in low-dimensional spaces. We provide conditions on the dimension of the random sketch for the resulting reduced order model to be quasi-optimal with high probability. Our approach can be used for reducing both complexity and memory requirements. The provided algorithms are well suited for any modern computational environment. Major operations, except solving linear systems of equations, are embarrassingly parallel. Our version of proper orthogonal decomposition can be computed on multiple workstations with a communication cost independent of the dimension of the full order model. The reduced order model can even be constructed in a so-called streaming environment, i.e., under extreme memory constraints. In addition, we provide an efficient way for estimating the error of the reduced order model, which is not only more efficient than the classical approach but is also less sensitive to round-off errors. Finally, the methodology is validated on benchmark problems.

math.NA↗

arXiv2019

Stochastic methods for solving high-dimensional partial differential equations

We propose algorithms for solving high-dimensional Partial Differential Equations (PDEs) that combine a probabilistic interpretation of PDEs, through Feynman-Kac representation, with sparse interpolation. Monte-Carlo methods and time-integration schemes are used to estimate pointwise evaluations of the solution of a PDE. We use a sequential control variates algorithm, where control variates are constructed based on successive approximations of the solution of the PDE. Two different algorithms are proposed, combining in different ways the sequential control variates algorithm and adaptive sparse interpolation. Numerical examples will illustrate the behavior of these algorithms.

math.NA↗

arXiv2019

Weakly intrusive low-rank approximation method for nonlinear parameter-dependent equations

This paper presents a weakly intrusive strategy for computing a low-rank approximation of the solution of a system of nonlinear parameter-dependent equations. The proposed strategy relies on a Newton-like iterative solver which only requires evaluations of the residual of the parameter-dependent equation and of a preconditioner (such as the differential of the residual) for instances of the parameters independently. The algorithm provides an approximation of the set of solutions associated with a possibly large number of instances of the parameters, with a computational complexity which can be orders of magnitude lower than when using the same Newton-like solver for all instances of the parameters. The reduction of complexity requires efficient strategies for obtaining low-rank approximations of the residual, of the preconditioner, and of the increment at each iteration of the algorithm. For the approximation of the residual and the preconditioner, weakly intrusive variants of the empirical interpolation method are introduced, which require evaluations of entries of the residual and the preconditioner. Then, an approximation of the increment is obtained by using a greedy algorithm for low-rank approximation, and a low-rank approximation of the iterate is finally obtained by using a truncated singular value decomposition. When the preconditioner is the differential of the residual, the proposed algorithm is interpreted as an inexact Newton solver for which a detailed convergence analysis is provided. Numerical examples illustrate the efficiency of the method.

math.NA↗

arXiv2019

Tree-based tensor formats

The main goal of this paper is to study the topological properties of tensors in tree-based Tucker format. These formats include the Tucker format and the Hierarchical Tucker format. A property of the so-called minimal subspaces is used for obtaining a representation of tensors with either bounded or fixed tree-based rank in the underlying algebraic tensor space. We provide a new characterisation of minimal subspaces which extends the existing characterisations. We also introduce a definition of topological tensor spaces in tree-based format, with the introduction of a norm at each vertex of the tree, and prove the existence of best approximations from sets of tensors with bounded tree-based rank, under some assumptions on the norms weaker than in the existing results.

math.NA↗

arXiv2019

A multiscale method for semi-linear elliptic equations with localized uncertainties and non-linearities

A multiscale numerical method is proposed for the solution of semi-linear elliptic stochastic partial differential equations with localized uncertainties and non-linearities, the uncertainties being modeled by a set of random parameters. It relies on a domain decomposition method which introduces several subdomains of interest (called patches) containing the different sources of uncertainties and non-linearities. An iterative algorithm is then introduced, which requires the solution of a sequence of linear global problems (with deterministic operators and uncertain right-hand sides), and non-linear local problems (with uncertain operators and/or right-hand sides) over the patches. Non-linear local problems are solved using an adaptive sampling-based least-squares method for the construction of sparse polynomial approximations of local solutions as functions of the random parameters. Consistency, convergence and robustness of the algorithm are proved under general assumptions on the semi-linear elliptic operator. A convergence acceleration technique (Aitken's dynamic relaxation) is also introduced to speed up the convergence of the algorithm. The performances of the proposed method are illustrated through numerical experiments carried out on a stationary non-linear diffusion-reaction problem.

math.NA↗

arXiv2019

Learning with tree-based tensor formats

This paper is concerned with the approximation of high-dimensional functions in a statistical learning setting, by empirical risk minimization over model classes of functions in tree-based tensor format. These are particular classes of rank-structured functions that can be seen as deep neural networks with a sparse architecture related to the tree and multilinear activation functions. For learning in a given model class, we exploit the fact that tree-based tensor formats are multilinear models and recast the problem of risk minimization over a nonlinear set into a succession of learning problems with linear models. Suitable changes of representation yield numerically stable learning problems and allow to exploit sparsity. For high-dimensional problems or when only a small data set is available, the selection of a good model class is a critical issue. For a given tree, the selection of the tuple of tree-based ranks that minimize the risk is a combinatorial problem. Here, we propose a rank adaptation strategy which provides in practice a good convergence of the risk as a function of the model class complexity. Finding a good tree is also a combinatorial problem, which can be related to the choice of a particular sparse architecture for deep neural networks. Here, we propose a stochastic algorithm for minimizing the complexity of the representation of a given function over a class of trees with a given arity, allowing changes in the topology of the tree. This tree optimization algorithm is then included in a learning scheme that successively adapts the tree and the corresponding tree-based ranks. Contrary to classical learning algorithms for nonlinear model classes, the proposed algorithms are numerically stable, reliable, and require only a low level expertise of the user.

stat.ML↗

arXiv2018

Low-rank approximation of linear parabolic equations by space-time tensor Galerkin methods

We devise a space-time tensor method for the low-rank approximation of linear parabolic evolution equations. The proposed method is a stable Galerkin method, uniformly in the discretization parameters, based on a Minimal Residual formulation of the evolution problem in Hilbert--Bochner spaces. The discrete solution is sought in a trial space composed of tensors of discrete functions in space and in time and is characterized as the unique minimizer of a discrete functional where the dual norm of the residual is evaluated in a space semi-discrete test space. The resulting global space-time linear system is solved iteratively by a greedy algorithm. Numerical results are presented to illustrate the performances of the proposed method on test cases including non-selfadjoint and time-dependent differential operators in space. The results are also compared to those obtained using a fully discrete Petrov--Galerkin setting to evaluate the dual residual norm.

math.NA↗