Searcharxiv⌕ Search

arXiv subjects

Guergana Petrova

Publications and source records attributed to Guergana Petrova.

At least 19 recordsLinked to original sources
arXiv2026

Constrained Kolmogorov widths

The main theme of approximation theory is to understand how well a general function $f$ can be approximated by a simpler function $g$ such as a polynomial or spline. In many applications, one wants $g$ to retain known properties of $f$ such as its inherent smoothness or a geometrical property such as monotonicity or convexity. Additional requirements on $g$ of this type are known as constraints. In this paper, we do a systematic study of constrained approximation to understand how the imposition of such constraints limits the efficiency of the approximation. We study constrained approximation in the setting of linear approximation where $g$ is to be taken from a finite dimensional linear space $V$ of a fixed dimension $n$. Kolmogorov widths describe how well one can approximate when using such linear spaces $V$. The first part of this paper introduces and studies several types of constrained widths, including the constrained Kolmogorov widths, and gives comparisons between them. The second part of the paper is restricted to classical settings where the constraint imposes a smoothness requirement on $g$. In this case, our results prove that the additional constraint can typically be imposed with no loss in the efficiency of the approximation.

math.NA↗
arXiv2026

Sampling and reconstruction of convex functions

We discuss optimal recovery for classes of multivariate convex functions from given point samples, as well as the sampling numbers of these classes, corresponding to optimal sample choices. Upper and lower bounds for either variant are established when the reconstruction error is measured in $L_p$ for $1\leq p\leq \infty$. These bounds match, sometimes up to logarithmic factors, and therefore characterize the respective optimal rate of decay. For classical smoothness classes such as Sobolev, Hölder or Besov spaces, it is well known that the optimal decay rate of sampling numbers can be achieved by sampling on uniform tensor product grids and using linear methods of reconstruction, such as piecewise polynomial interpolation. One of the main findings in this paper is that for classes of convex functions, these procedures generally produce suboptimal rates, except when $p=1$ and $p=\infty$, and are outperformed by nonlinear reconstruction methods that do not employ tensor product grids.

math.NA↗
arXiv2025

Optimal Recovery Meets Minimax Estimation

A fundamental problem in statistics and machine learning is to estimate a function $f$ from possibly noisy observations of its point samples. The goal is to design a numerical algorithm to construct an approximation $\hat f$ to $f$ in a prescribed norm that asymptotically achieves the best possible error (as a function of the number $m$ of observations and the variance $σ^2$ of the noise). This problem has received considerable attention in both nonparametric statistics (noisy observations) and optimal recovery (noiseless observations). Quantitative bounds require assumptions on $f$, known as model class assumptions. Classical results assume that $f$ is in the unit ball of a Besov space. In nonparametric statistics, the best possible performance of an algorithm for finding $\hat f$ is known as the minimax rate and has been studied in this setting under the assumption that the noise is Gaussian. In optimal recovery, the best possible performance of an algorithm is known as the optimal recovery rate and has also been determined in this setting. While one would expect that the minimax rate recovers the optimal recovery rate when the noise level $σ$ tends to zero, it turns out that the current results on minimax rates do not carefully determine the dependence on $σ$ and the limit cannot be taken. This paper handles this issue and determines the noise-level-aware (NLA) minimax rates for Besov classes when error is measured in an $L_q$-norm with matching upper and lower bounds. The end result is a reconciliation between minimax rates and optimal recovery rates. The NLA minimax rate continuously depends on the noise level and recovers the optimal recovery rate when $σ$ tends to zero.

math.ST↗
arXiv2025

Convergence and error control of consistent PINNs for elliptic PDEs

We provide an a priori analysis of collocation methods for solving elliptic boundary value problems. They begin with information in the form of point values of the data and utilize only this information to numerically approximate the solution u of the PDE. For such a method to provide an approximation with guaranteed error bounds, additional assumptions on the data, called model class assumptions, are needed. We determine the best error of approximating u in the energy norm, in terms of the total number of point samples, under all Besov class model assumptions for the right hand side and boundary data. We then turn to the study of numerical procedures and analyze whether a proposed numerical procedure achieves the optimal recovery error. We analyze numerical methods which generate the numerical approximation to $u$ by minimizing specified data driven loss functions over a set $Σ$ which is either a finite dimensional linear space, or more generally, a finite dimensional manifold. We show that the success of such a procedure depends critically on choosing a data driven loss function that is consistent with the PDE and provides sharp error control. Based on this analysis a new loss function is proposed. We also address the recent methods of Physics Informed Neural Networks. We prove that minimization of the new loss over restricted neural network spaces $Σ$ provides an optimal recovery of the solution $u$, provided that the optimization problem can be numerically executed and $Σ$ has sufficient approximation capabilities. We also analyze variants of the new loss function which are more practical for implementation. Finally, numerical examples illustrating the benefits of the proposed loss functions are given.

math.NA↗
arXiv2023

Solving PDEs with Incomplete Information

We consider the problem of numerically approximating the solutions to a partial differential equation (PDE) when there is insufficient information to determine a unique solution. Our main example is the Poisson boundary value problem, when the boundary data is unknown and instead one observes finitely many linear measurements of the solution. We view this setting as an optimal recovery problem and develop theory and numerical algorithms for its solution. The main vehicle employed is the derivation and approximation of the Riesz representers of these functionals with respect to relevant Hilbert spaces of harmonic functions.

math.NA↗
arXiv2023

Neural networks: deep, shallow, or in between?

We give estimates from below for the error of approximation of a compact subset from a Banach space by the outputs of feed-forward neural networks with width W, depth l and Lipschitz activation functions. We show that, modulo logarithmic factors, rates better that entropy numbers' rates are possibly attainable only for neural networks for which the depth l goes to infinity, and that there is no gain if we fix the depth and let the width W go to infinity.

stat.ML↗
arXiv2023

Optimal Learning

This paper studies the problem of learning an unknown function $f$ from given data about $f$. The learning problem is to give an approximation $\hat f$ to $f$ that predicts the values of $f$ away from the data. There are numerous settings for this learning problem depending on (i) what additional information we have about $f$ (known as a model class assumption), (ii) how we measure the accuracy of how well $\hat f$ predicts $f$, (iii) what is known about the data and data sites, (iv) whether the data observations are polluted by noise. A mathematical description of the optimal performance possible (the smallest possible error of recovery) is known in the presence of a model class assumption. Under standard model class assumptions, it is shown in this paper that a near optimal $\hat f$ can be found by solving a certain discrete over-parameterized optimization problem with a penalty term. Here, near optimal means that the error is bounded by a fixed constant times the optimal error. This explains the advantage of over-parameterization which is commonly used in modern machine learning. The main results of this paper prove that over-parameterized learning with an appropriate loss function gives a near optimal approximation $\hat f$ of the function $f$ from which the data is collected. Quantitative bounds are given for how much over-parameterization needs to be employed and how the penalization needs to be scaled in order to guarantee a near optimal recovery of $f$. An extension of these results to the case where the data is polluted by additive deterministic noise is also given.

cs.LG↗
arXiv2022

Limitations on approximation by deep and shallow neural networks

We prove Carl's type inequalities for the error of approximation of compact sets K by deep and shallow neural networks. This in turn gives lower bounds on how well we can approximate the functions in K when requiring the approximants to come from outputs of such networks. Our results are obtained as a byproduct of the study of the recently introduced Lipschitz widths.

stat.ML↗
arXiv2022

On the entropy numbers and the Kolmogorov widths

Direct estimates between linear or nonlinear Kolmogorov widths and entropy numbers are presented. These estimates are derived using the recently introduced Lipschitz widths. Applications for m-term approximation are obtained.

math.FA↗
arXiv2021

Lipschitz widths

This paper introduces a measure, called Lipschitz widths, of the optimal performance possible of certain nonlinear methods of approximation. It discusses their relation to entropy numbers and other well known widths such as the Kolmogorov and the stable manifold widths. It also shows that the Lipschitz widths provide a theoretical benchmark for the approximation quality achieved via deep neural networks.

math.NA↗
arXiv2021

Neural Network Approximation of Refinable Functions

In the desire to quantify the success of neural networks in deep learning and other applications, there is a great interest in understanding which functions are efficiently approximated by the outputs of neural networks. By now, there exists a variety of results which show that a wide range of functions can be approximated with sometimes surprising accuracy by these outputs. For example, it is known that the set of functions that can be approximated with exponential accuracy (in terms of the number of parameters used) includes, on one hand, very smooth functions such as polynomials and analytic functions (see e.g. \cite{E,S,Y}) and, on the other hand, very rough functions such as the Weierstrass function (see e.g. \cite{EPGB,DDFHP}), which is nowhere differentiable. In this paper, we add to the latter class of rough functions by showing that it also includes refinable functions. Namely, we show that refinable functions are approximated by the outputs of deep ReLU networks with a fixed width and increasing depth with accuracy exponential in terms of their number of parameters. Our results apply to functions used in the standard construction of wavelets as well as to functions constructed via subdivision algorithms in Computer Aided Geometric Design.

cs.LG↗
arXiv2020

Neural Network Approximation

Neural Networks (NNs) are the method of choice for building learning algorithms. Their popularity stems from their empirical success on several challenging learning problems. However, most scholars agree that a convincing theoretical explanation for this success is still lacking. This article surveys the known approximation properties of the outputs of NNs with the aim of uncovering the properties that are not present in the more traditional methods of approximation used in numerical analysis. Comparisons are made with traditional approximation methods from the viewpoint of rate distortion. Another major component in the analysis of numerical approximation is the computational time needed to construct the approximation and this in turn is intimately connected with the stability of the approximation algorithm. So the stability of numerical approximation using NNs is a large part of the analysis put forward. The survey, for the most part, is concerned with NNs using the popular ReLU activation function. In this case, the outputs of the NNs are piecewise linear functions on rather complicated partitions of the domain of $f$ into cells that are convex polytopes. When the architecture of the NN is fixed and the parameters are allowed to vary, the set of output functions of the NN is a parameterized nonlinear manifold. It is shown that this manifold has certain space filling properties leading to an increased ability to approximate (better rate distortion) but at the expense of numerical stability. The space filling creates a challenge to the numerical method in finding best or good parameter choices when trying to approximate.

math.NA↗
arXiv2020

Optimal Stable Nonlinear Approximation

While it is well known that nonlinear methods of approximation can often perform dramatically better than linear methods, there are still questions on how to measure the optimal performance possible for such methods. This paper studies nonlinear methods of approximation that are compatible with numerical implementation in that they are required to be numerically stable. A measure of optimal performance, called {\em stable manifold widths}, for approximating a model class $K$ in a Banach space $X$ by stable manifold methods is introduced. Fundamental inequalities between these stable manifold widths and the entropy of $K$ are established. The effects of requiring stability in the settings of deep learning and compressed sensing are discussed.

math.NA↗
arXiv2020

Nonlinear Methods for Model Reduction

The usual approach to model reduction for parametric partial differential equations (PDEs) is to construct a linear space $V_n$ which approximates well the solution manifold $\mathcal{M}$ consisting of all solutions $u(y)$ with $y$ the vector of parameters. This linear reduced model $V_n$ is then used for various tasks such as building an online forward solver for the PDE or estimating parameters from data observations. It is well understood in other problems of numerical computation that nonlinear methods such as adaptive approximation, $n$-term approximation, and certain tree-based methods may provide improved numerical efficiency. For model reduction, a nonlinear method would replace the linear space $V_n$ by a nonlinear space $Σ_n$. This idea has already been suggested in recent papers on model reduction where the parameter domain is decomposed into a finite number of cells and a linear space of low dimension is assigned to each cell. Up to this point, little is known in terms of performance guarantees for such a nonlinear strategy. Moreover, most numerical experiments for nonlinear model reduction use a parameter dimension of only one or two. In this work, a step is made towards a more cohesive theory for nonlinear model reduction. Framing these methods in the general setting of library approximation allows us to give a first comparison of their performance with those of standard linear approximation for any general compact set. We then turn to the study these methods for solution manifolds of parametrized elliptic PDEs. We study a very specific example of library approximation where the parameter domain is split into a finite number $N$ of rectangular cells and where different reduced affine spaces of dimension $m$ are assigned to each cell. The performance of this nonlinear procedure is analyzed from the viewpoint of accuracy of approximation versus $m$ and $N$.

math.NA↗
arXiv2020

Polynomial Approximation of Anisotropic Analytic Functions of Several Variables

Motivated by numerical methods for solving parametric partial differential equations, this paper studies the approximation of multivariate analytic functions by algebraic polynomials. We introduce various anisotropic model classes based on Taylor expansions, and study their approximation by finite dimensional polynomial spaces $\cal{P}_Λ$ described by lower sets $Λ$. Given a budget $n$ for the dimension of $\cal{P}_Λ$, we prove that certain lower sets $Λ_n$, with cardinality $n$, provide a certifiable approximation error that is in a certain sense optimal, and that these lower sets have a simple definition in terms of simplices. Our main goal is to obtain approximation results when the number of variables $d$ is large and even infinite, and so we concentrate almost exclusively on the case $d=\infty$. We also emphasize obtaining results which hold for the full range $n\ge 1$, rather than asymptotic results that only hold for $n$ sufficiently large. In applications, one typically wants $n$ small to comply with computational budgets.

math.NA↗
arXiv2016

Diffusion Coefficients Estimation for Elliptic Partial Differential Equations

This paper considers the Dirichlet problem $$ -\mathrm{div}(a\nabla u_a)=f \quad \hbox{on}\,\,\ D, \qquad u_a=0\quad \hbox{on}\,\,\partial D, $$ for a Lipschitz domain $D\subset \mathbb R^d$, where $a$ is a scalar diffusion function. For a fixed $f$, we discuss under which conditions is $a$ uniquely determined and when can $a$ be stably recovered from the knowledge of $u_a$. A first result is that whenever $a\in H^1(D)$, with $0<λ\le a\le Λ$ on $D$, and $f\in L_\infty(D)$ is strictly positive, then $$ \|a-b\|_{L_2(D)}\le C\|u_a-u_b\|_{H_0^1(D)}^{1/6}. $$ More generally, it is shown that the assumption $a\in H^1(D)$ can be weakened to $a\in H^s(D)$, for certain $s<1$, at the expense of lowering the exponent $1/6$ to a value that depends on $s$.

math.AP↗
arXiv2016

Data Assimilation and Sampling in Banach spaces

This paper studies the problem of approximating a function $f$ in a Banach space $X$ from measurements $l_j(f)$, $j=1,\dots,m$, where the $l_j$ are linear functionals from $X^*$. Most results study this problem for classical Banach spaces $X$ such as the $L_p$ spaces, $1\le p\le \infty$, and for $K$ the unit ball of a smoothness space in $X$. Our interest in this paper is in the model classes $K=K(ε,V)$, with $ε>0$ and $V$ a finite dimensional subspace of $X$, which consists of all $f\in X$ such that $dist(f,V)_X\le ε$. These model classes, called {\it approximation sets}, arise naturally in application domains such as parametric partial differential equations, uncertainty quantification, and signal processing. A general theory for the recovery of approximation sets in a Banach space is given. This theory includes tight a priori bounds on optimal performance, and algorithms for finding near optimal approximations. We show how the recovery problem for approximation sets is connected with well-studied concepts in Banach space theory such as liftings and the angle between spaces. Examples are given that show how this theory can be used to recover several recent results on sampling and data assimilation.

math.NA↗