SearcharxivSearch

arXiv subjects

Erich Novak

Publications and source records attributed to Erich Novak.

At least 19 recordsLinked to original sources

Tractability versus curse of dimensionality for geometric $L_p$-discrepancies

This paper studies tractability versus the curse of dimensionality for several geometric $L_p$-discrepancies through a unified discrepancy--integration duality framework, where worst case integration errors in suitable function spaces equal the corresponding discrepancies. A general lower bound method for non-negative linear rules in spaces under broad tensor-product assumptions establishes exponential information complexity in the dimension $d$, yielding the curse of dimensionality for the respective discrepancy. We complement this overview by new results on discrepancy--integration duality and the curse of dimensionality for the periodic $L_p$-discrepancy. The current state of research on this general problem is summarized in a clearly laid out table, which also highlights the remaining open questions.

math.NA

Extreme $L_p$ discrepancy, numerical integration and the curse of dimensionality

The classical notion of extreme $L_p$ discrepancy is a quantitative measure for the irregularity of distribution of finite point sets in the $d$-dimensinal unit cube. In this paper we find a dual integration problem whose worst-case error is exactly the extreme $L_p$ discrepancy of the underlying integration nodes. Studying this integration problem we show that the extreme $L_p$ discrepancy suffers from the curse of dimensionality for all $p \in (1,\infty)$. It is known that the problem is tractable for $p=\infty$; the case $p=1$ stays open.

math.NA

Generalized Discrepancy of Random Points

We study the $L_p$-discrepancy of random point sets in high dimensions, with emphasis on small values of $p$. Although the classical $L_p$-discrepancy suffers from the curse of dimensionality for all $p \in (1,\infty)$, the gap between known upper and lower bounds remains substantial, in particular for small $p \ge 1$. To clarify this picture, we review the existing results for i.i.d.\ uniformly distributed points and derive new upper bounds for \emph{generalized} $L_p$-discrepancies, obtained by allowing non-uniform sampling densities and corresponding non-negative quadrature weights. Using the probabilistic method, we show that random points drawn from optimally chosen product densities lead to significantly improved upper bounds. For $p=2$ these bounds are explicit and optimal; for general $p \in [1,\infty)$ we obtain sharp asymptotic estimates. The improvement can be interpreted as a form of importance sampling for the underlying Sobolev space $F_{d,q}$. Our results also reveal that, even with optimal densities, the curse of dimensionality persists for random points when $p\ge 1$, and it becomes most pronounced for small $p$. This suggests that the curse should also hold for the classical $L_1$-discrepancy for deterministic point sets.

math.NA

Noisy nonlinear information and entropy numbers

It is impossible to recover a vector from $\mathbb{R}^m$ with less than $m$ linear measurements, even if the measurements are chosen adaptively. Recently, it has been shown that one can recover vectors from $\mathbb{R}^m$ with arbitrary precision using only $O(\log m)$ continuous (even Lipschitz) adaptive measurements, resulting in an exponential speed-up of continuous information compared to linear information for various approximation problems. In this note, we characterize the quality of optimal (dis-)continuous information that is disturbed by deterministic noise in terms of entropy numbers. This shows that in the presence of noise the potential gain of continuous over linear measurements is limited, but significant in some cases.

math.NA

How many continuous measurements are needed to learn a vector?

One can recover vectors from $\mathbb{R}^m$ with arbitrary precision, using only $\lceil \log_2(m+1)\rceil +1$ continuous measurements that are chosen adaptively. This surprising result is explained and discussed, and we present applications to infinite-dimensional approximation problems.

math.NA

On the power of adaption and randomization

We present bounds on the maximal gain of adaptive and randomized algorithms over non-adaptive, deterministic ones for approximating linear operators on convex sets. If the sets are additionally symmetric, then our results are optimal. For non-symmetric sets, we unify some notions of $n$-widths and s-numbers, and show their connection to minimal errors. We also discuss extensions to non-linear widths and approximation based on function values, and conclude with a list of open problems.

math.NA

Conditions for tractability of the weighted $L_p$-discrepancy and integration in non-homogeneous tensor product spaces

We study tractability properties of the weighted $L_p$-discrepancy. The concept of {\it weighted} discrepancy was introduced by Sloan and Wo\'{z}\-nia\-kowski in 1998 in order to prove a weighted version of the Koksma-Hlawka inequality for the error of quasi-Monte Carlo integration rules. The weights have the aim to model the influence of different coordinates of integrands on the error. A discrepancy is said to be tractable if the information complexity, i.e., the minimal number $N$ of points such that the discrepancy is less than the initial discrepancy times an error threshold $\varepsilon$, does not grow exponentially fast with the dimension. In this case there are various notions of tractabilities used in order to classify the exact rate. For even integer parameters $p$ there are sufficient conditions on the weights available in literature, which guarantee the one or other notion of tractability. In the present paper we prove matching sufficient conditions (upper bounds) and neccessary conditions (lower bounds) for polynomial and weak tractability for all $p \in (1, \infty)$. The proofs of the lower bounds are based on a general result for the information complexity of integration with positive quadrature formulas for tensor product spaces. In order to demonstrate this lower bound we consider as a second application the integration of tensor products of polynomials of degree at most 2.

math.NA

Intractability results for integration in tensor product spaces

We study lower bounds on the worst-case error of numerical integration in tensor product spaces. As reference we use the $N$-th minimal error of linear rules that use $N$ function values. The information complexity is the minimal number $N$ of function evaluations that is necessary such that the $N$-th minimal error is less than a factor $\varepsilon$ times the initial error. We are interested to which extent the information complexity depends on the number $d$ of variables of the integrands. If the information complexity grows exponentially fast in $d$, then the integration problem is said to suffer from the curse of dimensionality. Under the assumption of the existence of a worst-case function for the uni-variate problem we present two methods for providing good lower bounds on the information complexity. The first method is based on a suitable decomposition of the worst-case function. This method can be seen as a generalization of the method of decomposable reproducing kernels, that is often successfully applied when integration in Hilbert spaces with a reproducing kernel is studied. The second method, although only applicable for positive quadrature rules, has the advantage, that it does not require a suitable decomposition of the worst-case function. Rather, it is based on a spline approximation of the worst-case function and can be used for analytic functions. The methods presented can be applied to problems beyond the Hilbert space setting. For demonstration purposes we apply them to several examples, notably to uniform integration over the unit-cube, weighted integration over the whole space, and integration of infinitely smooth functions over the cube. Some of these results have interesting consequences in discrepancy theory.

math.NA

The $L_p$-discrepancy for finite $p>1$ suffers from the curse of dimensionality

The $L_p$-discrepancy is a classical quantitative measure for the irregularity of distribution of an $N$-element point set in the $d$-dimensional unit cube. Its inverse for dimension $d$ and error threshold $\varepsilon \in (0,1)$ is the number of points in $[0,1)^d$ that is required such that the minimal normalized $L_p$-discrepancy is less or equal $\varepsilon$. It is well known, that the inverse of $L_2$-discrepancy grows exponentially fast with the dimension $d$, i.e., we have the curse of dimensionality, whereas the inverse of $L_{\infty}$-discrepancy depends exactly linearly on $d$. The behavior of inverse of $L_p$-discrepancy for general $p \not\in \{2,\infty\}$ was an open problem since many years. Recently, the curse of dimensionality for the $L_p$-discrepancy was shown for an infinite sequence of values $p$ in $(1,2]$, but the general result seemed to be out of reach. In the present paper we show that the $L_p$-discrepancy suffers from the curse of dimensionality for all $p$ in $(1,\infty)$ and only the case $p=1$ is still open. This result follows from a more general result that we show for the worst-case error of positive quadrature formulas for an anchored Sobolev space of once differentiable functions in each variable whose first mixed derivative has finite $L_q$-norm, where $q$ is the H\"older conjugate of $p$.

math.NA

Randomized approximation of summable sequences -- adaptive and non-adaptive

We prove lower bounds for the randomized approximation of the embedding $\ell_1^m \rightarrow \ell_\infty^m$ based on algorithms that use arbitrary linear (hence non-adaptive) information provided by a (randomized) measurement matrix $N \in \mathbb{R}^{n \times m}$. These lower bounds reflect the increasing difficulty of the problem for $m \to \infty$, namely, a term $\sqrt{\log m}$ in the complexity $n$. This result implies that non-compact operators between arbitrary Banach spaces are not approximable using non-adaptive Monte Carlo methods. We also compare these lower bounds for non-adaptive methods with upper bounds based on adaptive, randomized methods for recovery for which the complexity $n$ only exhibits a $(\log\log m)$-dependence. In doing so we give an example of linear problems where the error for adaptive vs. non-adaptive Monte Carlo methods shows a gap of order $n^{1/2} ( \log n)^{-1/2}$.

math.NA

The curse of dimensionality for the $L_p$-discrepancy with finite $p$

The $L_p$-discrepancy is a quantitative measure for the irregularity of distribution of an $N$-element point set in the $d$-dimensional unit cube, which is closely related to the worst-case error of quasi-Monte Carlo algorithms for numerical integration. Its inverse for dimension $d$ and error threshold $\varepsilon \in (0,1)$ is the minimal number of points in $[0,1)^d$ such that the minimal normalized $L_p$-discrepancy is less or equal $\varepsilon$. It is well known, that the inverse of $L_2$-discrepancy grows exponentially fast with the dimension $d$, i.e., we have the curse of dimensionality, whereas the inverse of $L_{\infty}$-discrepancy depends exactly linearly on $d$. The behavior of inverse of $L_p$-discrepancy for general $p \not\in \{2,\infty\}$ has been an open problem for many years. In this paper we show that the $L_p$-discrepancy suffers from the curse of dimensionality for all $p$ in $(1,2]$ which are of the form $p=2 \ell/(2 \ell -1)$ with $\ell \in \mathbb{N}$. This result follows from a more general result that we show for the worst-case error of numerical integration in an anchored Sobolev space with anchor 0 of once differentiable functions in each variable whose first derivative has finite $L_q$-norm, where $q$ is an even positive integer satisfying $1/p+1/q=1$.

math.NA

Lower bounds for integration and recovery in $L_2$

Function values are, in some sense, "almost as good" as general linear information for $L_2$-approximation (optimal recovery, data assimilation) of functions from a reproducing kernel Hilbert space. This was recently proved by new upper bounds on the sampling numbers under the assumption that the singular values of the embedding of this Hilbert space into $L_2$ are square-summable. Here we mainly prove new lower bounds. In particular we prove that the sampling numbers behave worse than the approximation numbers for Sobolev spaces with small smoothness. Hence there can be a logarithmic gap also in the case where the singular numbers of the embedding are square-summable. We first prove new lower bounds for the integration problem, again for rather classical Sobolev spaces of periodic univariate functions.

math.NA

Recovery of Sobolev functions restricted to iid sampling

We study $L_q$-approximation and integration for functions from the Sobolev space $W^s_p(\Omega)$ and compare optimal randomized (Monte Carlo) algorithms with algorithms that can only use iid sample points, uniformly distributed on the domain. The main result is that we obtain the same optimal rate of convergence if we restrict to iid sampling, a common assumption in learning and uncertainty quantification. The only exception is when $p=q=\infty$, where a logarithmic loss cannot be avoided.

math.NA

Lower Bounds for the Error of Quadrature Formulas for Hilbert Spaces

We prove lower bounds for the worst case error of quadrature formulas that use given sample points $\X_n = \{ x_1, \dots , x_n \}$. We are mainly interested in optimal point sets $\X_n$, but also prove lower bounds that hold with high probability for sets of independently and uniformly distributed points. As a tool, we use a recent result (and extensions thereof) of Vyb\'iral on the positive semi-definiteness of certain matrices related to the product theorem of Schur. The new technique also works for spaces of analytic functions where known methods based on decomposable kernels cannot be applied.

math.NA

Algorithms and Complexity for Functions on General Domains

Error bounds and complexity bounds in numerical analysis and information-based complexity are often proved for functions that are defined on very simple domains, such as a cube, a torus, or a sphere. We study optimal error bounds for the approximation or integration of functions defined on $D_d \subset R^d$ and only assume that $D_d$ is a bounded Lipschitz domain. Some results are even more general. We study three different concepts to measure the complexity: order of convergence, asymptotic constant, and explicit uniform bounds, i.e., bounds that hold for all $n$ (number of pieces of information) and all (normalized) domains. It is known for many problems that the order of convergence of optimal algorithms does not depend on the domain $D_d \subset R^d$. We present examples for which the following statements are true: 1) Also the asymptotic constant does not depend on the shape of $D_d$ or the imposed boundary values, it only depends on the volume of the domain. 2) There are explicit and uniform lower (or upper, respectively) bounds for the error that are only slightly smaller (or larger, respectively) than the asymptotic error bound.

math.NA

On the power of random information

We study approximation and integration problems and compare the quality of optimal information with the quality of random information. For some problems random information is almost optimal and for some other problems random information is much worse than optimal information. We prove new results and give a short survey of known results.

math.NA

Random sections of ellipsoids and the power of random information

We study the circumradius of the intersection of an $m$-dimensional ellipsoid $\mathcal E$ with semi-axes $\sigma_1\geq\dots\geq \sigma_m$ with random subspaces of codimension $n$. We find that, under certain assumptions on $\sigma$, this random radius $\mathcal{R}_n=\mathcal{R}_n(\sigma)$ is of the same order as the minimal such radius $\sigma_{n+1}$ with high probability. In other situations $\mathcal{R}_n$ is close to the maximum $\sigma_1$. The random variable $\mathcal{R}_n$ naturally corresponds to the worst-case error of the best algorithm based on random information for $L_2$-approximation of functions from a compactly embedded Hilbert space $H$ with unit ball $\mathcal E$. In particular, $\sigma_k$ is the $k$th largest singular value of the embedding $H\hookrightarrow L_2$. In this formulation, one can also consider the case $m=\infty$, and we prove that random information behaves very differently depending on whether $\sigma \in \ell_2$ or not. For $\sigma \notin \ell_2$ random information is completely useless, i.e., $\mathbb E[\mathcal{R}_n] = \sigma_1$. For $\sigma \in \ell_2$ the expected radius of random information tends to zero at least at rate $o(1/\sqrt{n})$ as $n\to\infty$. In the important case $\sigma_k \asymp k^{-\alpha} \ln^{-\beta}(k+1)$, where $\alpha > 0$ and $\beta\in\mathbb R$, we obtain that $$ \mathbb E [\mathcal{R}_n(\sigma)] \asymp \begin{cases} \sigma_1 & : \alpha<1/2 \,\text{ or }\, \beta\leq\alpha=1/2 \\ \sigma_n \, \sqrt{\ln(n+1)} & : \beta>\alpha=1/2 \\ \sigma_{n+1} & : \alpha>1/2. \end{cases} $$ In the proofs we use a comparison result for Gaussian processes \`a la Gordon, exponential estimates for sums of chi-squared random variables, and estimates for the extreme singular values of (structured) Gaussian random matrices. The upper bound is constructive. It is proven for the worst case error of a least squares estimator.

math.FA