arXiv subjects
Publications and source records attributed to Vladimir Temlyakov.
This paper aims to motivate stochastic optimization problems from a statistical perspective and a statistical learning perspective, where the goal is to maximize the log-likelihood or minimize the population risk. We briefly describe the two main approaches: offline (Monte Carlo / Sample Average Approximation) and online (Stochastic Approximation) approaches -- to solve the expectation minimization problems.
Recently, in a number of papers it was understood that results on sampling discretization and on the universal sampling discretization can be successfully used in the problem of sampling recovery. Moreover, it turns out that it is sufficient to only have a one-sided discretization inequality for some of those applications. This motivates us to write the present paper as a survey/research paper with the focus on the one-sided discretization inequalities and their applications in the sampling recovery. In this sense the paper complements the two existing survey papers on sampling discretization.
Although the basic idea behind the concept of a greedy basis had been around for some time, the formal development of a theory of greedy bases was initiated in 1999 with the publication of the article [S.~V.~Konyagin and V.~N.~Temlyakov, A remark on greedy approximation in Banach spaces, East J. Approx. 5 (1999), no. 3, 365--379]. The theoretical simplicity of the thresholding greedy algorithm became a model for a procedure widely used in numerical applications and the subject of greedy bases evolved very rapidly from the point of view of approximation theory. The idea of studying greedy bases and related greedy algorithms attracted also the attention of researchers with a classical Banach space theory background. From the more abstract point of functional analysis, the theory of greedy bases and its derivates evolved very fast as many fundamental results were discovered and new ramifications branched out. Hundreds of papers on greedy-like bases and several monographs have been written since the foundational paper mentioned above appeared. After twenty-five years, the theory is very much alive and it continues to be a very active research topic both for functional analysts and for researchers interested in the applied nature of nonlinear approximation alike. This is why we believe it is a good moment to gather a selection of 25 open problems (one per year since 1999!) whose solution would contribute to advance the state of art of this beautiful topic.
The study of greedy approximation in the context of convex optimization is becoming a promising research direction as greedy algorithms are actively being employed to construct sparse minimizers for convex functions with respect to given sets of elements. In this paper we propose a unified way of analyzing a certain kind of greedy-type algorithms for the minimization of convex functions on Banach spaces. Specifically, we define the class of Weak Biorthogonal Greedy Algorithms for convex optimization that contains a wide range of greedy algorithms. We analyze the introduced class of algorithms and establish the properties of convergence, rate of convergence, and numerical stability, which is understood in the sense that the steps of the algorithm are allowed to be performed not precisely but with controlled computational inaccuracies. We show that the following well-known algorithms for convex optimization -- the Weak Chebyshev Greedy Algorithm (co) and the Weak Greedy Algorithm with Free Relaxation (co) -- belong to this class, and introduce a new algorithm -- the Rescaled Weak Relaxed Greedy Algorithm (co). Presented numerical experiments demonstrate the practical performance of the aforementioned greedy algorithms in the setting of convex minimization as compared to optimization with regularization, which is the conventional approach of constructing sparse minimizers.
Discretization of the uniform norm of functions from a given finite dimensional subspace of continuous functions is studied. We pay special attention to the case of trigonometric polynomials with frequencies from an arbitrary finite set with fixed cardinality. We give two different proofs of the fact that for any $N$-dimensional subspace of the space of continuous functions it is sufficient to use $e^{CN}$ sample points for an accurate upper bound for the uniform norm. Previous known results show that one cannot improve on the exponential growth of the number of sampling points for a good discretization theorem in the uniform norm. Also, we prove a general result, which connects the upper bound on the number of sampling points in the discretization theorem for the uniform norm with the best $m$-term bilinear approximation of the Dirichlet kernel associated with the given subspace. We illustrate application of our technique on the example of trigonometric polynomials.
In this paper we present results on asymptotic characteristics of multivariate function classes in the uniform norm. Our main interest is the approximation of functions with mixed smoothness parameter not larger than $1/2$. Our focus will be on the behavior of the best $m$-term trigonometric approximation as well as the decay of Kolmogorov and entropy numbers in the uniform norm. It turns out that these quantities share a few fundamental abstract properties like their behavior under real interpolation, such that they can be treated simultaneously. We start with proving estimates on finite rank convolution operators with range in a step hyperbolic cross. These results imply bounds for the corresponding function space embeddings by a well-known decomposition technique. The decay of Kolmogorov numbers have direct implications for the problem of sampling recovery in $L_2$ in situations where recent results in the literature are not applicable since the corresponding approximation numbers are not square summable.
We prove a sampling discretization theorem for the square norm of functions from a finite dimensional subspace satisfying Nikol'skii's inequality with an upper bound on the number of sampling points of the order of the dimension of the subspace
Talagrand's fundamental result on the entropy numbers is slightly improved. Our proof uses different ideas based on results from greedy approximation.
The paper is devoted to discretization of integral norms of functions from a given finite dimensional subspace. We use recent general results on sampling discretization to derive a new Marcinkiewicz type discretization theorem for the multivariate trigonometric polynomials with frequencies from the hyperbolic crosses. It is shown that recently developed techniques allow us to improve the known results in this direction.
We consider numerical integration in classes, for which we do not impose any smoothness assumptions. We illustrate how nonlinear approximation, in particular greedy approximation, allows us to guarantee some rate of decay of errors of numerical integration even in such a general setting with no smoothness assumptions.
We present new results regarding Lebesgue-type inequalities for the Weak Chebyshev Greedy Algorithm (WCGA) in uniformly smooth Banach spaces. We improve earlier bounds in Temlyakov (Forum Math Sigma 2014), for dictionaries satisfying a new property introduced here. We apply these results to derive optimal bounds in two natural examples of sequence spaces. In particular, optimality is obtained in the case of the multivariate Haar system in Lp with 1<p<2, under the Littlewood-Paley norm.
This paper is devoted to the study of a discrepancy-type characteristic -- the fixed volume discrepancy -- of the Fibonacci point set in the unit square. It was observed recently that this new characteristic allows us to obtain optimal rate of dispersion from numerical integration results. This observation motivates us to thoroughly study this new version of discrepancy, which seems to be interesting by itself. The new ingredient of this paper is the use of the average over the shifts of hat functions instead of taking the supremum over the shifts. We show that this change in the setting results in an improvement of the upper bound for the smooth fixed volume discrepancy, similarly to the well-known results for the usual $L_p$-discrepancy. Interestingly, this shows that ``bad boxes'' for the usual discrepancy cannot be ``too small''. The known results on smooth discrepancy show that the obtained bounds cannot be improved in a certain sense.
In this paper we propose a unified way of analyzing a certain kind of greedy-type algorithms in Banach spaces. We define a class of the Weak Biorthogonal Greedy Algorithms that contains a wide range of greedy algorithms. In particular, we show that the following well-known algorithms --- the Weak Chebyshev Greedy Algorithm and the Weak Greedy Algorithm with Free Relaxation --- belong to this class. We investigate the properties of convergence, rate of convergence, and numerical stability of the Weak Biorthogonal Greedy Algorithms. Numerical stability is understood in the sense that the steps of the algorithm are allowed to be performed with controlled computational inaccuracies. We carry out a thorough analysis of the connection between the magnitude of those inaccuracies and the convergence properties of the algorithm. To emphasize the advantage of the proposed approach, we introduce here a new greedy algorithm --- the Rescaled Weak Relaxed Greedy Algorithm --- from the above class, and derive the convergence results without analyzing the algorithm explicitly. Additionally, we explain how the proposed approach can be extended to some other types of greedy algorithms.
The new ingredient of this paper is that we consider infinitely dimensional classes of functions and instead of the relative error setting, which was used in previous papers on norm discretization, we consider the absolute error setting. We demonstrate how known results from two areas of research -- supervised learning theory and numerical integration -- can be used in sampling discretization of the square norm on different function classes.
The main goal of this paper is to provide a brief survey of recent results which connect together results from different areas of research. It is well known that numerical integration of functions with mixed smoothness is closely related to the discrepancy theory. We discuss this connection in detail and provide a general view of this connection. It was established recently that the new concept of {\it fixed volume discrepancy} is very useful in proving the upper bounds for the dispersion. Also, it was understood recently that point sets with small dispersion are very good for the universal discretization of the uniform norm of trigonometric polynomials.
We introduce the notion of a \textit{weight-almost greedy} basis and show that a basis for a real Banach space is $w$-almost greedy if and only if it is both quasi-greedy and $w$-democratic. We also introduce the notion of \textit{weight-semi-greedy} basis and show that a $w$-almost greedy basis is $w$-semi-greedy and that the converse holds if the Banach space has finite cotype.
As a new type of series expansion, the so-called one-dimensional adaptive Fourier decomposition (AFD) and its variations (1D-AFDs) have effective applications in signal analysis and system identification. The 1D-AFDs have considerable influence to the rational approximation of one complex variable and phase retrieving problems, etc. In a recent paper, Qian developed 2D-AFDs for treating square images as the essential boundary of the 2-torus embedded into the space of two complex variables. This paper studies the numerical aspects of multi-dimensional AFDs, and in particular 2D-AFDs, which mainly include (i) Numerical algorithms of several types of 2D-AFDs in relation to image representation; (ii) Perform experiments for the algorithms with comparisons between 5 types of image reconstruction methods in the Fourier category; and (iii) New and sharper estimations for convergence rates of orthogonal greedy algorithm and pre-orthogonal greedy algorithm. The comparison shows that the 2D-AFD methods achieve optimal results among the others.
It is a survey on recent results in constructive sparse approximation. Three directions are discussed here: (1) Lebesgue-type inequalities for greedy algorithms with respect to a special class of dictionaries, (2) constructive sparse approximation with respect to the trigonometric system, (3) sparse approximation with respect to dictionaries with tensor product structure. In all three cases constructive ways are provided for sparse approximation. The technique used is based on fundamental results from the theory of greedy approximation. In particular, results in the direction (1) are based on deep methods developed recently in compressed sensing. We present some of these results with detailed proofs.