arXiv subjects

Vladimir Temlyakov

Publications and source records attributed to Vladimir Temlyakov.

26 records · Page 2Linked to original sources

arXiv2015

Dictionary descent in optimization

The problem of convex optimization is studied. Usually in convex optimization the minimization is over a d-dimensional domain. Very often the convergence rate of an optimization algorithm depends on the dimension d. The algorithms studied in this paper utilize dictionaries instead of a canonical basis used in the coordinate descent algorithms. We show how this approach allows us to reduce dimensionality of the problem. Also, we investigate which properties of a dictionary are beneficial for the convergence rate of typical greedy-type algorithms.

stat.ML↗

arXiv2014

Constructive sparse trigonometric approximation and other problems for functions with mixed smoothness

Our main interest in this paper is to study some approximation problems for classes of functions with mixed smoothness. We use technique, based on a combination of results from hyperbolic cross approximation, which were obtained in 1980s -- 1990s, and recent results on greedy approximation to obtain sharp estimates for best $m$-term approximation with respect to the trigonometric system. We give some observations on numerical integration and approximate recovery of functions with mixed smoothness. We prove lower bounds, which show that one cannot improve accuracy of sparse grids methods with $\asymp 2^nn^{d-1}$ points in the grid by adding $2^n$ arbitrary points. In case of numerical integration these lower bounds provide best known lower bounds for optimal cubature formulas and for sparse grids based cubature formulas.

math.NA↗

arXiv2014

Convergence and rate of convergence of some greedy algorithms in convex optimization

The paper gives a systematic study of the approximate versions of three greedy-type algorithms that are widely used in convex optimization. By approximate version we mean the one where some of evaluations are made with an error. Importance of such versions of greedy-type algorithms in convex optimization and in approximation theory was emphasized in previous literature.

stat.ML↗

arXiv2013

Chebushev Greedy Algorithm in convex optimization

Chebyshev Greedy Algorithm is a generalization of the well known Orthogonal Matching Pursuit defined in a Hilbert space to the case of Banach spaces. We apply this algorithm for constructing sparse approximate solutions (with respect to a given dictionary) to convex optimization problems. Rate of convergence results in a style of the Lebesgue-type inequalities are proved.

stat.ML↗

arXiv2013

Sparse approximation and recovery by greedy algorithms

We study sparse approximation by greedy algorithms. Our contribution is two-fold. First, we prove exact recovery with high probability of random $K$-sparse signals within $\lceil K(1+\e)\rceil$ iterations of the Orthogonal Matching Pursuit (OMP). This result shows that in a probabilistic sense the OMP is almost optimal for exact recovery. Second, we prove the Lebesgue-type inequalities for the Weak Chebyshev Greedy Algorithm, a generalization of the Weak Orthogonal Matching Pursuit to the case of a Banach space. The main novelty of these results is a Banach space setting instead of a Hilbert space setting. However, even in the case of a Hilbert space our results add some new elements to known results on the Lebesque-type inequalities for the RIP dictionaries. Our technique is a development of the recent technique created by Zhang.

math.NA↗

arXiv2013

Sparse approximation and recovery by greedy algorithms in Banach spaces

We study sparse approximation by greedy algorithms. We prove the Lebesgue-type inequalities for the Weak Chebyshev Greedy Algorithm (WCGA), a generalization of the Weak Orthogonal Matching Pursuit to the case of a Banach space. The main novelty of these results is a Banach space setting instead of a Hilbert space setting. The results are proved for redundant dictionaries satisfying certain conditions. Then we apply these general results to the case of bases. In particular, we prove that the WCGA provides almost optimal sparse approximation for the trigonometric system in $L_p$, $2\le p<\infty$.

stat.ML↗

arXiv2013

An inequality for the entropy numbers and its application

We prove an inequality for the entropy numbers in terms of nonlinear Kolmogorov's widths. This inequality is in a spirit of known inequalities of this type and it is adjusted to the form convenient in applications for $m$-term approximations with respect to a given system. Also, we obtain upper bounds for the $m$-term approximation by the Weak Relaxed Greedy Algorithm with respect to a system which is not a dictionary.

math.MG↗

arXiv2013

A remark on covering

We discuss construction of coverings of the unit ball of a finite dimensional Banach space. The well known technique of comparing volumes gives upper and lower bounds on covering numbers. This technique does not provide a construction of good coverings. Here we apply incoherent dictionaries for construction of good coverings. We use the following strategy. First, we build a good covering by balls with a radius close to one. Second, we iterate this construction to obtain a good covering for any radius. We mostly concentrate on the first step of this strategy.

math.MG↗