arXiv subjects

Leonid

Publications and source records attributed to Leonid.

6 recordsLinked to original sources

arXiv2012

Obtaining Measure Concentration from Markov Contraction

Concentration bounds for non-product, non-Haar measures are fairly recent: the first such result was obtained for contracting Markov chains by Marton in 1996 via the coupling method. The work that followed, with few exceptions, also used coupling. Although this technique is of unquestionable utility as a theoretical tool, it is not always simple to apply. As an alternative to coupling, we use the elementary Markov contraction lemma to obtain simple, useful, and apparently novel concentration results for various Markov-type processes. Our technique consists of expressing probabilities as matrix products and applying Markov contraction to these expressions; thus it is fairly general and holds the potential to yield further results in this vein.

math.PR↗

arXiv2011

Statistical estimation requires unbounded memory

We investigate the existence of bounded-memory consistent estimators of various statistical functionals. This question is resolved in the negative in a rather strong sense. We propose various bounded-memory approximations, using techniques from automata theory and stochastic processes. Some questions of potential interest are raised for future work.

stat.CO↗

arXiv2011

VC bounds on the cardinality of nearly orthogonal function classes

We bound the number of nearly orthogonal vectors with fixed VC-dimension over $\setpm^n$. Our bounds are of interest in machine learning and empirical process theory and improve previous bounds by Haussler. The bounds are based on a simple projection argument and the generalize to other product spaces. Along the way we derive tight bounds on the sum of binomial coefficients in terms of the entropy function.

math.CO↗

arXiv2009

Concentration inequalities for dependent Random variables via the martingale method

The martingale method is used to establish concentration inequalities for a class of dependent random sequences on a countable state space, with the constants in the inequalities expressed in terms of certain mixing coefficients. Along the way, bounds are obtained on martingale differences associated with the random sequences, which may be of independent interest. As applications of the main result, concentration inequalities are also derived for inhomogeneous Markov chains and hidden Markov chains, and an extremal property associated with their martingale difference bounds is established. This work complements and generalizes certain concentration inequalities obtained by Marton and Samson, while also providing different proofs of some known results.

math.PR↗

arXiv2007

A Universal Kernel for Learning Regular Languages

We give a universal kernel that renders all the regular languages linearly separable. We are not able to compute this kernel efficiently and conjecture that it is intractable, but we do have an efficient $\eps$-approximation.

cs.LG↗

arXiv2007

Constructing processes with prescribed mixing coefficients

The rate at which dependencies between future and past observations decay in a random process may be quantified in terms of mixing coefficients. The latter in turn appear in strong laws of large numbers and concentration of measure results for dependent random variables. Questions regarding what rates are possible for various notions of mixing have been posed since the 1960's, and have important implications for some open problems in the theory of strong mixing conditions. This paper deals with $η$-mixing, a notion defined in [Kontorovich and Ramanan], which is closely related to $ϕ$-mixing. We show that there exist measures on finite sequences with essentially arbitrary $η$-mixing coefficients, as well as processes with arbitrarily slow mixing rates.

math.PR↗