arXiv subjects

Shachar Lovett

Publications and source records attributed to Shachar Lovett.

At least 55 records · Page 3Linked to original sources

arXiv2020

Decision list compression by mild random restrictions

A decision list is an ordered list of rules. Each rule is specified by a term, which is a conjunction of literals, and a value. Given an input, the output of a decision list is the value corresponding to the first rule whose term is satisfied by the input. Decision lists generalize both CNFs and DNFs, and have been studied both in complexity theory and in learning theory. The size of a decision list is the number of rules, and its width is the maximal number of variables in a term. We prove that decision lists of small width can always be approximated by decision lists of small size, where we obtain sharp bounds. This in particular resolves a conjecture of Gopalan, Meka and Reingold (Computational Complexity, 2013) on DNF sparsification. An ingredient in our proof is a new random restriction lemma, which allows to analyze how DNFs (and more generally, decision lists) simplify if a small fraction of the variables are fixed. This is in contrast to the more commonly used switching lemma, which requires most of the variables to be fixed.

cs.CC↗

arXiv2020

Towards a combinatorial characterization of bounded memory learning

Combinatorial dimensions play an important role in the theory of machine learning. For example, VC dimension characterizes PAC learning, SQ dimension characterizes weak learning with statistical queries, and Littlestone dimension characterizes online learning. In this paper we aim to develop combinatorial dimensions that characterize bounded memory learning. We propose a candidate solution for the case of realizable strong learning under a known distribution, based on the SQ dimension of neighboring distributions. We prove both upper and lower bounds for our candidate solution, that match in some regime of parameters. In this parameter regime there is an equivalence between bounded memory and SQ learning. We conjecture that our characterization holds in a much wider regime of parameters.

cs.LG↗

arXiv2020

Noise-tolerant, Reliable Active Classification with Comparison Queries

With the explosion of massive, widely available unlabeled data in the past years, finding label and time efficient, robust learning algorithms has become ever more important in theory and in practice. We study the paradigm of active learning, in which algorithms with access to large pools of data may adaptively choose what samples to label in the hope of exponentially increasing efficiency. By introducing comparisons, an additional type of query comparing two points, we provide the first time and query efficient algorithms for learning non-homogeneous linear separators robust to bounded (Massart) noise. We further provide algorithms for a generalization of the popular Tsybakov low noise condition, and show how comparisons provide a strong reliability guarantee that is often impractical or impossible with only labels - returning a classifier that makes no errors with high probability.

cs.LG↗

arXiv2019

A bilinear Bogolyubov-Ruzsa lemma with poly-logarithmic bounds

The Bogolyubov-Ruzsa lemma, in particular the quantitative bounds obtained by Sanders, plays a central role in obtaining effective bounds for the inverse $U^3$ theorem for the Gowers norms. Recently, Gowers and Milićević applied a bilinear Bogolyubov-Ruzsa lemma as part of a proof of the inverse $U^4$ theorem with effective bounds. The goal of this note is to obtain quantitative bounds for the bilinear Bogolyubov-Ruzsa lemma which are similar to those obtained by Sanders for the Bogolyubov-Ruzsa lemma. We show that if a set $A \subset \mathbb{F}_p^n \times \mathbb{F}_p^n$ has density $α$, then after a constant number of horizontal and vertical sums, the set $A$ would contain a bilinear structure of co-dimension $r=\log^{O(1)} α^{-1}$. This improves the results of Gowers and Milićević which obtained similar results with a weaker bound of $r=\exp(\exp(\log^{O(1)} α^{-1}))$ and by Bienvenu and Lê which obtained $r=\exp(\exp(\exp(\log^{O(1)} α^{-1})))$.

math.CO↗

arXiv2019

From DNF compression to sunflower theorems via regularity

The sunflower conjecture is one of the most well-known open problems in combinatorics. It has several applications in theoretical computer science, one of which is DNF compression, due to Gopalan, Meka and Reingold [Computational Complexity 2013]. In this paper, we show that improved bounds for DNF compression imply improved bounds for the sunflower conjecture, which is the reverse direction of [Computational Complexity 2013]. The main approach is based on regularity of set systems and a structure-vs-pseudorandomness approach to the sunflower conjecture.

math.CO↗

arXiv2019

The analytic rank of tensors and its applications

The analytic rank of a tensor, first defined by Gowers and Wolf in the context of higher-order Fourier analysis, is defined to be the logarithm of the bias of the tensor. We prove that it is a subadditive measure of rank: that is, the analytic rank of the sum of two tensors is at most the sum of their individual analytic ranks. This analytic property turns out to have surprising applications: (i) common roots of tensors are always positively correlated; and (ii) the slice rank and partition rank, which were defined recently in the resolution of the cap-set problem in Ramsey theory, can be replaced by the analytic rank.

math.CO↗

arXiv2019

Torus polynomials: an algebraic approach to ACC lower bounds

We propose an algebraic approach to proving circuit lower bounds for ACC0 by defining and studying the notion of torus polynomials. We show how currently known polynomial-based approximation results for AC0 and ACC0 can be reformulated in this framework, implying that ACC0 can be approximated by low-degree torus polynomials. Furthermore, as a step towards proving ACC0 lower bounds for the majority function via our approach, we show that MAJORITY cannot be approximated by low-degree symmetric torus polynomials. We also pose several open problems related to our framework.

cs.CC↗

arXiv2018

Optimality of Linear Sketching under Modular Updates

We study the relation between streaming algorithms and linear sketching algorithms, in the context of binary updates. We show that for inputs in $n$ dimensions, the existence of efficient streaming algorithms which can process $Ω(n^2)$ updates implies efficient linear sketching algorithms with comparable cost. This improves upon the previous work of Li, Nguyen and Woodruff [LNW14] and Ai, Hu, Li and Woodruff [AHLW16] which required a triple-exponential number of updates to achieve a similar result for updates over integers. We extend our results to updates modulo $p$ for integers $p \ge 2$, and to approximation instead of exact computation.

cs.CC↗

arXiv2018

Generalized comparison trees for point-location problems

Let $H$ be an arbitrary family of hyper-planes in $d$-dimensions. We show that the point-location problem for $H$ can be solved by a linear decision tree that only uses a special type of queries called \emph{generalized comparison queries}. These queries correspond to hyperplanes that can be written as a linear combination of two hyperplanes from $H$; in particular, if all hyperplanes in $H$ are $k$-sparse then generalized comparisons are $2k$-sparse. The depth of the obtained linear decision tree is polynomial in $d$ and logarithmic in $|H|$, which is comparable to previous results in the literature that use general linear queries. This extends the study of comparison trees from a previous work by the authors [Kane {et al.}, FOCS 2017]. The main benefit is that using generalized comparison queries allows to overcome limitations that apply for the more restricted type of comparison queries. Our analysis combines a seminal result of Forster regarding sets in isotropic position [Forster, JCSS 2002], the margin-based inference dimension analysis for comparison queries from [Kane {et al.}, FOCS 2017], and compactness arguments.

cs.CG↗

arXiv2018

MDS matrices over small fields: A proof of the GM-MDS conjecture

An MDS matrix is a matrix whose minors all have full rank. A question arising in coding theory is what zero patterns can MDS matrices have. There is a natural combinatorial characterization (called the MDS condition) which is necessary over any field, as well as sufficient over very large fields by a probabilistic argument. Dau et al. (ISIT 2014) conjectured that the MDS condition is sufficient over small fields as well, where the construction of the matrix is algebraic instead of probabilistic. This is known as the GM-MDS conjecture. Concretely, if a $k \times n$ zero pattern satisfies the MDS condition, then they conjecture that there exists an MDS matrix with this zero pattern over any field of size $|\mathbb{F}| \ge n+k-1$. In recent years, this conjecture was proven in several special cases. In this work, we resolve the conjecture.

cs.DM↗

arXiv2017

The Gram-Schmidt Walk: A Cure for the Banaszczyk Blues

An important result in discrepancy due to Banaszczyk states that for any set of $n$ vectors in $\mathbb{R}^m$ of $\ell_2$ norm at most $1$ and any convex body $K$ in $\mathbb{R}^m$ of Gaussian measure at least half, there exists a $\pm 1$ combination of these vectors which lies in $5K$. This result implies the best known bounds for several problems in discrepancy. Banaszczyk's proof of this result is non-constructive and a major open problem has been to give an efficient algorithm to find such a $\pm 1$ combination of the vectors. In this paper, we resolve this question and give an efficient randomized algorithm to find a $\pm 1$ combination of the vectors which lies in $cK$ for $c>0$ an absolute constant. This leads to new efficient algorithms for several problems in discrepancy theory.

cs.DS↗

arXiv2017

Active classification with comparison queries

We study an extension of active learning in which the learning algorithm may ask the annotator to compare the distances of two examples from the boundary of their label-class. For example, in a recommendation system application (say for restaurants), the annotator may be asked whether she liked or disliked a specific restaurant (a label query); or which one of two restaurants did she like more (a comparison query). We focus on the class of half spaces, and show that under natural assumptions, such as large margin or bounded bit-description of the input examples, it is possible to reveal all the labels of a sample of size $n$ using approximately $O(\log n)$ queries. This implies an exponential improvement over classical active learning, where only label queries are allowed. We complement these results by showing that if any of these assumptions is removed then, in the worst case, $Ω(n)$ queries are required. Our results follow from a new general framework of active learning with additional queries. We identify a combinatorial dimension, called the \emph{inference dimension}, that captures the query complexity when each additional query is determined by $O(1)$ examples (such as comparison queries, each of which is determined by the two compared examples). Our results for half spaces follow by bounding the inference dimension in the cases discussed above.

cs.LG↗

arXiv2017

A counterexample to a strong variant of the Polynomial Freiman-Ruzsa conjecture in Euclidean space

The Polynomial Freiman-Ruzsa conjecture is one of the central open problems in additive combinatorics. If true, it would give tight quantitative bounds relating combinatorial and algebraic notions of approximate subgroups. In this note, we restrict our attention to subsets of Euclidean space. In this regime, the original conjecture considers approximate algebraic subgroups as the set of lattice points in a convex body. Green asked in 2007 whether this can be simplified to a generalized arithmetic progression, while not losing more than a polynomial factor in the underlying parameters. We give a negative answer to this question, based on a recent reverse Minkowski theorem combined with estimates for random lattices.

math.NT↗

arXiv2017

Near-optimal linear decision trees for k-SUM and related problems

We construct near optimal linear decision trees for a variety of decision problems in combinatorics and discrete geometry. For example, for any constant $k$, we construct linear decision trees that solve the $k$-SUM problem on $n$ elements using $O(n \log^2 n)$ linear queries. Moreover, the queries we use are comparison queries, which compare the sums of two $k$-subsets; when viewed as linear queries, comparison queries are $2k$-sparse and have only $\{-1,0,1\}$ coefficients. We give similar constructions for sorting sumsets $A+B$ and for solving the SUBSET-SUM problem, both with optimal number of queries, up to poly-logarithmic terms. Our constructions are based on the notion of "inference dimension", recently introduced by the authors in the context of active classification with comparison queries. This can be viewed as another contribution to the fruitful link between machine learning and discrete geometry, which goes back to the discovery of the VC dimension.

cs.CG↗

arXiv2017

Probabilistic existence of regular combinatorial structures

We show the existence of regular combinatorial objects which previously were not known to exist. Specifically, for a wide range of the underlying parameters, we show the existence of non-trivial orthogonal arrays, t-designs, and t-wise permutations. In all cases, the sizes of the objects are optimal up to polynomial overhead. The proof of existence is probabilistic. We show that a randomly chosen structure has the required properties with positive yet tiny probability. Our method allows also to give rather precise estimates on the number of objects of a given size and this is applied to count the number of orthogonal arrays, t-designs and regular hypergraphs. The main technical ingredient is a special local central limit theorem for suitable lattice random walks with finitely many steps.

math.CO↗

arXiv2017

The independence number of the Birkhoff polytope graph, and applications to maximally recoverable codes

Maximally recoverable codes are codes designed for distributed storage which combine quick recovery from single node failure and optimal recovery from catastrophic failure. Gopalan et al [SODA 2017] studied the alphabet size needed for such codes in grid topologies and gave a combinatorial characterization for it. Consider a labeling of the edges of the complete bipartite graph $K_{n,n}$ with labels coming from $F_2^d$ , that satisfies the following condition: for any simple cycle, the sum of the labels over its edges is nonzero. The minimal d where this is possible controls the alphabet size needed for maximally recoverable codes in n x n grid topologies. Prior to the current work, it was known that d is between $(\log n)^2$ and $n\log n$. We improve both bounds and show that d is linear in n. The upper bound is a recursive construction which beats the random construction. The lower bound follows by first relating the problem to the independence number of the Birkhoff polytope graph, and then providing tight bounds for it using the representation theory of the symmetric group.

math.CO↗

arXiv2016

Towards a Constructive Version of Banaszczyk's Vector Balancing Theorem

An important theorem of Banaszczyk (Random Structures & Algorithms `98) states that for any sequence of vectors of $\ell_2$ norm at most $1/5$ and any convex body $K$ of Gaussian measure $1/2$ in $\mathbb{R}^n$, there exists a signed combination of these vectors which lands inside $K$. A major open problem is to devise a constructive version of Banaszczyk's vector balancing theorem, i.e. to find an efficient algorithm which constructs the signed combination. We make progress towards this goal along several fronts. As our first contribution, we show an equivalence between Banaszczyk's theorem and the existence of $O(1)$-subgaussian distributions over signed combinations. For the case of symmetric convex bodies, our equivalence implies the existence of a universal signing algorithm (i.e. independent of the body), which simply samples from the subgaussian sign distribution and checks to see if the associated combination lands inside the body. For asymmetric convex bodies, we provide a novel recentering procedure, which allows us to reduce to the case where the body is symmetric. As our second main contribution, we show that the above framework can be efficiently implemented when the vectors have length $O(1/\sqrt{\log n})$, recovering Banaszczyk's results under this stronger assumption. More precisely, we use random walk techniques to produce the required $O(1)$-subgaussian signing distributions when the vectors have length $O(1/\sqrt{\log n})$, and use a stochastic gradient ascent method to implement the recentering procedure for asymmetric bodies.

cs.DS↗

arXiv2016

The Fourier structure of low degree polynomials

We study the structure of the Fourier coefficients of low degree multivariate polynomials over finite fields. We consider three properties: (i) the number of nonzero Fourier coefficients; (ii) the sum of the absolute value of the Fourier coefficients; and (iii) the size of the linear subspace spanned by the nonzero Fourier coefficients. For quadratic polynomials, tight relations are known between all three quantities. In this work, we extend this relation to higher degree polynomials. Specifically, for degree $d$ polynomials, we show that the three quantities are equivalent up to factors exponential in $d$.

math.CO↗