arXiv subjects

Robert Krauthgamer

Publications and source records attributed to Robert Krauthgamer.

At least 73 records · Page 4Linked to original sources

arXiv2020

Universal Streaming of Subset Norms

Most known algorithms in the streaming model of computation aim to approximate a single function such as an $\ell_p$-norm. In 2009, Nelson [\url{https://sublinear.info}, Open Problem 30] asked if it possible to design \emph{universal algorithms}, that simultaneously approximate multiple functions of the stream. In this paper we answer the question of Nelson for the class of \emph{subset $\ell_0$-norms} in the insertion-only frequency-vector model. Given a family of subsets $\mathcal{S}\subset 2^{[n]}$, we provide a single streaming algorithm that can $(1\pm ε)$-approximate the subset-norm for every $S\in\mathcal{S}$. Here, the subset-$\ell_p$-norm of $v\in \mathbb{R}^n$ with respect to set $S\subseteq [n]$ is the $\ell_p$-norm of vector $v_{|S}$ (which denotes restricting $v$ to $S$, by zeroing all other coordinates). Our main result is a near-tight characterization of the space complexity of every family $\mathcal{S}\subset 2^{[n]}$ of subset-$\ell_0$-norms in insertion-only streams, expressed in terms of the "heavy-hitter dimension" of $\mathcal{S}$, a new combinatorial quantity that is related to the VC-dimension of $\mathcal{S}$. In contrast, we show that the more general turnstile and sliding-window models require a much larger space usage. All these results easily extend to $\ell_1$. In addition, we design algorithms for two other subset-$\ell_p$-norm variants. These can be compared to the Priority Sampling algorithm of Duffield, Lund and Thorup [JACM 2007], which achieves additive approximation $ε\|{v}\|$ for all possible subsets ($\mathcal{S}=2^{[n]}$) in the entry-wise update model. One of our algorithms extends this algorithm to handle turnstile updates, and another one achieves multiplicative approximation given a family $\mathcal{S}$.

cs.DS↗

arXiv2020

Schatten Norms in Matrix Streams: Hello Sparsity, Goodbye Dimension

Spectral functions of large matrices contains important structural information about the underlying data, and is thus becoming increasingly important. Many times, large matrices representing real-world data are \emph{sparse} or \emph{doubly sparse} (i.e., sparse in both rows and columns), and are accessed as a \emph{stream} of updates, typically organized in \emph{row-order}. In this setting, where space (memory) is the limiting resource, all known algorithms require space that is polynomial in the dimension of the matrix, even for sparse matrices. We address this challenge by providing the first algorithms whose space requirement is \emph{independent of the matrix dimension}, assuming the matrix is doubly-sparse and presented in row-order. Our algorithms approximate the Schatten $p$-norms, which we use in turn to approximate other spectral functions, such as logarithm of the determinant, trace of matrix inverse, and Estrada index. We validate these theoretical performance bounds by numerical experiments on real-world matrices representing social networks. We further prove that multiple passes are unavoidable in this setting, and show extensions of our primary technique, including a trade-off between space requirements and number of passes.

cs.DS↗

arXiv2019

Refined Vertex Sparsifiers of Planar Graphs

We study the following version of cut sparsification. Given a large edge-weighted network $G$ with $k$ terminal vertices, compress it into a smaller network $H$ with the same terminals, such that every minimum terminal cut in $H$ approximates the corresponding one in $G$, up to a factor $q\geq 1$ that is called the quality. (The case $q=1$ is known also as a mimicking network). We provide new insights about the structure of minimum terminal cuts, leading to new results for cut sparsifiers of planar graphs. Our first contribution identifies a subset of the minimum terminal cuts, which we call elementary, that generates all the others. Consequently, $H$ is a cut sparsifier if and only if it preserves all the elementary terminal cuts (up to this factor $q$). This structural characterization lead to improved bounds on the size of $H$. For example, it improve the bound of mimicking-network size for planar graphs into a near-optimal one. Our second and main contribution is to refine the known bounds in terms of $γ=γ(G)$, which is defined as the minimum number of faces that are incident to all the terminals in a planar graph $G$. We prove that the number of elementary terminal cuts is $O((2k/γ)^{2γ})$ (compared to $O(2^k)$ terminal cuts), and furthermore obtain a mimicking-network of size $O(γ2^{2γ} k^4)$, which is near-optimal as a function of $γ$. In the analysis we break the elementary terminal cuts into fragments, and count them carefully. Our third contribution is a duality between cut sparsification and distance sparsification for certain planar graphs, when the sparsifier $H$ is required to be a minor of $G$. This duality connects problems that were previously studied separately, implying new results, new proofs of known results, and equivalences between open gaps.

cs.DS↗

arXiv2019

Sublinear Algorithms for Gap Edit Distance

The edit distance is a way of quantifying how similar two strings are to one another by counting the minimum number of character insertions, deletions, and substitutions required to transform one string into the other. A simple dynamic programming computes the edit distance between two strings of length $n$ in $O(n^2)$ time, and a more sophisticated algorithm runs in time $O(n+t^2)$ when the edit distance is $t$ [Landau, Myers and Schmidt, SICOMP 1998]. In pursuit of obtaining faster running time, the last couple of decades have seen a flurry of research on approximating edit distance, including polylogarithmic approximation in near-linear time [Andoni, Krauthgamer and Onak, FOCS 2010], and a constant-factor approximation in subquadratic time [Chakrabarty, Das, Goldenberg, Koucký and Saks, FOCS 2018]. We study sublinear-time algorithms for small edit distance, which was investigated extensively because of its numerous applications. Our main result is an algorithm for distinguishing whether the edit distance is at most $t$ or at least $t^2$ (the quadratic gap problem) in time $\tilde{O}(\frac{n}{t}+t^3)$. This time bound is sublinear roughly for all $t$ in $[ω(1), o(n^{1/3})]$, which was not known before. The best previous algorithms solve this problem in sublinear time only for $t=ω(n^{1/3})$ [Andoni and Onak, STOC 2009]. Our algorithm is based on a new approach that adaptively switches between uniform sampling and reading contiguous blocks of the input strings. In contrast, all previous algorithms choose which coordinates to query non-adaptively. Moreover, it can be extended to solve the $t$ vs $t^{2-ε}$ gap problem in time $\tilde{O}(\frac{n}{t^{1-ε}}+t^3)$.

cs.CC↗

arXiv2019

New Algorithms and Lower Bounds for All-Pairs Max-Flow in Undirected Graphs

We investigate the time-complexity of the All-Pairs Max-Flow problem: Given a graph with $n$ nodes and $m$ edges, compute for all pairs of nodes the maximum-flow value between them. If Max-Flow (the version with a given source-sink pair $s,t$) can be solved in time $T(m)$, then an $O(n^2) \cdot T(m)$ is a trivial upper bound. But can we do better? For directed graphs, recent results in fine-grained complexity suggest that this time bound is essentially optimal. In contrast, for undirected graphs with edge capacities, a seminal algorithm of Gomory and Hu (1961) runs in much faster time $O(n)\cdot T(m)$. Under the plausible assumption that Max-Flow can be solved in near-linear time $m^{1+o(1)}$, this half-century old algorithm yields an $nm^{1+o(1)}$ bound. Several other algorithms have been designed through the years, including $\tilde{O}(mn)$ time for unit-capacity edges (unconditionally), but none of them break the $O(mn)$ barrier. Meanwhile, no super-linear lower bound was shown for undirected graphs. We design the first hardness reductions for All-Pairs Max-Flow in undirected graphs, giving an essentially optimal lower bound for the $\textit{node-capacities}$ setting. For edge capacities, our efforts to prove similar lower bounds have failed, but we have discovered a surprising new algorithm that breaks the $O(mn)$ barrier for graphs with unit-capacity edges! Assuming $T(m)=m^{1+o(1)}$, our algorithm runs in time $m^{3/2 +o(1)}$ and outputs a cut-equivalent tree (similarly to the Gomory-Hu algorithm). Even with current Max-Flow algorithms we improve state-of-the-art as long as $m=O(n^{5/3-\varepsilon})$. Finally, we explain the lack of lower bounds by proving a $\textit{non-reducibility}$ result. This result is based on a new quasi-linear time $\tilde{O}(m)$ $\textit{non-deterministic}$ algorithm for constructing a cut-equivalent tree and may be of independent interest.

cs.DS↗

arXiv2019

Coresets for Ordered Weighted Clustering

We design coresets for Ordered k-Median, a generalization of classical clustering problems such as k-Median and k-Center, that offers a more flexible data analysis, like easily combining multiple objectives (e.g., to increase fairness or for Pareto optimization). Its objective function is defined via the Ordered Weighted Averaging (OWA) paradigm of Yager (1988), where data points are weighted according to a predefined weight vector, but in order of their contribution to the objective (distance from the centers). A powerful data-reduction technique, called a coreset, is to summarize a point set $X$ in $\mathbb{R}^d$ into a small (weighted) point set $X'$, such that for every set of $k$ potential centers, the objective value of the coreset $X'$ approximates that of $X$ within factor $1\pm ε$. When there are multiple objectives (weights), the above standard coreset might have limited usefulness, whereas in a \emph{simultaneous} coreset, which was introduced recently by Bachem and Lucic and Lattanzi (2018), the above approximation holds for all weights (in addition to all centers). Our main result is a construction of a simultaneous coreset of size $O_{ε, d}(k^2 \log^2 |X|)$ for Ordered k-Median. To validate the efficacy of our coreset construction we ran experiments on a real geographical data set. We find that our algorithm produces a small coreset, which translates to a massive speedup of clustering computations, while maintaining high accuracy for a range of weights.

cs.DS↗

arXiv2019

Faster Algorithms for All-Pairs Bounded Min-Cuts

The All-Pairs Min-Cut problem (aka All-Pairs Max-Flow) asks to compute a minimum $s$-$t$ cut (or just its value) for all pairs of vertices $s,t$. We study this problem in directed graphs with unit edge/vertex capacities (corresponding to edge/vertex connectivity). Our focus is on the $k$-bounded case, where the algorithm has to find all pairs with min-cut value less than $k$, and report only those. The most basic case $k=1$ is the Transitive Closure (TC) problem, which can be solved in graphs with $n$ vertices and $m$ edges in time $O(mn)$ combinatorially, and in time $O(n^ω)$ where $ω<2.38$ is the matrix-multiplication exponent. These time bounds are conjectured to be optimal. We present new algorithms and conditional lower bounds that advance the frontier for larger $k$, as follows: (i) A randomized algorithm for vertex capacities that runs in time $O((nk)^ω)$. (ii) Two deterministic algorithms for edge capacities (which is more general) that work in DAGs and further reports a minimum cut for each pair. The first algorithm is combinatorial (does not involve matrix multiplication) and runs in time $O(2^{O(k^2)}\cdot mn)$. The second algorithm can be faster on dense DAGs and runs in time $O((k\log n)^{4^k+o(k)} n^ω)$. (iii) The first super-cubic lower bound of $n^{ω-1-o(1)} k^2$ time under the $4$-Clique conjecture, which holds even in the simplest case of DAGs with unit vertex capacities. It improves on the previous (SETH-based) lower bounds even in the unbounded setting $k=n$. For combinatorial algorithms, our reduction implies an $n^{2-o(1)} k^2$ conditional lower bound. Thus, we identify new settings where the complexity of the problem is (conditionally) higher than that of TC.

cs.DS↗

arXiv2019

The Set Cover Conjecture and Subgraph Isomorphism with a Tree Pattern

In the Set Cover problem, the input is a ground set of $n$ elements and a collection of $m$ sets, and the goal is to find the smallest sub-collection of sets whose union is the entire ground set. The fastest algorithm known runs in time $O(mn2^n)$ [Fomin et al., WG 2004], and the Set Cover Conjecture (SeCoCo) [Cygan et al., TALG 2016] asserts that for every fixed $\varepsilon>0$, no algorithm can solve Set Cover in time $2^{(1-\varepsilon)n}poly(m)$, even if set sizes are bounded by $Δ=Δ(\varepsilon)$. We show strong connections between this problem and kTree, a special case of Subgraph Isomorphism where the input is an $n$-node graph $G$ and a $k$-node tree $T$, and the goal is to determine whether $G$ has a subgraph isomorphic to $T$. First, we propose a weaker conjecture Log-SeCoCo, that allows input sets of size $Δ=O(1/\varepsilon \cdot\log n)$, and show that an algorithm breaking Log-SeCoCo would imply a faster algorithm than the currently known $2^n poly(n)$-time algorithm [Koutis and Williams, TALG 2016] for Directed nTree, which is kTree with $k=n$ and arbitrary directions to the edges of $G$ and $T$. This would also improve the running time for Directed Hamiltonicity, for which no algorithm significantly faster than $2^n poly(n)$ is known despite extensive research. Second, we prove that if Set Cover cannot be solved significantly faster than $2^npoly(m)$ (an assumption even weaker than Log-SeCoCo), then kTree cannot be computed significantly faster than $2^kpoly(n)$, the running time of the Koutis and Williams' algorithm. Applying the same techniques to the p-Partial Cover problem, a parameterized version of Set Cover that requires covering at least $p$ elements, we obtain a new algorithm with running time $(2+\varepsilon)^p (m+n)^{O(1/\varepsilon)}$ for arbitrary $\varepsilon>0$, which improves previous work and is nearly optimal assuming say Log-SeCoCo.

cs.DS↗

arXiv2018

Relaxed Voronoi: a Simple Framework for Terminal-Clustering Problems

We reprove three known algorithmic bounds for terminal-clustering problems, using a single framework that leads to simpler proofs. In this genre of problems, the input is a metric space $(X,d)$ (possibly arising from a graph) and a subset of terminals $K\subset X$, and the goal is to partition the points $X$ such that each part, called a cluster, contains exactly one terminal (possibly with connectivity requirements) so as to minimize some objective. The three bounds we reprove are for Steiner Point Removal on trees [Gupta, SODA 2001], for Metric $0$-Extension in bounded doubling dimension [Lee and Naor, unpublished 2003], and for Connected Metric $0$-Extension [Englert et al., SICOMP 2014]. A natural approach is to cluster each point with its closest terminal, which would partition $X$ into so-called Voronoi cells, but this approach can fail miserably due to its stringent cluster boundaries. A now-standard fix, which we call the Relaxed-Voronoi framework, is to use enlarged Voronoi cells, but to obtain disjoint clusters, the cells are computed greedily according to some order. This method, first proposed by Calinescu, Karloff and Rabani [SICOMP 2004], was employed successfully to provide state-of-the-art results for terminal-clustering problems on general metrics. However, for restricted families of metrics, e.g., trees and doubling metrics, only more complicated, ad-hoc algorithms are known. Our main contribution is to demonstrate that the Relaxed-Voronoi algorithm is applicable to restricted metrics, and actually leads to relatively simple algorithms and analyses.

cs.DS↗

arXiv2018

Flow-Cut Gaps and Face Covers in Planar Graphs

The relationship between the sparsest cut and the maximum concurrent multi-flow in graphs has been studied extensively. For general graphs with $k$ terminal pairs, the flow-cut gap is $O(\log k)$, and this is tight. But when topological restrictions are placed on the flow network, the situation is far less clear. In particular, it has been conjectured that the flow-cut gap in planar networks is $O(1)$, while the known bounds place the gap somewhere between $2$ (Lee and Raghavendra, 2003) and $O(\sqrt{\log k})$ (Rao, 1999). A seminal result of Okamura and Seymour (1981) shows that when all the terminals of a planar network lie on a single face, the flow-cut gap is exactly $1$. This setting can be generalized by considering planar networks where the terminals lie on $γ>1$ faces in some fixed planar drawing. Lee and Sidiropoulos (2009) proved that the flow-cut gap is bounded by a function of $γ$, and Chekuri, Shepherd, and Weibel (2013) showed that the gap is at most $3γ$. We prove that the flow-cut gap is $O(\logγ)$, by showing that the edge-weighted shortest-path metric induced on the terminals admits a stochastic embedding into trees with distortion $O(\logγ)$, which is tight. The preceding results refer to the setting of edge-capacitated networks. For vertex-capacitated networks, it can be significantly more challenging to control flow-cut gaps. While there is no exact vertex-capacitated version of the Okamura-Seymour Theorem, an approximate version holds; Lee, Mendel, and Moharrami (2015) showed that the vertex-capacitated flow-cut gap is $O(1)$ on planar networks whose terminals lie on a single face. We prove that the flow-cut gap is $O(γ)$ for vertex-capacitated instances when the terminals lie on at most $γ$ faces. In fact, this result holds in the more general setting of submodular vertex capacities.

cs.DS↗

arXiv2018

Matrix Norms in Data Streams: Faster, Multi-Pass and Row-Order

A central problem in data streams is to characterize which functions of an underlying frequency vector can be approximated efficiently. Recently there has been considerable effort in extending this problem to that of estimating functions of a matrix that is presented as a data-stream. This setting generalizes classical problems to the analogous ones for matrices. For example, instead of estimating frequent-item counts, we now wish to estimate "frequent-direction" counts. A related example is to estimate norms, which now correspond to estimating a vector norm on the singular values of the matrix. Despite recent efforts, the current understanding for such matrix problems is considerably weaker than that for vector problems. We study a number of aspects of estimating matrix norms in a stream that have not previously been considered: (1) multi-pass algorithms, (2) algorithms that see the underlying matrix one row at a time, and (3) time-efficient algorithms. Our multi-pass and row-order algorithms use less memory than what is provably required in the single-pass and entrywise-update models, and thus give separations between these models (in terms of memory). Moreover, all of our algorithms are considerably faster than previous ones. We also prove a number of lower bounds, and obtain for instance, a near-complete characterization of the memory required of row-order algorithms for estimating Schatten $p$-norms of sparse matrices.

cs.DS↗

arXiv2018

On Solving Linear Systems in Sublinear Time

We study \emph{sublinear} algorithms that solve linear systems locally. In the classical version of this problem the input is a matrix $S\in \mathbb{R}^{n\times n}$ and a vector $b\in\mathbb{R}^n$ in the range of $S$, and the goal is to output $x\in \mathbb{R}^n$ satisfying $Sx=b$. For the case when the matrix $S$ is symmetric diagonally dominant (SDD), the breakthrough algorithm of Spielman and Teng [STOC 2004] approximately solves this problem in near-linear time (in the input size which is the number of non-zeros in $S$), and subsequent papers have further simplified, improved, and generalized the algorithms for this setting. Here we focus on computing one (or a few) coordinates of $x$, which potentially allows for sublinear algorithms. Formally, given an index $u\in [n]$ together with $S$ and $b$ as above, the goal is to output an approximation $\hat{x}_u$ for $x^*_u$, where $x^*$ is a fixed solution to $Sx=b$. Our results show that there is a qualitative gap between SDD matrices and the more general class of positive semidefinite (PSD) matrices. For SDD matrices, we develop an algorithm that approximates a single coordinate $x_{u}$ in time that is polylogarithmic in $n$, provided that $S$ is sparse and has a small condition number (e.g., Laplacian of an expander graph). The approximation guarantee is additive $| \hat{x}_u-x^*_u | \le ε\| x^* \|_\infty$ for accuracy parameter $ε>0$. We further prove that the condition-number assumption is necessary and tight. In contrast to the SDD matrices, we prove that for certain PSD matrices $S$, the running time must be at least polynomial in $n$. This holds even when one wants to obtain the same additive approximation, and $S$ has bounded sparsity and condition number.

cs.DS↗

arXiv2018

Batch Sparse Recovery, or How to Leverage the Average Sparsity

We introduce a \emph{batch} version of sparse recovery, where the goal is to report a sequence of vectors $A_1',\ldots,A_m' \in \mathbb{R}^n$ that estimate unknown signals $A_1,\ldots,A_m \in \mathbb{R}^n$ using a few linear measurements, each involving exactly one signal vector, under an assumption of \emph{average sparsity}. More precisely, we want to have \newline $(1) \;\;\; \sum_{j \in [m]}{\|A_j- A_j'\|_p^p} \le C \cdot \min \Big\{ \sum_{j \in [m]}{\|A_j - A_j^*\|_p^p} \Big\}$ for predetermined constants $C \ge 1$ and $p$, where the minimum is over all $A_1^*,\ldots,A_m^*\in\mathbb{R}^n$ that are $k$-sparse on average. We assume $k$ is given as input, and ask for the minimal number of measurements required to satisfy $(1)$. The special case $m=1$ is known as stable sparse recovery and has been studied extensively. We resolve the question for $p =1$ up to polylogarithmic factors, by presenting a randomized adaptive scheme that performs $\tilde{O}(km)$ measurements and with high probability has output satisfying $(1)$, for arbitrarily small $C > 1$. Finally, we show that adaptivity is necessary for every non-trivial scheme.

cs.DS↗

arXiv2018

Conditional Lower Bound for Subgraph Isomorphism with a Tree Pattern

The kTree problem is a special case of Subgraph Isomorphism where the pattern graph is a tree, that is, the input is an $n$-node graph $G$ and a $k$-node tree $T$, and the goal is to determine whether $G$ has a subgraph isomorphic to $T$. We provide evidence that this problem cannot be computed significantly faster than $2^{k} \textsf{poly}(n)$, which matches the fastest algorithm known for this problem by Koutis and Williams [ICALP 2009 and TALG 2016]. Specifically, we show that if kTree can be solved in time $(2-\varepsilon)^k \textsf{poly}(n)$ for some constant $\varepsilon>0$, then Set Cover with $n'$ elements and $m'$ sets can be solved in time $(2-δ)^{n'} \textsf{poly}(m')$ for a constant $δ(\varepsilon) > 0$, which would refute the Set Cover Conjecture by Cygan et al. [CCC 2012 and TALG 2016]. Our techniques yield a new algorithm for the p-Partial Cover problem, a parameterized version of Set Cover that requires covering at least $p$ elements (rather than all elements). Its running time is $(2+\varepsilon)^p (m')^{O(1/\varepsilon)}$ for any fixed $\varepsilon>0$, which improves the previous $2.597^p \textsf{poly}(m')$-time algorithm by Zehavi [ESA 2015]. Our running time is nearly optimal, as a $(2-\varepsilon')^p \textsf{poly}(m')$-time algorithm would refute the Set Cover Conjecture.

cs.DS↗

arXiv2017

Tight Bounds for Gomory-Hu-like Cut Counting

By a classical result of Gomory and Hu (1961), in every edge-weighted graph $G=(V,E,w)$, the minimum $st$-cut values, when ranging over all $s,t\in V$, take at most $|V|-1$ distinct values. That is, these $\binom{|V|}{2}$ instances exhibit redundancy factor $Ω(|V|)$. They further showed how to construct from $G$ a tree $(V,E',w')$ that stores all minimum $st$-cut values. Motivated by this result, we obtain tight bounds for the redundancy factor of several generalizations of the minimum $st$-cut problem. 1. Group-Cut: Consider the minimum $(A,B)$-cut, ranging over all subsets $A,B\subseteq V$ of given sizes $|A|=α$ and $|B|=β$. The redundancy factor is $Ω_{α,β}(|V|)$. 2. Multiway-Cut: Consider the minimum cut separating every two vertices of $S\subseteq V$, ranging over all subsets of a given size $|S|=k$. The redundancy factor is $Ω_{k}(|V|)$. 3. Multicut: Consider the minimum cut separating every demand-pair in $D\subseteq V\times V$, ranging over collections of $|D|=k$ demand pairs. The redundancy factor is $Ω_{k}(|V|^k)$. This result is a bit surprising, as the redundancy factor is much larger than in the first two problems. A natural application of these bounds is to construct small data structures that stores all relevant cut values, like the Gomory-Hu tree. We initiate this direction by giving some upper and lower bounds.

cs.DS↗

arXiv2017

Streaming Symmetric Norms via Measure Concentration

We characterize the streaming space complexity of every symmetric norm $l$ (a norm on $\mathbb{R}^n$ invariant under sign-flips and coordinate-permutations), by relating this space complexity to the measure-concentration characteristics of $l$. Specifically, we provide nearly matching upper and lower bounds on the space complexity of calculating a $(1\pmε)$-approximation to the norm of the stream, for every $0<ε\leq 1/2$. (The bounds match up to $poly(ε^{-1} \log n)$ factors.) We further extend those bounds to any large approximation ratio $D\geq 1.1$, showing that the decrease in space complexity is proportional to $D^2$, and that this factor the best possible. All of the bounds depend on the median of $l(x)$ when $x$ is drawn uniformly from the $l_2$ unit sphere. The same median governs many phenomena in high-dimensional spaces, such as large-deviation bounds and the critical dimension in Dvoretzky's Theorem. The family of symmetric norms contains several well-studied norms, such as all $l_p$~norms, and indeed we provide a new explanation for the disparity in space complexity between $p\le 2$ and $p>2$. In addition, we apply our general results to easily derive bounds for several norms that were not studied before in the streaming model, including the top-$k$ norm and the $k$-support norm, which was recently employed for machine learning tasks. Overall, these results make progress on two outstanding problems in the area of sublinear algorithms (Problems 5 and 30 in~\url{http://sublinear.info}).

cs.DS↗

arXiv2017

Efficient Regression in Metric Spaces via Approximate Lipschitz Extension

We present a framework for performing efficient regression in general metric spaces. Roughly speaking, our regressor predicts the value at a new point by computing a Lipschitz extension --- the smoothest function consistent with the observed data --- after performing structural risk minimization to avoid overfitting. We obtain finite-sample risk bounds with minimal structural and noise assumptions, and a natural speed-precision tradeoff. The offline (learning) and online (prediction) stages can be solved by convex programming, but this naive approach has runtime complexity $O(n^3)$, which is prohibitive for large datasets. We design instead a regression algorithm whose speed and generalization performance depend on the intrinsic dimension of the data, to which the algorithm adapts. While our main innovation is algorithmic, the statistical results may also be of independent interest.

cs.LG↗

arXiv2017

Sketching and Embedding are Equivalent for Norms

An outstanding open question posed by Guha and Indyk in 2006 asks to characterize metric spaces in which distances can be estimated using efficient sketches. Specifically, we say that a sketching algorithm is efficient if it achieves constant approximation using constant sketch size. A well-known result of Indyk (J. ACM, 2006) implies that a metric that admits a constant-distortion embedding into $\ell_p$ for $p\in(0,2]$ also admits an efficient sketching scheme. But is the converse true, i.e., is embedding into $\ell_p$ the only way to achieve efficient sketching? We address these questions for the important special case of normed spaces, by providing an almost complete characterization of sketching in terms of embeddings. In particular, we prove that a finite-dimensional normed space allows efficient sketches if and only if it embeds (linearly) into $\ell_{1-\varepsilon}$ with constant distortion. We further prove that for norms that are closed under sum-product, efficient sketching is equivalent to embedding into $\ell_1$ with constant distortion. Examples of such norms include the Earth Mover's Distance (specifically its norm variant, called Kantorovich-Rubinstein norm), and the trace norm (a.k.a. Schatten $1$-norm or the nuclear norm). Using known non-embeddability theorems for these norms by Naor and Schechtman (SICOMP, 2007) and by Pisier (Compositio. Math., 1978), we then conclude that these spaces do not admit efficient sketches either, making progress towards answering another open question posed by Indyk in 2006. Finally, we observe that resolving whether "sketching is equivalent to embedding into $\ell_1$ for general norms" (i.e., without the above restriction) is equivalent to resolving a well-known open problem in Functional Analysis posed by Kwapien in 1969.

cs.DS↗