arXiv subjects

Or Zamir

Publications and source records attributed to Or Zamir.

30 records · Page 2Linked to original sources

arXiv2024

Excuse me, sir? Your language model is leaking (information)

We introduce a cryptographic method to hide an arbitrary secret payload in the response of a Large Language Model (LLM). A secret key is required to extract the payload from the model's response, and without the key it is provably impossible to distinguish between the responses of the original LLM and the LLM that hides a payload. In particular, the quality of generated text is not affected by the payload. Our approach extends a recent result of Christ, Gunn and Zamir (2023) who introduced an undetectable watermarking scheme for LLMs.

cs.CR↗

arXiv2023

Undetectable Watermarks for Language Models

Recent advances in the capabilities of large language models such as GPT-4 have spurred increasing concern about our ability to detect AI-generated text. Prior works have suggested methods of embedding watermarks in model outputs, by noticeably altering the output distribution. We ask: Is it possible to introduce a watermark without incurring any detectable change to the output distribution? To this end we introduce a cryptographically-inspired notion of undetectable watermarks for language models. That is, watermarks can be detected only with the knowledge of a secret key; without the secret key, it is computationally intractable to distinguish watermarked outputs from those of the original model. In particular, it is impossible for a user to observe any degradation in the quality of the text. Crucially, watermarks should remain undetectable even when the user is allowed to adaptively query the model with arbitrarily chosen prompts. We construct undetectable watermarks based on the existence of one-way functions, a standard assumption in cryptography.

cs.CR↗

arXiv2022

Algorithmic Applications of Hypergraph and Partition Containers

We present a general method to convert algorithms into faster algorithms for almost-regular input instances. Informally, an almost-regular input is an input in which the maximum degree is larger than the average degree by at most a constant factor. This family of inputs vastly generalizes several families of inputs for which we commonly have improved algorithms, including bounded-degree inputs and random inputs. It also generalizes families of inputs for which we don't usually have faster algorithms, including regular-inputs of arbitrarily high degree and very dense inputs. We apply our method to achieve breakthroughs in exact algorithms for several central NP-Complete problems including $k$-SAT, Graph Coloring, and Maximum Independent Set. Our main tool is the first algorithmic application of the relatively new Hypergraph Container Method (Saxton and Thomason 2015, Balogh, Morris and Samotij 2015). This recent breakthrough, which generalizes an earlier version for graphs (Kleitman and Winston 1982, Sapozhenko 2001), has been used extensively in recent years in extremal combinatorics. An important component of our work is the generalization of (hyper-)graph containers to Partition Containers.

cs.DS↗

arXiv2022

The wrong direction of Jensen's inequality is algorithmically right

Let $\mathcal{A}$ be an algorithm with expected running time $e^X$, conditioned on the value of some random variable $X$. We construct an algorithm $\mathcal{A'}$ with expected running time $O(e^{E[X]})$, that fully executes $\mathcal{A}$. In particular, an algorithm whose running time is a random variable $T$ can be converted to one with expected running time $O(e^{E[\ln T]})$, which is never worse than $O(E[T])$. No information about the distribution of $X$ is required for the construction of $\mathcal{A}'$.

cs.CC↗

arXiv2022

Hardness of Approximation in P via Short Cycle Removal: Cycle Detection, Distance Oracles, and Beyond

We present a new technique for efficiently removing almost all short cycles in a graph without unintentionally removing its triangles. Consequently, triangle finding problems do not become easy even in almost $k$-cycle free graphs, for any constant $k\geq 4$. Triangle finding is at the base of many conditional lower bounds in P, mainly for distance computation problems, and the existence of many $4$- or $5$-cycles in a worst-case instance had been the obstacle towards resolving major open questions. Hardness of approximation: Are there distance oracles with $m^{1+o(1)}$ preprocessing time and $m^{o(1)}$ query time that achieve a constant approximation? Existing algorithms with such desirable time bounds only achieve super-constant approximation factors, while only $3-ε$ factors were conditionally ruled out (Pătraşcu, Roditty, and Thorup; FOCS 2012). We prove that no $O(1)$ approximations are possible, assuming the $3$-SUM or APSP conjectures. In particular, we prove that $k$-approximations require $Ω(m^{1+1/ck})$ time, which is tight up to the constant $c$. The lower bound holds even for the offline version where we are given the queries in advance, and extends to other problems such as dynamic shortest paths. The $4$-Cycle problem: An infamous open question in fine-grained complexity is to establish any surprising consequences from a subquadratic or even linear-time algorithm for detecting a $4$-cycle in a graph. We prove that $Ω(m^{1.1194})$ time is needed for $k$-cycle detection for all $k\geq 4$, unless we can detect a triangle in $\sqrt{n}$-degree graphs in $O(n^{2-δ})$ time; a breakthrough that is not known to follow even from optimal matrix multiplication algorithms.

cs.DS↗

arXiv2022

Faster algorithm for Unique $(k,2)$-CSP

In a $(k,2)$-Constraint Satisfaction Problem we are given a set of arbitrary constraints on pairs of $k$-ary variables, and are asked to find an assignment of values to these variables such that all constraints are satisfied. The $(k,2)$-CSP problem generalizes problems like $k$-coloring and $k$-list-coloring. In the Unique $(k,2)$-CSP problem, we add the assumption that the input set of constraints has at most one satisfying assignment. Beigel and Eppstein gave an algorithm for $(k,2)$-CSP running in time $O\left(\left(0.4518k\right)^n\right)$ for $k>3$ and $O\left(1.356^n\right)$ for $k=3$, where $n$ is the number of variables. Feder and Motwani improved upon the Beigel-Eppstein algorithm for $k\geq 11$. Hertli, Hurbain, Millius, Moser, Scheder and Szedl{á}k improved these bounds for Unique $(k,2)$-CSP for every $k\geq 5$. We improve the result of Hertli et al. and obtain better bounds for Unique~$(k,2)$-CSP for~$k\geq 5$. In particular, we improve the running time of Unique~$(5,2)$-CSP from~$O\left(2.254^n\right)$ to~$O\left(2.232^n\right)$ and Unique~$(6,2)$-CSP from~$O\left(2.652^n\right)$ to~$O\left(2.641^n\right)$.

cs.DS↗

arXiv2021

Randomized Dimensionality Reduction for Facility Location and Single-Linkage Clustering

Random dimensionality reduction is a versatile tool for speeding up algorithms for high-dimensional problems. We study its application to two clustering problems: the facility location problem, and the single-linkage hierarchical clustering problem, which is equivalent to computing the minimum spanning tree. We show that if we project the input pointset $X$ onto a random $d = O(d_X)$-dimensional subspace (where $d_X$ is the doubling dimension of $X$), then the optimum facility location cost in the projected space approximates the original cost up to a constant factor. We show an analogous statement for minimum spanning tree, but with the dimension $d$ having an extra $\log \log n$ term and the approximation factor being arbitrarily close to $1$. Furthermore, we extend these results to approximating solutions instead of just their costs. Lastly, we provide experimental results to validate the quality of solutions and the speedup due to the dimensionality reduction. Unlike several previous papers studying this approach in the context of $k$-means and $k$-medians, our dimension bound does not depend on the number of clusters but only on the intrinsic dimensionality of $X$.

cs.DS↗

arXiv2021

Breaking the $2^n$ barrier for 5-coloring and 6-coloring

The coloring problem (i.e., computing the chromatic number of a graph) can be solved in $O^*(2^n)$ time, as shown by Björklund, Husfeldt and Koivisto in 2009. For $k=3,4$, better algorithms are known for the $k$-coloring problem. $3$-coloring can be solved in $O(1.33^n)$ time (Beigel and Eppstein, 2005) and $4$-coloring can be solved in $O(1.73^n)$ time (Fomin, Gaspers and Saurabh, 2007). Surprisingly, for $k>4$ no improvements over the general $O^*(2^n)$ are known. We show that both $5$-coloring and $6$-coloring can also be solved in $O\left(\left(2-\varepsilon\right)^n\right)$ time for some $\varepsilon>0$. As a crucial step, we obtain an exponential improvement for computing the chromatic number of a very large family of graphs. In particular, for any constants $Δ,α>0$, the chromatic number of graphs with at least $α\cdot n$ vertices of degree at most $Δ$ can be computed in $O\left(\left(2-\varepsilon\right)^n\right)$ time, for some $\varepsilon = \varepsilon_{Δ,α} > 0$. This statement generalizes previous results for bounded-degree graphs (Björklund, Husfeldt, Kaski, and Koivisto, 2010) and graphs with bounded average degree (Golovnev, Kulikov and Mihajilin, 2016). We generalize the aforementioned statement to List Coloring, for which no previous improvements are known even for the case bounded-degree graphs.

cs.DS↗

arXiv2019

Random $k$-out subgraph leaves only $O(n/k)$ inter-component edges

Each vertex of an arbitrary simple graph on $n$ vertices chooses $k$ random incident edges. What is the expected number of edges in the original graph that connect different connected components of the sampled subgraph? We prove that the answer is $O(n/k)$, when $k\ge c\log n$, for some large enough $c$. We conjecture that the same holds for smaller values of $k$, possibly for any $k\ge 2$. Such a result is best possible for any $k\ge 2$. As an application, we use this sampling result to obtain a one-way communication protocol with \emph{private} randomness for finding a spanning forest of a graph in which each vertex sends only ${O}(\sqrt{n}\log n)$ bits to a referee.

cs.DM↗

arXiv2018

Selection from heaps, row-sorted matrices and $X+Y$ using soft heaps

We use soft heaps to obtain simpler optimal algorithms for selecting the $k$-th smallest item, and the set of~$k$ smallest items, from a heap-ordered tree, from a collection of sorted lists, and from $X+Y$, where $X$ and $Y$ are two unsorted sets. Our results match, and in some ways extend and improve, classical results of Frederickson (1993) and Frederickson and Johnson (1982). In particular, for selecting the $k$-th smallest item, or the set of~$k$ smallest items, from a collection of~$m$ sorted lists we obtain a new optimal "output-sensitive" algorithm that performs only $O(m+\sum_{i=1}^m \log(k_i+1))$ comparisons, where $k_i$ is the number of items of the $i$-th list that belong to the overall set of~$k$ smallest items.

cs.DS↗

arXiv2015

Subtree Isomorphism Revisited

The Subtree Isomorphism problem asks whether a given tree is contained in another given tree. The problem is of fundamental importance and has been studied since the 1960s. For some variants, e.g., ordered trees, near-linear time algorithms are known, but for the general case truly subquadratic algorithms remain elusive. Our first result is a reduction from the Orthogonal Vectors problem to Subtree Isomorphism, showing that a truly subquadratic algorithm for the latter refutes the Strong Exponential Time Hypothesis (SETH). In light of this conditional lower bound, we focus on natural special cases for which no truly subquadratic algorithms are known. We classify these cases against the quadratic barrier, showing in particular that: -- Even for binary, rooted trees, a truly subquadratic algorithm refutes SETH. -- Even for rooted trees of depth $O(\log\log{n})$, where $n$ is the total number of vertices, a truly subquadratic algorithm refutes SETH. -- For every constant $d$, there is a constant $ε_d>0$ and a randomized, truly subquadratic algorithm for degree-$d$ rooted trees of depth at most $(1+ ε_d) \log_{d}{n}$. In particular, there is an $O(\min\{ 2.85^h ,n^2 \})$ algorithm for binary trees of depth $h$. Our reductions utilize new "tree gadgets" that are likely useful for future SETH-based lower bounds for problems on trees. Our upper bounds apply a folklore result from randomized decision tree complexity.

cs.CC↗

arXiv2015

Motion Planning for Unlabeled Discs with Optimality Guarantees

We study the problem of path planning for unlabeled (indistinguishable) unit-disc robots in a planar environment cluttered with polygonal obstacles. We introduce an algorithm which minimizes the total path length, i.e., the sum of lengths of the individual paths. Our algorithm is guaranteed to find a solution if one exists, or report that none exists otherwise. It runs in time $\tilde{O}(m^4+m^2n^2)$, where $m$ is the number of robots and $n$ is the total complexity of the workspace. Moreover, the total length of the returned solution is at most $\text{OPT}+4m$, where OPT is the optimal solution cost. To the best of our knowledge this is the first algorithm for the problem that has such guarantees. The algorithm has been implemented in an exact manner and we present experimental results that attest to its efficiency.

cs.CG↗