arXiv subjects
Publications and source records attributed to Marius Zimand.
We show that one-way functions exist if and only if there exists an efficient distribution relative to which almost-optimal compression is hard on average. The result is obtained by combining a theorem of Ilango, Ren, and Santhanam and one by Bauwens and Zimand.
We put forth new models for universal channel coding. Unlike standard codes which are designed for a specific type of channel, our most general universal code makes communication resilient on every channel, provided the noise level is below the tolerated bound, where the noise level t of a channel is the logarithm of its ambiguity (the maximum number of strings that can be distorted into a given one). The other more restricted universal codes still work for large classes of natural channels. In a universal code, encoding is channel-independent, but the decoding function knows the type of channel. We allow the encoding and the decoding functions to share randomness, which is unavailable to the channel. There are two scenarios for the type of attack that a channel can perform. In the oblivious scenario, codewords belong to an additive group and the channel distorts a codeword by adding a vector from a fixed set. The selection is based on the message and the encoding function, but not on the codeword. In the Hamming scenario, the channel knows the codeword and is fully adversarial. For a universal code, there are two parameters of interest: the rate, which is the ratio between the message length k and the codeword length n, and the number of shared random bits. We show the existence in both scenarios of universal codes with rate 1-t/n - o(1), which is optimal modulo the o(1) term. The number of shared random bits is O(log n) in the oblivious scenario, and O(n) in the Hamming scenario, which, for typical values of the noise level, we show to be optimal, modulo the constant hidden in the O() notation. In both scenarios, the universal encoding is done in time polynomial in n, but the channel-dependent decoding procedures are in general not efficient. For some weaker classes of channels we construct universal codes with polynomial-time encoding and decoding.
The classical coding theorem in Kolmogorov complexity states that if an $n$-bit string $x$ is sampled with probability $δ$ by an algorithm with prefix-free domain then K$(x) \leq \log(1/δ) + O(1)$. In a recent work, Lu and Oliveira [LO21] established an unconditional time-bounded version of this result, by showing that if $x$ can be efficiently sampled with probability $δ$ then rKt$(x) = O(\log(1/δ)) + O(\log n)$, where rKt denotes the randomized analogue of Levin's Kt complexity. Unfortunately, this result is often insufficient when transferring applications of the classical coding theorem to the time-bounded setting, as it achieves a $O(\log(1/δ))$ bound instead of the information-theoretic optimal $\log(1/δ)$. We show a coding theorem for rKt with a factor of $2$. As in previous work, our coding theorem is efficient in the sense that it provides a polynomial-time probabilistic algorithm that, when given $x$, the code of the sampler, and $δ$, it outputs, with probability $\ge 0.99$, a probabilistic representation of $x$ that certifies this rKt complexity bound. Assuming the security of cryptographic pseudorandom generators, we show that no efficient coding theorem can achieve a bound of the form rKt$(x) \leq (2 - o(1)) \cdot \log(1/δ) +$ poly$(\log n)$. Under a weaker assumption, we exhibit a gap between efficient coding theorems and existential coding theorems with near-optimal parameters. We consider pK$^t$ complexity [GKLO22], a variant of rKt where the randomness is public and the time bound is fixed. We observe the existence of an optimal coding theorem for pK$^t$, and employ this result to establish an unconditional version of a theorem of Antunes and Fortnow [AF09] which characterizes the worst-case running times of languages that are in average polynomial-time over all P-samplable distributions.
We study connections between expansion in bipartite graphs and efficient online matching modeled via several games. In the basic game, an opponent switches {\em on} and {\em off} nodes on the left side and, at any moment, at most $K$ nodes may be on. Each time a node is switched on, it must be irrevocably matched with one of its neighbors. A bipartite graph has $e$-expansion up to $K$ if every set $S$ of at most $K$ left nodes has at least $e\#S$ neighbors. If all left nodes have degree $D$ and $e$ is close to $D$, then the graph is a lossless expander. We show that lossless expanders allow for a polynomial time strategy in the above game, and, furthermore, with a slight modification, they allow a strategy running in time $O(D \log N)$, where $N$ is the number of left nodes. Using this game and a few related variants, we derive applications in data structures and switching networks. Namely, (a) 1-query bitprobe storage schemes for dynamic sets (previous schemes work only for static sets),(b) explicit space- and time-efficient storage schemes for static and dynamic sets with non-adaptive access to memory (the first fully dynamic dictionary with non-adaptive probing using almost optimal space), and (c) non-explicit constant depth non-blocking $N$-connectors with poly$(\log N)$ time path finding algorithms whose size is optimal within a factor of $O(\log N)$ (previous connectors are double-exponentially slower).
The paper proposes open problems in classical Kolmogorov complexity. Each problem is presented with background information and thus the article also surveys some recent studies in the area.
Bauwens and Zimand [BZ 2019] have shown that lossless expanders have an interesting online matching property. The result appears in an implicit form in [BZ 2019]. We present an explicit version of this property which is directly amenable to typical applications, prove it in a self-contained manner that clarifies the role of some parameters, and give two applications. A $(K, ε)$ lossless expander is a bipartite graph such that any subset $S$ of size at most $K$ of nodes on the left side of the bipartition has at least $(1-ε) D |S|$ neighbors, where $D$ is the left degree.The main result is that any such graph, after a slight modification, admits $(1-O(ε)D, 1)$ online matching up to size $K$. This means that for any sequence $S=(x_1, \ldots, x_K)$ of nodes on the left side of the bipartition, one can assign in an online manner to each node $x_i$ in $S$ a set $A_i$ consisting of $(1-O(ε))$ fraction of its neighbors so that the sets $A_1, \ldots, A_K$ are pairwise disjoint. "Online manner" refers to the fact that, for every $i$, the set of nodes assigned to $x_i$ only depends on the nodes assigned to $x_1, \ldots, x_{i-1}$. The first application concerns storage schemes for representing a set $S$, so that a membership query "Is $x \in S$?" can be answered probabilistically by reading a single bit. All the previous one-probe storage schemes were for a static set $S$. We show that a lossless expander can be used to construct a one-probe storage scheme for dynamic sets, i.e., sets in which elements can be inserted and deleted without affecting the representation of other elements. The second application is about non-blocking networks.
A fundamental question that has been studied in cryptography and in information theory is whether two parties can communicate confidentially using exclusively an open channel. We consider the model in which the two parties hold inputs that are correlated in a certain sense. This model has been studied extensively in information theory, and communication protocols have been designed which exploit the correlation to extract from the inputs a shared secret key. However, all the existing protocols are not universal in the sense that they require that the two parties also know some attributes of the correlation. In other words, they require that each party knows something about the other party's input. We present a protocol that does not require any prior additional information. It uses space-bounded Kolmogorov complexity to measure correlation and it allows the two legal parties to obtain a common key that looks random to an eavesdropper that observes the communication and is restricted to use a bounded amount of space for the attack. Thus the protocol achieves complexity-theoretical security, but it does not use any unproven result from computational complexity. On the negative side, the protocol is not efficient in the sense that the computation of the two legal parties uses more space than the space allowed to the adversary.
In a lossless compression system with target lengths, a compressor ${\cal C}$ maps an integer $m$ and a binary string $x$ to an $m$-bit code $p$, and if $m$ is sufficiently large, a decompressor ${\cal D}$ reconstructs $x$ from $p$. We call a pair $(m,x)$ $\textit{achievable}$ for $({\cal C},{\cal D})$ if this reconstruction is successful. We introduce the notion of an optimal compressor ${\cal C}_\text{opt}$, by the following universality property: For any compressor-decompressor pair $({\cal C}, {\cal D})$, there exists a decompressor ${\cal D}'$ such that if $(m,x)$ is achievable for $({\cal C},{\cal D})$, then $(m+Δ, x)$ is achievable for $({\cal C}_\text{opt}, {\cal D}')$, where $Δ$ is some small value called the overhead. We show that there exists an optimal compressor that has only polylogarithmic overhead and works in probabilistic polynomial time. Differently said, for any pair $({\cal C}, {\cal D})$, no matter how slow ${\cal C}$ is, or even if ${\cal C}$ is non-computable, ${\cal C}_{\text{opt}}$ is a fixed compressor that in polynomial time produces codes almost as short as those of ${\cal C}$. The cost is that the corresponding decompressor is slower. We also show that each such optimal compressor can be used for distributed compression, in which case it can achieve optimal compression rates, as given in the Slepian-Wolf theorem, and even for the Kolmogorov complexity variant of this theorem. Moreover, the overhead is logarithmic in the number of sources, and unlike previous implementations of Slepian-Wolf coding, meaningful compression can still be achieved if the number of sources is much larger than the length of the compressed strings.
Romashchenko and Zimand~\cite{rom-zim:c:mutualinfo} have shown that if we partition the set of pairs $(x,y)$ of $n$-bit strings into combinatorial rectangles, then $I(x:y) \geq I(x:y \mid t(x,y)) - O(\log n)$, where $I$ denotes mutual information in the Kolmogorov complexity sense, and $t(x,y)$ is the rectangle containing $(x,y)$. We observe that this inequality can be extended to coverings with rectangles which may overlap. The new inequality essentially states that in case of a covering with combinatorial rectangles, $I(x:y) \geq I(x:y \mid t(x,y)) - \log ρ- O(\log n)$, where $t(x,y)$ is any rectangle containing $(x,y)$ and $ρ$ is the thickness of the covering, which is the maximum number of rectangles that overlap. We discuss applications to communication complexity of protocols that are nondeterministic, or randomized, or Arthur-Merlin, and also to the information complexity of interactive protocols.
We show that the mutual information, in the sense of Kolmogorov complexity, of any pair of strings $x$ and $y$ is equal, up to logarithmic precision, to the length of the longest shared secret key that two parties, one having $x$ and the complexity profile of the pair and the other one having $y$ and the complexity profile of the pair, can establish via a probabilistic protocol with interaction on a public channel. For $\ell > 2$, the longest shared secret that can be established from a tuple of strings $(x_1, \ldots , x_\ell)$ by $\ell$ parties, each one having one component of the tuple and the complexity profile of the tuple, is equal, up to logarithmic precision, to the complexity of the tuple minus the minimum communication necessary for distributing the tuple to all parties. We establish the communication complexity of secret key agreement protocols that produce a secret key of maximal length, for protocols with public randomness. We also show that if the communication complexity drops below the established threshold, then only very short secret keys can be obtained.
Distributed compression is the task of compressing correlated data by several parties, each one possessing one piece of data and acting separately. The classical Slepian-Wolf theorem (D. Slepian, J. K. Wolf, IEEE Transactions on Inf. Theory, 1973) shows that if data is generated by independent draws from a joint distribution, that is by a memoryless stochastic process, then distributed compression can achieve the same compression rates as centralized compression when the parties act together. Recently, the author (M. Zimand, STOC 2017) has obtained an analogue version of the Slepian-Wolf theorem in the framework of Algorithmic Information Theory (also known as Kolmogorov complexity). The advantage over the classical theorem, is that the AIT version works for individual strings, without any assumption regarding the generative process. The only requirement is that the parties know the complexity profile of the input strings, which is a simple quantitative measure of the data correlation. The goal of this paper is to present in an accessible form that omits some technical details the main ideas from the reference (M. Zimand, STOC 2017).
Given a machine $U$, a $c$-short program for $x$ is a string $p$ such that $U(p)=x$ and the length of $p$ is bounded by $c$ + (the length of a shortest program for $x$). We show that for any standard Turing machine, it is possible to compute in polynomial time on input $x$ a list of polynomial size guaranteed to contain a O$(\log |x|)$-short program for $x$. We also show that there exists a computable function that maps every $x$ to a list of size $|x|^2$ containing a O$(1)$-short program for $x$. This is essentially optimal because we prove that for each such function there is a $c$ and infinitely many $x$ for which the list has size at least $c|x|^2$. Finally we show that for some standard machines, computable functions generating lists with $0$-short programs, must have infinitely often list sizes proportional to $2^{|x|}$.
Alice and Bob are given two correlated n-bit strings x_1 and, respectively, x_2, which they want to losslessly compress and send to Zack. They can either collaborate by sharing their strings, or work separately. We show that there is no disadvantage in the second scenario: Alice and Bob, without knowing the other party's string, can achieve almost optimal compression in the sense of Kolmogorov complexity. Furthermore, compression takes polynomial time and can be made at any combination of lengths that satisfy some necessary conditions (modulo additive polylog terms). More precisely, there exist probabilistic algorithms E_1, E_2, and deterministic algorithm D, with E_1 and E_2 running in polynomial time, having the following behavior: if n_1, n_2 are two integers satisfying n_1 + n_2 \geq C(x_1,x_2), n_1 \geq C(x_1 | x_2), n_2 \geq C(x_2 | x_1), then for i \in {1,2}, E_i on input x_i and n_i outputs a string of length n_i + \polylog n such that D on input E_1(x_1), E_2(x_2) reconstructs (x_1,x_2) with high probability (where C(x) denotes the plain Kolmogorov complexity of x, and C(x \mid y) is the complexity of x conditioned by y). Our main result is more general, as it deals with the compression of any constant number of correlated strings. It is an analog in the framework of algorithmic information theory of the classic Slepian-Wolf Theorem, a fundamental result in network information theory, in which x_1 and x_2 are realizations of two discrete random variables formed by drawing independently n times from a joint distribution. Also, in the classical result, the decompressor needs to know the joint distribution of the sources. In our result no type of independence is assumed and the decompressor does not have any apriori information about the sources that are compressed, and it still is the case that distributed compression is on a par with centralized compression.
It is impossible to effectively modify a string in order to increase its Kolmogorov complexity. But is it possible to construct a few strings, not longer than the input string, so that most of them have larger complexity? We show that the answer is yes. We present an algorithm that on input a string $x$ of length $n$ returns a list with $O(n^2)$ many strings, all of length $n$, such that 99\% of them are more complex than $x$, provided the complexity of $x$ is less than $n - \log \log n - O(1)$. We obtain similar results for other parameters, including a polynomial-time construction.
A $c$-short program for a string $x$ is a description of $x$ of length at most $C(x) + c$, where $C(x)$ is the Kolmogorov complexity of $x$. We show that there exists a randomized algorithm that constructs a list of $n$ elements that contains a $O(\log n)$-short program for $x$. We also show a polynomial-time randomized construction that achieves the same list size for $O(\log^2 n)$-short programs. These results beat the lower bounds shown by Bauwens et al. \cite{bmvz:c:shortlist} for deterministic constructions of such lists. We also prove tight lower bounds for the main parameters of our result. The constructions use only $O(\log n)$ ($O(\log^2 n)$ for the polynomial-time result) random bits . Thus using only few random bits it is possible to do tasks that cannot be done by any deterministic algorithm regardless of its running time.
An index $e$ in a numbering of partial-recursive functions is called minimal if every lesser index computes a different function from $e$. Since the 1960's it has been known that, in any reasonable programming language, no effective procedure determines whether or not a given index is minimal. We investigate whether the task of determining minimal indices can be solved in an approximate sense. Our first question, regarding the set of minimal indices, is whether there exists an algorithm which can correctly label 1 out of $k$ indices as either minimal or non-minimal. Our second question, regarding the function which computes minimal indices, is whether one can compute a short list of candidate indices which includes a minimal index for a given program. We give some negative results and leave the possibility of positive results as open questions.
We show that if DTIME[2^O(n)] is not included in DSPACE[2^o(n)], then, for every set B in PSPACE/poly, all strings x in B of length n can be represented by a string compressed(x) of length at most log(|B^{=n}|)+O(log n), such that a polynomial-time algorithm, given compressed(x), can distinguish x from all the other strings in B^{=n}. Modulo the O(log n) additive term, this achieves the information-theoretic optimum for string compression. We also observe that optimal compression is not possible for sets more complex than PSPACE/poly because for any time-constructible superpolynomial function t, there is a set A computable in space t(n) such that at least one string x of length n requires compressed(x) to be of length 2 log(|A^=n|).
Bauwens, Mahklin, Vereshchagin and Zimand [ECCC TR13-007] and Teutsch [arxiv:1212.6104] have shown that given a string x it is possible to construct in polynomial time a list containing a short description of it. We simplify their technique and present a shorter proof of this result.