arXiv subjects

Andrei Romashchenko

Publications and source records attributed to Andrei Romashchenko.

At least 19 recordsLinked to original sources

arXiv2026

Size Bounds for CQs Under Acyclic Constraints

We study size bounds for conjunctive query (CQ) results which in recent years have played a crucial role in database theory. In particular, we compare the so-called entropic bound which is known to be asymptotically tight but not known to be computable, and its computable relaxation called the polymatroid bound, which is generally not tight. We focus here on conjunctive queries under acyclic functional dependencies. These queries are known to be well-behaved in the sense that, in the case without projections, both bounds coincide. We show that this picture changes when projections are allowed: in this case, even for acyclic functional dependencies, there is in general a polynomial gap between the two bounds. We complement this negative result by showing a special case for which the polymatroid bound is tight under acyclic functional dependencies and projections that is characterized by the position of the output variables in a topological order of the query variables.

cs.DB↗

arXiv2026

Beyond Mutual Information: Extension Profiles and Shape Functions of Random Variable Pairs

We study the extension profile of a pair of jointly distributed finite-valued random variables $(X,Y)$, defined as the set of all triples of numbers $ (H(X|W), H(Y|W), I(X:Y|W)) $ obtained by extending the pair with an auxiliary random variable $W$. This object captures structural properties of joint distributions that are not determined solely by the entropies of $X$ and $Y$ and their mutual information. To describe the boundary of the extension profile, we introduce the associated shape function, defined as the Legendre--Fenchel transform of the nontrivial part of the profile boundary. We establish general upper and lower bounds on the shape function in terms of classical information-theoretic quantities. For pairs that are uniform on their support, we interpret the support as a biregular bipartite graph and relate the extension profile to combinatorial and spectral properties of this graph. In this setting, we derive bounds on the shape function in terms of the second-largest eigenvalue of the graph. Thus, pairs whose support graphs have a small second eigenvalue admit only a restricted class of extensions. Our results provide a new perspective on the information-theoretic structure of joint distributions and highlight connections among non-Shannon-type information inequalities, the G\'acs--K\"orner common information, and spectral graph theory. We discuss several applications of the developed framework to problems concerning the structure and representation of mutual information.

cs.IT↗

arXiv2026

Spectral Conditions for the Ingleton Inequality

The Ingleton inequality is a classical linear information inequality that holds for representable matroids but fails to be universally valid for entropic vectors. Understanding the extent to which this inequality can be violated has been a longstanding problem in information theory. In this paper, we show that for a broad class of jointly distributed random variables $(X,Y)$ the Ingleton inequality holds up to a small additive error, even even though the mutual information between $X$ and $Y$ is far from being extractable. Contrary to common intuition, strongly non-extractable mutual information does not lead to large violations of the Ingleton inequality in this setting. More precisely, we consider pairs $(X,Y)$ that are uniformly distributed on their joint support and whose associated biregular bipartite graph is an expander. For all auxiliary random variables $A$ and $B$ jointly distributed with $(X,Y)$, we establish a lower bound on the Ingleton quantity $I(X;Y | A) + I(X;Y | B) + I(A;B) - I(X;Y)$ in terms of the spectral parameters of the underlying graph. Our proof combines the expander mixing lemma with a partitioning technique for finite sets.

cs.IT↗

arXiv2025

Structural Properties of Entropic Vectors and Stability of the Ingleton Inequality

We study constrained versions of the Ingleton inequality in the entropic setting and quantify its stability under small violations of conditional independence. Although the classical Ingleton inequality fails for general entropy profiles, it is known to hold under certain exact independence constraints. We focus on the regime where selected conditional mutual information terms are small (but not zero), and the inequality continues to hold up to controlled error terms. A central technical tool is a structural lemma that materializes part of the mutual information between two random variables, implicitly capturing the effect of infinitely many non-Shannon--type inequalities. This leads to conceptually transparent proofs without explicitly invoking such infinite families. Some of our bounds recover, in a unified way, what can also be deduced from the infinite families of inequalities of Mat\'u\v{s} (2007) and of Dougherty--Freiling--Zeger (2011), while others appear to be new.

cs.IT↗

arXiv2025

Algebraic Barriers to Halving Algorithmic Information Quantities in Correlated Strings

We study the possibility of scaling down algorithmic information quantities in tuples of correlated strings. In particular, we address a question raised by Alexander Shen: whether, for any triple of strings $(a, b, c)$, there exists a string $z$ such that each conditional Kolmogorov complexity $C(a|z), C(b|z), C(c|z)$ is approximately half of the corresponding unconditional Kolmogorov complexity. We give a negative answer to this question by constructing a triple $(a, b, c)$ for which no such string $z$ exists. Moreover, we construct a fully explicit example of such a tuple. Our construction is based on combinatorial properties of incidences in finite projective planes and relies on bounds for point-line incidences over prime fields. As an application, we show that this impossibility yields lower bounds on the communication complexity of secret key agreement protocols in certain settings. These results reveal algebraic obstructions to efficient information exchange and highlight a separation in information-theoretic behavior between fields with and without proper subfields.

cs.IT↗

arXiv2024

Communication Complexity of the Secret Key Agreement in Algorithmic Information Theory

It is known that the mutual information, in the sense of Kolmogorov complexity, of any pair of strings x and y is equal to the length of the longest shared secret key that two parties can establish via a probabilistic protocol with interaction on a public channel, assuming that the parties hold as their inputs x and y respectively. We determine the worst-case communication complexity of this problem for the setting where the parties can use private sources of random bits. We show that for some x, y the communication complexity of the secret key agreement does not decrease even if the parties have to agree on a secret key whose size is much smaller than the mutual information between x and y. On the other hand, we discuss examples of x, y such that the communication complexity of the protocol declines gradually with the size of the derived secret key. The proof of the main result uses spectral properties of appropriate graphs and the expander mixing lemma, as well as information theoretic techniques.

cs.IT↗

arXiv2024

Common information in well-mixing graphs and applications to information-theoretic cryptography

We study the connection between mixing properties for bipartite graphs and materialization of the mutual information in one-shot settings. We show that mixing properties of a graph imply impossibility to extract the mutual information shared by the ends of an edge randomly sampled in the graph. We apply these impossibility results to some questions motivated by information-theoretic cryptography. In particular, we show that communication complexity of a secret key agreement in one-shot setting is inherently uneven: for some inputs, almost all communication complexity inevitably falls on only one party.

cs.IT↗

arXiv2023

Spectral approach to the communication complexity of multi-party key agreement

We propose a linear algebraic method, rooted in the spectral properties of graphs, that can be used to prove lower bounds in communication complexity. Our proof technique effectively marries spectral bounds with information-theoretic inequalities. The key insight is the observation that, in specific settings, even when data sets $X$ and $Y$ are closely correlated and have high mutual information, the owner of $X$ cannot convey a reasonably short message that maintains substantial mutual information with $Y$. In essence, from the perspective of the owner of $Y$, any sufficiently brief message $m=m(X)$ would appear nearly indistinguishable from a random bit sequence. We employ this argument in several problems of communication complexity. Our main result concerns cryptographic protocols. We establish a lower bound for communication complexity of multi-party secret key agreement with unconditional, i.e., information-theoretic security. Specifically, for one-round protocols (simultaneous messages model) of secret key agreement with three participants we obtain an asymptotically tight lower bound. This bound implies optimality of the previously known omniscience communication protocol (this result applies to a non-interactive secret key agreement with three parties and input data sets with an arbitrary symmetric information profile). We consider communication problems in one-shot scenarios when the parties' inputs are not produced by any i.i.d. sources, and there are no ergodicity assumptions on the input data. In this setting, we found it natural to present our results using the framework of Kolmogorov complexity.

cs.IT↗

arXiv2022

Inequalities for space-bounded Kolmogorov complexity

There is a parallelism between Shannon information theory and algorithmic information theory. In particular, the same linear inequalities are true for Shannon entropies of tuples of random variables and Kolmogorov complexities of tuples of strings (Hammer et al., 1997), as well as for sizes of subgroups and projections of sets (Chan, Yeung, Romashchenko, Shen, Vereshchagin, 1998--2002). This parallelism started with the Kolmogorov-Levin formula (1968) for the complexity of pairs of strings with logarithmic precision. Longpré (1986) proved a version of this formula for space-bounded complexities. In this paper we prove an improved version of Longpré's result with a tighter space bound, using Sipser's trick (1980). Then, using this space bound, we show that every linear inequality that is true for complexities or entropies, is also true for space-bounded Kolmogorov complexities with a polynomial space overhead.

cs.IT↗

arXiv2022

Clustering with Respect to the Information Distance

We discuss the notion of a dense cluster with respect to the information distance and prove that all such clusters have an extractable core that represents the mutual information shared by the objects in the cluster.

cs.IT↗

arXiv2022

Resource-Bounded Kolmogorov Complexity Provides an Obstacle to Soficness of Multidimensional Shifts

We suggest necessary conditions of soficness of multidimensional shifts formulated in termsof resource-bounded Kolmogorov complexity. Using this technique we provide examples ofeffective and non-sofic shifts on $\mathbb{Z}^2$ with very low block complexity: the number of globallyadmissible patterns of size $n\times n$ grows only as a polynomial in $n$. We also show that moreconventional proofs of non-soficness for multi-dimensional effective shifts can be expressed interms of Kolmogorov complexity with unbounded computational resources.

cs.DM↗

arXiv2022

27 Open Problems in Kolmogorov Complexity

The paper proposes open problems in classical Kolmogorov complexity. Each problem is presented with background information and thus the article also surveys some recent studies in the area.

cs.IT↗

arXiv2019

The expressiveness of quasiperiodic and minimal shifts of finite type

We study multidimensional minimal and quasiperiodic shifts of finite type. We prove for these classes several results that were previously known for the shifts of finite type in general, without restriction. We show that some quasiperiodic shifts of finite type admit only non-computable configurations; we characterize the classes of Turing degrees that can be represented by quasiperiodic shifts of finite type. We also transpose to the classes of minimal/quasiperiodic shifts of finite type some results on subdynamics previously known for the effective shifts without restrictions: every effective minimal (quasiperiodic) shift of dimension $d$ can be represented as a projection of a subdynamics of a minimal (respectively, quasiperiodic) shift of finite type of dimension $d+1$.

cs.DM↗

arXiv2019

How to Use Undiscovered Information Inequalities: Direct Applications of the Copy Lemma

We discuss linear programming techniques that help to deduce corollaries of non classic inequalities for Shannon's entropy. We focus on direct applications of the copy lemma. These applications involve implicitly some (known or unknown) non-classic universal inequalities for Shannon's entropy, though we do not derive these inequalities explicitly. To reduce the computational complexity of these problems, we extensively use symmetry considerations. We present two examples of usage of these techniques: we provide a reduced size formal inference of the best known bound for the Ingleton score (originally proven by Dougherty \emph{et al.} with explicitly derived non Shannon type inequalities), and improve the lower bound for the optimal information ratio of the secret sharing scheme for an access structure based on the Vamos matroid.

cs.IT↗

arXiv2019

On a conditional inequality in Kolmogorov complexity and its applications in communication complexity

Romashchenko and Zimand~\cite{rom-zim:c:mutualinfo} have shown that if we partition the set of pairs $(x,y)$ of $n$-bit strings into combinatorial rectangles, then $I(x:y) \geq I(x:y \mid t(x,y)) - O(\log n)$, where $I$ denotes mutual information in the Kolmogorov complexity sense, and $t(x,y)$ is the rectangle containing $(x,y)$. We observe that this inequality can be extended to coverings with rectangles which may overlap. The new inequality essentially states that in case of a covering with combinatorial rectangles, $I(x:y) \geq I(x:y \mid t(x,y)) - \log ρ- O(\log n)$, where $t(x,y)$ is any rectangle containing $(x,y)$ and $ρ$ is the thickness of the covering, which is the maximum number of rectangles that overlap. We discuss applications to communication complexity of protocols that are nondeterministic, or randomized, or Arthur-Merlin, and also to the information complexity of interactive protocols.

cs.CC↗

arXiv2019

An operational characterization of mutual information in algorithmic information theory

We show that the mutual information, in the sense of Kolmogorov complexity, of any pair of strings $x$ and $y$ is equal, up to logarithmic precision, to the length of the longest shared secret key that two parties, one having $x$ and the complexity profile of the pair and the other one having $y$ and the complexity profile of the pair, can establish via a probabilistic protocol with interaction on a public channel. For $\ell > 2$, the longest shared secret that can be established from a tuple of strings $(x_1, \ldots , x_\ell)$ by $\ell$ parties, each one having one component of the tuple and the complexity profile of the tuple, is equal, up to logarithmic precision, to the complexity of the tuple minus the minimum communication necessary for distributing the tuple to all parties. We establish the communication complexity of secret key agreement protocols that produce a secret key of maximal length, for protocols with public randomness. We also show that if the communication complexity drops below the established threshold, then only very short secret keys can be obtained.

cs.IT↗

arXiv2018

On the Combinatorial Version of the Slepian-Wolf Problem

We study the following combinatorial version of the Slepian-Wolf coding scheme. Two isolated Senders are given binary strings $X$ and $Y$ respectively; the length of each string is equal to $n$, and the Hamming distance between the strings is at most $αn$. The Senders compress their strings and communicate the results to the Receiver. Then the Receiver must reconstruct both strings $X$ and $Y$. The aim is to minimise the lengths of the transmitted messages. For an asymmetric variant of this problem (where one of the Senders transmits the input string to the Receiver without compression) with deterministic encoding a nontrivial lower bound was found by A.Orlitsky and K.Viswanathany. In our paper we prove a new lower bound for the schemes with syndrome coding, where at least one of the Senders uses linear encoding of the input string. For the combinatorial Slepian-Wolf problem with randomized encoding the theoretical optimum of communication complexity was recently found by the first author, though effective protocols with optimal lengths of messages remained unknown. We close this gap and present a polynomial time randomized protocol that achieves the optimal communication complexity.

cs.IT↗

arXiv2017

Conditional Information Inequalities and Combinatorial Applications

We show that the inequality $H(A \mid B,X) + H(A \mid B,Y) \le H(A\mid B)$ for jointly distributed random variables $A,B,X,Y$, which does not hold in general case, holds under some natural condition on the support of the probability distribution of $A,B,X,Y$. This result generalizes a version of the conditional Ingleton inequality: if for some distribution $I(X: Y \mid A) = H(A\mid X,Y)=0$, then $I(A : B) \le I(A : B \mid X) + I(A: B \mid Y) + I(X : Y)$. We present two applications of our result. The first one is the following easy-to-formulate combinatorial theorem: assume that the edges of a bipartite graph are partitioned into $K$ matchings such that for each pair (left vertex $x$, right vertex $y$) there is at most one matching in the partition involving both $x$ and $y$; assume further that the degree of each left vertex is at least $L$ and the degree of each right vertex is at least $R$. Then $K\ge LR$. The second application is a new method to prove lower bounds for biclique coverings of bipartite graphs.

cs.IT↗