Entropy bounds for grammar compression

Abstract

Grammar compression represents a string as a context free grammar. Achieving compression requires encoding such grammar as a binary string; there are a few commonly used encodings. We bound the size of practically used encodings for several heuristical compression methods, including \RePair and \Greedy algorithms: the standard encoding of \RePair, which combines entropy coding and special encoding of a grammar, achieves $1.5|S|H_k(S)$, where $H_k(S)$ is $k$-th order entropy of $S$. We also show that by stopping after some iteration we can achieve $|S|H_k(S)$. This is particularly interesting, as it explains a phenomenon observed in practice: introducing too many nonterminals causes the bit-size to grow. We generalize our approach to other compression methods like \Greedy and a wide class of irreducible grammars as well as to other practically used bit encodings (including naive, which uses fixed-length codes). Our approach not only proves the bounds but also partially explains why \Greedy and \RePair are much better in practice than other grammar based methods. In some cases we argue that our estimations are optimal. The tools used in our analysis are of independent interest: we prove the new, optimal, bounds on the zeroth order entropy of parsing of a string.