Searcharxiv⌕ Search

arXiv subjects

Hiroki Shibata

Publications and source records attributed to Hiroki Shibata.

15 recordsLinked to original sources

Exact and Approximation Algorithms for Computing Smallest SESs and BMSs

Compression schemes represent repetitive strings compactly by using repeated substrings. Computing the minimum size of a representation of a given input string is important for compression and for understanding relationships between compression schemes, although this task is NP-hard for several schemes. In this paper, we introduce exact and approximation algorithms for finding smallest substring equation systems (SESs) and bidirectional macro schemes (BMSs). Specifically, for an input string of length $n$, we give deterministic and randomized algorithms for computing a smallest SES using $O^*(4^n)$ and $O^*(2^n)$ time and space, respectively. We also give a deterministic algorithm for computing a smallest BMS using $O^*(2^n)$ time and space. We further give deterministic and randomized $(1+α)$-approximation algorithms for smallest SESs using $O^*(4^{e^{-α}n})$ and $O^*(2^{e^{-α}n})$ time and space, respectively, for every $α\ge 0$. A conversion from SESs to BMSs gives $4(1+α)$-approximation algorithms for smallest BMSs with the same time and space bounds. All time bounds hold in the worst case, and the randomized algorithms return correct answers with high probability. Our algorithms adapt several approaches to exact and approximation algorithms for combinatorial problems, such as dynamic programming over determinant expansions and a greedy algorithm for monotone submodular functions.

cs.DS↗

The Greedy Superstring Conjecture Is False

In the Shortest Common Superstring problem (SCS), one is given a finite set of strings and is asked to find a shortest string containing every input string as a substring. Since the problem is NP-hard, various approximation algorithms have been proposed for it. The currently best known approximation ratio is $2.466$ due to Englert, Matsakis, and Veselý; a better approximation ratio $2$ has been announced by OpenAI just recently. In 1988, Tarhio and Ukkonen introduced a simple greedy algorithm that iteratively takes two strings with the longest overlap and merges them together. They showed that the greedy algorithm may produce a superstring that is about two times longer than the optimal one and conjectured that this bound is tight. This has become known as the Greedy Superstring Conjecture and has remained open for almost four decades. The best known upper bound on the approximation ratio of the greedy algorithm is $3.396$ due to Englert, Matsakis, and Veselý. In this paper, we present a series of results tightening the bounds of the greedy algorithm, with the main contributions being as follows: 1. We refute the Greedy Conjecture by constructing a family of instances for which a lower bound on the approximation ratio tends to $2.4$. 2. We prove that the greedy algorithm is $3$-approximate. 3. For input strings of fixed length $k$, we prove that the approximation guarantee is exactly $9/5$ for $k=3$ and at least $2$ for all $k \ge 6$.

cs.DS↗

String Representation Based on Substring Equation Systems

Repetitiveness measures quantify how much repetitive structure a string contains and serve as parameters for compressed representations and indexing data structures. Many compression schemes represent strings by recording equalities between identical substrings. We study string compression based on substring equation systems (SESs), which represent strings as unique solutions to substring-equality and character-assignment constraints. We show that every string $w$ has an SES of size $O(χ(w))$, where $χ(w)$ is the size of a smallest suffixient set for $w$ with a unique terminal symbol appended. This result establishes the reachability of $χ$, which had been an open problem. We also prove that computing the size $s(w)$ of the smallest SES that represents $w$ is NP-hard and $(1 + \varepsilon)$-inapproximable for some fixed constant $\varepsilon > 0$. Finally, we prove that the size $b(w)$ of the smallest bidirectional macro scheme (BMS) representing $w$ satisfies $s(w) \leq b(w) \leq 4s(w)$. Hence, SES and BMS are equivalent up to a constant factor, and this equivalence gives the new bound $b(w) \in O(χ(w))$.

cs.DS↗

Sensitivity and Size Relationships of the Lempel-Ziv Factorization

The Lempel-Ziv (LZ) factorization is a fundamental method for compressing repetitive strings. Its multiplicative sensitivity to an operation measures how much that operation can increase the number of phrases, as the maximum ratio of the phrase counts after and before the operation. While its sensitivity to single-character edits is known, its sensitivity to more global operations such as prefix deletion and string reversal remains unknown. Further open questions concern its size relationships with other compressed representations and whether it can be converted into an LZ-like encoding of polylogarithmic height without asymptotically increasing its size. In this paper, We determine the multiplicative sensitivity of the LZ factorization to prefix deletion, cyclic rotation, and string reversal by establishing an $Ω(\log n)$ lower bound for each operation that matches the known upper bound, where $n$ is the input length. Using the result for prefix deletion, we show that the combined size of the relative LZ factorization and the LZ encoding of its reference string can be asymptotically smaller than that of the LZ factorization. We also obtain asymptotically tight worst-case ratios of $Θ(\log n)$ for the size of the LZ factorization divided by the minimum size of a collage system and for the size of the lex-parse divided by that of the LZ factorization. We give lower and upper bounds on the worst-case ratio of the minimum size of an LZ-like encoding of height at most $h$ to the size of the LZ factorization, with matching bounds for $h \geq \lg n$. In particular, we show that the minimum size of an LZ-like encoding with height at most $h \in O({\rm poly}\,\log n)$ can be larger than the size of the LZ factorization by a factor of $Ω(\log n / \log\log n)$. All our lower bounds follow from a common string construction based on the bit-reversal permutation.

cs.DS↗

Online computation of maximal closed substrings

A non-empty string is closed if it has length one or its longest border appears exactly twice in the string. An occurrence of a closed substring is a maximal closed substring (MCS) if it cannot be extended to the left or to the right while preserving closedness. MCSs can be regarded as a general class of maximal repetitive structures including runs. In this paper, we study the computation of MCSs of a string given in an online manner, where one character is appended to the string at a time. Our algorithm detects newly formed MCSs after each append operation by using the rightmost previous occurrence of each suffix. To support this efficiently, we introduce the link-cut suffix tree (LCST), a novel data structure combining an online suffix tree with a link-cut tree. The LCST maintains rightmost occurrence information for substrings represented in the suffix tree in $O(n \log n)$ total time and $O(n)$ space, where $n$ is the length of the input string. Using the LCST, we obtain an $O(n \log n)$-time online algorithm for computing all MCSs, which is worst-case optimal. As further direct applications of the LCST, we obtain online algorithms for rightmost LZ77 factorizations and most recent match queries.

cs.DS↗

Compact Enumeration of Maximal Closed Substrings in Run-Length Encoded Strings

A string $w$ is closed if $|w|=1$, or if $w$ has a non-empty proper border occurring only as its prefix and suffix. A maximal closed substring (MCS) is a maximal occurrence of a closed string; equivalently, it is a maximal closed repeat (MCR). We study the problem of enumerating all MCS occurrences directly from the run-length encoding (RLE) of a string. For a string $T$ of length $n$ with RLE size $m$, we give a compact representation of all MCS occurrences whose worst-case size is $O(m^2)$, and show that this bound is tight for this representation. Our approach is based on a characterization of MCS occurrences in terms of consecutive occurrences of their longest borders, together with data structures built on the RLE of $T$. Denoting the resulting representation by $\mathcal{F}$, we compute it in $O(m\log^2 m + |\mathcal{F}|\log m)$ time using $O(m)$ working space.

cs.DS↗

Relaxation of Square-Freeness

We extend the analysis of nonrepetitive sequences of Entringer et al. [Journal of Combinatorial Theory, 1974] to relaxations of equality testing under nonstandard equivalence relations, in particular parameterized equivalence and order-preserving equivalence. For this setting, we introduce $\ell^+$-squares, defined as squares whose total length is at least $2\ell$. We obtain an infinite $3^+$-parameterized-square-free ternary word and an infinite $3^+$-order-preserving-square-free binary word through an approach based on combinatorics on words. In addition, we report the longest $\ell^+$-square-free words across several equivalence relations.

math.CO↗

Counting Distinct (Non-)Crossing Substrings in Optimal Time

Let $w$ be a string of length $n$. The problem of counting factors crossing a position -- Problem 64 from the textbook ``125 Problems in Text Algorithms'' [Crochemore, Lecroq, and Rytter, 2021] -- asks to count the number $\mathcal{C}(w,k)$ (resp. $\mathcal{N}(w,k)$) of distinct substrings in $w$ that have occurrences containing (resp. not containing) a position $k$ in $w$. The solutions provided in their textbook compute $\mathcal{C}(w,k)$ and $\mathcal{N}(w,k)$ in $O(n)$ time for a single position $k$ in $w$, and thus a direct application would require $O(n^2)$ time for all positions $k = 1, \ldots, n$ in $w$. Their solution is designed for constant-size alphabets. In this paper, we present new algorithms which compute $\mathcal{C}(w,k)$ in $O(n)$ total time for general ordered alphabets, and $\mathcal{N}(w,k)$ in $O(n)$ total time for linearly sortable alphabets,for all positions $k = 1, \ldots, n$ in $w$. We further derive model-dependent optimal bounds by separating the algorithms into preprocessing and linear-time postprocessing: for $\mathcal{C}$ the preprocessing is run reporting, and for $\mathcal{N}$ it is preprocessing based on longest previous non-overlapping factors (LPnF) and longest next factors (LNF). In particular, all values $\mathcal{C}(w,k)$ can be computed in $O(n\log n)$ time over general unordered alphabets in which direct accesses to alphabet characters are restricted to equality tests, and in $O(n\logσ)$ time in the word RAM model, where $σ$ denotes the number of distinct characters occurring in $w$. For $\mathcal{N}(w,k)$, the equality-testing complexity over general unordered alphabets is $Θ(n^2)$. We also show that our upper bounds are optimal for all of the aforementioned alphabet assumptions and computation models.

cs.DS↗

Online Computation of Palindromes and Suffix Trees on Tries

We consider the problems of computing maximal palindromes and distinct palindromes in a trie. A trie is a natural generalization of a string, which can be seen as a single-path tree. There is a linear-time offline algorithm to compute maximal palindromes and distinct palindromes in a given (static) trie whose edge-labels are drawn from a linearly-sortable alphabet [Mieno et al., ISAAC 2022]. In this paper, we tackle problems of palindrome enumeration on dynamic tries which support leaf additions and leaf deletions. We propose the first sub-quadratic algorithms to enumerate palindromes in a dynamic trie. For maximal palindromes, we propose an algorithm that runs in $O(N \min(\log h, σ))$ time and uses $O(N)$ space, where $N$ is the maximum number of edges in the trie, $σ$ is the size of the alphabet, and $h$ is the height of the trie. For distinct palindromes, we develop several online algorithms based on different algorithmic frameworks, including approaches using the EERTREE (a.k.a. palindromic tree) and the suffix tree of a trie. These algorithms support leaf insertions and deletions in the trie and achieve different time and space trade-offs. Furthermore, as a by-product, we present online algorithms to construct the suffix tree and the EERTREE of the input trie, which is of independent interest.

cs.DS↗

LZBE: an LZ-style compressor supporting $O(\log n)$-time random access

An LZ-like factorization of a string divides it into factors, each being either a single character or a copy of a preceding substring. While grammar-based compression schemes support efficient random access with space linear in the compressed size, no comparable guarantees are known for general LZ-like factorizations. This limitation motivated restricted variants such as LZ-End [Kreft and Navarro, 2013] and height-bounded LZ (LZHB) [Bannai et al., 2024], which trade off some compression efficiency for faster access. In this paper, we introduce LZ-Begin-End (LZBE), a new LZ-like variant in which every copy factor must refer to a contiguous sequence of preceding factors. This structural restriction ensures that any context-free grammar can be transformed into an LZBE factorization of the same size. We further study the greedy LZBE factorization, which selects each copy factor to be as long as possible while processing the input from left to right, and show that it can be computed in linear time. Moreover, we exhibit a family of strings for which the greedy LZBE factorization is asymptotically smaller than the smallest grammar. These results demonstrate that the LZBE scheme is strictly more expressive than grammar-based compression in the worst case. To support fast queries, we propose a data structure for LZBE-compressed strings that permits O(log n)-time random access within space linear in the compressed size, where n is the length of the input string.

cs.DS↗

Subsequence Matching and LCS under Cartesian-Tree Equivalence

Two strings of the same length are said to Cartesian-tree match (CT-match) if their Cartesian-trees are isomorphic [Park et al., TCS 2020]. Cartesian-tree matching is a natural model that allows for capturing similarities of numerical sequences. Oizumi et al. [CPM 2022] showed that subsequence pattern matching under CT-matching model (CT-MSeq) can be solved in $O(nm \log \log n)$ time, where $n$ and $m$ are text and pattern lengths, respectively. This current article follows this line of research, and gives the following new results: (1) An $O(nm)$-time CT-MSeq algorithm for binary alphabets; (2) An $O((nm)^{1-ε})$-time conditional lower bound for the CT-MSeq problem on alphabets of size 4, for any constant $ε> 0$, under the Orthogonal Vector Hypothesis (OVH). Further, we introduce the new problem of longest common subsequence under CT-matching (CT-LCS) for two given strings $S$ and $T$ of length $n$, and present the following results: (3) An $O(n^6)$-time CT-LCS algorithm for general ordered alphabets; (4) An $O(n^2 / \log n)$-time CT-LCS algorithm for binary alphabets; (5) An $O(n^{2-ε})$-time conditional lower bound for the CT-LCS problem on alphabets of size 5, for any constant $ε> 0$, under OVH.

cs.DS↗

LZ78 Substring Compression in Compressed Space

The Lempel--Ziv 78 (LZ78) factorization is a well-studied technique for data compression. It and its derivatives are used in compression formats such as "compress" or "gif". Although most research focuses on the factorization of plain data, not much research has been conducted on indexing the data for fast LZ78 factorization. Here, we study the LZ78 factorization and its derivatives in the substring compression model, where we are allowed to index the data and return the factorization of a substring specified at query time. In that model, we propose an algorithm that works in compressed space, computing the factorization with a logarithmic slowdown compared to the optimal time complexity.

cs.DS↗

Tight Additive Sensitivity on LZ-style Compressors and String Attractors

The worst-case additive sensitivity of a string repetitiveness measure $c$ is defined to be the largest difference between $c(w)$ and $c(w')$, where $w$ is a string of length $n$ and $w'$ is a string that can be obtained by performing a single-character edit operation on $w$. We present $O(\sqrt{n})$ upper bounds for the worst-case additive sensitivity of the smallest string attractor size $γ$ and the smallest bidirectional scheme size $b$, which match the known lower bounds $Ω(\sqrt{n})$ for $γ$ and $b$ [Akagi et al. 2023]. Further, we present matching upper and lower bounds for the worst-case additive sensitivity of the Lempel-Ziv family - $Θ(n^{\frac{2}{3}})$ for LZSS and LZ-End, and $Θ(n)$ for LZ78.

cs.DS↗

Packed Acyclic Deterministic Finite Automata

An acyclic deterministic finite automaton (ADFA) is a data structure that represents a set of strings (i.e., a dictionary) and facilitates a pattern searching problem of determining whether a given pattern string is present in the dictionary. We introduce the packed ADFA (PADFA), a compact variant of ADFA, which is designed to achieve more efficient pattern searching by encoding specific paths as packed strings stored in contiguous memory. We theoretically demonstrate that pattern searching in PADFA is near time-optimal with a small additional overhead and becomes fully time-optimal for sufficiently long patterns. Moreover, we prove that a PADFA requires fewer bits than a trie when the dictionary size is relatively smaller than the number of states in the PADFA. Lastly, we empirically show that PADFAs improve both the space and time efficiency of pattern searching on real-world datasets.

cs.DS↗

Properties and applications of Fisher distribution on the rotation group

We study properties of Fisher distribution (von Mises-Fisher distribution, matrix Langevin distribution) on the rotation group SO(3). In particular we apply the holonomic gradient descent, introduced by Nakayama et al. (2011), and a method of series expansion for evaluating the normalizing constant of the distribution and for computing the maximum likelihood estimate. The rotation group can be identified with the Stiefel manifold of two orthonormal vectors. Therefore from the viewpoint of statistical modeling, it is of interest to compare Fisher distributions on these manifolds. We illustrate the difference with an example of near-earth objects data.

stat.ME↗