Searcharxiv⌕ Search

arXiv · 2610.24707

Diverse Why-Provenance for Datalog Queries

Abstract

Datalog is a powerful rule-based language that allows us to express complex recursive queries and has found numerous applications over the years. Explaining why a result to a Datalog query is obtained is an essential task towards explainable and transparent data-intensive applications that rely on Datalog. A standard way of explaining a query result is the so-called why-provenance, which provides information about the witnesses to a query result in the form of subsets of the input database that as a whole can be used to derive that result. The state-of-the-art approach for efficiently computing the why-provenance explanations of a certain query result aims at enumerating, without repetitions, the relevant explanations by relying on off-the-shelf SAT solvers. However, whenever the set of explanations is very large, outputting the entire set might be prohibitively expensive. Moreover, it is questionable the practical usefulness of flooding the user with a very large number of explanations, many of them possibly differing only marginally. In this case, one can argue that it would be practically more useful to compute a small, sufficiently diverse subset of the set of why-provenance explanations. The present work initiates the study of the latter problem. In particular, we formalize the problem of interest as an optimization problem, and, after showing that it is highly intractable (it is NP-hard even in data complexity), we investigate whether ILP optimization solvers can be employed towards practical solutions. Our main finding is that this is achievable as long as we keep the requested set of explanations small, or we employ approximation algorithms (with error guarantees).

Explore related subjects

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Marco Calautti, Mostafa Milani, Andreas Pieris. 2026-10-17. Diverse Why-Provenance for Datalog Queries. https://arxiv.org/abs/2610.24707

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

ProvQML: Efficient Provenance Capture for Machine Learning as Recursive Aggregate SQL [Extended Version]

Expressing ML algorithms as recursive aggregate SQL queries over distributed systems enables large-scale declarative data analytics, yet no existing system supports scalable provenance capture, necessary for ML explainability. We present ProvQML, a provenance-aware recursive aggregate SQL engine that efficiently captures fine-grained provenance while evaluating ML algorithms as recursive aggregate queries. We capture two types of provenance: (i) local provenance that records the contribution of each training instance to gradient computation at a specific model state, and (ii) global provenance that encodes the sequence of model parameter updates across the iterations. Extensive experiments on widely used ML datasets demonstrate that our approach efficiently captures both provenance for various ML models over widely used ML datasets.

cs.DB↗

UBASE: An AI Search Engine for Trillion-Scale Vector Data Management at ByteDance

Since 2016, UBASE has been the foundation of ByteDance's search infrastructure, scaling to more than 7,000 clusters and 300 PB of indexed data. Driven by the demands of AI workloads, UBASE has evolved from a text search engine into a unified AI search system supporting vector retrieval, lexical matching, and predicate filtering. Its largest deployment indexes nearly one trillion high-dimensional vectors. This scale exposes two central bottlenecks in AI-era retrieval: memory-intensive graph-index construction under sustained ingestion, and the prohibitive cost of keeping vector indexes entirely in memory. UBASE addresses these bottlenecks with two techniques. First, it introduces a quantization-aware vector kernel based on SymRaBitQ, a new symmetric quantization scheme with tight theoretical guarantees that allows index construction to run directly in the quantized space accurately and efficiently without retaining a copy of full-precision vectors. Second, it provides a hybrid storage engine that supports memory-resident, hybrid, and SSD-resident deployments, with fine-grained record-level caching to trade memory for latency under operational control. On large-scale benchmarks, UBASE improves throughput by up to 3x, reduces indexing memory by 80%, and lowers serving cost by up to 99.4% compared with prior systems, while supporting trillion-vector scale, write-heavy or latency-sensitive workloads in production.

cs.DB↗

Contextual Utility of Quantization Moves in Extreme Low-Bit LLMs

Accurate measurements of isolated weight replacements need not predict their joint effect on a quantized model. We study this gap for legal replacements at a fixed bitrate, separating isolated pricing error, composition error, and changes in evaluation data. On Llama-3.2-1B and activation-ordered 3B GPTQ bases, option-KL composition residuals are 26x and 42x the sums of absolute midpoint pricing errors; large residuals remain with exact-singleton prices. Exhaustive Qwen3-0.6B lattices reveal context-dependent effects: six of eight moves in the stratified lattice have statistically resolved ARC-KL sign reversals, excluding an exact monotone additive representation. Pair measurements improve prediction of unqueried combinations on both Qwen and Llama. Their value for selection depends on the criterion: the stratified Qwen front retains a quality-coverage advantage after refitting, whereas the 3B additive and quadratic fronts are nearly equal in quality at a one-standard-error tolerance. Low-order endpoint structure leaves continuous gradient responses undetermined, and shared-midpoint prices can differ from those of the executed subset. Support-matched controls show that restricting the number of replacements curbs the failures of large combinations, while additional endpoint search gives bank-dependent NLL/ARC trade-offs.

cs.DB↗