Searcharxiv⌕ Search
Home/Archive/arXiv

arXiv · 2610.13756

ProvQML: Efficient Provenance Capture for Machine Learning as Recursive Aggregate SQL [Extended Version]

Nkechi J. Akinwale·Seokki Lee
View original source ↗Read source document ↗Export metadata ↓

Abstract

Expressing ML algorithms as recursive aggregate SQL queries over distributed systems enables large-scale declarative data analytics, yet no existing system supports scalable provenance capture, necessary for ML explainability. We present ProvQML, a provenance-aware recursive aggregate SQL engine that efficiently captures fine-grained provenance while evaluating ML algorithms as recursive aggregate queries. We capture two types of provenance: (i) local provenance that records the contribution of each training instance to gradient computation at a specific model state, and (ii) global provenance that encodes the sequence of model parameter updates across the iterations. Extensive experiments on widely used ML datasets demonstrate that our approach efficiently captures both provenance for various ML models over widely used ML datasets.

Explore related subjects

cs.DB

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Nkechi J. Akinwale, Seokki Lee. 2026-10-09. ProvQML: Efficient Provenance Capture for Machine Learning as Recursive Aggregate SQL [Extended Version]. https://arxiv.org/abs/2610.13756

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Access Paths for Efficient Ordering with Large Language Models

In this work, we present the \texttt{LLM ORDER BY} semantic operator as a logical abstraction and conduct a systematic study of its physical implementations. First, we propose several improvements to existing semantic sorting algorithms and introduce a semantic-aware external merge sort algorithm. Our extensive evaluation reveals that no single implementation offers universal optimality on all datasets. From our evaluations, we observe a general scaling relationship between sorting cost and the ordering quality for comparison-based algorithms. Building on these insights, we design a budget-aware optimizer that utilizes heuristic rules, LLM-as-Judge evaluation, and consensus aggregation to dynamically select the near-optimal access path for LLM ORDER BY. In our extensive evaluations, our optimizer consistently achieves ranking accuracy on par with or superior to the best static methods across all benchmarks. We believe that this work provides foundational insights into the principled optimization of semantic operators essential for building robust, large-scale LLM-powered analytic systems.

cs.DB↗
arXiv2026

MetaJoin: Adaptive Join Selections for Heterogeneous Tables

Joining heterogeneous tables requires different forms of correspondence, including exact agreement, syntactic transformation, semantic similarity, and entity-level matching. The appropriate join method is often unknown in advance, while running every candidate method incurs unnecessary cost. We introduce MetaJoin, a label-free framework for adaptive join selection. MetaJoin uses input evidence to choose an initial specialist, then uses specialist-specific output evidence to determine whether that result should be retained, compared with an alternative, escalated, or repaired. This allows join selection to adapt after execution rather than committing to a one-shot routing decision. We also introduce MNN Join, a transformation specialist that searches for string transformations by maximizing mutual-nearest-neighbor coverage between two unaligned columns and uses the same reciprocal structure to validate its output. We evaluate MetaJoin on 670 datasets from six benchmarks. It achieves 0.817 macro row-F1, slightly exceeding a benchmark-level oracle that is given the best standalone specialist for each benchmark (0.812), and outperforms label-free selection policies built from the same specialist portfolio by more than 0.13. Across the 609 datasets in the four transformation-dominant benchmarks, MNN Join outperforms every evaluated transformation baseline. Finally, MetaJoin's accuracy varies by only 0.015 as the controller backbone ranges from 3B to 14B parameters, showing that output validation and repair substantially reduce dependence on the initial routing decision.

cs.DB↗
arXiv2026

The Stretto Execution Engine for Semantic Data Systems

Semantic data systems enable querying over both structured and unstructured data using LLMs, but a central challenge is navigating the performance-accuracy trade-off when selecting among alternative strategies for executing semantic operators. In this paper, we present Stretto, a new execution engine for semantic query processing that combines a novel optimizer with a rich set of physical execution strategies for semantic operators. Unlike prior work, Stretto provides global - rather than per-operator - accuracy guarantees while optimizing query performance. To this end, Stretto formulates query planning as a constrained optimization problem and employs a gradient-based optimizer to jointly select operator implementations and allocate error budgets across the full query plan. Furthermore, to enable diverse execution choices, Stretto introduces a new technique that builds on KV-cache compression to realize a spectrum of physical execution strategies for semantic operators with controllable runtime-accuracy trade-offs. Experimental results demonstrate that Stretto outperforms state-of-the-art systems while consistently satisfying quality guarantees.

cs.DB↗