Searcharxiv⌕ Search
Home/Archive/arXiv

arXiv · 2609.29668

Graph, Loop, and Harness Engineering for Zero-Trust Agentic Data Engineering and Analytical Processing

Sagar Srinivas Sakhinana·Venkataramana Runkana
View original source ↗Read source document ↗Export metadata ↓

Abstract

Large language model agents increasingly automate data workflows, but end-to-end cloud data engineering and analytical execution require reliable coordination across code, data, infrastructure, and runtime environments. We present two zero-trust frameworks. Zero-Trust Agentic Data Engineering generates, deploys, and verifies complete cloud data-engineering solutions from natural-language tasks, with completion conditioned on repository, deployment, runtime, and policy evidence. Zero-Trust Agentic OLAP combines governed Data Preparation with verified Online Analytical Processing (OLAP), permitting production promotion only after validation and evidence-bound approval, and releasing analytical answers only after Same-Snapshot Execution, Exact Result Equivalence, deterministic grounding, and reflection. Both frameworks share three abstractions: graph engineering for evidence-gated workflow structure, loop engineering for bounded recovery, and agent-harness engineering for zero-trust execution. We evaluate both frameworks under nominal execution, controlled failures, bounded recovery, and policy-constrained conditions, measuring verified completion, recovery, authorization enforcement, production promotion, and verified OLAP execution.

Explore related subjects

cs.LGcs.AI

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Sagar Srinivas Sakhinana, Venkataramana Runkana. 2026-08-30. Graph, Loop, and Harness Engineering for Zero-Trust Agentic Data Engineering and Analytical Processing. https://arxiv.org/abs/2609.29668

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

The Re-Label Method For Data-Centric Machine Learning

In industry deep learning application, our manually labeled data has a certain number of noisy data. To solve this problem and achieve more than 90 score in dev dataset, we present a simple method to find the noisy data and re-label the noisy data by human, given the model predictions as references in human labeling. In this paper, we illustrate our idea for a broad set of deep learning tasks, includes classification, sequence tagging, object detection, sequence generation, click-through rate prediction. The dev dataset evaluation results and human evaluation results verify our idea.

cs.LG↗
arXiv2026

DeFlow: Decoupling Behavior-Prior Modeling and Value Maximization for Offline Policy Extraction

We present DeFlow, a decoupled offline RL framework that leverages flow matching to faithfully capture complex behavior manifolds. Optimizing generative policies is computationally prohibitive, typically necessitating backpropagation through ODE solvers. We address this by learning a lightweight refinement module within an explicit, data-derived trust region of the flow manifold, rather than sacrificing the iterative generation capability via single-step distillation. This way, we bypass solver differentiation and eliminate the need for balancing loss terms, ensuring stable improvement while fully preserving the flow's iterative expressivity. Empirically, DeFlow achieves superior performance on the challenging OGBench benchmark and demonstrates efficient offline-to-online adaptation.

cs.LG↗
arXiv2026

CoCopula: Conformal Calibration and Copula Aggregation for Reliable Renewable Energy Forecasting

The rapid growth of renewable energy penetration has intensified the need for reliable probabilistic forecasts to support grid operations at aggregated (fleet or system) levels. In practice, however, system operators often lack access to fleet-level probabilistic models and instead rely on site-level forecasts produced by heterogeneous third-party providers. Constructing coherent and calibrated fleet-level probabilistic forecasts from such inputs remains challenging due to complex cross-site dependencies and aggregation-induced miscalibration. Motivated by a real-world operational challenge identified through collaboration with an industry partner, this paper presents a probabilistic aggregation and calibration framework that directly converts site-level probabilistic forecasts into reliable fleet-level forecasts. The paper introduces \method~ (\textbf{Co}nformal Calibration and \textbf{Copula} Aggregation), which integrates Copula-based dependence modeling to capture cross-site correlations followed by Context-Aware Conformal Prediction (CACP) to mitigate miscalibration at the aggregated level. This combination enables dependence-aware aggregation while improving empirical coverage and maintaining sharp prediction intervals. Experiments on large-scale solar generation datasets from three system operators in the United States including MISO, ERCOT, and SPP demonstrate that the proposed \method{} approach achieves near-nominal empirical coverage and consistently lower Winkler scores than competing aggregation-based baselines across different target coverage levels. Importantly, \method{} achieves performance comparable to directly calibrated system-level forecasts, demonstrating its practical value when dedicated fleet-level forecasting models are unavailable.

cs.LG↗