Measuring and Exploiting Implicit Trust in LLM Tool-Calling Pipelines

Murali Ediga; Sudipta Chattopadhyay

Home/Archive/arXiv

arXiv · 2609.18217

Measuring and Exploiting Implicit Trust in LLM Tool-Calling Pipelines

Murali Ediga·Sudipta Chattopadhyay

View original source ↗Read source document ↗Export metadata ↓

Abstract

The Model Context Protocol (MCP) enables LLMs to invoke external tools, but every tool interaction exposes the model to attacker-controlled text through multiple input channels (tool descriptions, tool results, sampling messages) that share a single context window without privilege separation. In this paper, we present a framework to measure the trust profile of an arbitrary LLM based on a variety of payload framings sent through different channels. Following this assessment, we devise cross-channel fragmentation attacks that distribute seemingly benign payloads across two or three channels; no individual channel carries a complete injection, yet the LLM compiles the fragments into credential exfiltration. We evaluated our attacks across 12 frontier models, three production clients, and six payloads, totalling over 15,000 trials. Our evaluation reveals that cross-channel attacks are an unexplored attack surface: models that fully resist single-channel injection (0% compliance) exfiltrate sensitive data at up to 100% under two-channel fragmentation (e.g., GPT-4o, Llama 70B, Composer 2, Haiku 4.5). We further demonstrate value-aligned exploitation, where a tool's stated purpose requires the data the attacker targets, and a sampling system prompt override that injects persistent instructions via VS Code's MCP implementation. Finally, we evaluated our attacks against seven third-party MCP security tools and three prompt-based defenses. All tools failed to detect fragmented payloads, and prompt defenses proved model-specific rather than universal.

Explore related subjects

cs.CR

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Murali Ediga, Sudipta Chattopadhyay. 2026-09-16. Measuring and Exploiting Implicit Trust in LLM Tool-Calling Pipelines. https://arxiv.org/abs/2609.18217

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Spoofing Missed-Detection Bounds for PRF GNSS Ranging Authentication Under AWGN Models

Pseudorandom-function (PRF) ranging codes, such as those used in Galileo's encrypted E6-C under the Signal Authentication Service (SAS), enable a receiver to authenticate pseudoranges once the PRF secret is revealed. This work bounds how much authentication security the receiver obtains under Additive White Gaussian Noise (AWGN) assumptions. Against a spoofer that does not estimate the code before submitting its forgery, PRF security makes the forged correlation zero-mean up to the security of the underlying PRF, allowing integration time and C/N$_0$ to mostly determine probability of missed detection (PMD) and probability of false alarm (PFA). Against such a spoofer at a conservative 30 dB-Hz, 400 ms of E6-C aggregation certifies a PMD below $2^{-128}$ (plus any PRF advantage). For a spoofer that estimates chips before submitting a forgery, I derive the receiving-antenna gain at which authentication security breaks, which is about 12 dB for E6-C for the adversaries modeled. This work can be used to design a PRF GNSS ranging code protocol and a receiver capable of correctly asserting PRF ranging security assuming an AWGN model.

cs.CR↗

arXiv2026

First Attack, Final Offensive: The Dark Forest on an Open Roster

The Dark Forest argument holds that a civilization that detects another should strike it at once. Existing formal models make the detected civilization the object of the strike and play it on a roster the attacker knows to be complete. This paper changes both choices. The object of hostility is remaining uncontrolled capacity to retaliate or to warn someone who can, and the roster is open: no attacker ever knows it has met everyone. A first strike is then rational only if the timing benefit of what it removes now rather than later is at least the disclosure loss from every survivor that learns of it. A survivor that can bring about the attacker's destruction enters that loss as a lump, not a per-unit rate, and the actors the attacker has never found may be such a survivor, one that no strike removes. Their capacity cannot be estimated, but what they can do is capped at the attacker's destruction, so the test against them asks one answerable question: a first strike is rational only if the attacker accepts that the strike may be its last attack. The Dark Forest premises, read as hypotheses, fix what a general attacker cannot estimate: hidden hunters exist, a hunter that verifies a hostile acts against it with probability at least $q$, and a hider is rarely found, so a believer's first strike is rational only if what it removes is worth a $q$-share of its survival, the whole of it as $q$ approaches one. With survival as the payoff, the profile in which every hunter strikes what it finds is not a Nash equilibrium whenever a strike is more visible to unfound hunters than a hider is findable, while the profile in which every hunter hides is. That visibility comparison is the decisive physical question; an attacker that treats its strike as unseen has assumed the roster closed.

cs.CR↗

arXiv2026

From Capability to Assurance in Autonomous Penetration-Testing Harnesses: A Framework and Reference Implementation

Research on large language model agents for penetration testing is evaluated almost entirely by capability: whether the agent captures a flag or reproduces a proof of concept. That metric suits a benchmark but is silent on the properties that decide whether an autonomous agent can be used in an authorized engagement: whether a reported finding is true, whether the agent stayed inside its authorized scope, and whether an operator can audit what it did. We call these assurance properties and argue that they belong to the harness, the runtime wrapping the model, and can be enforced in code. This paper makes three contributions. First, we define a framework of five assurance properties (evidence grounding, non destructive claim reduction, computed severity, enforced authorization, and tamper evident accountability), each with a formal model and an explicit acceptance test, connected to prior work in capability based security, tamper evident logging, and software provenance. Second, we position representative systems (PentestGPT, the Cochise reference harness, MAPTA, and the trajectory judge PentestJudge) within the framework using published coding criteria, and identify a consistent assurance gap. Third, we study one open source implementation, NeuroSploit, pinned to an exact commit, reporting its architecture, its complexity cost, and a content addressed artifact bundle from a run against a public deliberately vulnerable target. We execute the deterministic authorization and audit acceptance tests directly and find and report a real enforcement gap, which we reflect by scoring both properties as partial. We therefore claim an initial existence argument that the properties are realizable together, not a comparative performance result, and we specify the multi target, ablation, and adversarial evaluation protocol required to turn the framework obligations into measurements.

cs.CR↗