Searcharxiv⌕ Search

arXiv subjects

Luman Zhao

Publications and source records attributed to Luman Zhao.

2 recordsLinked to original sources

LTBD: Learnable Trust-Boundary Delimiters for Prompt Injection Defense

Large language models (LLMs) perform remarkably well on complex tasks, yet remain highly vulnerable to prompt injection attacks, where malicious instructions embedded in external data can override user intent. Existing defenses remain limited by model fine-tuning requirements, vulnerability to adaptive attacks, or reliance on brittle handcrafted prompts. We argue that a fundamental source of this vulnerability is the lack of an explicit representation of trust provenance. To address this, we introduce Learnable Trust-Boundary Delimiters (LTBD), a lightweight defense that explicitly encodes trust boundaries in the input while keeping the LLM parameters unchanged. LTBD uses a small number of learnable delimiters to distinguish trusted user instructions from untrusted external data, enabling the model to better respect the intended trust hierarchy. Experimental results show that LTBD substantially outperforms inference-time defenses and performs competitively with training-based approaches, while preserving benign-task utility and introducing negligible inference overhead. In particular, LTBD achieves 0.00% ASR on AlpacaFarm and only 0.11-0.19% ASR on TaskTracker. LTBD also remains effective under adaptive attacks, where adversaries have full knowledge of the defense and explicitly attempt to bypass it.

cs.CR↗

Modular assurance of an Autonomous Ferry using Contract-Based Design and Simulation-based Verification Principles

With the introduction of autonomous technology into our society, e.g. autonomous shipping, it is important to assess and assure the safety of autonomous systems in a real-world context. Simulation-based testing is a common approach to attempt to verify performance of autonomous systems, but assurance also requires formal evidence. This paper introduces the Assurance of Digital Assets (ADA) framework, a structured method for the assurance of digital assets, i.e. novel, complex, or intelligent systems enabled by digital technologies, using contract-based design. Results are shown for an autonomous ferry assurance case, focusing on collision avoidance during the ferry's transit. Further, we discuss the role of simulation-based testing in verifying compliance to contract specifications, to build the necessary evidence for an assurance case.

cs.LO↗