SEARCH · Searcharxiv

Search Searcharxiv

Search indexed arXiv papers on artificial intelligence, large language models, computer vision and robotics. Read source abstracts and follow links to arXiv.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 523 records · Page 29Linked to original sources

arXiv2026

Progressive Behavioral Drift through Compression Valleys in Large Language Models

We show that attention sinks and compression valleys create a vulnerable region in decoder-only Transformers, where small activation perturbations can be amplified through the autoregressive trajectory. Based on this, we propose Sensitivity-Scaled Steering (SSS), a progressive activation-space attack that anchors perturbations at the beginning-of-sequence token and adaptively reinforces them at sensitive layers and tokens. Instead of forcing an abrupt behavioral change, SSS induces a staged drift, making outputs gradually shift toward the target behavior while remaining fluent and benign-looking in early generations. Across multiple open-weight models and four behavioral axes, SSS achieves high attack success, preserves coherence, and causes negligible degradation to general capabilities. These results show that attention sinks and compression valleys are not merely mechanistic features; rather, they expose exploitable amplification mechanisms that can be treated as hidden-state weaknesses for activation-space attacks in white-box and supply-chain LLM deployments.

cs.CR↗

arXiv2026

More Haste, Less Speed: Weaker Single-Layer Watermark Improves Distortion-Free Watermark Ensembles

Watermarking has emerged as a crucial technique for detecting and attributing content generated by large language models. While recent advancements have utilized watermark ensembles to enhance robustness, prevailing methods typically prioritize maximizing the strength of the watermark at every individual layer. In this work, we identify a critical limitation in this "stronger-is-better" approach: strong watermarks significantly reduce the entropy of the token distribution, which paradoxically weakens the effectiveness of watermarking in subsequent layers. We theoretically and empirically show that detectability is bounded by entropy and that watermark ensembles induce a monotonic decrease in both entropy and the expected green-list ratio across layers. To address this inherent trade-off, we propose a general framework that utilizes weaker single-layer watermarks to preserve the entropy required for effective multi-layer ensembling. Empirical evaluations demonstrate that this counter-intuitive strategy mitigates signal decay and consistently outperforms strong baselines in both detectability and robustness.

cs.CR↗

arXiv2026

Balancing the privacy-utility trade-off: How to draw reliable conclusions from private data

Absolute anonymization, conceived as an irreversible transformation preventing re-identification and sensitive value disclosure, has proven to be a broken promise. Modern data protection must therefore shift toward a privacy-utility trade-off grounded in risk mitigation. Differential Privacy (DP) offers a rigorous mathematical framework for balancing quantified disclosure risk with analytical usefulness. Nevertheless, widespread adoption remains limited, largely because complex technical concepts, such as privacy-loss parameters, have yet to be translated into forms meaningful to non-technical stakeholders. This difficulty arises from randomization itself: both analysts and adversaries must draw conclusions from uncertain observations rather than deterministic values. In this work, we adopt an interpretation of the privacy-utility trade-off based on hypothesis testing to measure the uncertainty introduced by randomized mechanisms. In particular, we use the concept of relative disclosure risk to quantify the maximum reduction in uncertainty an adversary can obtain from a membership attack on protected outputs, and show this measure relates directly to standard privacy-loss parameters. We further analyze how DP affects analytical validity via its impact on hypothesis tests assessing statistical significance. Building on these results, we provide practical guidance, accessible to non-experts such as data protection authorities, for navigating the trade-off and selecting protection mechanisms and parameter values.

stat.ME↗

arXiv2026

ROAST: Risk-aware Outlier-exposure for Adversarial Selective Training of Anomaly Detectors Against Evasion Attacks

Safety-critical domains like healthcare rely on deep neural networks (DNNs) for prediction, yet DNNs remain vulnerable to evasion attacks. Anomaly detectors (ADs) are widely used to protect DNNs, but conventional ADs are trained indiscriminately on benign data from all patients, overlooking physiological differences that introduce noise, degrade robustness, and reduce recall. In this paper, we propose ROAST, a novel risk-aware outlier exposure (OE) selective training framework that improves AD recall while largely preserving precision. ROAST identifies patients who are less vulnerable to attack and focuses training on these cleaner, more reliable data, thereby reducing false negatives and improving recall. To preserve precision, the framework applies OE by injecting adversarial samples into the training set of the less vulnerable patients, avoiding noisy data from others. Experiments show that ROAST increases recall by 16.2\% (black-box attack setting) and 5.89\% (white-box attack setting) on average while reducing the training time by 88.3\% on average compared to indiscriminate training, with minimal impact on precision.

cs.CR↗

arXiv2026

What's on Your Mind? Exploring Privacy of Mental Health Apps

Therapy and life-coaching apps have grown rapidly in number, variety, and popularity. At the same time, their users often share highly sensitive and personal information, including mental health issues, trauma experiences, fantasies, desires, and relationship difficulties. This prompts the need to examine privacy practices across the ecosystem. In this paper, we present a comprehensive analysis of a corpus of 25 popular Android mental health and life-coaching apps, such as Replika and Headspace. It builds on static analysis and dynamic network traffic analysis, coupled with identifying gaps between each app's observed behavior and its privacy policies. Our analysis highlights serious concerns and substantial transparency gaps. First, every app in our corpus embeds at least one tracker SDK not named in its privacy policy, and 85\% of the apps we instrument fail to disclose at least half of the trackers detected in their APKs. Second, more than half of the apps declare several dangerous permissions without corresponding privacy-policy disclosures, including camera or microphone permissions. Third, nearly half the apps disclose third-party AI processing (e.g., via OpenAI, Anthropic, and Groq) in their privacy policies, while several use only generic language (e.g., "AI services"), failing to identify which company receives user data. Overall, our results show that current disclosure practices fall short of the transparency needed for meaningful informed consent. We argue for a significantly updated regulatory framework for therapy apps that more closely aligns with the professional and ethical standards that bind licensed human therapists.

cs.CR↗

arXiv2026

Removing the Watermark Is Not Enough: Forensic Stealth in Generative-AI Watermark Removal

The literature on watermark removal has largely asked whether an attacker can make the watermark verifier fail while preserving the appearance of the image. This is a useful test, but it does not capture the purpose of removal in applications where watermarks support provenance. In such settings, the attacker wants the image to pass as ordinary content. If the removal process leaves a recognizable statistical trace, the watermark may have disappeared, but deniability has not been restored. We call this missing requirement forensic stealth. We evaluate six recent attacks spanning four different removal strategies and find that all leave strong forensic traces. At a 1% false-positive target, attack-specific detectors identify at least 99% of the removal outputs. In a separate image-by-image assessment of five attacks, only one of 750 outputs removes the watermark, remains within the fidelity budget, and evades forensic detection. The consistency of this result across different mechanisms shows that current evaluation practice overlooks a central part of the security problem. We also ask whether forensic stealth is possible in principle. Under explicit idealized assumptions, we show that exact forensic stealth is possible when a remover preserves source content and resamples the remaining detail from the corresponding clean distribution. In this model, the resulting outputs exactly match the clean-image distribution while remaining within the distortion budget. This shows that removal traces are not inevitable and places the practical difficulty in generating source-appropriate clean variation without damaging the image. We argue that forensic stealth should become part of the standard by which watermark removal is judged.

cs.CR↗

arXiv2026

SkillSafetyBench: Evaluating Agent Safety under Skill-Facing Attack Surfaces

Reusable skills are becoming a common interface for extending large language model agents, packaging procedural guidance with access to files, tools, memory, and execution environments. However, this modularity introduces attack surfaces that are largely missed by existing safety evaluations: even when the user request is benign, unsafe influence may reside in skill guidance, local artifacts, or execution-environment files that steer the agent toward unsafe actions. We present SkillSafetyBench, a runnable benchmark for evaluating such skill-facing safety failures. SkillSafetyBench includes 155 adversarial cases across 47 tasks, 6 risk domains, and 30 safety categories, each evaluated with a case-specific rule-based verifier. Experiments with multiple CLI agents and model backends show that non-user attacks can consistently induce unsafe behavior, with distinct failure patterns across domains, attack methods, and scaffold-model pairings. Our findings suggest that agent safety depends not only on model-level alignment, but also on how agents interpret skills, trust workflow context, and act through executable environments. The complete benchmark is available at https://github.com/AI45Lab/skill-safety-bench.

cs.CR↗

arXiv2026

Adversarial Trust Poisoning in Vehicular Collaborative Perception

Collaborative perception (CP) enables connected and autonomous vehicles to share sensor data and jointly reason about their environment. To defend against adversaries that fabricate or manipulate shared data, existing systems employ cross-vehicle inconsistency detection and trust estimation, penalizing vehicles whose observations conflict with the majority. In this work, we show that these defenses themselves introduce a new attack surface. We present TrustFlip, a novel attack that weaponizes consistency-based defenses to poison the trust assigned to benign vehicles. Instead of injecting false data into the collaboration pipeline, it deploys physical adversarial objects that are genuine but induce inconsistent observations among benign vehicles. The resulting inconsistencies are misattributed by the defense to the targeted vehicle, causing its trust score to degrade and eventually leading to its downweighting or exclusion from collaboration. Consequently, the system loses reliable sensing contributors, degrading perception capability and potentially inducing safety-critical failures. We evaluate TrustFlip across multiple collaborative perception architectures and defense mechanisms. Our results show that state-of-the-art defenses can be significantly affected: the attack removes the targeted benign vehicle from collaboration in up to 87.7% of scenarios and drops Average Precision (AP) by up to 13%. As an initial mitigation, we introduce TrustReflect, a lightweight self-reflection mechanism that marks disputed regions as uncertain and excludes them from trust evaluation, reducing the attack success rate by 35-100%.

cs.CR↗

arXiv2026

A Wolf in Sheep's Clothing: Targeted Routing Hijacking in Federated RAG

Federated Retrieval-Augmented Generation (FedRAG) is attractive for privacy-sensitive applications because full local corpora remain on clients. As a result, routing must rely on client-provided semantic profiles, creating a new opportunity for manipulation. We introduce Routing Hijacking, a routing-stage attack in which a malicious client forges its profile to attract target queries despite having irrelevant underlying data. We show that this vulnerability is severe. Across three representative FedRAG routing architectures, Routing Hijacking consistently misroutes target queries and leads to downstream disruptions and failures, including missing evidence, poisoning, incorrect answers, and hallucinations. In a controlled MedQA-USMLE stress test, we further show that poisoned retrieved evidence can mislead models across scales, leading to incorrect answers, hallucinations, and sycophantic failures. Existing defenses do not close this gap: encrypted routing preserves the exploited ranking, and Byzantine-robust Federated Learning (FL) rules transfer poorly to heterogeneous routing profiles. To address this gap, we propose a trust-aware post-routing framework that reweights clients using returned-evidence feedback, including retrieval relevance, profile consistency, and cross-client agreement; online experiments show that it suppresses persistent hijacking over recurring queries and transfers to a learned neural router. Our findings establish routing integrity as a security challenge in FedRAG and highlight the need for stronger defenses for secure federated retrieval.

cs.CR↗

arXiv2026

Clinically Grounded Privacy Evaluation of Medical LMs

Medical language models (LMs) can memorize and reproduce protected health information, but privacy evaluations often focus on recovery of training text rather than disclosure under realistic threat models. We introduce a clinically grounded framework that evaluates leakage along a graded axis of adversarial access, ranging from publicly inferable demographics to leaked note fragments. At each tier, we measure verbatim memorization of patient-specific text and semantic leakage of sensitive diagnoses. Applying the framework to an LM continually pretrained on 378k clinical notes, we find that routine encounter metadata (i.e., name, date of birth, visit date, provider name, and practice location) elicits high rates of verbatim memorization across a patient's timeline and sensitive-diagnosis recovery (AUROC 0.91 for abortion, 0.82 for HIV). At the same time, exact-match memorization can overstate disclosure: 36% of memorized tokens reflect templated documentation. Our work highlights the risks of training on longitudinal clinical data and provides a practical, reusable framework for contextual privacy evaluation of medical LMs.

cs.CL↗

arXiv2026

KidnapRAG: A Black-Box Attack for Hijacking Reasoning in Agentic Retrieval-Augmented Generation Systems

Retrieval-Augmented Generation (RAG) systems are vulnerable to poisoning attacks that inject malicious documents into the retrieval process to manipulate model outputs. Recent Agentic RAG systems are more robust to such attacks because they iteratively perform retrieval and reasoning, allowing them to ignore weakly relevant poisoned documents and preserve the reasoning chain induced by the user query. However, existing attacks on Agentic RAG systems often assume white-box access to system prompts, reasoning traces, retrievers, or model parameters, limiting their applicability in realistic settings. In this paper, we study black-box poisoning attacks against Agentic RAG systems, where the attacker can only publish externally retrievable poisoned documents. We propose KidnapRAG, a sequential poisoning attack that hijacks the agent's multi-step reasoning chain using three role-specific documents: Bait, Chain-Link, and Mal-Ins, which attract initial retrieval, induce query reformulation, and provide attacker-controlled evidence, respectively. Experiments across multiple Agentic RAG frameworks, LLM backbones, and benchmarks show that KidnapRAG consistently outperforms existing poisoning baselines under black-box conditions. Further analyses show that KidnapRAG progressively weakens the original retrieval intent, redirects retrieval behavior, and increases reliance on attacker-controlled evidence. Our code is publicly available at https://github.com/chanwoochoi316/KidnapRAG.

cs.CR↗

arXiv2026

MTD-Playground: An Attacker-Aware Evaluation Framework for Network Moving Target Defense

Moving Target Defense (MTD) has emerged as a proactive network cyber defense paradigm that increases attacker uncertainty through dynamic network reconfiguration techniques such as Software-Defined Networking (SDN)-enabled path randomization. However, existing evaluations remain fragmented due to inconsistent attacker assumptions, attack scenarios, and evaluation metrics, limiting reproducibility and deployment-oriented comparison. In this paper, we present MTD-Playground, an attacker-aware evaluation framework for benchmarking SDN-enabled path-randomization (PR) MTD techniques under realistic enterprise-style multi-stage attack scenarios. Beyond isolated security and performance metrics, MTD-Playground introduces a composite evaluation methodology for analyzing deployment effectiveness, mutation-interval trade-offs, and defender-attacker operational balance. Using periodic path randomization as a representative PR-MTD strategy, our evaluation shows that aggressive mutation intervals reduce attack success rates to 4-20% while increasing attack completion time to 160-311s across evaluated attack scenarios. At the same time, PR-MTD improves throughput by up to 30.9% and reduces internal-path latency without service interruption. Composite analysis further shows that shorter mutation intervals consistently achieve the highest deployment effectiveness and positive defender advantage. These results demonstrate that SDN-based PR-MTD can substantially disrupt multi-stage attack progression while remaining practically deployable in enterprise environments.

cs.CR↗

arXiv2026

Bit-Level Triangular Content-Aware Permutation for Fragile Image Watermarking: Zero False Positive Rate, Single-Bit Sensitivity, and Arbitrary Dimension Support

With the growth of digital document exchange, protecting image integrity against attacks such as Vector Quantization (VQ) and collage has become critical. Existing methods are vulnerable to these attacks and limited to fixed image dimensions. This paper presents a novel, dimension-agnostic, fragile watermarking algorithm that enhances security and tamper localization by replacing conventional hash functions with Triangular Content-Aware Permutation (TCA). The image is combined with key-based global noise and divided into blocks. The core innovation is applying content-dependent permutation with intrinsic avalanche effect (TCA) at the bit-plane level, generating a unique content-dependent watermark. For color images, a vertical sandwich transformation merges channels, preserving inter-channel dependency with only 1.62x time increase. The "remainder merging" strategy eliminates padding constraints. Experiments on 50 grayscale and 10 color images under 18 attacks show FPR=0% and FNR=0% for 17 attacks. Salt-and-pepper noise yields negligible FNR of 0.27% (grayscale) and 0.14% (color). Average PSNR is 51.14 dB (8-bit), 75.25 dB (12-bit), and 99.33 dB (16-bit). Embedding and extraction times are 1.61 s and 1.63 s, respectively. The algorithm achieves 100% accuracy against collage, VQ, copy-move, JPEG (quality 5-95), and geometric attacks, providing a secure solution for digital forensics, medical imaging, and legal document authentication.

cs.CR↗

arXiv2026

ContextLeak: Exfiltrating LLM Agent Context via Malicious Tools

Exfiltrating an LLM agent's runtime context -- such as the user prompt, execution trajectory, and tool list -- poses severe security and privacy risks to users. Such attacks can be carried out via malicious tools and typically require three conditions: (1) the agent selects the malicious tool for task execution, (2) the agent passes its runtime context as input arguments to the tool, and (3) the tool's implementation transmits these inputs to an attacker-controlled endpoint. Existing work primarily focuses on conditions (1) and (3), leaving condition (2) largely unexplored, despite its critical role in enabling successful context exfiltration. In this work, we bridge this gap by developing ContextLeak, a malicious tool attack that induces the agent to both select the tool and disclose its context as input arguments. We realize this attack by carefully crafting the tool's name and description using reinforcement learning. Specifically, ContextLeak employs an LLM, referred to as the attack LLM, to automatically generate the malicious tool's name and description. To improve attack effectiveness, we fine-tune the attack LLM via reinforcement learning on a set of shadow users with diverse, simulated agent contexts. Our key technical contribution is the design of novel reward functions tailored to the context exfiltration objective, enabling effective reinforcement-learning-based fine-tuning of the attack LLM. Extensive evaluation demonstrates that our attack remains highly effective even when the shadow users' contexts differ substantially from those of the victim users. Moreover, ContextLeak significantly outperforms existing malicious tool attacks when adapted to this setting.

cs.CR↗

arXiv2026

FISGuard: Defending Against Membership Inference via Fixed Input Subspaces

As large language models are increasingly adopted in federated learning, protecting user privacy while performing parameter-efficient fine-tuning on distributed private data has become an important challenge. Although clients only share gradients instead of directly uploading raw data, the shared gradients may still leak membership information about training samples. ProjRes (S&P, 2026) further increases this risk: with less information and without accessing model outputs, an attacker can effectively distinguish members from non-members solely based on the projection residual between a candidate representation and the subspace induced by server-observable gradients. Existing defenses against membership inference mostly rely on gradient perturbation or regularization, which can not only degrade model utility but also fail to effectively defend against the membership inference attack introduced by ProjRes, which exploits the geometric structure of gradients. To address this issue, we propose FISGuard, a lightweight defense. Its key idea is to construct and fix a low-dimensional representation subspace using independent public data, thereby restricting the space through which private representations are exposed via gradients while preserving the primary information required for downstream tasks. This substantially reduces the projection-residual discrepancy between members and non-members. We evaluate FISGuard against five representative defense methods across three NLP datasets, two LLMs, and two fine-tuning strategies, Adapter and LoRA. The results show that FISGuard reduces the ProjRes attack AUC to near the random-guessing level of 0.5 in most settings, while maintaining downstream task performance close to that of the undefended model and introducing only limited computational overhead, thereby achieving a favorable privacy--utility trade-off.

cs.CR↗

arXiv2026

Online Differentially Private Consistent Clustering

We study differentially private (DP) $k$-means and $k$-median clustering in the online streaming setting. In this model, points arrive sequentially, and at each time step, we need to output a set of $k$ centers that optimizes the clustering objective for all points seen so far. We give a generic reduction that transforms the (sensitive) input stream into a private stream, which is a semi-coreset of the input stream. This implies that any (non-private) online clustering algorithm, run as a post-processing step, can achieve good utility for the original clustering objective. Our algorithm matches or improves upon the approximation ratio, space usage, and running time of existing algorithms [Epasto et al., 2026, Dupré la Tour et al., 2024]. A key aspect of our reduction is that it inherits desirable properties of the underlying non-private clustering algorithm, such as consistency [Lattanzi and Vassilvitskii, 2017]--a property not satisfied by previous DP algorithms.

cs.DS↗

arXiv2026

A User-Centric Context-Aware Permission Governance Framework for Privacy Control in Default Mobile Applications

Mobile operating systems provide runtime permission controls intended to improve user control over sensitive data. However, default or pre-installed applications are deeply integrated into the system, may operate with elevated privileges, and are difficult for users to scrutinize. Existing permission models generally grant persistent or temporary access for an application session without distinguishing among individual features, leaving users uncertain about when and why data are accessed. This paper presents a context-sensitive, user-focused permission governance framework for default mobile applications. It introduces a feature-based authorization option, "Allow When Needed," that restricts access to the functionality requiring the data rather than the entire application session. A weighted scoring system estimates the privacy implications of user choices based on permission sensitivity and authorization type. A web-based simulation platform was developed to model 30 realistic permission-request situations across six commonly used default application types and support controlled early-stage evaluation before native implementation. The exploratory assessment combined a cross-sectional survey of 104 respondents examining permission awareness and behavior with formative usability testing involving eight participants interacting with the prototype. Survey findings indicate that users do not consistently examine default-application permissions and prefer contextual explanations before granting access. The results provide preliminary evidence that context-aware permission governance can improve user understanding and decision clarity. This simulation-based study represents an initial step toward evaluating feature-level authorization and privacy-feedback mechanisms before native mobile deployment.

cs.CR↗

arXiv2026

GraftyVul: Synthesising Insecure Programs Through Real-World Vulnerability Grafting

Vulnerability datasets underpin a wide range of security research, including vulnerability detection, automated remediation, and secure code generation. However, existing datasets sacrifice at least one of three desirable properties: diversity (of language or vulnerability type), reproducibility/executability, or realism. We therefore present GraftyVul, a system that constructs vulnerable programs by grafting real-world vulnerabilities into open-source projects. This grounds the dataset in vulnerabilities observed in real-world contexts while harnessing known good build and test environments, enabling exploit-verification scripts to guarantee that an introduced vulnerability successfully alters a program's behaviour. Using GraftyVul, we generate 212 verified and exploitable vulnerable programs spanning five programming languages (Python, TypeScript, Java, Go, and C#) across 23 CWE categories. To evaluate fidelity, we introduce a language- and context-agnostic semantic embedding that compares vulnerabilities by sink, mechanism and host-feature rather than surface code. This approach outperforms standard code embeddings on cross-language clone and CWE classification. These embeddings demonstrate that GraftyVul samples retain a strong semantic signature to their source vulnerability. We additionally compare GraftyVul against 13 widely used datasets, where it attains competitive diversity while being the only reproducible-exploit dataset with broad language and CWE coverage. Finally, we illustrate GraftyVul's practical utility through an industrial case study evaluating a production vulnerability remediation system.

cs.CR↗