Searcharxiv⌕ Search
Home/Archive/arXiv

arXiv · 2610.25791

Towards Architectural-Level Safety Control: Relocating Hazardous Knowledge into Removable Experts in Mixture-of-Experts Models

Cheng Jin·Zhongtian Ma·Xiao Wang·Han Qi·Xiaoyu Wen·Xiang Wang·Youbang Sun·Xia Hu·Bowen Zhou·Chaochao Lu·Qiaosheng Zhang
View original source ↗Read source document ↗Export metadata ↓

Abstract

Large language models acquire broad knowledge from web-scale corpora during pretraining, including knowledge that could be hazardous if misused. Removing such knowledge from the model itself without degrading general capability is desirable but difficult, as it is deeply entangled with general capability within the same parameters. Existing safety alignment and unlearning methods typically adjust these shared parameters, tending to suppress the expression of hazardous knowledge rather than remove it. As a result, the knowledge remains in the model and is readily recoverable by subsequent fine-tuning. To address this limitation, we propose REX (Relocating into Removable EXperts), a knowledge relocation method that separates hazardous knowledge from general capability at the architectural level. Exploiting the modular structure of Mixture-of-Experts models, REX expands the model with a set of new, physically removable experts designated to hold hazardous knowledge. Through expert expansion, selective gradient masking and routing differentiation, it relocates hazardous knowledge out of the original experts and concentrates it in the new ones. Excising these new experts yields a model suitable for public release. This removal holds consistently across Intern-S2, Qwen3.5-35B-A3B and Gemma-4-26B-A4B; on Qwen3.5-35B-A3B, for instance, excision lowers WMDP-Bio accuracy by 48.4 points while MMLU changes by less than 3 points. The excised model matches the strongest of nine unlearning baselines in removing hazardous knowledge, and it is the only one that preserves this removal under subsequent fine-tuning. After 250M tokens of standard fine-tuning on Qwen3.5, the best baseline rebounds on WMDP-Bio from 44.1% to 78.3%, whereas REX rises only from 32.4% to 42.8%.

Explore related subjects

cs.CR

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Cheng Jin, Zhongtian Ma, Xiao Wang, Han Qi, Xiaoyu Wen, Xiang Wang, Youbang Sun, Xia Hu, Bowen Zhou, Chaochao Lu, Qiaosheng Zhang. 2026-10-19. Towards Architectural-Level Safety Control: Relocating Hazardous Knowledge into Removable Experts in Mixture-of-Experts Models. https://arxiv.org/abs/2610.25791

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Can You Trust the Vectors in Your Vector Database? Black-Hole Attack from Embedding Space Defects

Vector databases serve as the retrieval backbone of modern AI applications, yet their security remains largely unexplored. We propose the Black-Hole Attack, a poisoning attack that injects a small number of malicious vectors near the geometric center of the stored vectors. These injected vectors attract queries like a black hole and frequently appear in the top-k retrieval results for most queries. This attack is enabled by a phenomenon we term centrality-driven hubness: in high-dimensional embedding spaces, vectors near the centroid become nearest neighbors of a disproportionately large number of other vectors, while this centroid region is nearly empty in practice. The attack shows that vectors in a vector database cannot be blindly trusted: geometric defects in high-dimensional embeddings make retrieval inherently vulnerable. Based on this insight, we propose four attack paths tailored to different attacker capabilities. Our experiments show that up to 94.4% of queries are successfully attacked. Additionally, we study two directions of defense: hubness mitigation and detection-based filtering. Hubness mitigation either significantly reduces retrieval accuracy or provides only limited protection, while the detection-based defense is effective against some attack paths but fails against others. A robust and adaptive defense thus remains an open problem, and our findings indicate that vector databases require more careful treatment of security.

cs.CR↗
arXiv2026

Robust LLM Watermarking with Minimal Semantic Distortion for IP Protection

Proprietary large language models (LLMs) face risks of intellectual property (IP) violation, as adversaries can replicate an LLM by collecting input-output pairs to train a surrogate model, causing financial setbacks. Watermarks offer a promising defense to verify ownership, but existing methods often struggle with semantic distortion, factual inconsistency, and adversarial attacks. In addition, key-conditioned watermarks for provider-specific detection, especially in cross-provider and multi-user scenarios, remain largely underexplored. To address these challenges, we propose SAFESEAL, a novel key-conditioned watermarking framework that achieves strong detectability with minimal impact on model utility, effectively balancing detectability, utility, and robustness. SAFESEAL preserves named entities while substituting linguistic terms with context-aware synonyms through a key-conditioned Tournament sampling mechanism, maintaining semantic fidelity and factual consistency. For detection, we introduce a key-conditioned contrastive detector that jointly encodes the text and key, enabling provider-specific and robust watermark verification. We derive theoretical bounds on the utility-detectability trade-off and significantly reduce latency through lightweight models, batching, and parallelism. Extensive experiments show that SAFESEAL outperforms baselines in utility, detectability, and robustness, achieving a BERTScore of 0.983, entity similarity of 0.963, a 98.2% detection rate, and the highest human ratings for text quality and content preservation, with latency comparable to the fastest baseline. To promote transparency and community-driven progress, we release the first public watermark leaderboard and an interactive demo.

cs.CR↗
arXiv2026

Anywhere, Any-Stymie: Remote Activation of Trojan Malware on LiDAR with Modulated Signals

LiDAR sensors are widely deployed in autonomous systems for safety-critical 3D perception. We identify a previously unexplored attack surface in which dormant malware, embedded in the LiDAR sensing pipeline, remains inactive during normal operation and is triggered externally after deployment, without access to sensor hardware or networking at attack time. This malware was developed in a closed research test environment with vendor technical support, rather than by exploiting an inherent production supply-chain vulnerability. To trigger the malware, we repurpose the sensor's own native dual-return signal processing as a covert channel for a modulated optical signal delivered into the environment. Once triggered, the malware performs false data injection and suppression on static and mobile platforms. Unlike optical spoofing, where the transmitted signal is itself the attack, here it carries only a small key that the sensor decodes with existing signal processing. We establish feasibility with static triggering at 300~ft and drive-by runs reaching 35 mph, and verify that injected artifacts are taken up as detections by a fine-tuned 3D object detector. On a deployed tactical autonomous vehicle running an unmodified production autonomy stack, we demonstrate both fail-safe and fail-dangerous impact: emergency stop, mission abort, and collision. These results motivate stronger integrity guarantees across LiDAR development and deployment pipelines.

cs.CR↗