Searcharxiv⌕ Search

arXiv · 2610.19948

When Protection Overrides Consequences: Understanding Protection Mode in Large Language Model Decision-Making

Abstract

Frontier language models excel at complex tasks yet may prioritize protection over larger physical consequences in simple, urgent decisions. GPT-5.6 Sol, for example, moves a girl playing in shallow water before retrieving a submerged camera. We call this Protection Mode and study it with ProtectionBench: 285 prompts, 34 physical families, and nine models. At lowest effort, at least one of four responses prioritizes the subject on 34.74% of Sol prompts and over 83% of MiniMax M3 and Qwen3.5-27B prompts; none of four independent human raters does so. Controls show that choices usually track physical risk. We elicit both risks before requesting an action. Some models assess risk incorrectly; others state the correct ordering but still choose the subject. Qwen3.5-27B with thinking shows 100% correct risk ordering but 57.81% subject choice among eligible conversations on eight selected scenes. Attention audits find no stable reduction in attention to relevant facts; removing the stated risk direction reduces object-first choice in dots3-note-prev but not detectably in Qwen3.5-27B. Adding a general "people or animals before property" rule increases subject choice by 19-38 percentage points, whereas factual reminders favor the object by 0.7-5.2 points. Removing naturally generated versions of that rule lowers subject choice by 10-15 points. We also examine whether post-training introduces this pattern. Base models already show Protection Mode; post-training changes justifications more than choices.

Explore related subjects

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Yuanfan Li, Junyi Chen, Qi Zhou, Rui Yang, ZhangLiujie, Weihang Chen. 2026-10-14. When Protection Overrides Consequences: Understanding Protection Mode in Large Language Model Decision-Making. https://arxiv.org/abs/2610.19948

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

SEDM: Selective Self-Evolving Distributed Memory for Agents

Agents in a society can reuse each other's experience, but indiscriminate memory use increases processing cost and can reduce task success. We present SEDM (Selective Self-Evolving Distributed Memory), a framework that selects both what agents share and when recipients use it. On the sender side, SEDM admits memories through paired execution tests, ranks them by relevance and measured utility, and abstracts them for cross-domain reuse. On the recipient side, Calibrated Memory Gating (CMG) predicts success with and without memory in a single forward pass, learning from paired outcomes including ties. It injects memory only when the calibrated success gain outweighs a context-cost penalty. SEDM exceeds the strongest evaluated baseline on LoCoMo open-domain questions by 15.1 F1 points and outperforms G-Memory on FEVER and HotpotQA with 16-32% fewer prompt tokens. In a controlled HotpotQA study, CMG achieves success comparable to always using memory with 18% fewer agent input tokens. Within the full SEDM pipeline on LoCoMo, CMG outperforms both always-use and no-memory policies for two SEDM memories while using 19-31% fewer input tokens than always-use. These results show that selective use can recover value from shared memories that hurt performance when injected indiscriminately.

cs.AI↗

Toward Agents That Reason About Their Computation

While reinforcement learning agents can achieve superhuman performance in many complex tasks, they typically do not become more computationally efficient as they improve. In contrast, humans gradually require less cognitive effort as they become more proficient at a task. If agents could reason about their compute as they learn, could they similarly reduce their computation footprint? If they could, we could have more energy efficient agents or free up compute cycles for other processes like planning. In this paper, we experiment with showing agents the cost of their computation and giving them the ability to control when they use compute. We conduct our experiments on the Arcade Learning Environment, and our results demonstrate that with the same training compute budget, agents that reason about their compute perform better on 75% of games. Furthermore, these agents use three times less compute on average. We analyze individual games and show where agents gain these efficiencies. An interactive version of this paper is available at https://adrianorenstein.com/paper/toward-compute.

cs.AI↗

LingLanMedBench: A Unified Benchmark for Evaluating Large Language Models in Traditional Chinese Medicine

Reliable evaluation of large language models in traditional Chinese medicine remains limited because existing benchmarks mainly focus on examination-style question answering or isolated generative tasks. We developed LingLanMedBench, a unified benchmark comprising 25,620 expert-verified items across 13 subtasks and five domains, covering knowledge, language understanding, information extraction, and diagnostic-therapeutic decision-making. LingLanMedBench standardizes task formats and evaluation metrics and further incorporates synonym-tolerant clinical-label matching, empirically derived hard subsets, and decision-recognition reformulations of open-ended clinical tasks. Fourteen open-source and proprietary models were evaluated under a standardized zero-shot protocol. Performance varied substantially across domains, model families, task formulations, and difficulty levels. The leading overall score decreased from 51.1 on the full benchmark to 31.9 on the hard subsets. Prescription performance also differed markedly between open-ended generation, with F1 scores of 32.7 and 20.3, and decision recognition, with accuracies of 90.6 and 56.8 on the full and hard subsets, respectively. These findings reveal a substantial gap between knowledge-oriented performance and clinically faithful reasoning. LingLanMedBench provides a standardized and reproducible framework for characterizing current capabilities, identifying systematic limitations, and tracking meaningful progress in large language models for traditional Chinese medicine.

cs.AI↗