arXiv · 2611.23096
Hidden-State Similarity Predicts Re-Elicitation After Inoculation Prompting
Abstract
Fine-tuning on narrow harmful tasks can cause emergent misalignment, where models generalize harmful behavior beyond the training distribution. Inoculation prompting can reduce this effect by explicitly eliciting the undesired behavior during training, but recent work shows that the behavior can reappear when evaluation prompts contain cues from the training context. We study what makes such prompts effective triggers. We find that, surprisingly, textual similarity to the inoculation prompt is an incomplete predictor. Instead, prompts are more likely to re-elicit suppressed behavior when they induce activation states similar to those produced by the inoculation context. These findings advance our understanding of how inoculation prompting modulates conditional misalignment, and suggest that activation space analysis can help identify when suppressed behaviors remain accessible under eval-time prompts.
Explore related subjects
Keep this discovery
Explore connections, maps & timelines
Rose N. Kwon, Samy Mammeri, Sabyasachi Sahoo, Christian Gagné, Thomas Jiralerspong. 2026-11-17. Hidden-State Similarity Predicts Re-Elicitation After Inoculation Prompting. https://searcharxiv.com/abs/2611.23096
Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite Searcharxiv by Indiciae: BibTeX · RIS.