TY - RPRT TI - Why Is RLHF Alignment Shallow? A Gradient Analysis AU - Robin Young PY - 2026 UR - https://arxiv.org/abs/2603.04851 ID - 2603.04851 ER -