TY - RPRT TI - Offline Safe Policy Optimization From Heterogeneous Feedback AU - Ze Gong AU - Pradeep Varakantham AU - Akshat Kumar PY - 2025 UR - https://arxiv.org/abs/2512.20173 ID - 2512.20173 ER -