arXiv · 2611.19787
Rethinking Post-Training for Expressive Zero-Shot Text-to-Speech
Abstract
Zero-shot voice cloning typically emphasizes preserving the timbre and prosodic characteristics of a reference speaker. However, in real-world use, speech should adapt its delivery to the target text rather than remain tied to a fixed speaking style. In this work, we formulate this problem as text-adaptive prosody synthesis for zero-shot voice cloning and study how post-training can endow a cloning model with this capability. We develop a three-stage framework. First, Direct Preference Optimization (DPO) provides a cold start by aligning the model with text-appropriate expressive speech. Second, Group Sequence Policy Optimization (GSPO) jointly optimizes prosodic expressiveness, acoustic quality, intelligibility, and speaker similarity. Finally, context-conditioned On-Policy Self-Distillation (OPSD) enables self-improvement by transferring expressive behavior from strong-reference to weak-reference conditions. On Qwen3-TTS, our framework improves Emotion from 18.5 to 81.2 and Paralinguistics from 38.4 to 73.9. Extensive ablations validate the role of each stage.
Explore related subjects
Keep this discovery
Explore connections, maps & timelines
Jingbin Hu, Xinfa Zhu, Hao Wang, Wenjie Tian, Kangxiang Xia, Hangrui Hu, Kexin Huang, Jin Xu, Liumeng Xue, Lei Xie. 2026-11-15. Rethinking Post-Training for Expressive Zero-Shot Text-to-Speech. https://searcharxiv.com/abs/2611.19787
Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite Searcharxiv by Indiciae: BibTeX · RIS.