arXiv · 2611.19145
A Systematic Study of Cross-Token Transfer for Data-Efficient Non-Verbal Text-to-Speech
Abstract
Non-verbal vocalizations (NVs), such as laughter and breath, carry affective cues that words alone do not convey. Yet NV data remains scarce because speech annotation pipelines often omit or normalize these events. Existing work has improved NV text-to-speech through model design and data curation, but whether one NV token can help a model learn another has received little systematic study. We investigate this cross-token transferability in a controlled CosyVoice 2 setup. We first trace a laughter-only scaling curve, then hold laughter at 200 or 1,600 examples while adding breath, sigh, cough, lip-smack, or neutral speech. With only 200 laughter examples, adding 3,200 cough examples raises end-to-end laughter F1 from 0.170 to 0.529, surpassing the 0.495 obtained from 1,600 laughter examples alone. At the 1,600-example anchor, cough reaches 0.695, nearly matching 6,400 laughter examples (0.697). Waveform-level Whisper-D scoring and a semantic-token transcriber show consistent trends. Cross-token transfer therefore offers a complementary route to low-resource NV generation.
Explore related subjects
Keep this discovery
Explore connections, maps & timelines
Jing Pan, Haibin Wu, Minxue Niu, Florian Metze, Yuzong Liu. 2026-11-14. A Systematic Study of Cross-Token Transfer for Data-Efficient Non-Verbal Text-to-Speech. https://searcharxiv.com/abs/2611.19145
Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite Searcharxiv by Indiciae: BibTeX · RIS.