Searcharxiv⌕ Search
Home/Archive/arXiv

arXiv · 2610.28035

LoCA: Local Credit Assignment for Reinforcement Learning of Autoregressive Flow-Matching TTS

Jie Liu·Junyu Zhou·Junhao Zhong·WeiSen Tang·Rukai Cai·Haoyue Zhan·Ruiqi Xing·Ling Long
View original source ↗Read source document ↗Export metadata ↓

Abstract

Reinforcement learning can improve text-to-speech synthesis through feedback on generated speech. However, sentence-level rewards do not identify local errors, while signed regression updates can lead to excessive fitting or repulsion of sampled outputs. We propose LoCA (Local Credit Assignment) for autoregressive flow-matching text-to-speech. LoCA uses ASR-based text alignment to locate content errors in generated speech and redistribute sentence-level advantages across acoustic frames. To improve training stability, we introduce reference-relative positive and negative thresholds that limit continued fitting and repulsion. Experiments on VoxCPM2 show that LoCA reduces sample-weighted content error on a multilingual CV3 evaluation subset by 20.5% and 14.3% relative to the pretrained backbone and GROW, respectively, while improving speaker similarity. LoCA also improves content accuracy on the Seed-TTS-Eval hard subset. Ablation results support the contributions of local credit and threshold-based regression control.

Explore related subjects

cs.SD

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Jie Liu, Junyu Zhou, Junhao Zhong, WeiSen Tang, Rukai Cai, Haoyue Zhan, Ruiqi Xing, Ling Long. 2026-10-20. LoCA: Local Credit Assignment for Reinforcement Learning of Autoregressive Flow-Matching TTS. https://arxiv.org/abs/2610.28035

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Deep learning-based multi-level airway collapse site localization on obstructive sleep apnea patients based on snoring sounds

Objective This study evaluated deep learning models, specifically ResNet-50 and Audio Spectrogram Transformer (AST), to identify multi-level upper airway collapses in obstructive sleep apnea (OSA) patients using snoring sounds. Methods Snoring recordings from 37 subjects undergoing drug-induced sleep endoscopy (DISE) between 2020 and 2021 were used to train models for two multi-label tasks: identifying obstructions at the Velum (V), Oropharynx (O), Tongue-base (T), and Epiglottis (E) levels, and the retropalatal (RP) vs. retroglossal (RG) levels. Results The AST model significantly outperformed ResNet-50 (P <0.05 after Bonferroni correction), effectively identifying V-, O-, and RP-level obstructions (F1-score: 0.59-0.87, Matthews Correlation Coefficient: 0.50-0.80, and AUC: 0.87-0.97). However, the performance of both models at the T, E, and RG levels remained limited due to small sample sizes, even with the Synthetic Minority Over-sampling Technique. A full-night recording analysis further demonstrated the feasibility of acoustic profiling for airway dynamics. Conclusion The AST models can effectively identify airway obstruction at V, O, and RP levels via snoring sounds. The proposed models offer a promising non-invasive tool for tracking overnight airway dynamics and assisting in clinical treatment planning.

cs.SD↗
arXiv2026

Speaker-Disentangled Remote Speech Detection of Asthma and COPD Exacerbations

Early detection of exacerbations in asthma and chronic obstructive pulmonary disease (COPD) is important for timely intervention. Speech has emerged as a promising tool for continuous, non-invasive respiratory disease monitoring. However, speech signals inherently carry speaker-identifiable attributes that may dominate model predictions, which may compromise both diagnostic accuracy and patient privacy. We propose an adversarial learning architecture that disentangles pathology-related acoustic patterns from speaker-identifiable attributes through gradient reversal-based training. To enhance clinical interpretability, we employ SHapley Additive exPlanations (SHAP) to quantify the contributions of acoustic features to pathology-related predictions versus speaker identity. On the TACTICAS dataset, the adversarial model significantly reduces speaker separability, while improving or maintaining classification performance. Specifically, it significantly improves the AUC for exacerbation type classification (from 0.65 to 0.73), enhances imbalance-sensitive metrics for respiratory status screening (PR-AUC and macro F1), and maintains cross-speaker generalization under the subject-wise split. Concurrently, the J-ratio decreases, confirming effective suppression of speaker information. This study demonstrates that adversarial disentanglement of speaker identity not only enhances patient privacy but also maintains or improves diagnostic accuracy for respiratory disease monitoring. Our framework offers a foundation for deployable, interpretable, and privacy-aware speech-based methods in chronic respiratory disease management.

cs.SD↗
arXiv2026

TTS-Q: Speaker-Aware Post-Training Quantization of Text-to-Speech Diffusion Transformers

Zero-shot text-to-speech (TTS) can clone a voice from seconds of audio. Yet the quality comes from billion-parameter diffusion transformers whose size and multi-step sampling put them out of reach of the on-device, assistive, and real-time applications that need them most. Post-training quantization (PTQ) shrinks such models without retraining, but it has been built for language and vision, not speech. At 4 bits, a strong quantizer leaves intelligibility close to full precision but consistently erodes speaker timbre, so error rates stay low while the voice drifts from the target speaker. We ask why quantization hurts speech, where the damage lives, and whether it can be repaired without retraining. To find out, we run the first 4-bit weight-and-activation (W4A4) PTQ study of a TTS diffusion transformer at 1B and 3.5B. The timbre cost comes from both the weight and the activation quantization, but only the activation part has the timestep structure a precision schedule can exploit. We therefore introduce TTS-Q, which selects hard-to-quantize items from task-matched calibration candidates and applies a budget-balanced mixed-precision schedule that assigns higher precision to late-step feed-forward activations. TTS-Q recovers timbre best at both scales, at no cost to intelligibility. Separately, the deployed calibrated-W4A4 reference reduces peak memory on GPU, providing a practical reference for resource-constrained TTS deployment.

cs.SD↗