arXiv subjects

Yechen Jin

Publications and source records attributed to Yechen Jin.

3 recordsLinked to original sources

arXiv2026

AST: Adaptive, Seamless, and Training-Free Precise Speech Editing

Text-based speech editing aims to modify specific segments while preserving speaker identity and acoustic context. Current approaches generally involve either expensive task-specific training or adapting pre-trained Text-to-Speech (TTS) models. However, both paradigms face challenges: task-specific methods often degrade fidelity in unedited regions, whereas TTS adaptations struggle with a trade-off between editing naturalness and temporal fidelity. To address these issues, we propose AST, an Adaptive, Seamless, and Training-free speech editing framework. Built upon pre-trained TTS, AST leverages Latent Recomposition to stitch preserved source segments with synthesized targets, guaranteeing fidelity in unedited regions. To break the quality-controllability trade-off, we introduce Adaptive Weak Fact Guidance (AWFG), which modulates a mel-space signal to ensure seamless boundary transitions without disrupting the generative manifold. Furthermore, to address evaluation gaps in temporal fidelity, we propose a new benchmark suite: the LibriSpeech-Edit dataset and a novel metric, Word-level Dynamic Time Warping (WDTW). Extensive experiments demonstrate that AST consistently outperforms existing task-specific and fine-tuned speech editing methods across content accuracy, perceptual quality, speaker preservation, and temporal fidelity. Remarkably, AST achieves state-of-the-art zero-shot speech editing performance without any task-specific training or paired editing data, validating the effectiveness of latent recomposition and AWFG in bridging the quality-controllability trade-off.

cs.SD↗

arXiv2025

JoyVoice: Long-Context Conditioning for Anthropomorphic Multi-Speaker Conversational Synthesis

Large speech generation models are evolving from single-speaker, short sentence synthesis to multi-speaker, long conversation geneartion. Current long-form speech generation models are predominately constrained to dyadic, turn-based interactions. To address this, we introduce JoyVoice, a novel anthropomorphic foundation model designed for flexible, boundary-free synthesis of up to eight speakers. Unlike conventional cascaded systems, JoyVoice employs a unified E2E-Transformer-DiT architecture that utilizes autoregressive hidden representations directly for diffusion inputs, enabling holistic end-to-end optimization. We further propose a MM-Tokenizer operating at a low bitrate of 12.5 Hz, which integrates multitask semantic and MMSE losses to effectively model both semantic and acoustic information. Additionally, the model incorporates robust text front-end processing via large-scale data perturbation. Experiments show that JoyVoice achieves state-of-the-art results in multilingual generation (Chinese, English, Japanese, Korean) and zero-shot voice cloning. JoyVoice achieves top-tier results on both the Seed-TTS-Eval Benchmark and multi-speaker long-form conversational voice cloning tasks, demonstrating superior audio quality and generalization. It achieves significant improvements in prosodic continuity for long-form speech, rhythm richness in multi-speaker conversations, paralinguistic naturalness, besides superior intelligibility. We encourage readers to listen to the demo at https://jea-speech.github.io/JoyVoice

cs.SD↗

arXiv2025

RIPRAG: Hack a Black-box Retrieval-Augmented Generation Question-Answering System with Reinforcement Learning

Retrieval-Augmented Generation (RAG) systems based on Large Language Models (LLMs) have become a core technology for tasks such as question-answering (QA) and content generation. RAG poisoning is an attack method to induce LLMs to generate the attacker's expected text by injecting poisoned documents into the database of RAG systems. Existing research can be broadly divided into two classes: white-box methods and black-box methods. White-box methods utilize gradient information to optimize poisoned documents, and black-box methods use a pre-trained LLM to generate them. However, existing white-box methods require knowledge of the RAG system's internal composition and implementation details, whereas black-box methods are unable to utilize interactive information. In this work, we propose the RIPRAG attack framework, an end-to-end attack pipeline that treats the target RAG system as a black box and leverages our proposed Reinforcement Learning from Black-box Feedback (RLBF) method to optimize the generation model for poisoned documents. We designed two kinds of rewards: similarity reward and attack reward. Experimental results demonstrate that this method can effectively execute poisoning attacks against most complex RAG systems, achieving an attack success rate (ASR) improvement of up to 0.72 compared to baseline methods. This highlights prevalent deficiencies in current defensive methods and provides critical insights for LLM security research.

cs.AI↗