arXiv subjects

Fanbo Meng

Publications and source records attributed to Fanbo Meng.

8 recordsLinked to original sources

arXiv2026

Hierarchical Acoustic-Semantic Modeling: Modality Separation and Semantic Coherence for Full-Duplex SLMs

Developing seamless, high-performance, native intelligent full-duplex Spoken Language Models (SLMs) remains a critical challenge and long-standing goal for the speech and NLP community. Despite notable progress, recent endeavors are fundamentally constrained by severe modality interference, which causes substantial knowledge degradation and compromises semantic integrity -- ultimately making full-duplex SLMs feel unnatural and unintelligent. In this paper, through an exhaustive fine-grained analysis of model optimization dynamics, we uncover the root cause of such performance degradation, revealing that modality interference arises from inherent gradient conflicts between acoustic and semantic modeling when the two modalities are forced to share a deep parameter space. Guided by this key insight, we introduce Lychee-FD, a native end-to-end full-duplex framework designed to mitigate modality interference. Importantly, we propose a hierarchical parameter separation strategy that decouples conflicting modalities in deep layers while preserving cross-modality coherence via a dedicated semantic alignment channel. Extensive experiments on multiple full-duplex benchmarks demonstrate that our method significantly advances the state of the art, yielding substantial improvements in both speech intelligence (+7.4% on Spoken QA) and full-duplex interaction fluidity (+28.5% on FullDuplexBench 1.5) without compromising inference efficiency. To the best of our knowledge, this work is the first to achieve two key advances: 1) uncovering and elucidating the root cause of modality interference in full-duplex SLMs, and 2) designing an elegant hierarchical model together with a practical solution for seamless, high-performance, native intelligent full-duplex SLMs.

cs.CL↗

arXiv2025

UniMoE-Audio: Unified Speech and Music Generation with Dynamic-Capacity MoE

Recent advances in unified multimodal models indicate a clear trend towards comprehensive content generation. However, the auditory domain remains a significant challenge, with music and speech often developed in isolation, hindering progress towards universal audio synthesis. This separation stems from inherent task conflicts and severe data imbalances, which impede the development of a truly unified audio generation model. To address this challenge, we propose UniMoE-Audio, a unified speech and music generation model within a novel Dynamic-Capacity Mixture-of-Experts (MoE) framework. Architecturally, UniMoE-Audio introduces a Top-P routing strategy for dynamic expert number allocation, and a hybrid expert design comprising routed experts for domain-specific knowledge, shared experts for domain-agnostic features, and null experts for adaptive computation skipping. To tackle data imbalance, we introduce a three-stage training curriculum: 1) Independent Specialist Training leverages original datasets to instill domain-specific knowledge into each "proto-expert" without interference; 2) MoE Integration and Warmup incorporates these specialists into the UniMoE-Audio architecture, warming up the gate module and shared expert using a subset of balanced dataset; and 3) Synergistic Joint Training trains the entire model end-to-end on the fully balanced dataset, fostering enhanced cross-domain synergy. Extensive experiments show that UniMoE-Audio not only achieves state-of-the-art performance on major speech and music generation benchmarks, but also demonstrates superior synergistic learning, mitigating the performance degradation typically seen in naive joint training. Our findings highlight the substantial potential of specialized MoE architecture and curated training strategies in advancing the field of universal audio generation. Homepage: https://mukioxun.github.io/Uni-MoE-site/home.html

cs.SD↗

arXiv2022

View-Invariant Skeleton-based Action Recognition via Global-Local Contrastive Learning

Skeleton-based human action recognition has been drawing more interest recently due to its low sensitivity to appearance changes and the accessibility of more skeleton data. However, even the 3D skeletons captured in practice are still sensitive to the viewpoint and direction gave the occlusion of different human-body joints and the errors in human joint localization. Such view variance of skeleton data may significantly affect the performance of action recognition. To address this issue, we propose in this paper a new view-invariant representation learning approach, without any manual action labeling, for skeleton-based human action recognition. Specifically, we leverage the multi-view skeleton data simultaneously taken for the same person in the network training, by maximizing the mutual information between the representations extracted from different views, and then propose a global-local contrastive loss to model the multi-scale co-occurrence relationships in both spatial and temporal domains. Extensive experimental results show that the proposed method is robust to the view difference of the input skeleton data and significantly boosts the performance of unsupervised skeleton-based human action methods, resulting in new state-of-the-art accuracies on two challenging multi-view benchmarks of PKUMMD and NTU RGB+D.

cs.CV↗

arXiv2022

Improving Adversarial Waveform Generation based Singing Voice Conversion with Harmonic Signals

Adversarial waveform generation has been a popular approach as the backend of singing voice conversion (SVC) to generate high-quality singing audio. However, the instability of GAN also leads to other problems, such as pitch jitters and U/V errors. It affects the smoothness and continuity of harmonics, hence degrades the conversion quality seriously. This paper proposes to feed harmonic signals to the SVC model in advance to enhance audio generation. We extract the sine excitation from the pitch, and filter it with a linear time-varying (LTV) filter estimated by a neural network. Both these two harmonic signals are adopted as the inputs to generate the singing waveform. In our experiments, two mainstream models, MelGAN and ParallelWaveGAN, are investigated to validate the effectiveness of the proposed approach. We conduct a MOS test on clean and noisy test sets. The result shows that both signals significantly improve SVC in fidelity and timbre similarity. Besides, the case analysis further validates that this method enhances the smoothness and continuity of harmonics in the generated audio, and the filtered excitation better matches the target audio.

cs.SD↗

arXiv2021

Design of a HOM-Damped 166.6 MHz Compact Quarter-Wave beta=1 Superconducting Cavity for High Energy Photon Source

Superconducting cavities with low RF frequencies and heavy damping of higher order modes (HOM) are desired for the main accelerator of High Energy Photon Source (HEPS), a 6 GeV synchrotron light source promising ultralow emittance currently under construction in Beijing. A compact 166.6 MHz superconducting cavity was proposed adopting a quarter-wave beta=1 geometry. Based on the successful development of a proof-of-principle cavity, a HOM-damped 166.6 MHz compact superconducting cavity was subsequently designed. A ferrite damper was installed on the beam pipe to reduce HOM impedance below the stringent threshold of coupled-bunch instabilities. Being compact, RF field heating on the cavity vacuum seal was carefully examined against quenching the NbTi flange. The cavity was later dressed with a helium vessel and the tuning mechanism was also realized. Excellent RF and mechanical properties were eventually achieved. Finally, the two-cavity string was designed to ensure smooth transitions among components and proper shielding of synchrotron light. This paper presents a complete design of a fully dressed HOM-damped low-frequency beta=1 superconducting cavity for HEPS.

physics.acc-ph↗

arXiv2020

ChoreoNet: Towards Music to Dance Synthesis with Choreographic Action Unit

Dance and music are two highly correlated artistic forms. Synthesizing dance motions has attracted much attention recently. Most previous works conduct music-to-dance synthesis via directly music to human skeleton keypoints mapping. Meanwhile, human choreographers design dance motions from music in a two-stage manner: they firstly devise multiple choreographic dance units (CAUs), each with a series of dance motions, and then arrange the CAU sequence according to the rhythm, melody and emotion of the music. Inspired by these, we systematically study such two-stage choreography approach and construct a dataset to incorporate such choreography knowledge. Based on the constructed dataset, we design a two-stage music-to-dance synthesis framework ChoreoNet to imitate human choreography procedure. Our framework firstly devises a CAU prediction model to learn the mapping relationship between music and CAU sequences. Afterwards, we devise a spatial-temporal inpainting model to convert the CAU sequence into continuous dance motions. Experimental results demonstrate that the proposed ChoreoNet outperforms baseline methods (0.622 in terms of CAU BLEU score and 1.59 in terms of user study score).

cs.CV↗

arXiv2013

The study and design of RF coupler for Chinese ADS HWR Superconducting Cavity

RF power coupler is a key component of the superconducting accelerating system in Chinese ADS proton linac injector I, which is used to transmit 15kW RF power from the power source to the superconducting HWR cavity. According to the requirement of working frequency, power level, transmission capability and cooling condition, the physics design of coupler has been finished, which includes RF structure optimization, thermal simulation, thermal stress analysis and so on. Based on this design, the prototype of HWR coupler has been fabricated, and then has passed the high power test successfully.

physics.acc-ph↗

arXiv2013

Coupler Conditioning and High Power Testing of ADS Spoke Cavity

Power couplers, used in China-ADS proton linac injector I, are required to transfer 6kW RF power to the superconducting Spoke cavities. At present, first two couplers of coaxial design have been fabricated and accomplished high power test at IHEP. The test results indicated that couplers of this design are qualified to deliver 10kW RF power in continuous travelling wave mode. This paper described the coupler's room temperature test procedures and results, discussed the original high power test terminated due to serious out-gassing and after some modifications. In the final test, the couplers smoothly exceeded the design power level.

physics.ins-det↗