arXiv subjects

Xian Shi

Publications and source records attributed to Xian Shi.

At least 37 records · Page 2Linked to original sources

arXiv2025

CosyVoice 3: Towards In-the-wild Speech Generation via Scaling-up and Post-training

In our prior works, we introduced a scalable streaming speech synthesis model, CosyVoice 2, which integrates a large language model (LLM) and a chunk-aware flow matching (FM) model, and achieves low-latency bi-streaming speech synthesis and human-parity quality. Despite these advancements, CosyVoice 2 exhibits limitations in language coverage, domain diversity, data volume, text formats, and post-training techniques. In this paper, we present CosyVoice 3, an improved model designed for zero-shot multilingual speech synthesis in the wild, surpassing its predecessor in content consistency, speaker similarity, and prosody naturalness. Key features of CosyVoice 3 include: 1) A novel speech tokenizer to improve prosody naturalness, developed via supervised multi-task training, including automatic speech recognition, speech emotion recognition, language identification, audio event detection, and speaker analysis. 2) A new differentiable reward model for post-training applicable not only to CosyVoice 3 but also to other LLM-based speech synthesis models. 3) Dataset Size Scaling: Training data is expanded from ten thousand hours to one million hours, encompassing 9 languages and 18 Chinese dialects across various domains and text formats. 4) Model Size Scaling: Model parameters are increased from 0.5 billion to 1.5 billion, resulting in enhanced performance on our multilingual benchmark due to the larger model capacity. These advancements contribute significantly to the progress of speech synthesis in the wild. We encourage readers to listen to the demo at https://funaudiollm.github.io/cosyvoice3.

cs.SD↗

arXiv2025

ThermoONet -- a deep learning-based small body thermophysical network: applications to modelling water activity of comets

Cometary activity is a compelling subject of study, with thermophysical models playing a pivotal role in its understanding. However, traditional numerical solutions for small body thermophysical models are computationally intensive, posing challenges for investigations requiring high-resolution or repetitive modeling. To address this limitation, we employed a machine learning approach to develop ThermoONet - a neural network designed to predict the temperature and water ice sublimation flux of comets. Performance evaluations indicate that ThermoONet achieves a low average error in subsurface temperature of approximately 2% relative to the numerical simulation, while reducing computational time by nearly six orders of magnitude. We applied ThermoONet to model the water activity of comets 67P/Churyumov-Gerasimenko and 21P/Giacobini-Zinner. By successfully fitting the water production rate curves of these comets, as obtained by the Rosetta mission and the SOHO telescope, respectively, we demonstrate the network's effectiveness and efficiency. Furthermore, when combined with a global optimization algorithm, ThermoONet proves capable of retrieving the physical properties of target bodies.

astro-ph.EP↗

arXiv2025

The Deimginarity Cost of Quantum States

Here we address a task denoted as deimaginarity, which is to transform a state into a real state with the aid of random covariant-free unitary operations. We consider the minimum cost of randomness required for deimaginarity in the scenario of infinite copies and suiciently small error, and we prove that the deimaginarity cost of a state $ρ$ is equal to its regularized relative entropy of imaginarity, which can be seen as an operational interpretation of the relative entropy of imaginarity.

quant-ph↗

arXiv2025

MinMo: A Multimodal Large Language Model for Seamless Voice Interaction

Recent advancements in large language models (LLMs) and multimodal speech-text models have laid the groundwork for seamless voice interactions, enabling real-time, natural, and human-like conversations. Previous models for voice interactions are categorized as native and aligned. Native models integrate speech and text processing in one framework but struggle with issues like differing sequence lengths and insufficient pre-training. Aligned models maintain text LLM capabilities but are often limited by small datasets and a narrow focus on speech tasks. In this work, we introduce MinMo, a Multimodal Large Language Model with approximately 8B parameters for seamless voice interaction. We address the main limitations of prior aligned multimodal models. We train MinMo through multiple stages of speech-to-text alignment, text-to-speech alignment, speech-to-speech alignment, and duplex interaction alignment, on 1.4 million hours of diverse speech data and a broad range of speech tasks. After the multi-stage training, MinMo achieves state-of-the-art performance across various benchmarks for voice comprehension and generation while maintaining the capabilities of text LLMs, and also facilitates full-duplex conversation, that is, simultaneous two-way communication between the user and the system. Moreover, we propose a novel and simple voice decoder that outperforms prior models in voice generation. The enhanced instruction-following capabilities of MinMo supports controlling speech generation based on user instructions, with various nuances including emotions, dialects, and speaking rates, and mimicking specific voices. For MinMo, the speech-to-text latency is approximately 100ms, full-duplex latency is approximately 600ms in theory and 800ms in practice. The MinMo project web page is https://funaudiollm.github.io/minmo, and the code and models will be released soon.

cs.CL↗

arXiv2024

CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Models

In our previous work, we introduced CosyVoice, a multilingual speech synthesis model based on supervised discrete speech tokens. By employing progressive semantic decoding with two popular generative models, language models (LMs) and Flow Matching, CosyVoice demonstrated high prosody naturalness, content consistency, and speaker similarity in speech in-context learning. Recently, significant progress has been made in multi-modal large language models (LLMs), where the response latency and real-time factor of speech synthesis play a crucial role in the interactive experience. Therefore, in this report, we present an improved streaming speech synthesis model, CosyVoice 2, which incorporates comprehensive and systematic optimizations. Specifically, we introduce finite-scalar quantization to improve the codebook utilization of speech tokens. For the text-speech LM, we streamline the model architecture to allow direct use of a pre-trained LLM as the backbone. In addition, we develop a chunk-aware causal flow matching model to support various synthesis scenarios, enabling both streaming and non-streaming synthesis within a single model. By training on a large-scale multilingual dataset, CosyVoice 2 achieves human-parity naturalness, minimal response latency, and virtually lossless synthesis quality in the streaming mode. We invite readers to listen to the demos at https://funaudiollm.github.io/cosyvoice2.

cs.SD↗

arXiv2024

Boulder migration in the Khonsu region of comet 67P/Churyumov-Gerasimenko

European Space Agency's Rosetta mission is the only space mission that performed long-term monitoring of comet at close distances. Its over two years' rendezvous with comet 67P/Churyumov-Gerasimenko revealed diverse evolutionary processes of the cometary nucleus. One of the most striking events is the migration of a 30-m boulder in the southern hemisphere region of Khonsu. Previous works found the boulder's 140-m displacement occurred during the three months from August to October 2015, and several triggering mechanisms were proposed, including outburst at the boulder site, seismic vibrations from nearby activities, or surface erosion of the slope beneath the boulder. In this work, we further analyze this impressive event by analysing imaging data from Rosetta's OSIRIS camera. We constrained the boulder's migration time to within 14 hours and derived a detailed timeline of the boulder migration event and local dust activities. High-resolution thermophysical modelling shows significant dichotomy in the thermal history of the boulder's southern and northern sides, which could have triggered or facilitated its migration via its own volatile activity.

astro-ph.EP↗

arXiv2024

Deep operator neural network applied to efficient computation of asteroid surface temperature and the Yarkovsky effect

Surface temperature distribution is crucial for thermal property-based studies about irregular asteroids in our Solar System. While direct numerical simulations could model surface temperatures with high fidelity, they often take a significant amount of computational time, especially for problems where temperature distributions are required to be repeatedly calculated. To this end, deep operator neural network (DeepONet) provides a powerful tool due to its high computational efficiency and generalization ability. In this work, we applied DeepONet to the modelling of asteroid surface temperatures. Results show that the trained network is able to predict temperature with an accuracy of ~1% on average, while the computational cost is five orders of magnitude lower, hence enabling thermal property analysis in a multidimensional parameter space. As a preliminary application, we analyzed the orbital evolution of asteroids through direct N-body simulations embedded with instantaneous Yarkovsky effect inferred by DeepONet-based thermophysical modelling.Taking asteroids (3200) Phaethon and (89433) 2001 WM41 as examples, we show the efficacy and efficiency of our AI-based approach.

astro-ph.EP↗

arXiv2024

Three-dimensional Vorticity Effects on Extinction Behavior of Laminar Flamelets

A recent rotational flamelet model is developed and tested with an improved framework of detailed chemistry and transport. The rotational flamelet model incorporates the effects of shear strain and vorticity on local flame behavior and is three-dimensional by nature. A similarity solution reduces the three-dimensional governing equations to ODEs involving a transformation to a non-Newtonian reference frame. A 9-species chemical kinetics model is used for H2-O2 combustion with non-reacting N2. Multiple flamelet cases including non-premixed, premixed, and partially-premixed flames are performed. Across all cases, vorticity extends flammability limits by up to 30% in terms of the ambient extinction strain rate and modifies both local flame structure and mixture composition. For non-premixed flames, where the location of minimum density coincides with the location of peak temperature, the centrifugal force induced by vorticity reduces the mass flow rate through the flame, effectively lowering the local strain rate. This increases residence time, thus extending flammability limits and reducing burning rates. This analysis is done also for premixed and partially-premixed flames. If minimum density lies between the flame zone and the fuel inlet boundary, centrifugal forces do not significantly modify flame behavior. Stable and unstable branches of S-curves for non-premixed and partially-premixed flames and stable branches for premixed flames show extended flammability limits due to vorticity. The capabilities of the rotational flamelet model reveal that vital physics are currently missing from two-dimensional, irrotational, constant-density, flamelet models. Improvements of detailed chemical kinetics, transport formulation, and thermo-physical properties bring the new flamelet model to par in these areas with existing models, while adding new features in terms of physical emulation.

physics.flu-dyn↗

arXiv2024

Vortex Stretching of Non-premixed, Diluted Hydrogen/Oxygen Flamelets

A three-dimensional flamelet model considering vortex stretching with unitary Lewis number is used to simulate diluted hydrogen-oxygen diffusion flames. Non-reacting nitrogen is used as the diluent gas in the fuel stream. Unitary Lewis number provides a common thermal and mass diffusivity from which to create scalar dissipation rate. Both stable and unstable branches of flammability curves (S-curves) are calculated with three vorticity levels and plotted against multiple input and output parameters. The description of the three-dimensional flamelet structure, allowing vorticity and variable density to produce a centrifugal effect, is seen to be necessary for an accurate determination of the $\mathrm{H_2O}$ production rate when ambient inflow strain rate $(S^*)$ and vorticity $(ω)$ are chosen as the key parameters. Maximum temperature and integrated $\mathrm{H_2O}$ production rate each nearly collapse to a single curve when plotted versus maximum scalar dissipation rate $(χ_{max})$ but do not collapse when plotted versus the local maximum strain rate $(S^*_{local})$ or $S^*$. Additionally, $S^*_{local}$ and scalar dissipation rate $(χ)$ depend strongly on vorticity and ambient inflow strain rate. It is argued that the controlling inputs for a flamelet embedded in a turbulent eddy are the ambient vorticity and strain rate which are thus the natural choice of parameterizing variables. These ambient quantities can be readily linked to the averaged or filtered turbulent flow by leveraging cascade theory, as opposed to local strain rate or scalar dissipation rate within the flame zone, which do not have a widely accepted, first-principles scaling connection to the turbulence cascade.

physics.flu-dyn↗

arXiv2024

Bounds for Revised Unambiguous Discrimination Tasks of Quantum Resources

Quantum state discrimination is a fundamental task that is meaningful in quantum information theory. In this manuscript, we consider a revised unambiguous discrimination of quantum resources. First, we present an upper bound of the success probability for a revised unambiguous discrimination task in the unasymptotic and asymptotic scenarios. Next, we generalize the task from quantum states to quantum channels. We present an upper bound of the success probability for the task under the adaptive strategy. Furthermore, we show the bound can be computed efficiently. Finally, compared with the classical unambiguous discrimination, we show the advantage of the quantum by considering a quantifier on a set of semidefinite positive operators.

quant-ph↗

arXiv2024

FunAudioLLM: Voice Understanding and Generation Foundation Models for Natural Interaction Between Humans and LLMs

This report introduces FunAudioLLM, a model family designed to enhance natural voice interactions between humans and large language models (LLMs). At its core are two innovative models: SenseVoice, which handles multilingual speech recognition, emotion recognition, and audio event detection; and CosyVoice, which facilitates natural speech generation with control over multiple languages, timbre, speaking style, and speaker identity. SenseVoice-Small delivers exceptionally low-latency ASR for 5 languages, and SenseVoice-Large supports high-precision ASR for over 50 languages, while CosyVoice excels in multi-lingual voice generation, zero-shot in-context learning, cross-lingual voice cloning, and instruction-following capabilities. The models related to SenseVoice and CosyVoice have been open-sourced on Modelscope and Huggingface, along with the corresponding training, inference, and fine-tuning codes released on GitHub. By integrating these models with LLMs, FunAudioLLM enables applications such as speech-to-speech translation, emotional voice chat, interactive podcasts, and expressive audiobook narration, thereby pushing the boundaries of voice interaction technology. Demos are available at https://fun-audio-llm.github.io, and the code can be accessed at https://github.com/FunAudioLLM.

cs.SD↗

arXiv2024

Entanglement Polygon Inequalities for A Class of Mixed States

The study on the entanglement polygon inequality of multipartite systems has attracted much attention. However, most of the results are on pure states. Here we consider the property for a class of mixed states, which are the reduced density matrices of generalized W-class states in multipartite higher dimensional systems. First we show the class of mixed states satisfies the entanglement polygon inequalities in terms of Tsallis-q entanglement, then we propose a class of tighter inequalities for mixed states in terms of Tsallis-q entanglement. At last, we get an inequality for the mixed states, which can be regarded as a relation for bipartite entanglement.

quant-ph↗

arXiv2024

Tianyu: search for the second solar system and explore the dynamic universe

Giant planets like Jupiter and Saturn, play important roles in the formation and habitability of Earth-like planets. The detection of solar system analogs that have multiple cold giant planets is essential for our understanding of planet habitability and planet formation. Although transit surveys such as Kepler and TESS have discovered thousands of exoplanets, these missions are not sensitive to long period planets due to their limited observation baseline. The Tianyu project, comprising two 1-meter telescopes (Tianyu-I and II), is designed to detect transiting cold giant planets in order to find solar system analogs. Featuring a large field of view and equipped with a high-speed CMOS camera, Tianyu-I will perform a high-precision photometric survey of about 100 million stars, measuring light curves at hour-long cadence. The candidates found by Tianyu-I will be confirmed by Tianyu-II and other surveys and follow-up facilities through multi-band photometry, spectroscopy, and high resolution imaging. Tianyu telescopes will be situated at an elevation about 4000 meters in Lenghu, China. With a photometric precision of 1% for stars with V < 18 mag, Tianyu is expected to find more than 300 transiting exoplanets, including about 12 cold giant planets, over five years. A five-year survey of Tianyu would discover 1-2 solar system analogs. Moreover, Tianyu is also designed for non-exoplanetary exploration, incorporating multiple survey modes covering timescales from sub-seconds to months, with a particular emphasis on events occurring within the sub-second to hour range. It excels in observing areas such as infant supernovae, rare variable stars and binaries, tidal disruption events, Be stars, cometary activities, and interstellar objects. These discoveries not only enhance our comprehension of the universe but also offer compelling opportunities for public engagement in scientific exploration.

astro-ph.IM↗

arXiv2024

Families of Schmidt-number witnesses for high dimensional quantum states

Higher dimensional entangled states demonstrate significant advantages in quantum information processing tasks. Schmidt number is a quantity on the entanglement dimension of a bipartite state. Here we build families of k-positive maps from the symmetric information complete positive operator-valued measurements and mutually unbiased bases, and we also present the Schmidt number witnesses, correspondingly. At last, based on the witnesses obtained from mutually unbiased bases, we show the distance between a bipartite state and the set of states with Schmidt number less than k.

quant-ph↗

arXiv2024

Diurnal ejection of boulder clusters on comet 67P lasting beyond 3 AU

Ejection of large boulder-like debris is a vigorous form of cometary activity that is unlikely induced by water ice out-gassing alone but rather associated with the sublimation of super-volatile ices. Though perceived on several comets, actual pattern and mechanism of such activity are still unclear. Here we report on a specialized observation of ejections of decimeter- to meter-sized boulders on comet 67P/Churyumov-Gerasimenko outbound between 2.5 and 3.3 AU from the Sun. With a common source region, these events recurred in local morning. The boulders of elongated shapes were ejected in clusters at low inclinations comparable to the solar elevation below 40 degrees at the time. We show that these chunks could be propelled by the surrounding, asymmetric gas field that produced a distinct lateral acceleration. Possibly both water and carbon dioxide have contributed to their mobilization, while the season and local topography are among deciding factors. The mechanisms for sustaining regular activity of comets at large heliocentric distances are likely more diverse and intricate than previously thought.

astro-ph.EP↗

arXiv2024

LCB-net: Long-Context Biasing for Audio-Visual Speech Recognition

The growing prevalence of online conferences and courses presents a new challenge in improving automatic speech recognition (ASR) with enriched textual information from video slides. In contrast to rare phrase lists, the slides within videos are synchronized in real-time with the speech, enabling the extraction of long contextual bias. Therefore, we propose a novel long-context biasing network (LCB-net) for audio-visual speech recognition (AVSR) to leverage the long-context information available in videos effectively. Specifically, we adopt a bi-encoder architecture to simultaneously model audio and long-context biasing. Besides, we also propose a biasing prediction module that utilizes binary cross entropy (BCE) loss to explicitly determine biased phrases in the long-context biasing. Furthermore, we introduce a dynamic contextual phrases simulation to enhance the generalization and robustness of our LCB-net. Experiments on the SlideSpeech, a large-scale audio-visual corpus enriched with slides, reveal that our proposed LCB-net outperforms general ASR model by 9.4%/9.1%/10.9% relative WER/U-WER/B-WER reduction on test set, which enjoys high unbiased and biased performance. Moreover, we also evaluate our model on LibriSpeech corpus, leading to 23.8%/19.2%/35.4% relative WER/U-WER/B-WER reduction over the ASR model.

cs.SD↗

arXiv2023

Lower Bounds of Entanglement Quantifiers Based On Entanglement Witnesses

To quantify the entanglement of bipartite systems in terms of some entanglement measure is a challenging problem in general, and it is much worse when the information about the system is less. In this manuscript, based on two classes of entanglement criteria, we present a method to obtain the lower bounds of the entanglement measures, concurrence, entanglement of formation, and geometrical entanglement measure.

quant-ph↗

arXiv2023

SeACo-Paraformer: A Non-Autoregressive ASR System with Flexible and Effective Hotword Customization Ability

Hotword customization is one of the concerned issues remained in ASR field - it is of value to enable users of ASR systems to customize names of entities, persons and other phrases to obtain better experience. The past few years have seen effective modeling strategies for ASR contextualization developed, but they still exhibit space for improvement about training stability and the invisible activation process. In this paper we propose Semantic-Augmented Contextual-Paraformer (SeACo-Paraformer) a novel NAR based ASR system with flexible and effective hotword customization ability. It possesses the advantages of AED-based model's accuracy, NAR model's efficiency, and explicit customization capacity of superior performance. Through extensive experiments with 50,000 hours of industrial big data, our proposed model outperforms strong baselines in customization. Besides, we explore an efficient way to filter large-scale incoming hotwords for further improvement. The industrial models compared, source codes and two hotword test sets are all open source.

cs.SD↗