arXiv subjects

Xian Shi

Publications and source records attributed to Xian Shi.

At least 55 records · Page 3Linked to original sources

arXiv2023

SlideSpeech: A Large-Scale Slide-Enriched Audio-Visual Corpus

Multi-Modal automatic speech recognition (ASR) techniques aim to leverage additional modalities to improve the performance of speech recognition systems. While existing approaches primarily focus on video or contextual information, the utilization of extra supplementary textual information has been overlooked. Recognizing the abundance of online conference videos with slides, which provide rich domain-specific information in the form of text and images, we release SlideSpeech, a large-scale audio-visual corpus enriched with slides. The corpus contains 1,705 videos, 1,000+ hours, with 473 hours of high-quality transcribed speech. Moreover, the corpus contains a significant amount of real-time synchronized slides. In this work, we present the pipeline for constructing the corpus and propose baseline methods for utilizing text information in the visual slide context. Through the application of keyword extraction and contextual ASR methods in the benchmark system, we demonstrate the potential of improving speech recognition performance by incorporating textual information from supplementary video slides.

cs.SD↗

arXiv2023

A coherence quantifier based on the quantum optimal transport cost

In this manuscript, we present a coherence measure based on the quantum optimal transport cost in terms of convex roof extended method. We also obtain the analytical solutions of the quantifier for pure states. At last, we propose an operational interpretation of the coherence measure for pure states.

quant-ph↗

arXiv2023

Evolution of Quantum Resources in Quantum-walk-based Search Algorithm

Quantum walk is fundamental to designing many quantum algorithms. Here we consider the effects of quantum coherence and quantum entanglement for the quantum walk search on the complete bipartite graph. First, we numerically show the complementary relationship between the success probability and the two quantum resources (quantum coherence and quantum entanglement). We also provide theoretical analysis in the asymptotic scenarios. At last, we discuss the role played by generalized depolarizing noises and find that it would influence the dynamics of success probability and quantum coherence sharply, which is demonstrated by theoretical derivation and numerical simulation.

quant-ph↗

arXiv2023

The entanglement criteria based on equiangular tight frames

Finite tight frames play an important role in miscellaneous areas, including quantum information theory. Here we apply a class of tight frames, equiangular tight frames, to address the problem of detecting the entanglement of bipartite states. Here we derive some entanglement criteria based on positive operator-valued measurements built from equiangular tight frames. We also present a class of entanglement witnesses based on the equiangular tight frames. At last, we generalize the entanglement criterion for bipartite systems to multipartite systems.

quant-ph↗

arXiv2023

Accurate and Reliable Confidence Estimation Based on Non-Autoregressive End-to-End Speech Recognition System

Estimating confidence scores for recognition results is a classic task in ASR field and of vital importance for kinds of downstream tasks and training strategies. Previous end-to-end~(E2E) based confidence estimation models (CEM) predict score sequences of equal length with input transcriptions, leading to unreliable estimation when deletion and insertion errors occur. In this paper we proposed CIF-Aligned confidence estimation model (CA-CEM) to achieve accurate and reliable confidence estimation based on novel non-autoregressive E2E ASR model - Paraformer. CA-CEM utilizes the modeling character of continuous integrate-and-fire (CIF) mechanism to generate token-synchronous acoustic embedding, which solves the estimation failure issue above. We measure the quality of estimation with AUC and RMSE in token level and ECE-U - a proposed metrics in utterance level. CA-CEM gains 24% and 19% relative reduction on ECE-U and also better AUC and RMSE on two test sets. Furthermore, we conduct analysis to explore the potential of CEM for different ASR related usage.

cs.SD↗

arXiv2023

BAT: Boundary aware transducer for memory-efficient and low-latency ASR

Recently, recurrent neural network transducer (RNN-T) gains increasing popularity due to its natural streaming capability as well as superior performance. Nevertheless, RNN-T training requires large time and computation resources as RNN-T loss calculation is slow and consumes a lot of memory. Another limitation of RNN-T is that it tends to access more contexts for better performance, thus leading to higher emission latency in streaming ASR. In this paper we propose boundary-aware transducer (BAT) for memory-efficient and low-latency ASR. In BAT, the lattice for RNN-T loss computation is reduced to a restricted region selected by the alignment from continuous integrate-and-fire (CIF), which is jointly optimized with the RNN-T model. Extensive experiments demonstrate that compared to RNN-T, BAT reduces time and memory consumption significantly in training, and achieves good CER-latency trade-offs in inference for streaming ASR.

eess.AS↗

arXiv2023

FunASR: A Fundamental End-to-End Speech Recognition Toolkit

This paper introduces FunASR, an open-source speech recognition toolkit designed to bridge the gap between academic research and industrial applications. FunASR offers models trained on large-scale industrial corpora and the ability to deploy them in applications. The toolkit's flagship model, Paraformer, is a non-autoregressive end-to-end speech recognition model that has been trained on a manually annotated Mandarin speech recognition dataset that contains 60,000 hours of speech. To improve the performance of Paraformer, we have added timestamp prediction and hotword customization capabilities to the standard Paraformer backbone. In addition, to facilitate model deployment, we have open-sourced a voice activity detection model based on the Feedforward Sequential Memory Network (FSMN-VAD) and a text post-processing punctuation model based on the controllable time-delay Transformer (CT-Transformer), both of which were trained on industrial corpora. These functional modules provide a solid foundation for building high-precision long audio speech recognition services. Compared to other models trained on open datasets, Paraformer demonstrates superior performance.

cs.SD↗

arXiv2023

Quantifying the entanglement of quantum states under the geometry method

Quantifying entanglement is an important issue in quantum information theory. Here we consider the entanglement measures through the trace norm in terms of two methods, the modified measure and the extended measure for bipartite states. We present the analytical formula for the pure states in terms of the modified measure and the mixed states of two-qubit systems for the extended measure. We also generalize the modified measure from bipartite states to tripartite states.

quant-ph↗

arXiv2023

Achieving Timestamp Prediction While Recognizing with Non-Autoregressive End-to-End ASR Model

Conventional ASR systems use frame-level phoneme posterior to conduct force-alignment~(FA) and provide timestamps, while end-to-end ASR systems especially AED based ones are short of such ability. This paper proposes to perform timestamp prediction~(TP) while recognizing by utilizing continuous integrate-and-fire~(CIF) mechanism in non-autoregressive ASR model - Paraformer. Foucing on the fire place bias issue of CIF, we conduct post-processing strategies including fire-delay and silence insertion. Besides, we propose to use scaled-CIF to smooth the weights of CIF output, which is proved beneficial for both ASR and TP task. Accumulated averaging shift~(AAS) and diarization error rate~(DER) are adopted to measure the quality of timestamps and we compare these metrics of proposed system and conventional hybrid force-alignment system. The experiment results over manually-marked timestamps testset show that the proposed optimization methods significantly improve the accuracy of CIF timestamps, reducing 66.7\% and 82.1\% of AAS and DER respectively. Comparing to Kaldi force-alignment trained with the same data, optimized CIF timestamps achieved 12.3\% relative AAS reduction.

cs.SD↗

arXiv2022

A family of separability criteria and lower bounds of concurrence

The problem on detecting the entanglement of a bipartite state is significant in quantum information theory. In this article, we apply the Ky Fan norm to the revised realignment matrix of a bipartite state. Specifially, we consider a family of separable criteria for bipartite states, and present when the density matrix corresponds to a state is real, the criteria is equivalent to the enhanced realignment criterion. Moreover, we present analytical lower bounds of concurrence and the convex-roof extended negativity for arbitrary dimensional systems.

quant-ph↗

arXiv2022

Linguistic-Acoustic Similarity Based Accent Shift for Accent Recognition

General accent recognition (AR) models tend to directly extract low-level information from spectrums, which always significantly overfit on speakers or channels. Considering accent can be regarded as a series of shifts relative to native pronunciation, distinguishing accents will be an easier task with accent shift as input. But due to the lack of native utterance as an anchor, estimating the accent shift is difficult. In this paper, we propose linguistic-acoustic similarity based accent shift (LASAS) for AR tasks. For an accent speech utterance, after mapping the corresponding text vector to multiple accent-associated spaces as anchors, its accent shift could be estimated by the similarities between the acoustic embedding and those anchors. Then, we concatenate the accent shift with a dimension-reduced text vector to obtain a linguistic-acoustic bimodal representation. Compared with pure acoustic embedding, the bimodal representation is richer and more clear by taking full advantage of both linguistic and acoustic information, which can effectively improve AR performance. Experiments on Accented English Speech Recognition Challenge (AESRC) dataset show that our method achieves 77.42% accuracy on Test set, obtaining a 6.94% relative improvement over a competitive system in the challenge.

cs.SD↗

arXiv2022

ET White Paper: To Find the First Earth 2.0

We propose to develop a wide-field and ultra-high-precision photometric survey mission, temporarily named "Earth 2.0 (ET)". This mission is designed to measure, for the first time, the occurrence rate and the orbital distributions of Earth-sized planets. ET consists of seven 30cm telescopes, to be launched to the Earth-Sun's L2 point. Six of these are transit telescopes with a field of view of 500 square degrees. Staring in the direction that encompasses the original Kepler field for four continuous years, this monitoring will return tens of thousands of transiting planets, including the elusive Earth twins orbiting solar-type stars. The seventh telescope is a 30cm microlensing telescope that will monitor an area of 4 square degrees toward the galactic bulge. This, combined with simultaneous ground-based KMTNet observations, will measure masses for hundreds of long-period and free-floating planets. Together, the transit and the microlensing telescopes will revolutionize our understandings of terrestrial planets across a large swath of orbital distances and free space. In addition, the survey data will also facilitate studies in the fields of asteroseismology, Galactic archeology, time-domain sciences, and black holes in binaries.

astro-ph.IM↗

arXiv2022

A Genuine Multipartite Entanglement Measure Generated by the Parametrized Entanglement Measure

In this paper, we investigate a genuine multipartite entanglement measure based on the geometric method. This measure arrives at the maximal value for the absolutely maximally entangled states and has desirable properties for quantifying the genuine multipartite entanglement. We present a lower bound of the genuine multipartite entanglement measure. At last, we present some examples to show that the genuine entanglement measure is with distinct entanglement ordering from other measures, and we also present the advantages of the measure proposed here with other measures.

quant-ph↗

arXiv2022

Entanglement polygon inequalities for pure states in qudit systems

Entanglement is one of the important resources in quantum tasks. Recently, Yang $et$ $al.$ [arXiv:2205.08801] proposed an entanglement polygon inequalities (EPI) in terms of some entanglement measures for $n$-qudit pure states. Here we continue to consider the entanglement polygon inequalities. Specifially, we show that the EPI is valid for $n$-qudit pure states in terms of geometric entanglement measure (GEM), then we study the residual entanglement in terms of GEM for pure states in three-qubit systems. At last, we present counterexamples showing that the EPI is invalid for higher dimensional systems in terms of negativity, we also present a class of states beyond qubits satisfy the EPI in terms of negativity.

quant-ph↗

arXiv2022

Open-Set Semi-Supervised Learning for 3D Point Cloud Understanding

Semantic understanding of 3D point cloud relies on learning models with massively annotated data, which, in many cases, are expensive or difficult to collect. This has led to an emerging research interest in semi-supervised learning (SSL) for 3D point cloud. It is commonly assumed in SSL that the unlabeled data are drawn from the same distribution as that of the labeled ones; This assumption, however, rarely holds true in realistic environments. Blindly using out-of-distribution (OOD) unlabeled data could harm SSL performance. In this work, we propose to selectively utilize unlabeled data through sample weighting, so that only conducive unlabeled data would be prioritized. To estimate the weights, we adopt a bi-level optimization framework which iteratively optimizes a metaobjective on a held-out validation set and a task-objective on a training set. Faced with the instability of efficient bi-level optimizers, we further propose three regularization techniques to enhance the training stability. Extensive experiments on 3D point cloud classification and segmentation tasks verify the effectiveness of our proposed method. We also demonstrate the feasibility of a more efficient training strategy.

cs.CV↗

arXiv2021

Darts-Conformer: Towards Efficient Gradient-Based Neural Architecture Search For End-to-End ASR

Neural architecture search (NAS) has been successfully applied to tasks like image classification and language modeling for finding efficient high-performance network architectures. In ASR field especially end-to-end ASR, the related research is still in its infancy. In this work, we focus on applying NAS on the most popular manually designed model: Conformer, and then propose an efficient ASR model searching method that benefits from the natural advantage of differentiable architecture search (Darts) in reducing computational overheads. We fuse Darts mutator and Conformer blocks to form a complete search space, within which a modified architecture called Darts-Conformer cell is found automatically. The entire searching process on AISHELL-1 dataset costs only 0.7 GPU days. Replacing the Conformer encoder by stacking searched cell, we get an end-to-end ASR model (named as Darts-Conformner) that outperforms the Conformer baseline by 4.7\% on the open-source AISHELL-1 dataset. Besides, we verify the transferability of the architecture searched on a small dataset to a larger 2k-hour dataset. To the best of our knowledge, this is the first successful attempt to apply gradient-based architecture search in the attention-based encoder-decoder ASR model.

cs.SD↗

arXiv2021

Multi-linear Monogamy Relations for Three Qubit States

The monogamy of entanglement means that entanglement cannot be freely shared. In 2014, Oliveira et al. [ Oliveira et al., Phys. Rev. A. 89, 034303 (2014)] proposed a monogamy relation in the linear version and considered it in terms of entanglement of formation. Here we generalize the above version and consider a multi-linear monogamy relation for a multi-qubit system in terms of entanglement of formation and concurrence. Based on the above results, we present an entanglement criterion for genuine entangled states, also we consider the absolutely maximally entangled states and present what an absolutely maximally entangled state is for a three-qubit system. At last, we apply our results to a three-qubit pure state in terms of quantum discord.

quant-ph↗

arXiv2021

Label-Efficient Point Cloud Semantic Segmentation: An Active Learning Approach

Deep learning models are the state-of-the-art methods for semantic point cloud segmentation, the success of which relies on the availability of large-scale annotated datasets. However, it can be extremely time-consuming and prohibitively expensive to compile such datasets. In this work, we propose an active learning approach to maximize model performance given limited annotation budgets. We investigate the appropriate sample granularity for active selection under realistic annotation cost measurement (clicks), and demonstrate that super-point based selection allows for more efficient usage of the limited budget compared to point-level and instance-level selection. We further exploit local consistency constraints to boost the performance of the super-point based approach. We evaluate our methods on two benchmarking datasets (ShapeNet and S3DIS) and the results demonstrate that active learning is an effective strategy to address the high annotation costs in semantic point cloud segmentation.

cs.CV↗