arXiv subjects

Binbin Zhang

Publications and source records attributed to Binbin Zhang.

At least 55 records · Page 3Linked to original sources

arXiv2022

Wespeaker: A Research and Production oriented Speaker Embedding Learning Toolkit

Speaker modeling is essential for many related tasks, such as speaker recognition and speaker diarization. The dominant modeling approach is fixed-dimensional vector representation, i.e., speaker embedding. This paper introduces a research and production oriented speaker embedding learning toolkit, Wespeaker. Wespeaker contains the implementation of scalable data management, state-of-the-art speaker embedding models, loss functions, and scoring back-ends, with highly competitive results achieved by structured recipes which were adopted in the winning systems in several speaker verification challenges. The application to other downstream tasks such as speaker diarization is also exhibited in the related recipe. Moreover, CPU- and GPU-compatible deployment codes are integrated for production-oriented development. The toolkit is publicly available at https://github.com/wenet-e2e/wespeaker.

cs.SD↗

arXiv2022

FusionFormer: Fusing Operations in Transformer for Efficient Streaming Speech Recognition

The recently proposed Conformer architecture which combines convolution with attention to capture both local and global dependencies has become the \textit{de facto} backbone model for Automatic Speech Recognition~(ASR). Inherited from the Natural Language Processing (NLP) tasks, the architecture takes Layer Normalization~(LN) as a default normalization technique. However, through a series of systematic studies, we find that LN might take 10\% of the inference time despite that it only contributes to 0.1\% of the FLOPs. This motivates us to replace LN with other normalization techniques, e.g., Batch Normalization~(BN), to speed up inference with the help of operator fusion methods and the avoidance of calculating the mean and variance statistics during inference. After examining several plain attempts which directly remove all LN layers or replace them with BN in the same place, we find that the divergence issue is mainly caused by the unstable layer output. We therefore propose to append a BN layer to each linear or convolution layer where stabilized training results are observed. We also propose to simplify the activations in Conformer, such as Swish and GLU, by replacing them with ReLU. All these exchanged modules can be fused into the weights of the adjacent linear/convolution layers and hence have zero inference cost. Therefore, we name it FusionFormer. Our experiments indicate that FusionFormer is as effective as the LN-based Conformer and is about 10\% faster.

cs.SD↗

arXiv2022

WeKws: A production first small-footprint end-to-end Keyword Spotting Toolkit

Keyword spotting (KWS) enables speech-based user interaction and gradually becomes an indispensable component of smart devices. Recently, end-to-end (E2E) methods have become the most popular approach for on-device KWS tasks. However, there is still a gap between the research and deployment of E2E KWS methods. In this paper, we introduce WeKws, a production-quality, easy-to-build, and convenient-to-be-applied E2E KWS toolkit. WeKws contains the implementations of several state-of-the-art backbone networks, making it achieve highly competitive results on three publicly available datasets. To make WeKws a pure E2E toolkit, we utilize a refined max-pooling loss to make the model learn the ending position of the keyword by itself, which significantly simplifies the training pipeline and makes WeKws very efficient to be applied in real-world scenarios. The toolkit is publicly available at https://github.com/wenet-e2e/wekws.

eess.AS↗

arXiv2022

An Insight-HXMT view of the mHz quasi-regular modulation phenomenon in the black hole X-ray binary 4U 1630-47

Here we report the spectral-timing results of the black hole X-ray binary 4U 1630-47 during its 2021 outburst using observations from the Hard X-ray Modulation Telescope. Type-C quasi-periodic oscillations (QPOs) in 1.6--4.2 Hz and quasi-regular modulation (QRM) near 60 mHz are detected during the outburst. The mHz QRM has a fractional rms of 10%--16% in the 8--35 keV energy band with a Q factor (frequency/width) of 2--4. Benefiting from the broad energy band of hxmt, we study the energy dependence of the 60 mHz QRM in 1--100 keV for the first time. We find that the fractional rms of the mHz QRM increases with photon energy, while the time lags of the mHz QRM are soft and decrease with photon energy. Fast recurrence of the mHz QRM, in a timescale of less than one hour, has been observed during the outburst. During this period, the corresponding energy spectra moderately change when the source transitions from the QRM state to the non-QRM state. The QRM phenomena also shows a dependence with the accretion rate. We suggest that the QRM could be caused by an unknown accretion instability aroused from the corona.

astro-ph.HE↗

arXiv2022

WeNet 2.0: More Productive End-to-End Speech Recognition Toolkit

Recently, we made available WeNet, a production-oriented end-to-end speech recognition toolkit, which introduces a unified two-pass (U2) framework and a built-in runtime to address the streaming and non-streaming decoding modes in a single model. To further improve ASR performance and facilitate various production requirements, in this paper, we present WeNet 2.0 with four important updates. (1) We propose U2++, a unified two-pass framework with bidirectional attention decoders, which includes the future contextual information by a right-to-left attention decoder to improve the representative ability of the shared encoder and the performance during the rescoring stage. (2) We introduce an n-gram based language model and a WFST-based decoder into WeNet 2.0, promoting the use of rich text data in production scenarios. (3) We design a unified contextual biasing framework, which leverages user-specific context (e.g., contact lists) to provide rapid adaptation ability for production and improves ASR accuracy in both with-LM and without-LM scenarios. (4) We design a unified IO to support large-scale data for effective model training. In summary, the brand-new WeNet 2.0 achieves up to 10\% relative recognition performance improvement over the original WeNet on various corpora and makes available several important production-oriented features.

cs.SD↗

arXiv2022

Quasi-periodic oscillations of the X-ray burst from the magnetar SGR J1935+2154 and associated with the fast radio burst FRB 200428

The origin(s) and mechanism(s) of fast radio bursts (FRBs), which are short radio pulses from cosmological distances, have remained a major puzzle since their discovery. We report a strong Quasi-Periodic Oscillation(QPO) of 40 Hz in the X-ray burst from the magnetar SGR J1935+2154 and associated with FRB 200428, significantly detected with the Hard X-ray Modulation Telescope (Insight-HXMT) and also hinted by the Konus-Wind data. QPOs from magnetar bursts have only been rarely detected; our 3.4 sigma (p-value is 2.9e-4) detection of the QPO reported here reveals the strongest QPO signal observed from magnetars (except in some very rare giant flares), making this X-ray burst unique among magnetar bursts. The two X-ray spikes coinciding with the two FRB pulses are also among the peaks of the QPO. Our results suggest that at least some FRBs are related to strong oscillation processes of neutron stars. We also show that we may overestimate the significance of the QPO signal and underestimate the errors of QPO parameters if QPO exists only in a fraction of the time series of a X-ray burst which we use to calculate the Leahy-normalized periodogram.

astro-ph.HE↗

arXiv2022

WenetSpeech: A 10000+ Hours Multi-domain Mandarin Corpus for Speech Recognition

In this paper, we present WenetSpeech, a multi-domain Mandarin corpus consisting of 10000+ hours high-quality labeled speech, 2400+ hours weakly labeled speech, and about 10000 hours unlabeled speech, with 22400+ hours in total. We collect the data from YouTube and Podcast, which covers a variety of speaking styles, scenarios, domains, topics, and noisy conditions. An optical character recognition (OCR) based method is introduced to generate the audio/text segmentation candidates for the YouTube data on its corresponding video captions, while a high-quality ASR transcription system is used to generate audio/text pair candidates for the Podcast data. Then we propose a novel end-to-end label error detection approach to further validate and filter the candidates. We also provide three manually labelled high-quality test sets along with WenetSpeech for evaluation -- Dev for cross-validation purpose in training, Test_Net, collected from Internet for matched test, and Test\_Meeting, recorded from real meetings for more challenging mismatched test. Baseline systems trained with WenetSpeech are provided for three popular speech recognition toolkits, namely Kaldi, ESPnet, and WeNet, and recognition results on the three test sets are also provided as benchmarks. To the best of our knowledge, WenetSpeech is the current largest open-sourced Mandarin speech corpus with transcriptions, which benefits research on production-level speech recognition.

cs.SD↗

arXiv2021

U2++: Unified Two-pass Bidirectional End-to-end Model for Speech Recognition

The unified streaming and non-streaming two-pass (U2) end-to-end model for speech recognition has shown great performance in terms of streaming capability, accuracy, real-time factor (RTF), and latency. In this paper, we present U2++, an enhanced version of U2 to further improve the accuracy. The core idea of U2++ is to use the forward and the backward information of the labeling sequences at the same time at training to learn richer information, and combine the forward and backward prediction at decoding to give more accurate recognition results. We also proposed a new data augmentation method called SpecSub to help the U2++ model to be more accurate and robust. Our experiments show that, compared with U2, U2++ shows faster convergence at training, better robustness to the decoding method, as well as consistent 5\% - 8\% word error rate reduction gain over U2. On the experiment of AISHELL-1, we achieve a 4.63\% character error rate (CER) with a non-streaming setup and 5.05\% with a streaming setup with 320ms latency by U2++. To the best of our knowledge, 5.05\% is the best-published streaming result on the AISHELL-1 test set.

cs.SD↗

arXiv2021

Unified Streaming and Non-streaming Two-pass End-to-end Model for Speech Recognition

In this paper, we present a novel two-pass approach to unify streaming and non-streaming end-to-end (E2E) speech recognition in a single model. Our model adopts the hybrid CTC/attention architecture, in which the conformer layers in the encoder are modified. We propose a dynamic chunk-based attention strategy to allow arbitrary right context length. At inference time, the CTC decoder generates n-best hypotheses in a streaming way. The inference latency could be easily controlled by only changing the chunk size. The CTC hypotheses are then rescored by the attention decoder to get the final result. This efficient rescoring process causes very little sentence-level latency. Our experiments on the open 170-hour AISHELL-1 dataset show that, the proposed method can unify the streaming and non-streaming model simply and efficiently. On the AISHELL-1 test set, our unified model achieves 5.60% relative character error rate (CER) reduction in non-streaming ASR compared to a standard non-streaming transformer. The same model achieves 5.42% CER with 640ms latency in a streaming ASR system.

cs.SD↗

arXiv2021

WeNet: Production oriented Streaming and Non-streaming End-to-End Speech Recognition Toolkit

In this paper, we propose an open source, production first, and production ready speech recognition toolkit called WeNet in which a new two-pass approach is implemented to unify streaming and non-streaming end-to-end (E2E) speech recognition in a single model. The main motivation of WeNet is to close the gap between the research and the production of E2E speechrecognition models. WeNet provides an efficient way to ship ASR applications in several real-world scenarios, which is the main difference and advantage to other open source E2E speech recognition toolkits. In our toolkit, a new two-pass method is implemented. Our method propose a dynamic chunk-based attention strategy of the the transformer layers to allow arbitrary right context length modifies in hybrid CTC/attention architecture. The inference latency could be easily controlled by only changing the chunk size. The CTC hypotheses are then rescored by the attention decoder to get the final result. Our experiments on the AISHELL-1 dataset using WeNet show that, our model achieves 5.03\% relative character error rate (CER) reduction in non-streaming ASR compared to a standard non-streaming transformer. After model quantification, our model perform reasonable RTF and latency.

cs.SD↗

arXiv2021

On-ground calibrations of the GRID-02 gamma-ray detector

The Gamma-Ray Integrated Detectors (GRID) are a space project to monitor the transient gamma-ray sky in the multi-messenger astronomy era using multiple detectors on-board CubeSats. The second GRID detector, GRID-02, was launched in 2020. The performance of the detector, including the energy response, effective area, angular response, and temperature-bias dependence, is calibrated in the laboratory and presented here. These measurements are compared with particle tracing simulations and validate the Geant4 model that will be used for generating detector responses.

astro-ph.IM↗

arXiv2021

Interpretable Compositional Convolutional Neural Networks

The reasonable definition of semantic interpretability presents the core challenge in explainable AI. This paper proposes a method to modify a traditional convolutional neural network (CNN) into an interpretable compositional CNN, in order to learn filters that encode meaningful visual patterns in intermediate convolutional layers. In a compositional CNN, each filter is supposed to consistently represent a specific compositional object part or image region with a clear meaning. The compositional CNN learns from image labels for classification without any annotations of parts or regions for supervision. Our method can be broadly applied to different types of CNNs. Experiments have demonstrated the effectiveness of our method.

cs.CV↗

arXiv2021

Verifiability and Predictability: Interpreting Utilities of Network Architectures for Point Cloud Processing

In this paper, we diagnose deep neural networks for 3D point cloud processing to explore utilities of different intermediate-layer network architectures. We propose a number of hypotheses on the effects of specific intermediate-layer network architectures on the representation capacity of DNNs. In order to prove the hypotheses, we design five metrics to diagnose various types of DNNs from the following perspectives, information discarding, information concentration, rotation robustness, adversarial robustness, and neighborhood inconsistency. We conduct comparative studies based on such metrics to verify the hypotheses. We further use the verified hypotheses to revise intermediate-layer architectures of existing DNNs and improve their utilities. Experiments demonstrate the effectiveness of our method.

cs.CV↗

arXiv2020

3D-Rotation-Equivariant Quaternion Neural Networks

This paper proposes a set of rules to revise various neural networks for 3D point cloud processing to rotation-equivariant quaternion neural networks (REQNNs). We find that when a neural network uses quaternion features under certain conditions, the network feature naturally has the rotation-equivariance property. Rotation equivariance means that applying a specific rotation transformation to the input point cloud is equivalent to applying the same rotation transformation to all intermediate-layer quaternion features. Besides, the REQNN also ensures that the intermediate-layer features are invariant to the permutation of input points. Compared with the original neural network, the REQNN exhibits higher rotation robustness.

cs.CV↗

arXiv2020

International Coordination of Multi-Messenger Transient Observations in the 2020s and Beyond: Kavli-IAU White Paper

This White Paper summarizes the discussions from a five-day workshop, involving 50 people from 18 countries, held in Cape Town, South Africa in February 2020. Convened by the International Astronomical Union's Executive Committee Working Group on Global Coordination of Ground and Space Astrophysics and sponsored by the Kavli Foundation, we discussed existing and potential bottlenecks for transient and multi-messenger astronomy, identifying eight broad areas of concern. Some of these are very similar to the challenges faced by many astronomers engaging in international collaboration, for example, data access policies, funding, theoretical and computational resources and workforce equity. Others, including, alerts, telescope coordination and target-of-opportunity implementation, are strongly linked to the time domain and are particularly challenging as we respond to transients. To address these bottlenecks we offer thirty-five specific recommendations, some of which are simply starting points and require development. These recommendations are not only aimed at collaborative groups and individuals, but also at the various organizations who are essential to making transient collaborations efficient and effective: including the International Astronomical Union, observatories, projects, scientific journals and funding agencies. We hope those involved in transient research will find them constructive and use them to develop collaborations with greater impact and more inclusive teams.

astro-ph.IM↗

arXiv2019

A unified binary neutron star merger magnetar model for the Chandra X-ray transients CDF-S XT1 and XT2

Two bright X-ray transients were reported from the Chandra Deep Field South archival data, namely CDF-S XT1 and XT2. Whereas the nature of the former is not identified, the latter was suggested as an excellent candidate for a rapidly spinning magnetar born from a binary neutron star (BNS) merger. Here we propose a unified model to interpret both transients within the framework of the BNS merger magnetar model. According to our picture, CDF-S XT2 is observed from the "free zone" where the magnetar spindown powered X-ray emission escapes freely, whereas CDF-S XT1 originates from the "trapped zone" where the X-ray emission is initially blocked by the dynamical ejecta and becomes transparent after the ejecta is pushed to a distance where Thomson optical depth drops below unity. We fit the magnetar model to the light curves of both transients and derived consistent parameters for the two events, with magnetic field, initial spin period and X-ray emission efficiency being ($B_p=10^{16}\,G$, $P=1.2\,\rm ms$, $η= 0.001$) and ($B_p=10^{15.8}\,G$, $P=4.4\, \rm ms$, $η= 0.001$) for XT1 and XT2, respectively. The "isotropic equivalent" ejecta mass of XT1 is $M_{\rm ej} \sim 10^{-3}$ $M_{\odot}$, while it is not constrained for XT2. Our results suggest that more extreme magnetar parameters are required to have XT1 detected from the trapped zone. The model parameters for both events are generally consistent with those derived from SGRB X-ray plateau observations. The host galaxy properties of both transients are also consistent with those of SGRBs. The event rate densities of both XT1 and XT2 are consistent with that of BNS mergers.

astro-ph.HE↗

arXiv2019

The shallow decay segment of GRB X-ray afterglow revisited

Based on the early-year observations from Neil Gehrels \emph{Swift} Observatory, \cite{liang07} performed a systematic analysis for the shallow decay component of gamma-ray bursts (GRBs) X-ray afterglow, in order to explore its physical origin. Here we revisit the analysis with an updated sample (with Swift/XRT GRBs between February 2004 and July 2017). We find that with a larger sample, 1) the distributions of the characteristic properties of the shallow decay phase (e.g. $t_{b}$ , $S_{X}$, $Γ_{X,1}$, and $α_{X,1}$) still accords with normal or lognormal distribution; 2) $Γ_{X,1}$ and $Γ_γ$ still show no correlation, but the tentative correlations of durations, energy fluences, and isotropic energies between the gamma-ray and X-ray phases still exist; 3) for most GRBs, there is no significant spectral evolution between the shallow decay segment and its follow-up segment, and the latter is usually consistent with the external-shock models; 4) assuming that the central engine has a power-law luminosity release history as $L(t)=L_{0}(\frac{t}{t_{0}})^{-q}$, we find that the value $q$ is mainly distributed between -0.5 and 0.5, with an average value of 0.16$\pm$ 0.12; 5) the tentative correlation between $E_{\rm{iso},X}$ and $t'_{b}$ disappears, so that the global 3-parameter correlation ($E_{\rm{iso},X}-E'_{p}-t'_{b}$) becomes less significant; 6) the anti-correlation between $L_{X}$ and $t'_{b}$ and the three-parameter correlation ($E_{\rm{iso},γ}-L_{X}-t_{b}$) indeed exist with a high confidence level. Overall, our results are generally consistent with \cite{liang07}, confirming their suggestion that the shallow decay segment in most bursts is consistent with an external forward shock origin, probably due to a continuous energy injection from a long-lived central engine.

astro-ph.HE↗

arXiv2018

A Low-latency Pipeline for GRB Light Curve and Spectrum using Fermi/GBM Near Real-time Data

Rapid response and short time latency are very important for Time Domain Astronomy, such as the observations of Gamma-ray Bursts (GRBs) and electromagnetic (EM) counterparts of gravitational waves (GWs). Based on the near real-time Fermi/GBM data, we developed a low-latency pipeline to automatically calculate the temporal and spectral properties of GRBs. With this pipeline, some important parameters can be obtained, such as T90 and fluence, within ~20 minutes after the GRB trigger. For ~90% GRBs, T90 and fluence are consistent with the GBM catalog results within 2 sigma errors. This pipeline has been used by the Gamma-ray Bursts Polarimeter (POLAR) and the Insight Hard X-ray Modulation Telescope (Insight-HXMT) to follow up the bursts of interest. For GRB 170817A, the first EM counterpart of GW events detected by Fermi/GBM and INTEGRAL/SPI-ACS, the pipeline gave T90 and spectral information in 21 minutes after the GBM trigger, providing important information for POLAR and Insight-HXMT observations.

astro-ph.IM↗