arXiv subjects

Yafeng Chen

Publications and source records attributed to Yafeng Chen.

32 records · Page 2Linked to original sources

arXiv2023

3D-Speaker: A Large-Scale Multi-Device, Multi-Distance, and Multi-Dialect Corpus for Speech Representation Disentanglement

Disentangling uncorrelated information in speech utterances is a crucial research topic within speech community. Different speech-related tasks focus on extracting distinct speech representations while minimizing the affects of other uncorrelated information. We present a large-scale speech corpus to facilitate the research of speech representation disentanglement. 3D-Speaker contains over 10,000 speakers, each of whom are simultaneously recorded by multiple Devices, locating at different Distances, and some speakers are speaking multiple Dialects. The controlled combinations of multi-dimensional audio data yield a matrix of a diverse blend of speech representation entanglement, thereby motivating intriguing methods to untangle them. The multi-domain nature of 3D-Speaker also makes it a suitable resource to evaluate large universal speech models and experiment methods of out-of-domain learning and self-supervised learning. https://3dspeaker.github.io/

cs.CL↗

arXiv2023

Improved Meta-Learning Training for Speaker Verification

Meta-learning has recently become a research hotspot in speaker verification (SV). We introduce two methods to improve the meta-learning training for SV in this paper. For the first method, a backbone embedding network is first jointly trained with the conventional cross entropy loss and prototypical networks (PN) loss. Then, inspired by speaker adaptive training in speech recognition, additional transformation coefficients are trained with only the PN loss. The transformation coefficients are used to modify the original backbone embedding network in the x-vector extraction process. Furthermore, the random erasing data augmentation technique is applied to all support samples in each episode to construct positive pairs, and a contrastive loss between the augmented and the original support samples is added to the objective in model training. Experiments are carried out on the SITW and VOiCES databases. Both of the methods can obtain consistent improvements over existing meta-learning training frameworks. By combining these two methods, we can observe further improvements on these two databases.

eess.AS↗

arXiv2023

Pushing the limits of self-supervised speaker verification using regularized distillation framework

Training robust speaker verification systems without speaker labels has long been a challenging task. Previous studies observed a large performance gap between self-supervised and fully supervised methods. In this paper, we apply a non-contrastive self-supervised learning framework called DIstillation with NO labels (DINO) and propose two regularization terms applied to embeddings in DINO. One regularization term guarantees the diversity of the embeddings, while the other regularization term decorrelates the variables of each embedding. The effectiveness of various data augmentation techniques are explored, on both time and frequency domain. A range of experiments conducted on the VoxCeleb datasets demonstrate the superiority of the regularized DINO framework in speaker verification. Our method achieves the state-of-the-art speaker verification performance under a single-stage self-supervised setting on VoxCeleb. Code has been made publicly available at https://github.com/alibaba-damo-academy/3D-Speaker.

eess.AS↗

arXiv2023

An Enhanced Res2Net with Local and Global Feature Fusion for Speaker Verification

Effective fusion of multi-scale features is crucial for improving speaker verification performance. While most existing methods aggregate multi-scale features in a layer-wise manner via simple operations, such as summation or concatenation. This paper proposes a novel architecture called Enhanced Res2Net (ERes2Net), which incorporates both local and global feature fusion techniques to improve the performance. The local feature fusion (LFF) fuses the features within one single residual block to extract the local signal. The global feature fusion (GFF) takes acoustic features of different scales as input to aggregate global signal. To facilitate effective feature fusion in both LFF and GFF, an attentional feature fusion module is employed in the ERes2Net architecture, replacing summation or concatenation operations. A range of experiments conducted on the VoxCeleb datasets demonstrate the superiority of the ERes2Net in speaker verification. Code has been made publicly available at https://github.com/alibaba-damo-academy/3D-Speaker.

eess.AS↗

arXiv2023

CAM++: A Fast and Efficient Network for Speaker Verification Using Context-Aware Masking

Time delay neural network (TDNN) has been proven to be efficient for speaker verification. One of its successful variants, ECAPA-TDNN, achieved state-of-the-art performance at the cost of much higher computational complexity and slower inference speed. This makes it inadequate for scenarios with demanding inference rate and limited computational resources. We are thus interested in finding an architecture that can achieve the performance of ECAPA-TDNN and the efficiency of vanilla TDNN. In this paper, we propose an efficient network based on context-aware masking, namely CAM++, which uses densely connected time delay neural network (D-TDNN) as backbone and adopts a novel multi-granularity pooling to capture contextual information at different levels. Extensive experiments on two public benchmarks, VoxCeleb and CN-Celeb, demonstrate that the proposed architecture outperforms other mainstream speaker verification systems with lower computational cost and faster inference speed.

cs.SD↗

arXiv2023

Exploring Universal Speech Attributes for Speaker Verification with an Improved Cross-stitch Network

The universal speech attributes for x-vector based speaker verification (SV) are addressed in this paper. The manner and place of articulation form the fundamental speech attribute unit (SAU), and then new speech attribute (NSA) units for acoustic modeling are generated by tied tri-SAU states. An improved cross-stitch network is adopted as a multitask learning (MTL) framework for integrating these universal speech attributes into the x-vector network training process. Experiments are conducted on common condition 5 (CC5) of the core-core and the 10 s-10 s tests of the NIST SRE10 evaluation set, and the proposed algorithm can achieve consistent improvements over the baseline x-vector on both these tasks.

eess.AS↗

arXiv2023

Exploring Speaker-Related Information in Spoken Language Understanding for Better Speaker Diarization

Speaker diarization(SD) is a classic task in speech processing and is crucial in multi-party scenarios such as meetings and conversations. Current mainstream speaker diarization approaches consider acoustic information only, which result in performance degradation when encountering adverse acoustic conditions. In this paper, we propose methods to extract speaker-related information from semantic content in multi-party meetings, which, as we will show, can further benefit speaker diarization. We introduce two sub-tasks, Dialogue Detection and Speaker-Turn Detection, in which we effectively extract speaker information from conversational semantics. We also propose a simple yet effective algorithm to jointly model acoustic and semantic information and obtain speaker-identified texts. Experiments on both AISHELL-4 and AliMeeting datasets show that our method achieves consistent improvements over acoustic-only speaker diarization systems.

cs.CL↗

arXiv2023

Multi-task Metric Learning for Text-independent Speaker Verification

In this work, we introduce metric learning (ML) to enhance the deep embedding learning for text-independent speaker verification (SV). Specifically, the deep speaker embedding network is trained with conventional cross entropy loss and auxiliary pair-based ML loss function. For the auxiliary ML task, training samples of a mini-batch are first arranged into pairs, then positive and negative pairs are selected and weighted through their own and relative similarities, and finally the auxiliary ML loss is calculated by the similarity of the selected pairs. To evaluate the proposed method, we conduct experiments on the Speaker in the Wild (SITW) dataset. The results demonstrate the effectiveness of the proposed method.

eess.AS↗

arXiv2022

Fast localization and single-pixel imaging of the moving object using time-division multiplexing

When imaging moving objects, single-pixel imaging produces motion blur. This paper proposes a new single-pixel imaging method, which can achieve anti-motion blur imaging of a fast-moving object. The geometric moment patterns and Hadamard patterns are used to alternately encode the position information and the image information of the object with time-division multiplexing. In the reconstruction process, the object position information is extracted independently and combining motion-compensation reconstruction algorithm to decouple the object motion from image information. As a result, the anti-motion blur image and the high frame rate object positions are obtained. Experimental results show that for a moving object with an angular velocity of up to 0.5rad/s relative to the imaging system, the proposed method achieves a localization frequency of 5.55kHz, and gradually reconstructs a clear image of the fast-moving object with a pseudo resolution of 512x512. The method has application prospects in single-pixel imaging of the fast-moving object.

eess.IV↗

arXiv2022

Graph Convolutional Network Based Semi-Supervised Learning on Multi-Speaker Meeting Data

Unsupervised clustering on speakers is becoming increasingly important for its potential uses in semi-supervised learning. In reality, we are often presented with enormous amounts of unlabeled data from multi-party meetings and discussions. An effective unsupervised clustering approach would allow us to significantly increase the amount of training data without additional costs for annotations. Recently, methods based on graph convolutional networks (GCN) have received growing attention for unsupervised clustering, as these methods exploit the connectivity patterns between nodes to improve learning performance. In this work, we present a GCN-based approach for semi-supervised learning. Given a pre-trained embedding extractor, a graph convolutional network is trained on the labeled data and clusters unlabeled data with "pseudo-labels". We present a self-correcting training mechanism that iteratively runs the cluster-train-correct process on pseudo-labels. We show that this proposed approach effectively uses unlabeled data and improves speaker recognition accuracy.

eess.AS↗

arXiv2021

Dual-Polarization Second-Order Photonic Topological Insulators

Second-order photonic topological insulators that host highly localized corner states resilient to defects, are opening new routes towards developing fascinating photonic devices. However, the existing works on second-order photonic topological insulators have mainly focused on either transverse magnetic or transverse electric modes. In this paper, we propose a dual-polarization topological photonic crystal structure for both transverse magnetic and transverse electric modes through topology optimization. Simple tight-binding lattice models are constructed to reveal the topological features of the optimized photonic crystal structure in a transparent way. The optimized dual-polarization second-order photonic topological insulator hosts four groups of corner states with different profiles and eigenfrequencies for both the transverse magnetic and transverse electric modes. Moreover, the robustness of theses corner states against defects is explicitly demonstrated. Our results offer opportunities for developing polarization-independent topological photonic devices.

physics.optics↗

arXiv2021

Topologically protected second-harmonic generation via doubly resonant high-order photonic modes

Topology-driven nonlinear light-matter effects open up new paradigms for both topological photonics and nonlinear optics. Here, we propose to achieve high-efficiency second-harmonic generation in a second-order photonic topological insulator. Such system hosts highly localized topological corner states with large quality factors for both fundamental and second harmonic waves, which could be matched perfectly in frequency by simply tuning the structural parameters. Through the nonlinear interaction of the doubly resonant topological corner states, unprecedented frequency conversion efficiency is predicted. In addition, the robustness of the nonlinear process against defects is also demonstrated. Our work opens up new avenues toward topologically protected nonlinear frequency conversion using high-order photonic topological modes.

physics.optics↗

arXiv2020

Multidimensional sound propagation in 3D high-order topological sonic insulator

High-order topological insulators (TIs) develop the conventional bulk-boundary correspondence theory and rise the interest in searching innovative topological materials. To realize a high-order TI with a wide passband of 1D and 2D transportation modes, we design non-trivial and trivial 3D sonic crystals whose combination mimics the Su-Schrieffer-Heeger model. The high-order topological boundary states can be found at the interfaces, including 0D corner state, 1D hinge state, and 2D surface state. The fabricated sample with the bent two-dimensional and one-dimensional acoustic channels exhibits the multidimensional sound propagation in space, and also verifies the transition between the complete band gap, hinge states, and surface states within the bulk band gap. Among them, the bandwidth of the single-mode hinge state achieves a large relative bandwidth 9.1%, in which sound transports one-dimensionally without significant leak into the surfaces or the bulk. The high-order topological states in the study pave the way for multidimensional sound manipulation in space.

physics.app-ph↗

arXiv2020

Inverse design of higher-order photonic topological insulators

Topological photonics revolutionizes some of the traditional approaches to light propagation and manipulation, and it provides unprecedented means for developing novel photonic devices. Recently discovered higher-order topological phases go beyond the conventional bulk-edge correspondence for photonic crystals and introduce novel opportunities for topological protection. Here, we introduce an intelligent numerical approach for inverse design of higher-order photonic topological insulators with great flexibility for controlling both topological edge and topological corner states. In particular, we consider the second-order photonic topological insulator and design several structures supporting both edge and corner states at different frequencies. By carefully programming these structures, we suggest a novel approach for topological routing of the edge and corner states by changing the operational frequency. Our finding paves the way to integrated topological photonic devices with novel functionalities.

physics.app-ph↗