arXiv subjects
Publications and source records attributed to Liang He.
This paper describes the systems submitted by the department of electronic engineering, institute of microelectronics of Tsinghua university and TsingMicro Co. Ltd. (THUEE) to the NIST 2019 speaker recognition evaluation CTS challenge. Six subsystems, including etdnn/ams, ftdnn/as, eftdnn/ams, resnet, multitask and c-vector are developed in this evaluation.
WiFi-based indoor localization has received extensive attentions from both academia and industry. However, the overhead of constructing and maintaining the WiFi fingerprint map remains a bottleneck for the wide-deployment of WiFi-based indoor localization systems. Recently, robots are adopted as the professional surveyor to fingerprint the environment autonomously. But the time and energy cost still limit the coverage of the robot surveyor, thus reduce its scalability. To fill this need, we design an AutonomousWiFi Fingerprinting system, called AuF, which autonomously constructs the fingerprint database with time and energy efficiency. AuF first conduct an automatic initialization process in the target indoor environment, then constructs the WiFi fingerprint database of in two steps: (i) surveying the site without sojourn, (ii) recovering unreliable signals in the database with two methods. We have implemented and evaluated AuF using a Pioneer 3-DX robot, on two sites of our $70$$\times$$90$m$^2$ Department building with different structures and deployments of access points (APs). The results show AuF finishes the fingerprint database construction in 43/51 minutes, and consumes 60/82 Wh on the two floors respectively, which is a 64%/71% and 61%/64% reduction when compared to traditional site survey methods, without degrading the localization accuracy.
The accuracy of OCR is usually affected by the quality of the input document image and different kinds of marred document images hamper the OCR results. Among these scenarios, the low-resolution image is a common and challenging case. In this paper, we propose the cascaded networks for document image super-resolution. Our model is composed by the Detail-Preserving Networks with small magnification. The loss function with perceptual terms is designed to simultaneously preserve the original patterns and enhance the edge of the characters. These networks are trained with the same architecture and different parameters and then assembled into a pipeline model with a larger magnification. The low-resolution images can upscale gradually by passing through each Detail-Preserving Network until the final high-resolution images. Through extensive experiments on two scanning document image datasets, we demonstrate that the proposed approach outperforms recent state-of-the-art image super-resolution methods, and combining it with standard OCR system lead to signification improvements on the recognition results.
The goal of acoustic (or sound) events detection (AED or SED) is to predict the temporal position of target events in given audio segments. This task plays a significant role in safety monitoring, acoustic early warning and other scenarios. However, the deficiency of data and diversity of acoustic event sources make the AED task a tough issue, especially for prevalent data-driven methods. In this paper, we start by analyzing acoustic events according to their time-frequency domain properties, showing that different acoustic events have different time-frequency scale characteristics. Inspired by the analysis, we propose an adaptive multi-scale detection (AdaMD) method. By taking advantage of the hourglass neural network and gated recurrent unit (GRU) module, our AdaMD produces multiple predictions at different temporal and frequency resolutions. An adaptive training algorithm is subsequently adopted to combine multi-scale predictions to enhance its overall capability. Experimental results on Detection and Classification of Acoustic Scenes and Events 2017 (DCASE 2017) Task 2, DCASE 2016 Task 3 and DCASE 2017 Task 3 demonstrate that the AdaMD outperforms published state-of-the-art competitors in terms of the metrics of event error rate (ER) and F1-score. The verification experiment on our collected factory mechanical dataset also proves the noise-resistant capability of the AdaMD, providing the possibility for it to be deployed in the complex environment.
Even though contact electrification has been studied for a long time, the mechanism of charge transfer still remains elusive. Most of previous reports only focus on the driving force of charge transfer. However, to better understand the mechanism of charge transfer, we believe contact itself for supplying transferring path that charges taking also need to be understood. Here, we focus on the in-plane symmetry of contact in contact electrification by utilizing a uniaxial strain to change the material's isotropic nature to anisotropic. A clear anisotropic charge transfer is observed by contacting axially stretched rubber films at different rotational angles, which could arise from fluctuation of contacting area in microscale. A universal ellipse model is also proposed for qualitatively describing the anisotropy of contact regardless of the specific driving force of contact electrification.
Most attention-based methods only concentrate along the time axis, which is insufficient for Acoustic Event Detection (AED). Meanwhile, previous methods for AED rarely considered that target events possess distinct temporal and frequential scales. In this work, we propose a Multi-Scale Time-Frequency Attention (MTFA) module for AED. MTFA gathers information at multiple resolutions to generate a time-frequency attention mask which tells the model where to focus along both time and frequency axis. With MTFA, the model could capture the characteristics of target events with different scales. We demonstrate the proposed method on Task 2 of Detection and Classification of Acoustic Scenes and Events (DCASE) 2017 Challenge. Our method achieves competitive results on both development dataset and evaluation dataset.
In this paper, we apply a latent class model (LCM) to the task of speaker diarization. LCM is similar to Patrick Kenny's variational Bayes (VB) method in that it uses soft information and avoids premature hard decisions in its iterations. In contrast to the VB method, which is based on a generative model, LCM provides a framework allowing both generative and discriminative models. The discriminative property is realized through the use of i-vector (Ivec), probabilistic linear discriminative analysis (PLDA), and a support vector machine (SVM) in this work. Systems denoted as LCM-Ivec-PLDA, LCM-Ivec-SVM, and LCM-Ivec-Hybrid are introduced. In addition, three further improvements are applied to enhance its performance. 1) Adding neighbor windows to extract more speaker information for each short segment. 2) Using a hidden Markov model to avoid frequent speaker change points. 3) Using an agglomerative hierarchical cluster to do initialization and present hard and soft priors, in order to overcome the problem of initial sensitivity. Experiments on the National Institute of Standards and Technology Rich Transcription 2009 speaker diarization database, under the condition of a single distant microphone, show that the diarization error rate (DER) of the proposed methods has substantial relative improvements compared with mainstream systems. Compared to the VB method, the relative improvements of LCM-Ivec-PLDA, LCM-Ivec-SVM, and LCM-Ivec-Hybrid systems are 23.5%, 27.1%, and 43.0%, respectively. Experiments on our collected database, CALLHOME97, CALLHOME00 and SRE08 short2-summed trial conditions also show that the proposed LCM-Ivec-Hybrid system has the best overall performance.
In neural network based speaker verification, speaker embedding is expected to be discriminative between speakers while the intra-speaker distance should remain small. A variety of loss functions have been proposed to achieve this goal. In this paper, we investigate the large margin softmax loss with different configurations in speaker verification. Ring loss and minimum hyperspherical energy criterion are introduced to further improve the performance. Results on VoxCeleb show that our best system outperforms the baseline approach by 15\% in EER, and by 13\%, 33\% in minDCF08 and minDCF10, respectively.
Crowd counting, i.e., estimation number of the pedestrian in crowd images, is emerging as an important research problem with the public security applications. A key component for the crowd counting systems is the construction of counting models which are robust to various scenarios under facts such as camera perspective and physical barriers. In this paper, we present an adaptive scenario discovery framework for crowd counting. The system is structured with two parallel pathways that are trained with different sizes of the receptive field to represent different scales and crowd densities. After ensuring that these components are present in the proper geometric configuration, a third branch is designed to adaptively recalibrate the pathway-wise responses by discovering and modeling the dynamic scenarios implicitly. Our system is able to represent highly variable crowd images and achieves state-of-the-art results in two challenging benchmarks.
Charm quarks, owing to their large mass, are produced predominantly in the initial hard scatterings in heavy-ion collisions, and therefore can be a valuable tool for studying the early time dynamics of these collisions. The rapidity-odd directed flow at mid-rapidity in heavy-ion collisions originates from a tilt in the reaction plane of the thermalized medium caused by the asymmetry between the number of participants from projectile and target nuclei as a function of rapidity. Recently, it has been predicted that the slope of the directed flow at mid-rapidity for $D^0$ mesons, arising from the transport of charm quarks in the tilted medium, can be several times larger than that of light flavor hadrons. The magnitude of the slope is expected to be sensitive to the magnitude of the tilt and the charm quark drag coefficient in the medium. It has also been predicted that the transient electromagnetic field generated at early time can induce a much larger directed flow for heavy quarks than for light quarks. In these proceedings, we will report on the first measurement of the directed flow for the $D^0$ and $\bar{D}^0$ mesons as a function of rapidity in Au+Au collisions at $\sqrt{s_{_{\rm NN}}}$ = 200 GeV using high statistics data collected with the Heavy Flavor Tracker in 2014 and 2016 RHIC runs.
Most current state-of-the-art text-independent speaker verification systems take probabilistic linear discriminant analysis (PLDA) as their backend classifiers. The parameters of PLDA are often estimated by maximizing the objective function, which focuses on increasing the value of log-likelihood function, but ignoring the distinction between speakers. In order to better distinguish speakers, we propose a multi-objective optimization training for PLDA. Experiment results show that the proposed method has more than 10% relative performance improvement in both EER and MinDCF on the NIST SRE14 i-vector challenge dataset, and about 20% relative performance improvement in EER on the MCE18 dataset.
Frame alignments can be computed by different methods in GMM-based speaker verification. By incorporating a phonetic Gaussian mixture model (PGMM), we are able to compare the performance using alignments extracted from the deep neural networks (DNN) and the conventional hidden Markov model (HMM) in digit-prompted speaker verification. Based on the different characteristics of these two alignments, we present a novel content verification method to improve the system security without much computational overhead. Our experiments on the RSR2015 Part-3 digit-prompted task show that, the DNN based alignment performs on par with the HMM alignment. The results also demonstrate the effectiveness of the proposed Kullback-Leibler (KL) divergence based scoring to reject speech with incorrect pass-phrases.
The pooling layer is an essential component in the neural network based speaker verification. Most of the current networks in speaker verification use average pooling to derive the utterance-level speaker representations. Average pooling takes every frame as equally important, which is suboptimal since the speaker-discriminant power is different between speech segments. In this paper, we present a unified attention-based pooling framework and combine it with the multi-head attention. Experiments on the Fisher and NIST SRE 2010 dataset show that involving outputs from lower layers to compute the attention weights can outperform average pooling and achieve better results than vanilla attention method. The multi-head attention further improves the performance.
Speaker embeddings achieve promising results on many speaker verification tasks. Phonetic information, as an important component of speech, is rarely considered in the extraction of speaker embeddings. In this paper, we introduce phonetic information to the speaker embedding extraction based on the x-vector architecture. Two methods using phonetic vectors and multi-task learning are proposed. On the Fisher dataset, our best system outperforms the original x-vector approach by 20% in EER, and by 15%, 15% in minDCF08 and minDCF10, respectively. Experiments conducted on NIST SRE10 further demonstrate the effectiveness of the proposed methods.
Locating actions in long untrimmed videos has been a challenging problem in video content analysis. The performances of existing action localization approaches remain unsatisfactory in precisely determining the beginning and the end of an action. Imitating the human perception procedure with observations and refinements, we propose a novel three-phase action localization framework. Our framework is embedded with an Actionness Network to generate initial proposals through frame-wise similarity grouping, and then a Refinement Network to conduct boundary adjustment on these proposals. Finally, the refined proposals are sent to a Localization Network for further fine-grained location regression. The whole process can be deemed as multi-stage refinement using a novel non-local pyramid feature under various temporal granularities. We evaluate our framework on THUMOS14 benchmark and obtain a significant improvement over the state-of-the-arts approaches. Specifically, the performance gain is remarkable under precise localization with high IoU thresholds. Our proposed framework achieves mAP@IoU=0.5 of 34.2%.
In this paper, we propose an enhanced triplet method that improves the encoding process of embeddings by jointly utilizing generative adversarial mechanism and multitasking optimization. We extend our triplet encoder with Generative Adversarial Networks (GANs) and softmax loss function. GAN is introduced for increasing the generality and diversity of samples, while softmax is for reinforcing features about speakers. For simplification, we term our method Multitasking Triplet Generative Adversarial Networks (MTGAN). Experiment on short utterances demonstrates that MTGAN reduces the verification equal error rate (EER) by 67% (relatively) and 32% (relatively) over conventional i-vector method and state-of-the-art triplet loss method respectively. This effectively indicates that MTGAN outperforms triplet methods in the aspect of expressing the high-level feature of speaker information.
Adaptive modulation and coding (AMC) is widely employed in modern wireless communication systems to improve the transmission efficiency by adjusting the transmission rate according to the channel conditions. Thus, AMC can provide very efficient use of channel resources especially over fading channels. Quadrature Amplitude Modulation (QAM) is an ef- ficient and widely employed digital modulation technique. It typically employs a rectangular signal constellation. Therefore the decision regions of the constellation are square partitions of the two-dimensional signal space. However, it is well known that hexagons rather than squares provide the most compact regular tiling in two dimensions. A compact tiling means a dense packing of the constellation points and thus more energy efficient data transmission. Hexagonal modulation can be difficult to implement because it does not fit well with the usual power- of-two symbol sizes employed with binary data. To overcome this problem, non-binary coding is combined with hexagonal modulation in this paper to provide a system which is compatible with binary data. The feasibility and efficiency are evaluated using a software-defined radio (SDR) based prototype. Extensive simulation results are presented which show that this approach can provide improved energy efficiency and spectrum utilization in wireless communication systems.
We investigate the steady state phase diagram of two-component driven open condensates in one dimension. We identify a miscible-immiscible transition which is predominantly driven by gapped density fluctuations and occurs upon increasing the inter-component inelastic coupling. Below the transition in the miscible phase, we find the effective long wavelength dynamics to be described by a two-component Kardar-Parisi-Zhang (KPZ) equation that belongs to the nonequilibrium universality class of the one-dimensional single-component KPZ equation at generic choices of parameters. Our results are relevant for different experimental realizations for two-component driven open condensates in exciton-polariton systems.