arXiv subjects

Junbo Zhang

Publications and source records attributed to Junbo Zhang.

103 records · Page 6Linked to original sources

arXiv2018

End-to-end Models with auditory attention in Multi-channel Keyword Spotting

In this paper, we propose an attention-based end-to-end model for multi-channel keyword spotting (KWS), which is trained to optimize the KWS result directly. As a result, our model outperforms the baseline model with signal pre-processing techniques in both the clean and noisy testing data. We also found that multi-task learning results in a better performance when the training and testing data are similar. Transfer learning and multi-target spectral mapping can dramatically enhance the robustness to the noisy environment. At 0.1 false alarm (FA) per hour, the model with transfer learning and multi-target mapping gain an absolute 30% improvement in the wake-up rate in the noisy data with SNR about -20.

cs.SD↗

arXiv2018

Sequence-to-sequence Models for Small-Footprint Keyword Spotting

In this paper, we propose a sequence-to-sequence model for keyword spotting (KWS). Compared with other end-to-end architectures for KWS, our model simplifies the pipelines of production-quality KWS system and satisfies the requirement of high accuracy, low-latency, and small-footprint. We also evaluate the performances of different encoder architectures, which include LSTM and GRU. Experiments on the real-world wake-up data show that our approach outperforms the recently proposed attention-based end-to-end model. Specifically speaking, with 73K parameters, our sequence-to-sequence model achieves $\sim$3.05\% false rejection rate (FRR) at 0.1 false alarm (FA) per hour.

cs.SD↗

arXiv2018

Investigating Generative Adversarial Networks based Speech Dereverberation for Robust Speech Recognition

We investigate the use of generative adversarial networks (GANs) in speech dereverberation for robust speech recognition. GANs have been recently studied for speech enhancement to remove additive noises, but there still lacks of a work to examine their ability in speech dereverberation and the advantages of using GANs have not been fully established. In this paper, we provide deep investigations in the use of GAN-based dereverberation front-end in ASR. First, we study the effectiveness of different dereverberation networks (the generator in GAN) and find that LSTM leads a significant improvement as compared with feed-forward DNN and CNN in our dataset. Second, further adding residual connections in the deep LSTMs can boost the performance as well. Finally, we find that, for the success of GAN, it is important to update the generator and the discriminator using the same mini-batch data during training. Moreover, using reverberant spectrogram as a condition to discriminator, as suggested in previous studies, may degrade the performance. In summary, our GAN-based dereverberation front-end achieves 14%-19% relative CER reduction as compared to the baseline DNN dereverberation network when tested on a strong multi-condition training acoustic model.

cs.SD↗

arXiv2018

Empirical Evaluation of Speaker Adaptation on DNN based Acoustic Model

Speaker adaptation aims to estimate a speaker specific acoustic model from a speaker independent one to minimize the mismatch between the training and testing conditions arisen from speaker variabilities. A variety of neural network adaptation methods have been proposed since deep learning models have become the main stream. But there still lacks an experimental comparison between different methods, especially when DNN-based acoustic models have been advanced greatly. In this paper, we aim to close this gap by providing an empirical evaluation of three typical speaker adaptation methods: LIN, LHUC and KLD. Adaptation experiments, with different size of adaptation data, are conducted on a strong TDNN-LSTM acoustic model. More challengingly, here, the source and target we are concerned with are standard Mandarin speaker model and accented Mandarin speaker model. We compare the performances of different methods and their combinations. Speaker adaptation performance is also examined by speaker's accent degree.

cs.SD↗

arXiv2018

HyperST-Net: Hypernetworks for Spatio-Temporal Forecasting

Spatio-temporal (ST) data, which represent multiple time series data corresponding to different spatial locations, are ubiquitous in real-world dynamic systems, such as air quality readings. Forecasting over ST data is of great importance but challenging as it is affected by many complex factors, including spatial characteristics, temporal characteristics and the intrinsic causality between them. In this paper, we propose a general framework (HyperST-Net) based on hypernetworks for deep ST models. More specifically, it consists of three major modules: a spatial module, a temporal module and a deduction module. Among them, the deduction module derives the parameter weights of the temporal module from the spatial characteristics, which are extracted by the spatial module. Then, we design a general form of HyperST layer as well as different forms for several basic layers in neural networks, including the dense layer (HyperST-Dense) and the convolutional layer (HyperST-Conv). Experiments on three types of real-world tasks demonstrate that the predictive models integrated with our framework achieve significant improvements, and outperform the state-of-the-art baselines as well.

cs.LG↗

arXiv2018

Attention-based End-to-End Models for Small-Footprint Keyword Spotting

In this paper, we propose an attention-based end-to-end neural approach for small-footprint keyword spotting (KWS), which aims to simplify the pipelines of building a production-quality KWS system. Our model consists of an encoder and an attention mechanism. The encoder transforms the input signal into a high level representation using RNNs. Then the attention mechanism weights the encoder features and generates a fixed-length vector. Finally, by linear transformation and softmax function, the vector becomes a score used for keyword detection. We also evaluate the performance of different encoder architectures, including LSTM, GRU and CRNN. Experiments on real-world wake-up data show that our approach outperforms the recent Deep KWS approach by a large margin and the best performance is achieved by CRNN. To be more specific, with ~84K parameters, our attention-based model achieves 1.02% false rejection rate (FRR) at 1.0 false alarm (FA) per hour.

cs.SD↗

arXiv2018

Optimal Joint Bidding and Pricing of Profit-seeking Load Serving Entity

The demand response provides an opportunity for load serving entities (LSEs) that operate retail electricity markets (REMs) to strategically purchase energy and provide reserves in wholesale electricity markets (WEMs). This paper concerns with the problem of simultaneously determining the optimal energy bids and reserve offers an LSE submits to the WEM as well as the optimal energy and reserve prices it sets in the REM so as to maximize its profit. To this end, we explicitly model the tri-layer market structure that consists of a WEM, a REM, and a set of end user customers, so as to capture the coupling between the bidding problem and the pricing problem. Based on the tri-layer market model, we then formulate the joint bidding and pricing problem as a bi-level programming problem and further transform it into a single-level mixed integer linear programming problem, which can be solved efficiently. Numerical studies using the IEEE test cases are presented to illustrate the application of the proposed methodology as well as to reveal several interesting characteristics of the LSE's profit-seeking behavior.

math.OC↗

arXiv2018

Attention-Based End-to-End Speech Recognition on Voice Search

Recently, there has been a growing interest in end-to-end speech recognition that directly transcribes speech to text without any predefined alignments. In this paper, we explore the use of attention-based encoder-decoder model for Mandarin speech recognition on a voice search task. Previous attempts have shown that applying attention-based encoder-decoder to Mandarin speech recognition was quite difficult due to the logographic orthography of Mandarin, the large vocabulary and the conditional dependency of the attention model. In this paper, we use character embedding to deal with the large vocabulary. Several tricks are used for effective model training, including L2 regularization, Gaussian weight noise and frame skipping. We compare two attention mechanisms and use attention smoothing to cover long context in the attention model. Taken together, these tricks allow us to finally achieve a character error rate (CER) of 3.58% and a sentence error rate (SER) of 7.43% on the MiTV voice search dataset. While together with a trigram language model, CER and SER reach 2.81% and 5.77%, respectively.

cs.CL↗

arXiv2017

Deep Spatio-Temporal Residual Networks for Citywide Crowd Flows Prediction

Forecasting the flow of crowds is of great importance to traffic management and public safety, yet a very challenging task affected by many complex factors, such as inter-region traffic, events and weather. In this paper, we propose a deep-learning-based approach, called ST-ResNet, to collectively forecast the in-flow and out-flow of crowds in each and every region through a city. We design an end-to-end structure of ST-ResNet based on unique properties of spatio-temporal data. More specifically, we employ the framework of the residual neural networks to model the temporal closeness, period, and trend properties of the crowd traffic, respectively. For each property, we design a branch of residual convolutional units, each of which models the spatial properties of the crowd traffic. ST-ResNet learns to dynamically aggregate the output of the three residual neural networks based on data, assigning different weights to different branches and regions. The aggregation is further combined with external factors, such as weather and day of the week, to predict the final traffic of crowds in each and every region. We evaluate ST-ResNet based on two types of crowd flows in Beijing and NYC, finding that its performance exceeds six well-know methods.

cs.AI↗

arXiv2017

Predicting Citywide Crowd Flows Using Deep Spatio-Temporal Residual Networks

Forecasting the flow of crowds is of great importance to traffic management and public safety, and very challenging as it is affected by many complex factors, including spatial dependencies (nearby and distant), temporal dependencies (closeness, period, trend), and external conditions (e.g., weather and events). We propose a deep-learning-based approach, called ST-ResNet, to collectively forecast two types of crowd flows (i.e. inflow and outflow) in each and every region of a city. We design an end-to-end structure of ST-ResNet based on unique properties of spatio-temporal data. More specifically, we employ the residual neural network framework to model the temporal closeness, period, and trend properties of crowd traffic. For each property, we design a branch of residual convolutional units, each of which models the spatial properties of crowd traffic. ST-ResNet learns to dynamically aggregate the output of the three residual neural networks based on data, assigning different weights to different branches and regions. The aggregation is further combined with external factors, such as weather and day of the week, to predict the final traffic of crowds in each and every region. We have developed a real-time system based on Microsoft Azure Cloud, called UrbanFlow, providing the crowd flow monitoring and forecasting in Guiyang City of China. In addition, we present an extensive experimental evaluation using two types of crowd flows in Beijing and New York City (NYC), where ST-ResNet outperforms nine well-known baselines.

cs.AI↗

arXiv2016

NLTE Analysis of High Resolution H-band Spectra. I. Neutral Silicon

We investigated the reliability of our silicon atomic model and the influence of non-local thermodynamical equilibrium (NLTE) on the formation of neutral silicon (Si I) lines in the near-infrared (near-IR) H-band. We derived the differential Si abundances for 13 sample stars with high-resolution H-band spectra from the Apache Point Observatory Galactic Evolution Experiment (APOGEE), as well as from optical spectra, both under local thermodynamical equilibrium (LTE) and NLTE conditions. We found that the differences between the Si abundances derived from the H-band and from optical lines for the same stars are less than 0.1 dex when the NLTE effects included, and that NLTE reduces the line-to-line scatter in the H-band spectra for most sample stars. These results suggest that our Si atomic model is appropriate for studying the formation of H-band Si lines. Our calculations show that the NLTE corrections of the Si I H-band lines are negative, i.e. the final Si abundances will be overestimated in LTE. The corrections for strong lines depend on surface gravity, and tend to be larger for giants, reaching ~ -0.2 dex in our sample, and up to ~ -0.4 dex in extreme cases of APOGEE targets. Thus, the NLTE effects should be included in deriving silicon abundances from H-band Si I lines, especially for the cases where only strong lines are available.

astro-ph.SR↗

arXiv2016

NLTE Analysis of High Resolution H-band Spectra. II. Neutral Magnesium

Aiming at testing the validity of our magnesium atomic model and investigating the effects of non-local thermodynamical equilibrium (NLTE) on the formation of the H-band neutral magnesium lines, we derive the differential Mg abundances from selected transitions for 13 stars either adopting or relaxing the assumption of local thermodynamical equilibrium (LTE). Our analysis is based on high-resolution and high signal-to-noise ratio H-band spectra from the Apache Point Observatory Galactic Evolution Experiment (APOGEE) and optical spectra from several instruments. The absolute differences between the Mg abundances derived from the two wavelength bands are always less than 0.1 dex in the NLTE analysis, while they are slightly larger for the LTE case. This suggests that our Mg atomic model is appropriate for investigating the NLTE formation of the H-band Mg lines. The NLTE corrections for the Mg I H-band lines are sensitive to the surface gravity, becoming larger for smaller log g values, and strong lines are more susceptible to departures from LTE. For cool giants, NLTE corrections tend to be negative, and for the strong line at 15765 Å they reach -0.14 dex in our sample, and up to -0.22 dex for other APOGEE stars. Our results suggest that it is important to include NLTE corrections in determining Mg abundances from the H-band Mg I transitions, especially when strong lines are used.

astro-ph.SR↗

arXiv2016

Parallel Large-Scale Attribute Reduction on Cloud Systems

The rapid growth of emerging information technologies and application patterns in modern society, e.g., Internet, Internet of Things, Cloud Computing and Tri-network Convergence, has caused the advent of the era of big data. Big data contains huge values, however, mining knowledge from big data is a tremendously challenging task because of data uncertainty and inconsistency. Attribute reduction (also known as feature selection) can not only be used as an effective preprocessing step, but also exploits the data redundancy to reduce the uncertainty. However, existing solutions are designed 1) either for a single machine that means the entire data must fit in the main memory and the parallelism is limited; 2) or for the Hadoop platform which means that the data have to be loaded into the distributed memory frequently and therefore become inefficient. In this paper, we overcome these shortcomings for maximum efficiency possible, and propose a unified framework for Parallel Large-scale Attribute Reduction, termed PLAR, for big data analysis. PLAR consists of three components: 1) Granular Computing (GrC)-based initialization: it converts a decision table (i.e., original data representation) into a granularity representation which reduces the amount of space and hence can be easily cached in the distributed memory: 2) model-parallelism: it simultaneously evaluates all feature candidates and makes attribute reduction highly parallelizable; 3) data-parallelism: it computes the significance of an attribute in parallel using a MapReduce-style manner. We implement PLAR with four representative heuristic feature selection algorithms on Spark, and evaluate them on various huge datasets, including UCI and astronomical datasets, finding our method's advantages beyond existing solutions.

cs.DC↗