Phase-Aware Speech Enhancement with a Recurrent Two Stage Net work

Abstract

We propose a neural network-based speech enhancement (SE) method called the phase-aware recurrent two stage network (rTSN). The rTSN is an extension of our previously proposed two stage network (TSN) framework. This TSN framework was equipped with a boosting strategy (BS) that initially estimates the multiple base predictions (MBPs) from a prior neural network (pri-NN) and then the MBPs are aggregated by a posterior neural network (post-NN) to obtain the final prediction. The TSN outperformed various state-of-the-art methods; however, it adopted the simple deep neural network as pri-NN. We have found that the pri-NN affects the performance (in perceptual quality), more than post-NN; therefore we adopted the long short-term memory recurrent neural network (LSTM-RNN) as pri-NN to boost the context information usage within speech signals. Further, the TSN framework did not consider the phase reconstruction, though phase information affected the perceptual quality. Therefore, we proposed to adopt the phase reconstruction method based on the Griffin-Lim algorithm. Finally, we evaluated rTSN with baselines such as TSN in perceptual quality related metrics as well as the phone recognition error rate.