arXiv subjects

Xiaofei Li

Publications and source records attributed to Xiaofei Li.

At least 73 records · Page 4Linked to original sources

arXiv2021

FullSubNet: A Full-Band and Sub-Band Fusion Model for Real-Time Single-Channel Speech Enhancement

This paper proposes a full-band and sub-band fusion model, named as FullSubNet, for single-channel real-time speech enhancement. Full-band and sub-band refer to the models that input full-band and sub-band noisy spectral feature, output full-band and sub-band speech target, respectively. The sub-band model processes each frequency independently. Its input consists of one frequency and several context frequencies. The output is the prediction of the clean speech target for the corresponding frequency. These two types of models have distinct characteristics. The full-band model can capture the global spectral context and the long-distance cross-band dependencies. However, it lacks the ability to modeling signal stationarity and attending the local spectral pattern. The sub-band model is just the opposite. In our proposed FullSubNet, we connect a pure full-band model and a pure sub-band model sequentially and use practical joint training to integrate these two types of models' advantages. We conducted experiments on the DNS challenge (INTERSPEECH 2020) dataset to evaluate the proposed method. Experimental results show that full-band and sub-band information are complementary, and the FullSubNet can effectively integrate them. Besides, the performance of the FullSubNet also exceeds that of the top-ranked methods in the DNS Challenge (INTERSPEECH 2020).

eess.AS↗

arXiv2020

Reverberant Sound Localization with a Robot Head Based on Direct-Path Relative Transfer Function

This paper addresses the problem of sound-source localization (SSL) with a robot head, which remains a challenge in real-world environments. In particular we are interested in locating speech sources, as they are of high interest for human-robot interaction. The microphone-pair response corresponding to the direct-path sound propagation is a function of the source direction. In practice, this response is contaminated by noise and reverberations. The direct-path relative transfer function (DP-RTF) is defined as the ratio between the direct-path acoustic transfer function (ATF) of the two microphones, and it is an important feature for SSL. We propose a method to estimate the DP-RTF from noisy and reverberant signals in the short-time Fourier transform (STFT) domain. First, the convolutive transfer function (CTF) approximation is adopted to accurately represent the impulse response of the microphone array, and the first coefficient of the CTF is mainly composed of the direct-path ATF. At each frequency, the frame-wise speech auto- and cross-power spectral density (PSD) are obtained by spectral subtraction. Then a set of linear equations is constructed by the speech auto- and cross-PSD of multiple frames, in which the DP-RTF is an unknown variable, and is estimated by solving the equations. Finally, the estimated DP-RTFs are concatenated across frequencies and used as a feature vector for SSL. Experiments with a robot, placed in various reverberant environments, show that the proposed method outperforms two state-of-the-art methods.

cs.SD↗

arXiv2020

Multichannel Online Dereverberation based on Spectral Magnitude Inverse Filtering

This paper addresses the problem of multichannel online dereverberation. The proposed method is carried out in the short-time Fourier transform (STFT) domain, and for each frequency band independently. In the STFT domain, the time-domain room impulse response is approximately represented by the convolutive transfer function (CTF). The multichannel CTFs are adaptively identified based on the cross-relation method, and using the recursive least square criterion. Instead of the complex-valued CTF convolution model, we use a nonnegative convolution model between the STFT magnitude of the source signal and the CTF magnitude, which is just a coarse approximation of the former model, but is shown to be more robust against the CTF perturbations. Based on this nonnegative model, we propose an online STFT magnitude inverse filtering method. The inverse filters of the CTF magnitude are formulated based on the multiple-input/output inverse theorem (MINT), and adaptively estimated based on the gradient descent criterion. Finally, the inverse filtering is applied to the STFT magnitude of the microphone signals, obtaining an estimate of the STFT magnitude of the source signal. Experiments regarding both speech enhancement and automatic speech recognition are conducted, which demonstrate that the proposed method can effectively suppress reverberation, even for the difficult case of a moving speaker.

cs.SD↗

arXiv2020

Sub-Band Knowledge Distillation Framework for Speech Enhancement

In single-channel speech enhancement, methods based on full-band spectral features have been widely studied. However, only a few methods pay attention to non-full-band spectral features. In this paper, we explore a knowledge distillation framework based on sub-band spectral mapping for single-channel speech enhancement. Specifically, we divide the full frequency band into multiple sub-bands and pre-train an elite-level sub-band enhancement model (teacher model) for each sub-band. These teacher models are dedicated to processing their own sub-bands. Next, under the teacher models' guidance, we train a general sub-band enhancement model (student model) that works for all sub-bands. Without increasing the number of model parameters and computational complexity, the student model's performance is further improved. To evaluate our proposed method, we conducted a large number of experiments on an open-source data set. The final experimental results show that the guidance from the elite-level teacher models dramatically improves the student model's performance, which exceeds the full-band model by employing fewer parameters.

eess.AS↗

arXiv2020

Online Monaural Speech Enhancement Using Delayed Subband LSTM

This paper proposes a delayed subband LSTM network for online monaural (single-channel) speech enhancement. The proposed method is developed in the short time Fourier transform (STFT) domain. Online processing requires frame-by-frame signal reception and processing. A paramount feature of the proposed method is that the same LSTM is used across frequencies, which drastically reduces the number of network parameters, the amount of training data and the computational burden. Training is performed in a subband manner: the input consists of one frequency, together with a few context frequencies. The network learns a speech-to-noise discriminative function relying on the signal stationarity and on the local spectral pattern, based on which it predicts a clean-speech mask at each frequency. To exploit future information, i.e. look-ahead, we propose an output-delayed subband architecture, which allows the unidirectional forward network to process a few future frames in addition to the current frame. We leverage the proposed method to participate to the DNS real-time speech enhancement challenge. Experiments with the DNS dataset show that the proposed method achieves better performance-measuring scores than the DNS baseline method, which learns the full-band spectra using a gated recurrent unit network.

cs.SD↗

arXiv2020

Polarization tensor vanishing structure of general shape: Existence for small perturbations of balls

The polarization tensor is a geometric quantity associated with a domain. It is a signature of the small inclusion's existence inside a domain and used in the small volume expansion method to reconstruct small inclusions by boundary measurements. In this paper, we consider the question of the polarization tensor vanishing structure of general shape. The only known examples of the polarization tensor vanishing structure are concentric disks and balls. We prove, by the implicit function theorem on Banach spaces, that a small perturbation of a ball can be enclosed by a domain so that the resulting inclusion of the core-shell structure becomes polarization tensor vanishing. The boundary of the enclosing domain is given by a sphere perturbed by spherical harmonics of degree zero and two. This is a continuation of the earlier work \cite{KLS2D} for two dimensions.

math.AP↗

arXiv2020

Neutral inclusions, weakly neutral inclusions, and an over-determined problem for confocal ellipsoids

An inclusion is said to be neutral to uniform fields if upon insertion into a homogenous medium with a uniform field it does not perturb the uniform field at all. It is said to be weakly neutral if it perturbs the uniform field mildly. Such inclusions are of interest in relation to invisibility cloaking and effective medium theory. There have been some attempts lately to construct or to show existence of such inclusions in the form of core-shell structure or a single inclusion with the imperfect bonding parameter attached to its boundary. The purpose of this paper is to review recent progress in such attempts. We also discuss about the over-determined problem for confocal ellipsoids which is closely related with the neutral inclusion, and its equivalent formulation in terms of Newtonian potentials. The main body of this paper consists of reviews on known results, but some new results are also included.

math.AP↗

arXiv2019

Expectation-Maximization for Speech Source Separation Using Convolutive Transfer Function

This paper addresses the problem of under-determinded speech source separation from multichannel microphone singals, i.e. the convolutive mixtures of multiple sources. The time-domain signals are first transformed to the short-time Fourier transform (STFT) domain. To represent the room filters in the STFT domain, instead of the widely-used narrowband assumption, we propose to use a more accurate model, i.e. the convolutive transfer function (CTF). At each frequency band, the CTF coefficients of the mixing filters and the STFT coefficients of the sources are jointly estimated by maximizing the likelihood of the microphone signals, which is resolved by an Expectation-Maximization (EM) algorithm. Experiments show that the proposed method provides very satisfactory performance under highly reverberant environments.

cs.SD↗

arXiv2019

Audio-noise Power Spectral Density Estimation Using Long Short-term Memory

We propose a method using a long short-term memory (LSTM) network to estimate the noise power spectral density (PSD) of single-channel audio signals represented in the short time Fourier transform (STFT) domain. An LSTM network common to all frequency bands is trained, which processes each frequency band individually by mapping the noisy STFT magnitude sequence to its corresponding noise PSD sequence. Unlike deep-learning-based speech enhancement methods that learn the full-band spectral structure of speech segments, the proposed method exploits the sub-band STFT magnitude evolution of noise with a long time dependency, in the spirit of the unsupervised noise estimators described in the literature. Speaker- and speech-independent experiments with different types of noise show that the proposed method outperforms the unsupervised estimators, and generalizes well to noise types that are not present in the training set.

eess.SP↗

arXiv2019

Online Localization and Tracking of Multiple Moving Speakers in Reverberant Environments

We address the problem of online localization and tracking of multiple moving speakers in reverberant environments. The paper has the following contributions. We use the direct-path relative transfer function (DP-RTF), an inter-channel feature that encodes acoustic information robust against reverberation, and we propose an online algorithm well suited for estimating DP-RTFs associated with moving audio sources. Another crucial ingredient of the proposed method is its ability to properly assign DP-RTFs to audio-source directions. Towards this goal, we adopt a maximum-likelihood formulation and we propose to use an exponentiated gradient (EG) to efficiently update source-direction estimates starting from their currently available values. The problem of multiple speaker tracking is computationally intractable because the number of possible associations between observed source directions and physical speakers grows exponentially with time. We adopt a Bayesian framework and we propose a variational approximation of the posterior filtering distribution associated with multiple speaker tracking, as well as an efficient variational expectation-maximization (VEM) solver. The proposed online localization and tracking method is thoroughly evaluated using two datasets that contain recordings performed in real environments.

cs.SD↗

arXiv2018

A cascaded multiple-speaker localization and tracking system

This paper presents an online multiple-speaker localization and tracking method, as the INRIA-Perception contribution to the LOCATA Challenge 2018. First, the recursive least-square method is used to adaptively estimate the direct-path relative transfer function as an interchannel localization feature. The feature is assumed to associate with a single speaker at each time-frequency bin. Second, a complex Gaussian mixture model (CGMM) is used as a generative model of the features. The weight of each CGMM component represents the probability that this component corresponds to an active speaker, and is adaptively estimated with an online optimization algorithm. Finally, taking the CGMM component weights as observations, a Bayesian multiple-speaker tracking method based on the variational expectation maximization algorithm is used. The tracker accounts for the variation of active speakers and the localization miss measurements, by introducing speaker birth and sleeping processes. The experiments carried out on the development dataset of the challenge are reported.

cs.SD↗

arXiv2018

Existence of coated inclusions of general shape weakly neutral to multiple fields in two dimensions

A two dimensional inclusion of core-shell structure is neutral to multiple uniform fields if and only if the core and the shell are concentric disks, provided that the conductivity of the matrix is isotropic. An inclusion is said to be neutral if upon its insertion the uniform field is not perturbed at all. In this paper we consider inclusions of core-shell structure of general shape which are weakly neutral to multiple uniform fields. An inclusion is said to be weakly neutral if the field perturbation is mild. We show, by an implicit function theorem, that if the core is a small perturbation of a disk then we can coat it by a shell so that the resulting structure becomes weakly neutral to multiple uniform fields.

math.AP↗

arXiv2018

Construction of weakly neutral inclusions of general shape by imperfect interfaces

Upon insertion of an inclusion into a medium with the uniform field, if the field is not perturbed at all outside the inclusion, then it is called a neutral inclusion. It is called a weakly neutral inclusion if the field is perturbed weakly. The inclusions neutral to multiple uniform fields are of circular shape if the medium is isotropic, and any other shape cannot be neutral. We consider in this paper the problem of constructing inclusions of general shape which are weakly neutral to multiple fields in two dimensions. We show that a simply connected domain satisfying a certain geometric condition can be realized as a weakly neutral inclusion to multiple fields by introducing an imperfect interface parameter on the boundary. The geometric condition on the domain and the imperfect interface parameter are determined by the first coefficient of the conformal mapping from the exterior of the unit disk onto the exterior of the domain. We provide some numerical examples to compare field perturbations by weakly neutral inclusions and perfectly bonding interfaces. They clearly show that the field perturbation by weakly neutral inclusions is much weaker.

math.AP↗

arXiv2018

Multichannel Speech Separation and Enhancement Using the Convolutive Transfer Function

This paper addresses the problem of speech separation and enhancement from multichannel convolutive and noisy mixtures, \emph{assuming known mixing filters}. We propose to perform the speech separation and enhancement task in the short-time Fourier transform domain, using the convolutive transfer function (CTF) approximation. Compared to time-domain filters, CTF has much less taps, consequently it has less near-common zeros among channels and less computational complexity. The work proposes three speech-source recovery methods, namely: i) the multichannel inverse filtering method, i.e. the multiple input/output inverse theorem (MINT), is exploited in the CTF domain, and for the multi-source case, ii) a beamforming-like multichannel inverse filtering method applying single source MINT and using power minimization, which is suitable whenever the source CTFs are not all known, and iii) a constrained Lasso method, where the sources are recovered by minimizing the $\ell_1$-norm to impose their spectral sparsity, with the constraint that the $\ell_2$-norm fitting cost, between the microphone signals and the mixing model involving the unknown source signals, is less than a tolerance. The noise can be reduced by setting a tolerance onto the noise power. Experiments under various acoustic conditions are carried out to evaluate the three proposed methods. The comparison between them as well as with the baseline methods is presented.

cs.SD↗

arXiv2017

On the quantification of the dissolved hydroxyl radicals in the plasma-liquid system using the molecular probe method

Hydroxyl (OH) radical is the most important reactive species produced by the plasma-liquid interactions, and the OH in the liquid phase (dissolved OH radical, OHdis) takes effect in many plasma-based applications due to its high reactivity. Therefore, the quantification of the OHdis in the plasma-liquid system is of great importance, and a molecular probe method usually used for the OHdis detection might be applied. Herein we investigate the validity of using the molecular probe method to estimate the [OHdis] in the plasma-liquid system. Dimethyl sulfoxide is used as the molecular probe to estimate the [OHdis] in an air plasma-liquid system, and the partial OHdis is related to the formed formaldehyde (HCHO) which is the OHdis-induced derivative. The analysis indicates that the true concentration of the OHdis should be estimated from the sum of three terms: the formed HCHO, the existing OH scavengers, and the OHdis generated H2O2. The results show that the measured [HCHO] needs to be corrected since the HCHO destruction is not negligible in the plasma-liquid system. We conclude from the results and the analysis that the molecular probe method generally underestimates the [OHdis] in the plasma-liquid system. If one wants to obtain the true concentration of the OHdis in the plasma-liquid system, one needs to know the destruction behavior of the OHdis-induced derivatives, the information of the OH scavengers (such as hydrated electron, atomic hydrogen besides the molecular probe), and also the knowledge of the OHdis generated H2O2.

physics.chem-ph↗

arXiv2017

On vanishing and localizing of transmission eigenfunctions near singular points: a numerical study

This paper is concerned with the intrinsic geometric structure of interior transmission eigenfunctions arising in wave scattering theory. We numerically show that the aforementioned geometric structure can be much delicate and intriguing. The major findings can be roughly summarized as follows. If there is a cusp on the support of the underlying potential function, then the interior transmission eigenfunction vanishes near the cusp if its interior angle is less than $π$, whereas the interior transmission eigenfunction localizes near the cusp if its interior angle is bigger than $π$. Furthermore, we show that the vanishing and blowup orders are inversely proportional to the interior angle of the cusp: the sharper the angle, the higher the convergence order. Our results are first of its type in the spectral theory for transmission eigenvalue problems, and the existing studies in the literature concentrate more on the intrinsic properties of the transmission eigenvalues instead of the transmission eigenfunctions. Due to the limitedness of the computing resources, our study is by no means exclusive and complete. We consider our study only in a certain geometric setup including corner, curved corner and edge singularities. Nevertheless, we believe that similar results hold for more general cusp singularities and rigorous theoretical justifications are much desirable. Our study enriches the spectral theory for transmission eigenvalue problems. We also discuss its implication to inverse scattering theory.

math.NA↗

arXiv2017

Reconstruction via the intrinsic geometric structures of interior transmission eigenfunctions

We are concerned with the inverse scattering problem of extracting the geometric structures of an unknown/inaccessible inhomogeneous medium by using the corresponding acoustic far-field measurement. Using the intrinsic geometric properties of the so-called interior transmission eigenfunctions, we develop a novel inverse scattering scheme. The proposed method can efficiently capture the cusp singularities of the support of the inhomogeneous medium. If further a priori information is available on the support of the medium, say, it is a convex polyhedron, then one can actually recover its shape. Both theoretical analysis and numerical experiments are provided. Our reconstruction method is new to the literature and opens up a new direction in the study of inverse scattering problems.

math.AP↗

arXiv2017

Multiple-Speaker Localization Based on Direct-Path Features and Likelihood Maximization with Spatial Sparsity Regularization

This paper addresses the problem of multiple-speaker localization in noisy and reverberant environments, using binaural recordings of an acoustic scene. A Gaussian mixture model (GMM) is adopted, whose components correspond to all the possible candidate source locations defined on a grid. After optimizing the GMM-based objective function, given an observed set of binaural features, both the number of sources and their locations are estimated by selecting the GMM components with the largest priors. This is achieved by enforcing a sparse solution, thus favoring a small number of speakers with respect to the large number of initial candidate source locations. An entropy-based penalty term is added to the likelihood, thus imposing sparsity over the set of GMM priors. In addition, the direct-path relative transfer function (DP-RTF) is used to build robust binaural features. The DP-RTF, recently proposed for single-source localization, was shown to be robust to reverberations, since it encodes inter-channel information corresponding to the direct-path of sound propagation. In this paper, we extend the DP-RTF estimation to the case of multiple sources. In the short-time Fourier transform domain, a consistency test is proposed to check whether a set of consecutive frames is associated to the same source or not. Reliable DP-RTF features are selected from the frames that pass the consistency test to be used for source localization. Experiments carried out using both simulation data and real data gathered with a robotic head confirm the efficiency of the proposed multi-source localization method.

cs.SD↗