arXiv subjects

Zhuo Chen

Publications and source records attributed to Zhuo Chen.

At least 397 records · Page 22Linked to original sources

arXiv2020

Neural Speech Separation Using Spatially Distributed Microphones

This paper proposes a neural network based speech separation method using spatially distributed microphones. Unlike with traditional microphone array settings, neither the number of microphones nor their spatial arrangement is known in advance, which hinders the use of conventional multi-channel speech separation neural networks based on fixed size input. To overcome this, a novel network architecture is proposed that interleaves inter-channel processing layers and temporal processing layers. The inter-channel processing layers apply a self-attention mechanism along the channel dimension to exploit the information obtained with a varying number of microphones. The temporal processing layers are based on a bidirectional long short term memory (BLSTM) model and applied to each channel independently. The proposed network leverages information across time and space by stacking these two kinds of layers alternately. Our network estimates time-frequency (TF) masks for each speaker, which are then used to generate enhanced speech signals either with TF masking or beamforming. Speech recognition experimental results show that the proposed method significantly outperforms baseline multi-channel speech separation systems.

eess.AS↗

arXiv2020

Generative Adversarial Zero-shot Learning via Knowledge Graphs

Zero-shot learning (ZSL) is to handle the prediction of those unseen classes that have no labeled training data. Recently, generative methods like Generative Adversarial Networks (GANs) are being widely investigated for ZSL due to their high accuracy, generalization capability and so on. However, the side information of classes used now is limited to text descriptions and attribute annotations, which are in short of semantics of the classes. In this paper, we introduce a new generative ZSL method named KG-GAN by incorporating rich semantics in a knowledge graph (KG) into GANs. Specifically, we build upon Graph Neural Networks and encode KG from two views: class view and attribute view considering the different semantics of KG. With well-learned semantic embeddings for each node (representing a visual category), we leverage GANs to synthesize compelling visual features for unseen classes. According to our evaluation with multiple image classification datasets, KG-GAN can achieve better performance than the state-of-the-art baselines.

cs.CV↗

arXiv2020

Dual-path RNN: efficient long sequence modeling for time-domain single-channel speech separation

Recent studies in deep learning-based speech separation have proven the superiority of time-domain approaches to conventional time-frequency-based methods. Unlike the time-frequency domain approaches, the time-domain separation systems often receive input sequences consisting of a huge number of time steps, which introduces challenges for modeling extremely long sequences. Conventional recurrent neural networks (RNNs) are not effective for modeling such long sequences due to optimization difficulties, while one-dimensional convolutional neural networks (1-D CNNs) cannot perform utterance-level sequence modeling when its receptive field is smaller than the sequence length. In this paper, we propose dual-path recurrent neural network (DPRNN), a simple yet effective method for organizing RNN layers in a deep structure to model extremely long sequences. DPRNN splits the long sequential input into smaller chunks and applies intra- and inter-chunk operations iteratively, where the input length can be made proportional to the square root of the original sequence length in each operation. Experiments show that by replacing 1-D CNN with DPRNN and apply sample-level modeling in the time-domain audio separation network (TasNet), a new state-of-the-art performance on WSJ0-2mix is achieved with a 20 times smaller model than the previous best system.

eess.AS↗

arXiv2020

End-to-end Microphone Permutation and Number Invariant Multi-channel Speech Separation

An important problem in ad-hoc microphone speech separation is how to guarantee the robustness of a system with respect to the locations and numbers of microphones. The former requires the system to be invariant to different indexing of the microphones with the same locations, while the latter requires the system to be able to process inputs with varying dimensions. Conventional optimization-based beamforming techniques satisfy these requirements by definition, while for deep learning-based end-to-end systems those constraints are not fully addressed. In this paper, we propose transform-average-concatenate (TAC), a simple design paradigm for channel permutation and number invariant multi-channel speech separation. Based on the filter-and-sum network (FaSNet), a recently proposed end-to-end time-domain beamforming system, we show how TAC significantly improves the separation performance across various numbers of microphones in noisy reverberant separation tasks with ad-hoc arrays. Moreover, we show that TAC also significantly improves the separation performance with fixed geometry array configuration, further proving the effectiveness of the proposed paradigm in the general problem of multi-microphone speech separation.

eess.AS↗

arXiv2020

A 3D radiation-hydrodynamic AGB binary model

The origin of chemically peculiar stars and non-zero eccentricity in evolved close binaries have been long-standing problems in binary stellar evolution. Answers to these questions may trace back to an intense mass transfer during AGB binary phase. In this work, we use AstroBEAR to solve the 3D radiation-hydrodynamic equations and calculate the mass transfer rate in asymptotic-giant-branch (AGB) binaries that undergo the wind-Roche-lobe-overflow or Bondi-Hoyle-Lyttleton (BHL) accretion. MESA produces the density and temperature of the boundary condition of the AGB star. To improve the resolution of the dynamics of a circumbinary disk, we implement an azimuthal angle-dependent 3D radiation transfer. We consider optically thin cooling and obtain the number density of the coolants by solving Saha equations. One of the goals of this work is to illustrate the transition from the wind-Roche-lobe-overflow to BHL accretion. Both circumbinary disks and spiral structure outflows can appear in the simulations. Circumbinary disks may form when the optical thickness in the equatorial region increases. The increase of the optical thickness is due to the deflected wind. The resulting mass transfer efficiency in our models is up to a factor of 8 times higher than what the standard BHL accretion scenario predicts, and the outflow gains up to 91% of its initial angular momentum when it reaches 1.3 binary separations. Consequently, some AGB binaries may undergo orbit shrinkage, and some will expand. The high mass transfer efficiency is closely related to the presence of the circumbinary disks.

astro-ph.SR↗

arXiv2020

ViP: Virtual Pooling for Accelerating CNN-based Image Classification and Object Detection

In recent years, Convolutional Neural Networks (CNNs) have shown superior capability in visual learning tasks. While accuracy-wise CNNs provide unprecedented performance, they are also known to be computationally intensive and energy demanding for modern computer systems. In this paper, we propose Virtual Pooling (ViP), a model-level approach to improve speed and energy consumption of CNN-based image classification and object detection tasks, with a provable error bound. We show the efficacy of ViP through experiments on four CNN models, three representative datasets, both desktop and mobile platforms, and two visual learning tasks, i.e., image classification and object detection. For example, ViP delivers 2.1x speedup with less than 1.5% accuracy degradation in ImageNet classification on VGG-16, and 1.8x speedup with 0.025 mAP degradation in PASCAL VOC object detection with Faster-RCNN. ViP also reduces mobile GPU and CPU energy consumption by up to 55% and 70%, respectively. As a complementary method to existing acceleration approaches, ViP achieves 1.9x speedup on ThiNet leading to a combined speedup of 5.23x on VGG-16. Furthermore, ViP provides a knob for machine learning practitioners to generate a set of CNN models with varying trade-offs between system speed/energy consumption and accuracy to better accommodate the requirements of their tasks. Code is available at https://github.com/cmu-enyac/VirtualPooling.

cs.CV↗

arXiv2020

Spectrum Intelligent Radio: Technology, Development, and Future Trends

The advent of Industry 4.0 with massive connectivity places significant strains on the current spectrum resources, and challenges the industry and regulators to respond promptly with new disruptive spectrum management strategies. The current radio development, with certain elements of intelligence, is nowhere near showing an agile response to the complex radio environments. Following the line of intelligence, we propose to classify spectrum intelligent radio into three streams: classical signal processing, machine learning (ML), and contextual adaptation. We focus on the ML approach, and propose a new intelligent radio architecture with three hierarchical forms: perception, understanding, and reasoning. The proposed perception method achieves fully blind multi-level spectrum sensing. The understanding method accurately predicts the primary users' coverage across a large area, and the reasoning method performs a near-optimal idle channel selection. Opportunities, challenges, and future visions are also discussed for the realization of a fully intelligent radio.

cs.IT↗

arXiv2019

Advances in Online Audio-Visual Meeting Transcription

This paper describes a system that generates speaker-annotated transcripts of meetings by using a microphone array and a 360-degree camera. The hallmark of the system is its ability to handle overlapped speech, which has been an unsolved problem in realistic settings for over a decade. We show that this problem can be addressed by using a continuous speech separation approach. In addition, we describe an online audio-visual speaker diarization method that leverages face tracking and identification, sound source localization, speaker identification, and, if available, prior speaker information for robustness to various real world challenges. All components are integrated in a meeting transcription framework called SRD, which stands for "separate, recognize, and diarize". Experimental results using recordings of natural meetings involving up to 11 attendees are reported. The continuous speech separation improves a word error rate (WER) by 16.1% compared with a highly tuned beamformer. When a complete list of meeting attendees is available, the discrepancy between WER and speaker-attributed WER is only 1.0%, indicating accurate word-to-speaker association. This increases marginally to 1.6% when 50% of the attendees are unknown to the system.

eess.AS↗

arXiv2019

Solving the Riemann Problem for Realistic Astrophysical Fluids

We present new methods to solve the Riemann problem both exactly and approximately for general equations of state (EoS) to facilitate realistic modeling and understanding of astrophysical flows. The existence and uniqueness of the new exact general EoS Riemann solution can be guaranteed if the EoS is monotone regardless of the physical validity of the EoS. We confirm that: (1) the solution of the new exact general EoS Riemann solver and the solution of the original exact Riemann solver match when calculating perfect gas Euler equations; (2) the solution of the new Harten-Lax-van Leer-Contact (HLLC) general EoS Riemann solver and the solution of the original HLLC Riemann solver match when working with perfect gas EoS; and (3) the solution of the new HLLC general EoS Riemann solver approaches the new exact solution. We solve the EoS with two methods, one is to interpolate 2D EoS tables by the bi-linear interpolation method, and the other is to analytically calculate thermodynamic variables at run-time. The interpolation method is more general as it can work with other monotone and realistic EoS while the analytic EoS solver introduced here works with a relatively idealized EoS. Numerical results confirm that the accuracy of the two EoS solvers is similar. We study the efficiency of these two methods with the HLLC general EoS Riemann solver and find that analytic EoS solver is faster in the test problems. However, we point out that a combination of the two EoS solvers may become favorable in some specific problems. Throughout this research, we assume local thermal equilibrium.

physics.comp-ph↗

arXiv2019

PyKaldi2: Yet another speech toolkit based on Kaldi and PyTorch

We introduce PyKaldi2 speech recognition toolkit implemented based on Kaldi and PyTorch. While similar toolkits are available built on top of the two, a key feature of PyKaldi2 is sequence training with criteria such as MMI, sMBR and MPE. In particular, we implemented the sequence training module with on-the-fly lattice generation during model training in order to simplify the training pipeline. To address the challenging acoustic environments in real applications, PyKaldi2 also supports on-the-fly noise and reverberation simulation to improve the model robustness. With this feature, it is possible to backpropogate the gradients from the sequence-level loss to the front-end feature extraction module, which, hopefully, can foster more research in the direction of joint front-end and backend learning. We performed benchmark experiments on Librispeech, and show that PyKaldi2 can achieve reasonable recognition accuracy. The toolkit is released under the MIT license.

cs.CL↗

arXiv2019

Consistency of the Infrared Variability of Sgr A* over 22 years

We report new infrared measurements of the supermassive black hole at the Galactic Center, Sgr A*, over a decade that was previously inaccessible at these wavelengths. This enables a variability study that addresses variability timescales that are ten times longer than earlier published studies. Sgr A* was initially detected in the near-infrared with adaptive optics observations in 2002. While earlier data exists in form of speckle imaging (1995 - 2005), Sgr A* was not detected in the initial analysis. Here, we improved our speckle holography analysis techniques. This has improved the sensitivity of the resulting speckle images by up to a factor of three. Sgr A* is now detectable in the majority of epochs covering 7 years. The brightness of Sgr A* in the speckle data has an average observed K magnitude of 16.0, which corresponds to a dereddened flux density of $3.4$ mJy. Furthermore, the flat power spectral density (PSD) of Sgr A* between $\sim$80 days and 7 years shows its uncorrelation in time beyond the proposed single power-law break of $\sim$245 minutes. We report that the brightness and its variability is consistent over 22 years. This analysis is based on simulations using Witzel et al. (2018) model to characterize infrared variability from 2006 to 2016. Finally, we note that the 2001 periapse of the extended, dusty object G1 had no apparent effect on the near-infrared emission from accretion flow onto Sgr A*. The result is consistent with G1 being a self-gravitating object rather than a disrupting gas cloud.

astro-ph.GA↗

arXiv2019

Unprecedented variability of Sgr A* in NIR

The electromagnetic counterpart to the Galactic center supermassive black hole, Sgr A*, has been observed in the near-infrared for over 20 years and is known to be highly variable. We report new Keck Telescope observations showing that Sgr A* reached much brighter flux levels in 2019 than ever measured at near-infrared wavelengths. In the K$^\prime$ band, Sgr A* reached flux levels of $\sim6$ mJy, twice the level of the previously observed peak flux from $>13,000$ measurements over 130 nights with the VLT and Keck Telescopes. We also observe a factor of 75 change in flux over a 2-hour time span with no obvious color changes between 1.6 $μ$m and 2.1 $μ$m. The distribution of flux variations observed this year is also significantly different than the historical distribution. Using the most comprehensive statistical model published, the probability of a single night exhibiting peak flux levels observed this year, given historical Keck observations, is less than $0.3\%$. The probability to observe the flux levels similar to all 4 nights of data in 2019 is less than $0.05\%$. This increase in brightness and variability may indicate a period of heightened activity from Sgr A* or a change in its accretion state. It may also indicate that the current model is not sufficient to model Sgr A* at high flux levels and should be updated. Potential physical origins of Sgr A*'s unprecedented brightness may be from changes in the accretion-flow as a result of the star S0-2's closest passage to the black hole in 2018 or from a delayed reaction to the approach of the dusty object G2 in 2014. Additional multi-wavelength observations will be necessary to both monitor Sgr A* for potential state changes and to constrain the physical processes responsible for its current variability.

astro-ph.GA↗

arXiv2019

Hydrodynamic Simulations of Disrupted Planetary Accretion Discs Inside the Core of an AGB Star

Volume complete sky surveys provide evidence for a binary origin for the formation of isolated white dwarfs with magnetic fields in excess of a MegaGauss. Interestingly, not a single high-field magnetic white dwarf has been found in a detached system suggesting that if the progenitors are indeed binaries, the companion must be removed or merge during formation. An origin scenario consistent with observations involves the engulfment, inspiral, and subsequent tidal disruption of a low-mass companion in the interior of a giant star during a common envelope phase. Material from the shredded companion forms a cold accretion disc embedded in the hot ambient around the proto-white dwarf. Entrainment of hot material may evaporate the disc before it can sufficiently amplify the magnetic field, which typically requires at least a few orbits of the disc.Using three-dimensional hydrodynamic simulations of accretion discs with masses between 1 and 10 times the mass of Jupiter inside the core of an Asymptotic Giant Branch star, we find that the discs survive for at least 10 orbits (and likely for 100 orbits), sufficient for strong magnetic fields to develop.

astro-ph.SR↗

arXiv2019

Relativistic redshift of the star S0-2 orbiting the Galactic center supermassive black hole

General Relativity predicts that a star passing close to a supermassive black hole should exhibit a relativistic redshift. We test this using observations of the Galactic center star S0-2. We combine existing spectroscopic and astrometric measurements from 1995-2017, which cover S0-2's 16-year orbit, with measurements in 2018 March to September which cover three events during its closest approach to the black hole. We detect the combination of special relativistic- and gravitational-redshift, quantified using a redshift parameter, $Υ$. Our result, $Υ=0.88 \pm 0.17$, is consistent with General Relativity ($Υ=1$) and excludes a Newtonian model ($Υ=0$ ) with a statistical significance of 5 $σ$.

astro-ph.GA↗

arXiv2019

A Learning-Based Two-Stage Spectrum Sharing Strategy with Multiple Primary Transmit Power Levels

Multi-parameter cognition in a cognitive radio network (CRN) provides a more thorough understanding of the radio environments, and could potentially lead to far more intelligent and efficient spectrum usage for a secondary user. In this paper, we investigate the multi-parameter cognition problem for a CRN where the primary transmitter (PT) radiates multiple transmit power levels, and propose a learning-based two-stage spectrum sharing strategy. We first propose a data-driven/machine learning based multi-level spectrum sensing scheme, including the spectrum learning (Stage I) and prediction (the first part in Stage II). This fully blind sensing scheme does not require any prior knowledge of the PT power characteristics. Then, based on a novel normalized power level alignment metric, we propose two prediction-transmission structures, namely periodic and non-periodic, for spectrum access (the second part in Stage II), which enable the secondary transmitter (ST) to closely follow the PT power level variation. The periodic structure features a fixed prediction interval, while the non-periodic one dynamically determines the interval with a proposed reinforcement learning algorithm to further improve the alignment metric. Finally, we extend the prediction-transmission structure to an online scenario, where the number of PT power levels might change as a consequence of PT adapting to the environment fluctuation or quality of service variation. The simulation results demonstrate the effectiveness of the proposed strategy in various scenarios.

eess.SP↗

arXiv2019

Meeting Transcription Using Virtual Microphone Arrays

We describe a system that generates speaker-annotated transcripts of meetings by using a virtual microphone array, a set of spatially distributed asynchronous recording devices such as laptops and mobile phones. The system is composed of continuous audio stream alignment, blind beamforming, speech recognition, speaker diarization using prior speaker information, and system combination. When utilizing seven input audio streams, our system achieves a word error rate (WER) of 22.3% and comes within 3% of the close-talking microphone WER on the non-overlapping speech segments. The speaker-attributed WER (SAWER) is 26.7%. The relative gains in SAWER over the single-device system are 14.8%, 20.3%, and 22.4% for three, five, and seven microphones, respectively. The presented system achieves a 13.6% diarization error rate when 10% of the speech duration contains more than one speaker. The contribution of each component to the overall performance is also investigated, and we validate the system with experiments on the NIST RT-07 conference meeting test set.

eess.AS↗

arXiv2019

Kapranov's construction of sh Leibniz algebras

Motivated by Kapranov's discovery of an sh Lie algebra structure on the tangent complex of a Kähler manifold and Chen-Stiénon-Xu's construction of sh Leibniz algebras associated with a Lie pair, we find a general method to construct sh Leibniz algebras. Let $\mathcal{A}$ be a commutative dg algebra. Given a derivation of $\mathcal{A}$ valued in a dg module $Ω$, we show that there exist sh Leibniz algebra structures on the dual module of $Ω$. Moreover, we prove that this process establishes a functor from the category of dg module valued derivations to the category of sh Leibniz algebras over $\mathcal{A}$.

math.QA↗

arXiv2019

Atiyah classes of strongly homotopy Lie pairs

The subject of this paper is strongly homotopy (SH) Lie algebras, also known as $L_\infty$-algebras. We extract an intrinsic character, the Atiyah class, which measures the nontriviality of an (SH) Lie algebra $A$ when it is extended to $L$. In fact, given such an SH Lie pair $(L, A)$, and any $A$-module $E$, there associates a canonical cohomology class, the Atiyah class $[α^E]$, which generalizes earlier known Atiyah classes out of Lie algebra pairs. We show that the Atiyah class $[α^{L/A}]$ induces a graded Lie algebra structure on $\operatorname{H}^\bullet_{\mathrm{CE}}(A,L/A[-2])$, and the Atiyah class $[α^E]$ of any $A$-module $E$ induces a Lie algebra module structure on $\operatorname{H}^\bullet_{\mathrm{CE}}(A,E)$. Moreover, Atiyah classes are invariant under gauge equivalent $A$-compatible infinitesimal deformations of $L$.

math.QA↗