arXiv subjects

Yi Ren

Publications and source records attributed to Yi Ren.

At least 217 records · Page 12Linked to original sources

arXiv2021

EMOVIE: A Mandarin Emotion Speech Dataset with a Simple Emotional Text-to-Speech Model

Recently, there has been an increasing interest in neural speech synthesis. While the deep neural network achieves the state-of-the-art result in text-to-speech (TTS) tasks, how to generate a more emotional and more expressive speech is becoming a new challenge to researchers due to the scarcity of high-quality emotion speech dataset and the lack of advanced emotional TTS model. In this paper, we first briefly introduce and publicly release a Mandarin emotion speech dataset including 9,724 samples with audio files and its emotion human-labeled annotation. After that, we propose a simple but efficient architecture for emotional speech synthesis called EMSpeech. Unlike those models which need additional reference audio as input, our model could predict emotion labels just from the input text and generate more expressive speech conditioned on the emotion embedding. In the experiment phase, we first validate the effectiveness of our dataset by an emotion classification task. Then we train our model on the proposed dataset and conduct a series of subjective evaluations. Finally, by showing a comparable performance in the emotional speech synthesis task, we successfully demonstrate the ability of the proposed model.

cs.CL↗

arXiv2021

WSRGlow: A Glow-based Waveform Generative Model for Audio Super-Resolution

Audio super-resolution is the task of constructing a high-resolution (HR) audio from a low-resolution (LR) audio by adding the missing band. Previous methods based on convolutional neural networks and mean squared error training objective have relatively low performance, while adversarial generative models are difficult to train and tune. Recently, normalizing flow has attracted a lot of attention for its high performance, simple training and fast inference. In this paper, we propose WSRGlow, a Glow-based waveform generative model to perform audio super-resolution. Specifically, 1) we integrate WaveNet and Glow to directly maximize the exact likelihood of the target HR audio conditioned on LR information; and 2) to exploit the audio information from low-resolution audio, we propose an LR audio encoder and an STFT encoder, which encode the LR information from the time domain and frequency domain respectively. The experimental results show that the proposed model is easier to train and outperforms the previous works in terms of both objective and perceptual quality. WSRGlow is also the first model to produce 48kHz waveforms from 12kHz LR audio.

cs.SD↗

arXiv2021

Decentralized Attribution of Generative Models

Growing applications of generative models have led to new threats such as malicious personation and digital copyright infringement. One solution to these threats is model attribution, i.e., the identification of user-end models where the contents under question are generated from. Existing studies showed empirical feasibility of attribution through a centralized classifier trained on all user-end models. However, this approach is not scalable in reality as the number of models ever grows. Neither does it provide an attributability guarantee. To this end, this paper studies decentralized attribution, which relies on binary classifiers associated with each user-end model. Each binary classifier is parameterized by a user-specific key and distinguishes its associated model distribution from the authentic data distribution. We develop sufficient conditions of the keys that guarantee an attributability lower bound. Our method is validated on MNIST, CelebA, and FFHQ datasets. We also examine the trade-off between generation quality and robustness of attribution against adversarial post-processes.

cs.CV↗

arXiv2021

Graph Intention Network for Click-through Rate Prediction in Sponsored Search

Estimating click-through rate (CTR) accurately has an essential impact on improving user experience and revenue in sponsored search. For CTR prediction model, it is necessary to make out user real-time search intention. Most of the current work is to mine their intentions based on user real-time behaviors. However, it is difficult to capture the intention when user behaviors are sparse, causing the behavior sparsity problem. Moreover, it is difficult for user to jump out of their specific historical behaviors for possible interest exploration, namely weak generalization problem. We propose a new approach Graph Intention Network (GIN) based on co-occurrence commodity graph to mine user intention. By adopting multi-layered graph diffusion, GIN enriches user behaviors to solve the behavior sparsity problem. By introducing co-occurrence relationship of commodities to explore the potential preferences, the weak generalization problem is also alleviated. To the best of our knowledge, the GIN method is the first to introduce graph learning for user intention mining in CTR prediction and propose end-to-end joint training of graph learning and CTR prediction tasks in sponsored search. At present, GIN has achieved excellent offline results on the real-world data of the e-commerce platform outperforming existing deep learning models, and has been running stable tests online and achieved significant CTR improvements.

cs.LG↗

arXiv2021

FastLR: Non-Autoregressive Lipreading Model with Integrate-and-Fire

Lipreading is an impressive technique and there has been a definite improvement of accuracy in recent years. However, existing methods for lipreading mainly build on autoregressive (AR) model, which generate target tokens one by one and suffer from high inference latency. To breakthrough this constraint, we propose FastLR, a non-autoregressive (NAR) lipreading model which generates all target tokens simultaneously. NAR lipreading is a challenging task that has many difficulties: 1) the discrepancy of sequence lengths between source and target makes it difficult to estimate the length of the output sequence; 2) the conditionally independent behavior of NAR generation lacks the correlation across time which leads to a poor approximation of target distribution; 3) the feature representation ability of encoder can be weak due to lack of effective alignment mechanism; and 4) the removal of AR language model exacerbates the inherent ambiguity problem of lipreading. Thus, in this paper, we introduce three methods to reduce the gap between FastLR and AR model: 1) to address challenges 1 and 2, we leverage integrate-and-fire (I\&F) module to model the correspondence between source video frames and output text sequence. 2) To tackle challenge 3, we add an auxiliary connectionist temporal classification (CTC) decoder to the top of the encoder and optimize it with extra CTC loss. We also add an auxiliary autoregressive decoder to help the feature extraction of encoder. 3) To overcome challenge 4, we propose a novel Noisy Parallel Decoding (NPD) for I\&F and bring Byte-Pair Encoding (BPE) into lipreading. Our experiments exhibit that FastLR achieves the speedup up to 10.97$\times$ comparing with state-of-the-art lipreading model with slight WER absolute increase of 1.5\% and 5.5\% on GRID and LRS2 lipreading datasets respectively, which demonstrates the effectiveness of our proposed method.

eess.AS↗

arXiv2021

Red supergiants in M31 and M33 II. The Mass Loss Rate

Mass loss is an important activity for red supergiants (RSGs) which can influence their evolution and final fate. Previous estimations of mass loss rates (MLRs) of RSGs exhibit significant dispersion due to the difference in method and the incompleteness of sample. With the improved quality and depth of the surveys including the UKIRT/WFCAM observation in near infrared, LGGS and PS1 in optical, a rather complete sample of RSGs is identified in M31 and M33 according to their brightness and colors. For about 2000 objects in either galaxy from this ever largest sample, the MLR is derived by fitting the observational optical-to-mid infrared spectral energy distribution (SED) with the DUSTY code of a 1-D dust radiative transfer model. The average MLR of RSGs is found to be around $2.0\times10^{-5}{\text{M}_\odot}/\text{yr}$ with a gas-to-dust ratio of 100, which yields a total contribution to the interstellar dust by RSGs of about $1.1\times10^{-3}{\text{M}_\odot}/\text{yr}$ in M31 and $6.0 \times10^{-4}{\text{M}_\odot}/\text{yr}$ in M33, a non-negligible source in comparison with evolved low-mass stars. The MLRs are divided into three types by the dust properties, i.e. amorphous silicate, amorphous carbon and optically thin, and the relations of MLR with stellar parameters, infrared flux and colors are discussed and compared with previous works for the silicate and carbon dust group respectively.

astro-ph.SR↗

arXiv2021

Evolved Massive Stars at Low-metallicity IV. Using 1.6 $μ$m "H-bump" to identify red supergiant stars: a case study of NGC 6822

We present a case study of using a novel method to identify red supergiant (RSG) candidates in NGC 6822, based on their 1.6 $μ$m "H-bump". We collected 32 bands of photometric data for NGC 6822 ranging from optical to MIR. By using the theoretical spectra from MARCS, we demonstrate that there is a prominent difference around 1.6 $μ$m ("H-bump") between low-surface-gravity (LSG) and high-surface-gravity (HSG) targets. Taking advantage of this feature, we identify efficient color-color diagrams (CCDs) of rzH and rzK to separate HSG and LSG targets from crossmatching of optical and NIR data. Moreover, synthetic photometry from ATLAS9 also give similar results. Further separating RSG candidates from the rest of the LSG candidates is done by using semi-empirical criteria on NIR CMDs and resulted in 323 RSG candidates. Meanwhile, the simulation of foreground stars from Besançon models also indicates that our selection criteria is largely free from the contamination of Galactic giants. In addition to the "H-bump" method, we also use the traditional BVR method as a comparison and/or supplement, by applying a slightly aggressive cut to select as much as possible RSG candidates (358 targets). Furthermore, the Gaia astrometric solution is used to constrain the sample, where 181 and 193 targets were selected from the "H-bump" and BVR method, respectively. The percentages of selected targets in both methods are similar as $\sim$60\%, indicating the comparable accuracy of the two methods. In total, there are 234 RSG candidates after combining targets from both methods with 140 ($\sim$60\%) of them in common. The final RSG candidates are in the expected locations on the MIR CMDs, while the spatial distribution is also coincident with the FUV-selected star formation regions, suggesting the selection is reasonable and reliable.

astro-ph.SR↗

arXiv2021

Edge and sublayer degrees of freedom for phosphorene nanoribbons with twofold-degenerate edge bands via electric field

For the pristine phosphorene nanoribbons (PNRs) with edge states, there exist two categories of edge bands near the Fermi energy (EF), i.e., the shuttle-shaped twofold-degenerate and the near-flat simple degenerate edge bands. However, the usual experimental measurement may not distinguish the difference between the two categories of edge bands. Here we study the varying rule for the edge bands of PNRs under an external electrostatic field. By using the KWANT code based on the tight-binding approach, we find that the twofold-degenerate edge bands can be divided into two separated shuttles until the degeneracy is completely removed and a gap near EFis opened under a sufficiently strong in-plane electric field. Importantly, each shuttle from the ribbon upper or lower edge outmost atoms is identified according to the local density of states. However, under a small off-plane field the shuttle-shaped bands are easily induced into two near-flat bands contributed from the edge atoms of the top and bottom sublayers, respectively. The evidence provides the edge and sublayer degrees of freedom (DOF) for the PNRs with shuttle-shaped edge bands, of which is obviously different from another category PNRs intrinsically with near-flat edge bands. This is because that the former category of ribbons solely have four zigzag-like atomic configurations at the edges in each unit cell, which also results in that the property is robust against the point defect in the ribbon center area. As an application, furthermore, based on this issue we propose a homogenous junction of a shuttle-edge-band PNR attached by two electric gates. Interestingly, the transport property of the junction with field manipulation well reflects the characteristics of the two DOFs. These findings may provide a further understanding on PNRs and initiate new developments in PNR-based electronics.

cond-mat.mes-hall↗

arXiv2021

Practical Quasi-Newton Methods for Training Deep Neural Networks

We consider the development of practical stochastic quasi-Newton, and in particular Kronecker-factored block-diagonal BFGS and L-BFGS methods, for training deep neural networks (DNNs). In DNN training, the number of variables and components of the gradient $n$ is often of the order of tens of millions and the Hessian has $n^2$ elements. Consequently, computing and storing a full $n \times n$ BFGS approximation or storing a modest number of (step, change in gradient) vector pairs for use in an L-BFGS implementation is out of the question. In our proposed methods, we approximate the Hessian by a block-diagonal matrix and use the structure of the gradient and Hessian to further approximate these blocks, each of which corresponds to a layer, as the Kronecker product of two much smaller matrices. This is analogous to the approach in KFAC, which computes a Kronecker-factored block-diagonal approximation to the Fisher matrix in a stochastic natural gradient method. Because the indefinite and highly variable nature of the Hessian in a DNN, we also propose a new damping approach to keep the upper as well as the lower bounds of the BFGS and L-BFGS approximations bounded. In tests on autoencoder feed-forward neural network models with either nine or thirteen layers applied to three datasets, our methods outperformed or performed comparably to KFAC and state-of-the-art first-order stochastic methods.

cs.LG↗

arXiv2020

DenoiSpeech: Denoising Text to Speech with Frame-Level Noise Modeling

While neural-based text to speech (TTS) models can synthesize natural and intelligible voice, they usually require high-quality speech data, which is costly to collect. In many scenarios, only noisy speech of a target speaker is available, which presents challenges for TTS model training for this speaker. Previous works usually address the challenge using two methods: 1) training the TTS model using the speech denoised with an enhancement model; 2) taking a single noise embedding as input when training with noisy speech. However, they usually cannot handle speech with real-world complicated noise such as those with high variations along time. In this paper, we develop DenoiSpeech, a TTS system that can synthesize clean speech for a speaker with noisy speech data. In DenoiSpeech, we handle real-world noisy speech by modeling the fine-grained frame-level noise with a noise condition module, which is jointly trained with the TTS model. Experimental results on real-world data show that DenoiSpeech outperforms the previous two methods by 0.31 and 0.66 MOS respectively.

eess.AS↗

arXiv2020

UWSpeech: Speech to Speech Translation for Unwritten Languages

Existing speech to speech translation systems heavily rely on the text of target language: they usually translate source language either to target text and then synthesize target speech from text, or directly to target speech with target text for auxiliary training. However, those methods cannot be applied to unwritten target languages, which have no written text or phoneme available. In this paper, we develop a translation system for unwritten languages, named as UWSpeech, which converts target unwritten speech into discrete tokens with a converter, and then translates source-language speech into target discrete tokens with a translator, and finally synthesizes target speech from target discrete tokens with an inverter. We propose a method called XL-VAE, which enhances vector quantized variational autoencoder (VQ-VAE) with cross-lingual (XL) speech recognition, to train the converter and inverter of UWSpeech jointly. Experiments on Fisher Spanish-English conversation translation dataset show that UWSpeech outperforms direct translation and VQ-VAE baseline by about 16 and 10 BLEU points respectively, which demonstrate the advantages and potentials of UWSpeech.

eess.AS↗

arXiv2020

Evolved Massive Stars at Low-metallicity III. A Source Catalog for the Large Magellanic Cloud

We present a clean, magnitude-limited (IRAC1 or WISE1$\leq$15.0 mag) multiwavelength source catalog for the LMC. The catalog was built upon crossmatching ($1''$) and deblending ($3''$) between the SEIP source list and Gaia DR2, with strict constraints on the Gaia astrometric solution to remove the foreground contamination. The catalog contains 197,004 targets in 52 different bands including 2 ultraviolet, 21 optical, and 29 infrared bands. Additional information about radial velocities and spectral/photometric classifications were collected from the literature. The bright end of our sample is mostly comprised of blue helium-burning stars (BHeBs) and red HeBs with inevitable contamination of main sequence stars at the blue end. After applying modified magnitude and color cuts based on previous studies, we identify and rank 2,974 RSG, 508 YSG, and 4,786 BSG candidates in the LMC in six CMDs. The comparison between the CMDs of the LMC and SMC indicates that the most distinct difference appears at the bright red end of the optical and near-infrared CMDs, where the cool evolved stars (e.g., RSGs, AGB, and RGs) are located, which is likely due to the effect of metallicity and SFH. Further quantitative comparison of colors of massive star candidates in equal absolute magnitude bins suggests that, there is basically no difference for the BSG candidates, but large discrepancy for the RSG candidates as LMC targets are redder than the SMC ones, which may be due to the combined effect of metallicity on both spectral type and mass-loss rate, and also the age effect. The $T_{\rm eff}$ of massive star populations are also derived from reddening-free color of $(J-K_{\rm S})_0$. The $T_{\rm eff}$ ranges are $3500 8000$ K for BSG population, with larger uncertainties towards the hotter stars.

astro-ph.SR↗

arXiv2020

Improving Redundancy Availability: Dynamic Subtasks Modulation for Robots with Redundancy Insufficiency

This work presents an approach for robots to suitably carry out complex applications characterized by the presence of multiple additional constraints or subtasks (e.g. obstacle and self-collision avoidance) but subject to redundancy insufficiency. The proposed approach, based on a novel subtask merging strategy, enforces all subtasks in due course by dynamically modulating a virtual secondary task, where the task status and soft priority are incorporated to improve the overall efficiency of redundancy resolution. The proposed approach greatly improves the redundancy availability by unitizing and deploying subtasks in a fine-grained and compact manner. We build up our control framework on the null space projection, which guarantees the execution of subtasks does not interfere with the primary task. Experimental results on two case studies are presented to show the performance of our approach.

cs.RO↗

arXiv2020

SongMASS: Automatic Song Writing with Pre-training and Alignment Constraint

Automatic song writing aims to compose a song (lyric and/or melody) by machine, which is an interesting topic in both academia and industry. In automatic song writing, lyric-to-melody generation and melody-to-lyric generation are two important tasks, both of which usually suffer from the following challenges: 1) the paired lyric and melody data are limited, which affects the generation quality of the two tasks, considering a lot of paired training data are needed due to the weak correlation between lyric and melody; 2) Strict alignments are required between lyric and melody, which relies on specific alignment modeling. In this paper, we propose SongMASS to address the above challenges, which leverages masked sequence to sequence (MASS) pre-training and attention based alignment modeling for lyric-to-melody and melody-to-lyric generation. Specifically, 1) we extend the original sentence-level MASS pre-training to song level to better capture long contextual information in music, and use a separate encoder and decoder for each modality (lyric or melody); 2) we leverage sentence-level attention mask and token-level attention constraint during training to enhance the alignment between lyric and melody. During inference, we use a dynamic programming strategy to obtain the alignment between each word/syllable in lyric and note in melody. We pre-train SongMASS on unpaired lyric and melody datasets, and both objective and subjective evaluations demonstrate that SongMASS generates lyric and melody with significantly better quality than the baseline method without pre-training or alignment constraint.

cs.SD↗

arXiv2020

Inductive Bias and Language Expressivity in Emergent Communication

Referential games and reconstruction games are the most common game types for studying emergent languages. We investigate how the type of the language game affects the emergent language in terms of: i) language compositionality and ii) transfer of an emergent language to a task different from its origin, which we refer to as language expressivity. With empirical experiments on a handcrafted symbolic dataset, we show that languages emerged from different games have different compositionality and further different expressivity.

cs.CL↗

arXiv2020

Antibunched photon-pair source based on photon blockade effect in a nondegenerate optical parametric oscillator

Nonclassical light sources, such as correlated photon-pairs, play an important role in quantum optics and quantum information processing systems. This study proposes a process to generate antibunched photon-pairs in a nondegenerate optical parametric oscillator. It is found that when the parameters of the system satisfy certain conditions, the generated photons in subharmonic modes exhibit a strong antibunching behavior and are strongly correlated with one another. In particular, the average photon-pair number is resonantly enhanced. It is also observed that the conventional photon blockade contributes to this phenomenon. In addition, it is interesting to note that fundamental mode photons can blockade the subharmonic mode photons. We refer to this phenomenon as a heterogeneous photon blockade.

quant-ph↗

arXiv2020

TSSRGCN: Temporal Spectral Spatial Retrieval Graph Convolutional Network for Traffic Flow Forecasting

Traffic flow forecasting is of great significance for improving the efficiency of transportation systems and preventing emergencies. Due to the highly non-linearity and intricate evolutionary patterns of short-term and long-term traffic flow, existing methods often fail to take full advantage of spatial-temporal information, especially the various temporal patterns with different period shifting and the characteristics of road segments. Besides, the globality representing the absolute value of traffic status indicators and the locality representing the relative value have not been considered simultaneously. This paper proposes a neural network model that focuses on the globality and locality of traffic networks as well as the temporal patterns of traffic data. The cycle-based dilated deformable convolution block is designed to capture different time-varying trends on each node accurately. Our model can extract both global and local spatial information since we combine two graph convolutional network methods to learn the representations of nodes and edges. Experiments on two real-world datasets show that the model can scrutinize the spatial-temporal correlation of traffic data, and its performance is better than the compared state-of-the-art methods. Further analysis indicates that the locality and globality of the traffic networks are critical to traffic flow prediction and the proposed TSSRGCN model can adapt to the various temporal traffic patterns.

cs.LG↗

arXiv2020

Red Supergiants in M31 and M33 I. The Complete Sample

The aim of this paper is to establish a complete sample of red supergiants (RSGs) in M31 and M33. The member stars of the two galaxies are selected from the near-infrared (NIR) point sources after removing the foreground dwarfs from their obvious branch in the $J-H/H-K$ diagram with the archival photometric data taken by the UKIRT/WFCAM. This separation by NIR colors of dwarfs from giants is confirmed by the optical/infrared color-color diagrams ($r-z/z-H$ and $B-V/V-R$), and the Gaia measurement of parallax and proper motion. The RSGs are then identified by their outstanding location in the members' $J-K/K$ diagram due to high luminosity and low effective temperature. The resultant sample has 5,498 and 3,055 RSGs in M31 and M33 respectively, which should be complete because the lower limiting $K$ magnitude of RSGs in both cases is brighter than the complete magnitude of the UKIRT photometry. Analysis of the control fields finds that the pollution rate in the RSGs sample is less than 1\%. The by-product is the complete sample of oxygen-rich asymptotic giant branch stars (AGBs), carbon-rich AGBs, thermally pulsing AGBs and extreme AGBs. In addition, the tip-RGB is determined together with its implication on the distance modulus to M31 and M33.

astro-ph.SR↗