arXiv subjects

Yi Ren

Publications and source records attributed to Yi Ren.

At least 235 records · Page 13Linked to original sources

arXiv2020

Invariant Deep Compressible Covariance Pooling for Aerial Scene Categorization

Learning discriminative and invariant feature representation is the key to visual image categorization. In this article, we propose a novel invariant deep compressible covariance pooling (IDCCP) to solve nuisance variations in aerial scene categorization. We consider transforming the input image according to a finite transformation group that consists of multiple confounding orthogonal matrices, such as the D4 group. Then, we adopt a Siamese-style network to transfer the group structure to the representation space, where we can derive a trivial representation that is invariant under the group action. The linear classifier trained with trivial representation will also be possessed with invariance. To further improve the discriminative power of representation, we extend the representation to the tensor space while imposing orthogonal constraints on the transformation matrix to effectively reduce feature dimensions. We conduct extensive experiments on the publicly released aerial scene image data sets and demonstrate the superiority of this method compared with state-of-the-art methods. In particular, with using ResNet architecture, our IDCCP model can reduce the dimension of the tensor representation by about 98% without sacrificing accuracy (i.e., <0.5%).

cs.CV↗

arXiv2020

When Shall I Be Empathetic? The Utility of Empathetic Parameter Estimation in Multi-Agent Interactions

Human-robot interactions (HRI) can be modeled as dynamic or differential games with incomplete information, where each agent holds private reward parameters. Due to the open challenge in finding perfect Bayesian equilibria of such games, existing studies often consider approximated solutions composed of parameter estimation and motion planning steps, in order to decouple the belief and physical dynamics. In parameter estimation, current approaches often assume that the reward parameters of the robot are known by the humans. We argue that by falsely conditioning on this assumption, the robot performs non-empathetic estimation of the humans' parameters, leading to undesirable values even in the simplest interactions. We test this argument by studying a two-vehicle uncontrolled intersection case with short reaction time. Results show that when both agents are unknowingly aggressive (or non-aggressive), empathy leads to more effective parameter estimation and higher reward values, suggesting that empathy is necessary when the true parameters of agents mismatch with their common belief. The proposed estimation and planning algorithms are therefore more robust than the existing approaches, by fully acknowledging the nature of information asymmetry in HRI. Lastly, we introduce value approximation techniques for real-time execution of the proposed algorithms.

cs.RO↗

arXiv2020

PopMAG: Pop Music Accompaniment Generation

In pop music, accompaniments are usually played by multiple instruments (tracks) such as drum, bass, string and guitar, and can make a song more expressive and contagious by arranging together with its melody. Previous works usually generate multiple tracks separately and the music notes from different tracks not explicitly depend on each other, which hurts the harmony modeling. To improve harmony, in this paper, we propose a novel MUlti-track MIDI representation (MuMIDI), which enables simultaneous multi-track generation in a single sequence and explicitly models the dependency of the notes from different tracks. While this greatly improves harmony, unfortunately, it enlarges the sequence length and brings the new challenge of long-term music modeling. We further introduce two new techniques to address this challenge: 1) We model multiple note attributes (e.g., pitch, duration, velocity) of a musical note in one step instead of multiple steps, which can shorten the length of a MuMIDI sequence. 2) We introduce extra long-context as memory to capture long-term dependency in music. We call our system for pop music accompaniment generation as PopMAG. We evaluate PopMAG on multiple datasets (LMD, FreeMidi and CPMD, a private dataset of Chinese pop songs) with both subjective and objective metrics. The results demonstrate the effectiveness of PopMAG for multi-track harmony modeling and long-term context modeling. Specifically, PopMAG wins 42\%/38\%/40\% votes when comparing with ground truth musical pieces on LMD, FreeMidi and CPMD datasets respectively and largely outperforms other state-of-the-art music accompaniment generation models and multi-track MIDI representations in terms of subjective and objective metrics.

cs.SD↗

arXiv2020

LRSpeech: Extremely Low-Resource Speech Synthesis and Recognition

Speech synthesis (text to speech, TTS) and recognition (automatic speech recognition, ASR) are important speech tasks, and require a large amount of text and speech pairs for model training. However, there are more than 6,000 languages in the world and most languages are lack of speech training data, which poses significant challenges when building TTS and ASR systems for extremely low-resource languages. In this paper, we develop LRSpeech, a TTS and ASR system under the extremely low-resource setting, which can support rare languages with low data cost. LRSpeech consists of three key techniques: 1) pre-training on rich-resource languages and fine-tuning on low-resource languages; 2) dual transformation between TTS and ASR to iteratively boost the accuracy of each other; 3) knowledge distillation to customize the TTS model on a high-quality target-speaker voice and improve the ASR model on multiple voices. We conduct experiments on an experimental language (English) and a truly low-resource language (Lithuanian) to verify the effectiveness of LRSpeech. Experimental results show that LRSpeech 1) achieves high quality for TTS in terms of both intelligibility (more than 98% intelligibility rate) and naturalness (above 3.5 mean opinion score (MOS)) of the synthesized speech, which satisfy the requirements for industrial deployment, 2) achieves promising recognition accuracy for ASR, and 3) last but not least, uses extremely low-resource training data. We also conduct comprehensive analyses on LRSpeech with different amounts of data resources, and provide valuable insights and guidances for industrial deployment. We are currently deploying LRSpeech into a commercialized cloud speech service to support TTS on more rare languages.

eess.AS↗

arXiv2020

MultiSpeech: Multi-Speaker Text to Speech with Transformer

Transformer-based text to speech (TTS) model (e.g., Transformer TTS~\cite{li2019neural}, FastSpeech~\cite{ren2019fastspeech}) has shown the advantages of training and inference efficiency over RNN-based model (e.g., Tacotron~\cite{shen2018natural}) due to its parallel computation in training and/or inference. However, the parallel computation increases the difficulty while learning the alignment between text and speech in Transformer, which is further magnified in the multi-speaker scenario with noisy data and diverse speakers, and hinders the applicability of Transformer for multi-speaker TTS. In this paper, we develop a robust and high-quality multi-speaker Transformer TTS system called MultiSpeech, with several specially designed components/techniques to improve text-to-speech alignment: 1) a diagonal constraint on the weight matrix of encoder-decoder attention in both training and inference; 2) layer normalization on phoneme embedding in encoder to better preserve position information; 3) a bottleneck in decoder pre-net to prevent copy between consecutive speech frames. Experiments on VCTK and LibriTTS multi-speaker datasets demonstrate the effectiveness of MultiSpeech: 1) it synthesizes more robust and better quality multi-speaker voice than naive Transformer based TTS; 2) with a MutiSpeech model as the teacher, we obtain a strong multi-speaker FastSpeech model with almost zero quality degradation while enjoying extremely fast inference speed.

eess.AS↗

arXiv2020

Degrees of symmetric Grothendieck polynomials and Castelnuovo-Mumford regularity

We give an explicit formula for the degree of the Grothendieck polynomial of a Grassmannian permutation and a closely related formula for the Castelnuovo-Mumford regularity of the Schubert determinantal ideal of a Grassmannian permutation. We then provide a counterexample to a conjecture of Kummini-Lakshmibai-Sastry-Seshadri on a formula for regularities of standard open patches of particular Grassmannian Schubert varieties and show that our work gives rise to an alternate explicit formula in these cases. We end with a new conjecture on the regularities of standard open patches of arbitrary Grassmannian Schubert varieties.

math.CO↗

arXiv2020

Almost Unsupervised Text to Speech and Automatic Speech Recognition

Text to speech (TTS) and automatic speech recognition (ASR) are two dual tasks in speech processing and both achieve impressive performance thanks to the recent advance in deep learning and large amount of aligned speech and text data. However, the lack of aligned data poses a major practical problem for TTS and ASR on low-resource languages. In this paper, by leveraging the dual nature of the two tasks, we propose an almost unsupervised learning method that only leverages few hundreds of paired data and extra unpaired data for TTS and ASR. Our method consists of the following components: (1) a denoising auto-encoder, which reconstructs speech and text sequences respectively to develop the capability of language modeling both in speech and text domain; (2) dual transformation, where the TTS model transforms the text $y$ into speech $\hat{x}$, and the ASR model leverages the transformed pair $(\hat{x},y)$ for training, and vice versa, to boost the accuracy of the two tasks; (3) bidirectional sequence modeling, which addresses error propagation especially in the long speech and text sequence when training with few paired data; (4) a unified model structure, which combines all the above components for TTS and ASR based on Transformer model. Our method achieves 99.84% in terms of word level intelligible rate and 2.68 MOS for TTS, and 11.7% PER for ASR on LJSpeech dataset, by leveraging only 200 paired speech and text data (about 20 minutes audio), together with extra unpaired speech and text data.

eess.AS↗

arXiv2020

Task-Level Curriculum Learning for Non-Autoregressive Neural Machine Translation

Non-autoregressive translation (NAT) achieves faster inference speed but at the cost of worse accuracy compared with autoregressive translation (AT). Since AT and NAT can share model structure and AT is an easier task than NAT due to the explicit dependency on previous target-side tokens, a natural idea is to gradually shift the model training from the easier AT task to the harder NAT task. To smooth the shift from AT training to NAT training, in this paper, we introduce semi-autoregressive translation (SAT) as intermediate tasks. SAT contains a hyperparameter k, and each k value defines a SAT task with different degrees of parallelism. Specially, SAT covers AT and NAT as its special cases: it reduces to AT when k = 1 and to NAT when k = N (N is the length of target sentence). We design curriculum schedules to gradually shift k from 1 to N, with different pacing functions and number of tasks trained at the same time. We called our method as task-level curriculum learning for NAT (TCL-NAT). Experiments on IWSLT14 De-En, IWSLT16 En-De, WMT14 En-De and De-En datasets show that TCL-NAT achieves significant accuracy improvements over previous NAT baselines and reduces the performance gap between NAT and AT models to 1-2 BLEU points, demonstrating the effectiveness of our proposed method.

cs.CL↗

arXiv2020

DeepSinger: Singing Voice Synthesis with Data Mined From the Web

In this paper, we develop DeepSinger, a multi-lingual multi-singer singing voice synthesis (SVS) system, which is built from scratch using singing training data mined from music websites. The pipeline of DeepSinger consists of several steps, including data crawling, singing and accompaniment separation, lyrics-to-singing alignment, data filtration, and singing modeling. Specifically, we design a lyrics-to-singing alignment model to automatically extract the duration of each phoneme in lyrics starting from coarse-grained sentence level to fine-grained phoneme level, and further design a multi-lingual multi-singer singing model based on a feed-forward Transformer to directly generate linear-spectrograms from lyrics, and synthesize voices using Griffin-Lim. DeepSinger has several advantages over previous SVS systems: 1) to the best of our knowledge, it is the first SVS system that directly mines training data from music websites, 2) the lyrics-to-singing alignment model further avoids any human efforts for alignment labeling and greatly reduces labeling cost, 3) the singing model based on a feed-forward Transformer is simple and efficient, by removing the complicated acoustic feature modeling in parametric synthesis and leveraging a reference encoder to capture the timbre of a singer from noisy singing data, and 4) it can synthesize singing voices in multiple languages and multiple singers. We evaluate DeepSinger on our mined singing dataset that consists of about 92 hours data from 89 singers on three languages (Chinese, Cantonese and English). The results demonstrate that with the singing data purely mined from the Web, DeepSinger can synthesize high-quality singing voices in terms of both pitch accuracy and voice naturalness (footnote: Our audio samples are shown in https://speechresearch.github.io/deepsinger/.)

eess.AS↗

arXiv2020

On Granulation and Irregular Variation of Red Supergiants

The mechanism and characteristics of the irregular variations of red supergiants (RSGs) are studied based on the RSG samples in Small Magellanic Cloud (SMC), Large Magellanic Cloud (LMC) and M31. With the time-series data from All-Sky Automated Survey for SuperNovae (ASAS-SN) and Intermediate Palomar Transient Factory survey, we use the continuous time autoregressive moving average model to estimate the variability features of the light curves and their power spectral density. The characteristic evolution timescale and amplitude of granulations are further derived from fitting the posterior power spectral density with the COR function, which is a Harvey-like granulation model. The consistency of theoretical predictions and results is checked to verify the correctness of the assumption that granulations on RSGs contribute to irregular variation. The relations between granulation and stellar parameters are obtained and compared with the results of red giant branch stars and Betelgeuse. It is found that the relations are in agreement with predictions from basic physical process of granulation and fall close to the extrapolated relations of RGB stars. The granulations in most of the RSGs evolve at a timescale of several days to a year with the characteristic amplitude of 10-1000 mmag. The results imply that the irregular variations of RSGs can be attributed to the evolution of granulations. When comparing the results from SMC, LMC and M31, the timescale and amplitude of granulation seem to increase with metallicity. The analytical relations of the granulation parameters with stellar parameters are derived for the RSG sample of each galaxy.

astro-ph.SR↗

arXiv2020

Targeting SARS-CoV-2 with AI- and HPC-enabled Lead Generation: A First Data Release

Researchers across the globe are seeking to rapidly repurpose existing drugs or discover new drugs to counter the the novel coronavirus disease (COVID-19) caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). One promising approach is to train machine learning (ML) and artificial intelligence (AI) tools to screen large numbers of small molecules. As a contribution to that effort, we are aggregating numerous small molecules from a variety of sources, using high-performance computing (HPC) to computer diverse properties of those molecules, using the computed properties to train ML/AI models, and then using the resulting models for screening. In this first data release, we make available 23 datasets collected from community sources representing over 4.2 B molecules enriched with pre-computed: 1) molecular fingerprints to aid similarity searches, 2) 2D images of molecules to enable exploration and application of image-based deep learning methods, and 3) 2D and 3D molecular descriptors to speed development of machine learning models. This data release encompasses structural information on the 4.2 B molecules and 60 TB of pre-computed data. Future releases will expand the data to include more detailed molecular simulations, computed models, and other products.

q-bio.BM↗

arXiv2020

Evolved Massive Stars at Low-metallicity II. Red Supergiant Stars in the Small Magellanic Cloud

We present the most comprehensive RSG sample for the SMC up to now, including 1,239 RSG candidates. The initial sample is derived based on a source catalog for the SMC with conservative ranking. Additional spectroscopic RSGs are retrieved from the literature, as well as RSG candidates selected from the inspection of CMDs. We estimate that there are in total $\sim$ 1,800 or more RSGs in the SMC. We purify the sample by studying the infrared CMDs and the variability of the objects, though there is still an ambiguity between AGBs and RSGs. There are much less RSGs candidates ($\sim$4\%) showing PAH emission features compared to the Milky Way and LMC ($\sim$15\%). The MIR variability of RSG sample increases with luminosity. We separate the RSG sample into two subsamples ("risky" and "safe") and identify one M5e AGB star in the "risky" subsample. Most of the targets with large variability are also the bright ones with large MLR. Some targets show excessive dust emission, which may be related to previous episodic mass loss events. We also roughly estimate the total gas and dust budget produced by entire RSG population as $\rm \sim1.9^{+2.4}_{-1.1}\times10^{-6}~M_{\odot}/yr$ in the most conservative case. Based on the MIST models, we derive a linear relation between $T_{\rm eff}$ and observed $\rm J-K_S$ color with reddening correction for the RSG sample. By using a constant bolometric correction and this relation, the Geneva evolutionary model is compared with our RSG sample, showing a good agreement and a lower initial mass limit of $\sim$7 $\rm M_\odot $ for the RSG population. Finally, we compare the RSG sample in the SMC and the LMC. Despite the incompleteness of LMC sample in the faint end, the result indicates that the LMC sample always shows redder color (except for the $\rm IRAC1-IRAC2$ and $\rm WISE1-WISE2$ colors due to CO absorption) and larger variability than the SMC sample.

astro-ph.SR↗

arXiv2020

A Study of Non-autoregressive Model for Sequence Generation

Non-autoregressive (NAR) models generate all the tokens of a sequence in parallel, resulting in faster generation speed compared to their autoregressive (AR) counterparts but at the cost of lower accuracy. Different techniques including knowledge distillation and source-target alignment have been proposed to bridge the gap between AR and NAR models in various tasks such as neural machine translation (NMT), automatic speech recognition (ASR), and text to speech (TTS). With the help of those techniques, NAR models can catch up with the accuracy of AR models in some tasks but not in some others. In this work, we conduct a study to understand the difficulty of NAR sequence generation and try to answer: (1) Why NAR models can catch up with AR models in some tasks but not all? (2) Why techniques like knowledge distillation and source-target alignment can help NAR models. Since the main difference between AR and NAR models is that NAR models do not use dependency among target tokens while AR models do, intuitively the difficulty of NAR sequence generation heavily depends on the strongness of dependency among target tokens. To quantify such dependency, we propose an analysis model called CoMMA to characterize the difficulty of different NAR sequence generation tasks. We have several interesting findings: 1) Among the NMT, ASR and TTS tasks, ASR has the most target-token dependency while TTS has the least. 2) Knowledge distillation reduces the target-token dependency in target sequence and thus improves the accuracy of NAR models. 3) Source-target alignment constraint encourages dependency of a target token on source tokens and thus eases the training of NAR models.

cs.CL↗

arXiv2020

Information Content of Hierarchical n-Point Polytope Functions for Quantifying and Reconstructing Disordered Systems

Disordered systems are ubiquitous in physical, biological and material sciences. Examples include liquid and glassy states of condensed matter, colloids, granular materials, porous media, composites, alloys, packings of cells in avian retina and tumor spheroids, to name but a few. A comprehensive understanding of such disordered systems requires, as the first step, systematic quantification, modeling and representation of the underlying complex configurations and microstructure, which is generally very challenging to achieve. Recently, we introduce a set of hierarchical statistical microstructural descriptors, i.e., the n-point polytope functions Pn, which are derived from the standard n-point correlation functions Sn, and successively include higher-order n-point statistics of the morphological features of interest in a concise, explainable, and expressive manner. Here we investigate the information content of the Pn functions via optimization-based realization rendering. This is achieved by successively incorporating higher order Pn functions up to n = 8 and quantitatively assessing the accuracy of the reconstructed systems via un-constrained statistical morphological descriptors (e.g., the lineal-path function). We examine a wide spectrum of representative random systems with distinct geometrical and topological features. We find that generally, successively incorporating higher order Pn functions, and thus, the higher-order morphological information encoded in these descriptors, leads to superior accuracy of the reconstructions. However, incorporating more Pn functions into the reconstruction also significantly increases the complexity and roughness of the associated energy landscape for the underlying stochastic optimization, making it difficult to convergence numerically.

cond-mat.stat-mech↗

arXiv2020

Microstructure Design of Low-Melting-Point Alloy (LMPA)/ Polymer Composites for Dynamic Dry Adhesion Tuning in Soft Gripping

Tunable dry adhesion is a crucial mechanism in compliant manipulation. The gripping force, mainly originated from the van der Waals force between the adhesive composite and the object to be gripped, can be controlled by reversibly varying the physical properties (e.g., stiffness) of the composite via external stimuli. The maximal gripping force Fmax and its tunability depend on, among other factors, the stress distribution on the gripping interface and its fracture dynamics (during detaching), which in turn are determined by the composite microstructure. Here, we present a computational framework for the modeling and design of a class of binary smart composites containing a porous low-melting-point alloy (LMPA) phase and a polymer phase, in order to achieve desirable dynamically tunable dry adhesion. In particular, we employ spatial correlation functions to quantify, model and represent the complex bi-continuous microstructure of the composites, from which a wide spectrum of realistic virtual 3D composite microstructures can be generated using stochastic optimization. A recently developed volume-compensated lattice-particle (VCLP) method is then employed to model the dynamic interfacial fracture process to compute Fmax for different composite microstructures. We focus on the interface defect tuning (IDT) mechanism for dry adhesion tuning enabled by the composite, in which the thermal expansion of the LMPA phase due to Joule heating initializes small cracks on the adhesion interface, subsequently causing the detachment of the gripper from the object due to interfacial fracture. We find that for an optimal microstructure among the ones studied here, a 10-fold dynamic tuning of Fmax before and after the thermal expansion of the LMPA phase can be achieved. Our computational results can provide valuable guidance for experimental fabrication of the LMPA-polymer composites.

physics.app-ph↗

arXiv2020

Compositional Languages Emerge in a Neural Iterated Learning Model

The principle of compositionality, which enables natural language to represent complex concepts via a structured combination of simpler ones, allows us to convey an open-ended set of messages using a limited vocabulary. If compositionality is indeed a natural property of language, we may expect it to appear in communication protocols that are created by neural agents in language games. In this paper, we propose an effective neural iterated learning (NIL) algorithm that, when applied to interacting neural agents, facilitates the emergence of a more structured type of language. Indeed, these languages provide learning speed advantages to neural agents during training, which can be incrementally amplified via NIL. We provide a probabilistic model of NIL and an explanation of why the advantage of compositional language exist. Our experiments confirm our analysis, and also demonstrate that the emerged languages largely improve the generalizing power of the neural agent communication.

cs.CL↗

arXiv2020

Fully distributed cooperation for networked uncertain mobile manipulators

This paper investigates a fully distributed cooperation scheme for networked mobile manipulators. To achieve cooperative task allocation in a distributed way, an adaptation-based estimation law is established for each robotic agent to estimate the desired local trajectory. In addition, wrench synthesis is analyzed in detail to lay a solid foundation for tight cooperation tasks. Together with the estimated task, a set of distributed adaptive controllers is proposed to achieve motion synchronization of the mobile manipulator ensemble over a directed graph with a spanning tree irrespective of the kinematic and dynamic uncertainties in both the mobile manipulators and the tightly grasped object. The controlled synchronization alleviates the performance degradation caused by the estimation/tracking discrepancy during the transient phase. The proposed scheme requires no persistent excitation condition and avoids the use of noisy Cartesian-space velocities. Furthermore, it is independent from the object's center of mass by employing formation-based task allocation and a task-oriented strategy. These attractive attributes facilitate the practical application of the scheme. It is theoretically proven that convergence of the cooperative task tracking error is guaranteed. Simulation results validate the efficacy and demonstrate the expected performance of the proposed scheme.

cs.RO↗

arXiv2019

A Study of Multilingual Neural Machine Translation

Multilingual neural machine translation (NMT) has recently been investigated from different aspects (e.g., pivot translation, zero-shot translation, fine-tuning, or training from scratch) and in different settings (e.g., rich resource and low resource, one-to-many, and many-to-one translation). This paper concentrates on a deep understanding of multilingual NMT and conducts a comprehensive study on a multilingual dataset with more than 20 languages. Our results show that (1) low-resource language pairs benefit much from multilingual training, while rich-resource language pairs may get hurt under limited model capacity and training with similar languages benefits more than dissimilar languages; (2) fine-tuning performs better than training from scratch in the one-to-many setting while training from scratch performs better in the many-to-one setting; (3) the bottom layers of the encoder and top layers of the decoder capture more language-specific information, and just fine-tuning these parts can achieve good accuracy for low-resource language pairs; (4) direct translation is better than pivot translation when the source language is similar to the target language (e.g., in the same language branch), even when the size of direct training data is much smaller; (5) given a fixed training data budget, it is better to introduce more languages into multilingual training for zero-shot translation.

cs.CL↗