arXiv subjects

Yi Ren

Publications and source records attributed to Yi Ren.

At least 199 records · Page 11Linked to original sources

arXiv2022

Kronecker-factored Quasi-Newton Methods for Deep Learning

Second-order methods have the capability of accelerating optimization by using much richer curvature information than first-order methods. However, most are impractical for deep learning, where the number of training parameters is huge. In Goldfarb et al. (2020), practical quasi-Newton methods were proposed that approximate the Hessian of a multilayer perceptron (MLP) model by a layer-wise block diagonal matrix where each layer's block is further approximated by a Kronecker product corresponding to the structure of the Hessian restricted to that layer. Here, we extend these methods to enable them to be applied to convolutional neural networks (CNNs), by analyzing the Kronecker-factored structure of the Hessian matrix of convolutional layers. Several improvements to the methods in Goldfarb et al. (2020) are also proposed that can be applied to both MLPs and CNNs. These new methods have memory requirements comparable to first-order methods and much less per-iteration time complexity than those in Goldfarb et al. (2020). Moreover, convergence results are proved for a variant under relatively mild conditions. Finally, we compared the performance of our new methods against several state-of-the-art (SOTA) methods on MLP autoencoder and CNN problems, and found that they outperformed the first-order SOTA methods and performed comparably to the second-order SOTA methods.

cs.LG↗

arXiv2022

Dependence of pulsation mode of Cepheids on metallicity

The Cepheid variables in SMC, LMC, the Milky Way, M33 and M31 are used to examine the dependence of pulsation mode on metallicity which was previously found in red supergiants. The initial samples of Cepheids are collected from the Cepheid catalogs identified from the OGLE, PS1, DIRECT, WISE and ZTF surveys. The contaminants are removed with the help of the Gaia/EDR3 astrometric information for extra galaxies or by comparing the geometric distance and the distance from the P-L relation for the Milky Way. The division of fundamental and first-overtone mode is refined according to the gap between the two modes in the P-L diagram of the objects in each galaxy. The ratio of FU/(FU+1O) is found to be 0.59, 0.60, 0.69, 0.83 and 0.85 for SMC, LMC, the Milky Way, M33 and M31 respectively in order of metallicity, which confirms that the pulsation mode depends on metallicity in the way that the ratio of FU/(FU+1O) increases with metallicity. This dependence is not changed if the incompleteness of the samples is taken into account.

astro-ph.SR↗

arXiv2022

ProsoSpeech: Enhancing Prosody With Quantized Vector Pre-training in Text-to-Speech

Expressive text-to-speech (TTS) has become a hot research topic recently, mainly focusing on modeling prosody in speech. Prosody modeling has several challenges: 1) the extracted pitch used in previous prosody modeling works have inevitable errors, which hurts the prosody modeling; 2) different attributes of prosody (e.g., pitch, duration and energy) are dependent on each other and produce the natural prosody together; and 3) due to high variability of prosody and the limited amount of high-quality data for TTS training, the distribution of prosody cannot be fully shaped. To tackle these issues, we propose ProsoSpeech, which enhances the prosody using quantized latent vectors pre-trained on large-scale unpaired and low-quality text and speech data. Specifically, we first introduce a word-level prosody encoder, which quantizes the low-frequency band of the speech and compresses prosody attributes in the latent prosody vector (LPV). Then we introduce an LPV predictor, which predicts LPV given word sequence. We pre-train the LPV predictor on large-scale text and low-quality speech data and fine-tune it on the high-quality TTS dataset. Finally, our model can generate expressive speech conditioned on the predicted LPV. Experimental results show that ProsoSpeech can generate speech with richer prosody compared with baseline methods.

eess.AS↗

arXiv2022

Video-based Facial Micro-Expression Analysis: A Survey of Datasets, Features and Algorithms

Unlike the conventional facial expressions, micro-expressions are involuntary and transient facial expressions capable of revealing the genuine emotions that people attempt to hide. Therefore, they can provide important information in a broad range of applications such as lie detection, criminal detection, etc. Since micro-expressions are transient and of low intensity, however, their detection and recognition is difficult and relies heavily on expert experiences. Due to its intrinsic particularity and complexity, video-based micro-expression analysis is attractive but challenging, and has recently become an active area of research. Although there have been numerous developments in this area, thus far there has been no comprehensive survey that provides researchers with a systematic overview of these developments with a unified evaluation. Accordingly, in this survey paper, we first highlight the key differences between macro- and micro-expressions, then use these differences to guide our research survey of video-based micro-expression analysis in a cascaded structure, encompassing the neuropsychological basis, datasets, features, spotting algorithms, recognition algorithms, applications and evaluation of state-of-the-art approaches. For each aspect, the basic techniques, advanced developments and major challenges are addressed and discussed. Furthermore, after considering the limitations of existing micro-expression datasets, we present and release a new dataset - called micro-and-macro expression warehouse (MMEW) - containing more video samples and more labeled emotion types. We then perform a unified comparison of representative methods on CAS(ME)2 for spotting, and on MMEW and SAMM for recognition, respectively. Finally, some potential future research directions are explored and outlined.

cs.CV↗

arXiv2022

MIC: Model-agnostic Integrated Cross-channel Recommenders

Semantically connecting users and items is a fundamental problem for the matching stage of an industrial recommender system. Recent advances in this topic are based on multi-channel retrieval to efficiently measure users' interest on items from the massive candidate pool. However, existing work are primarily built upon pre-defined retrieval channels, including User-CF (U2U), Item-CF (I2I), and Embedding-based Retrieval (U2I), thus access to the limited correlation between users and items which solely entail from partial information of latent interactions. In this paper, we propose a model-agnostic integrated cross-channel (MIC) approach for the large-scale recommendation, which maximally leverages the inherent multi-channel mutual information to enhance the matching performance. Specifically, MIC robustly models correlation within user-item, user-user, and item-item from latent interactions in a universal schema. For each channel, MIC naturally aligns pairs with semantic similarity and distinguishes them otherwise with more uniform anisotropic representation space. While state-of-the-art methods require specific architectural design, MIC intuitively considers them as a whole by enabling the complete information flow among users and items. Thus MIC can be easily plugged into other retrieval recommender systems. Extensive experiments show that our MIC helps several state-of-the-art models boost their performance on two real-world benchmarks. The satisfactory deployment of the proposed MIC on industrial online services empirically proves its scalability and flexibility.

cs.IR↗

arXiv2022

PortaSpeech: Portable and High-Quality Generative Text-to-Speech

Non-autoregressive text-to-speech (NAR-TTS) models such as FastSpeech 2 and Glow-TTS can synthesize high-quality speech from the given text in parallel. After analyzing two kinds of generative NAR-TTS models (VAE and normalizing flow), we find that: VAE is good at capturing the long-range semantics features (e.g., prosody) even with small model size but suffers from blurry and unnatural results; and normalizing flow is good at reconstructing the frequency bin-wise details but performs poorly when the number of model parameters is limited. Inspired by these observations, to generate diverse speech with natural details and rich prosody using a lightweight architecture, we propose PortaSpeech, a portable and high-quality generative text-to-speech model. Specifically, 1) to model both the prosody and mel-spectrogram details accurately, we adopt a lightweight VAE with an enhanced prior followed by a flow-based post-net with strong conditional inputs as the main architecture. 2) To further compress the model size and memory footprint, we introduce the grouped parameter sharing mechanism to the affine coupling layers in the post-net. 3) To improve the expressiveness of synthesized speech and reduce the dependency on accurate fine-grained alignment between text and speech, we propose a linguistic encoder with mixture alignment combining hard inter-word alignment and soft intra-word alignment, which explicitly extracts word-level semantic information. Experimental results show that PortaSpeech outperforms other TTS models in both voice quality and prosody modeling in terms of subjective and objective evaluation metrics, and shows only a slight performance degradation when reducing the model parameters to 6.7M (about 4x model size and 3x runtime memory compression ratio compared with FastSpeech 2). Our extensive ablation studies demonstrate that each design in PortaSpeech is effective.

eess.AS↗

arXiv2022

MR-SVS: Singing Voice Synthesis with Multi-Reference Encoder

Multi-speaker singing voice synthesis is to generate the singing voice sung by different speakers. To generalize to new speakers, previous zero-shot singing adaptation methods obtain the timbre of the target speaker with a fixed-size embedding from single reference audio. However, they face several challenges: 1) the fixed-size speaker embedding is not powerful enough to capture full details of the target timbre; 2) single reference audio does not contain sufficient timbre information of the target speaker; 3) the pitch inconsistency between different speakers also leads to a degradation in the generated voice. In this paper, we propose a new model called MR-SVS to tackle these problems. Specifically, we employ both a multi-reference encoder and a fixed-size encoder to encode the timbre of the target speaker from multiple reference audios. The Multi-reference encoder can capture more details and variations of the target timbre. Besides, we propose a well-designed pitch shift method to address the pitch inconsistency problem. Experiments indicate that our method outperforms the baseline method both in naturalness and similarity.

eess.AS↗

arXiv2021

Tensor Normal Training for Deep Learning Models

Despite the predominant use of first-order methods for training deep learning models, second-order methods, and in particular, natural gradient methods, remain of interest because of their potential for accelerating training through the use of curvature information. Several methods with non-diagonal preconditioning matrices, including KFAC, Shampoo, and K-BFGS, have been proposed and shown to be effective. Based on the so-called tensor normal (TN) distribution, we propose and analyze a brand new approximate natural gradient method, Tensor Normal Training (TNT), which like Shampoo, only requires knowledge of the shape of the training parameters. By approximating the probabilistically based Fisher matrix, as opposed to the empirical Fisher matrix, our method uses the block-wise covariance of the sampling based gradient as the pre-conditioning matrix. Moreover, the assumption that the sampling-based (tensor) gradient follows a TN distribution, ensures that its covariance has a Kronecker separable structure, which leads to a tractable approximation to the Fisher matrix. Consequently, TNT's memory requirements and per-iteration computational costs are only slightly higher than those for first-order methods. In our experiments, TNT exhibited superior optimization performance to state-of-the-art first-order methods, and comparable optimization performance to the state-of-the-art second-order methods KFAC and Shampoo. Moreover, TNT demonstrated its ability to generalize as well as first-order methods, while using fewer epochs.

cs.LG↗

arXiv2021

Parallel and High-Fidelity Text-to-Lip Generation

As a key component of talking face generation, lip movements generation determines the naturalness and coherence of the generated talking face video. Prior literature mainly focuses on speech-to-lip generation while there is a paucity in text-to-lip (T2L) generation. T2L is a challenging task and existing end-to-end works depend on the attention mechanism and autoregressive (AR) decoding manner. However, the AR decoding manner generates current lip frame conditioned on frames generated previously, which inherently hinders the inference speed, and also has a detrimental effect on the quality of generated lip frames due to error propagation. This encourages the research of parallel T2L generation. In this work, we propose a parallel decoding model for fast and high-fidelity text-to-lip generation (ParaLip). Specifically, we predict the duration of the encoded linguistic features and model the target lip frames conditioned on the encoded linguistic features with their duration in a non-autoregressive manner. Furthermore, we incorporate the structural similarity index loss and adversarial learning to improve perceptual quality of generated lip frames and alleviate the blurry prediction problem. Extensive experiments conducted on GRID and TCD-TIMIT datasets demonstrate the superiority of proposed methods. Video samples are available via \url{https://paralip.github.io/}.

cs.MM↗

arXiv2021

Multi-Singer: Fast Multi-Singer Singing Voice Vocoder With A Large-Scale Corpus

High-fidelity multi-singer singing voice synthesis is challenging for neural vocoder due to the singing voice data shortage, limited singer generalization, and large computational cost. Existing open corpora could not meet requirements for high-fidelity singing voice synthesis because of the scale and quality weaknesses. Previous vocoders have difficulty in multi-singer modeling, and a distinct degradation emerges when conducting unseen singer singing voice generation. To accelerate singing voice researches in the community, we release a large-scale, multi-singer Chinese singing voice dataset OpenSinger. To tackle the difficulty in unseen singer modeling, we propose Multi-Singer, a fast multi-singer vocoder with generative adversarial networks. Specifically, 1) Multi-Singer uses a multi-band generator to speed up both training and inference procedure. 2) to capture and rebuild singer identity from the acoustic feature (i.e., mel-spectrogram), Multi-Singer adopts a singer conditional discriminator and conditional adversarial training objective. 3) to supervise the reconstruction of singer identity in the spectrum envelopes in frequency domain, we propose an auxiliary singer perceptual loss. The joint training approach effectively works in GANs for multi-singer voices modeling. Experimental results verify the effectiveness of OpenSinger and show that Multi-Singer improves unseen singer singing voices modeling in both speed and quality over previous methods. The further experiment proves that combined with FastSpeech 2 as the acoustic model, Multi-Singer achieves strong robustness in the multi-singer singing voice synthesis pipeline. Samples are available at https://Multi-Singer.github.io/

eess.AS↗

arXiv2021

Electrical detection of the spin-flop and room-temperature magnetic ordering in van der Waals CrPS$_{4}$/(Pt, Pd) heterostructures

We study magneto-transport in heterostructures composed of the van der Waals antiferromagnet CrPS$_{4}$ and the heavy metals Pt and Pd. The transverse resistance (R$_{xy}$) signal reveals the spin-flop transition of CrPS$_{4}$ and a strongly enhanced magnetic ordering temperature (>300 K), which might originate from a strong spin-orbit coupling at the interface. While CrPS$_{4}$/Pt devices allow for easy detection of the spin-flop transition, CrPS$_{4}$/Pd devices show a more substantial enhancement in magnetic ordering temperature and exhibit a topological Hall effect signal, possibly related to chiral spin structures at the interface. The longitudinal magnetoresistance (R$_{xx}$) results from a combination of spin-Hall magnetoresistance and the negative magnetoresistance that can be explained by a field-induced change of the electronic band structure of CrPS$_{4}$.

physics.app-ph↗

arXiv2021

Unbiased Pairwise Learning to Rank in Recommender Systems

Nowadays, recommender systems already impact almost every facet of peoples lives. To provide personalized high quality recommendation results, conventional systems usually train pointwise rankers to predict the absolute value of objectives and leverage a distinct shallow tower to estimate and alleviate the impact of position bias. However, with such a training paradigm, the optimization target differs a lot from the ranking metrics valuing the relative order of top ranked items rather than the prediction precision of each item. Moreover, as the existing system tends to recommend more relevant items at higher positions, it is difficult for the shallow tower based methods to precisely attribute the user feedback to the impact of position or relevance. Therefore, there exists an exciting opportunity for us to get enhanced performance if we manage to solve the aforementioned issues. Unbiased learning to rank algorithms, which are verified to model the relative relevance accurately based on noisy feedback, are appealing candidates and have already been applied in many applications with single categorical labels, such as user click signals. Nevertheless, the existing unbiased LTR methods cannot properly handle multiple feedback incorporating both categorical and continuous labels. Accordingly, we design a novel unbiased LTR algorithm to tackle the challenges, which innovatively models position bias in the pairwise fashion and introduces the pairwise trust bias to separate the position bias, trust bias, and user relevance explicitly. Experiment results on public benchmark datasets and internal live traffic show the superior results of the proposed method for both categorical and continuous labels.

cs.IR↗

arXiv2021

Dust Extinction Law in Nearby Star-Resolved Galaxies. I. M31 Traced by Supergiants

The dust extinction laws and dust properties in M31 are explored with a sample of reddened O-type and B-type supergiants obtained from the LGGS. The observed spectral energy distributions (SEDs) for each tracer are constructed with multiband photometry from the LGGS, PS1 Survey, UKIRT, PHAT Survey, Swift/UVOT and XMM-SUSS. We model the SED for each tracer in combination with the intrinsic spectrum obtained from the stellar model atmosphere extinguished by the model extinction curves. Instead of mathematically parameterizing the extinction functions, the model extinction curves in this work are directly derived from the silicate-graphite dust model with a dust size distribution of $dn/da \sim a^{-α}{\rm exp}(-a/0.25),~0.005 < a < 5~μ{\rm m}$. The extinction tracers are distributed along the arms in M31, with the derived MW-type extinction curves covering a wide range of $R_V$ ($\approx 2 - 6$), indicating the complexity of the interstellar environment and the inhomogeneous distribution of interstellar dust in M31. The average extinction curve with $R_V \approx 3.51$ and dust size distribution $dn/da \sim a^{-3.35}{\rm exp}(-a/0.25)$ is similar to those of the MW but rises slightly less steeply in the far-UV bands, implying that the overall interstellar environment in M31 resembles the diffuse region in the MW. The extinction in the $V$ band of M31 is up to 3 mag, with a median value of $ A_V \approx 1$ mag. The multiband extinction values from the UV to IR bands are also predicted for M31, which will provide a general extinction correction for future works.

astro-ph.GA↗

arXiv2021

The Dust Mass of Supernova Remnants in M31

The dust temperature and mass of the supernova remnants (SNRs) in M31 are estimated by fitting the infrared spectral energy distribution calculated from the images in the Spitzer/IRAC4 and MIPS24, Herschel/PACS70, 100, 160, and Herschel/SPIRE250, 350$μ$m band. Twenty SNRs with relatively reliable photometry exhibit an average dust temperature of $20.1^{+1.8}_{-1.5}$K, which is higher than the surrounding and indicating the heating effect of supernova explosion. The dust mass of these SNRs ranges from about 100 to 800$ M_{\odot}$, much bigger than the SNRs in the Milky Way. On the other hand, this yields the dust surface density of $0.10^{+0.07}_{-0.04}{ M_{\odot} \rm pc^{-2}}$, about half of the surrounding area, which implies that about half dust in the SNRs is destroyed by the supernova explosion. The dust temperature, the radius, and thus the dust mass all demonstrate that the studied SNRs are old and very likely in the snowplow or even fade away phase because of the limitation by the far distance and observation resolution of M31, and the results can serve as a reference to the final effect of supernova explosion on the surrounding dust.

astro-ph.GA↗

arXiv2021

The Sample of Red Supergiants in Twelve Low-Mass Galaxies of the Local Group

This work establishes the most complete sample of red supergiants (RSGs) in twelve low-mass galaxies (WLM, IC 10, NGC 147, NGC 185, IC 1613, Leo A, Sextans B, Sextans A, NGC 6822, Pegasus Dwarf, SMC and LMC) of the Local Group, which forms the solid basis to study the properties of RSGs as well as the star formation rate (SFR) and initial mass function (IMF) of the galaxies. After removing the foreground dwarf stars by their obvious branch in the near-infrared color-color diagram ($(J-H)_0/(H-K)_0$) with the UKIRT/WFCAM and 2MASS photometry as well as the Gaia/EDR3 measurements of proper motion and parallax, RSGs are identified from their location in the color-magnitude diagram $(J-K)_{0}/K_{0}$ of the member stars of the specific galaxy. A total of 2,190 RSGs are found in ten dwarf galaxies, and additionally 4,823 and 2,138 RSGs in the LMC and SMC respectively. The locations of the tip of the red giant branch in the $(J-K)_{0}/K_{0}$ diagram are determined to serve as an indicator of the metallicity and distance modulus of the galaxies.

astro-ph.GA↗

arXiv2021

Data Generation Method for Learning a Low-dimensional Safe Region in Safe Reinforcement Learning

Safe reinforcement learning aims to learn a control policy while ensuring that neither the system nor the environment gets damaged during the learning process. For implementing safe reinforcement learning on highly nonlinear and high-dimensional dynamical systems, one possible approach is to find a low-dimensional safe region via data-driven feature extraction methods, which provides safety estimates to the learning algorithm. As the reliability of the learned safety estimates is data-dependent, we investigate in this work how different training data will affect the safe reinforcement learning approach. By balancing between the learning performance and the risk of being unsafe, a data generation method that combines two sampling methods is proposed to generate representative training data. The performance of the method is demonstrated with a three-link inverted pendulum example.

eess.SY↗

arXiv2021

Data-Driven Learning of 3-Point Correlation Functions as Microstructure Representations

This paper considers the open challenge of identifying complete, concise, and explainable quantitative microstructure representations for disordered heterogeneous material systems. Completeness and conciseness have been achieved through existing data-driven methods, e.g., deep generative models, which, however, do not provide mathematically explainable latent representations. This study investigates representations composed of three-point correlation functions, which are a special type of spatial convolutions. We show that a variety of microstructures can be characterized by a concise subset of three-point correlations, and the identification of such subsets can be achieved by Bayesian optimization. Lastly, we show that the proposed representation can directly be used to compute material properties based on the effective medium theory.

cond-mat.mtrl-sci↗

arXiv2021

The Extinction and Distance of the MBM Molecular Clouds at High Galactic Latitude

Based on the accurate color excess $E_{\rm G_{BP},G_{RP}}$ of more than 4 million stars and $E_{\rm NUV,G_{BP}}$ of more than 1 million stars from \citet{2021ApJS..254...38S}, the distance and the extinction of the molecular clouds in the MBM catalog at $|b|>20^{\circ}$ are studied in combination with the distance measurement of \emph{Gaia}/EDR3. The distance as well as the color excess is determined for 66 molecular clouds. The color excess ratio $E_{\rm G_{BP},G_{RP}}/E_{\rm NUV,G_{BP}}$ is derived for 39 of them, which is obviously larger and implies more small particles at smaller extinction. In addition, the scale height of the dust disk is found to be about 100 pc and becomes large at the anticenter direction due to the disk flaring.

astro-ph.GA↗