arXiv subjects

Soo-Young Lee

Publications and source records attributed to Soo-Young Lee.

At least 19 recordsLinked to original sources

arXiv2021

Multi-speaker Emotional Text-to-speech Synthesizer

We present a methodology to train our multi-speaker emotional text-to-speech synthesizer that can express speech for 10 speakers' 7 different emotions. All silences from audio samples are removed prior to learning. This results in fast learning by our model. Curriculum learning is applied to train our model efficiently. Our model is first trained with a large single-speaker neutral dataset, and then trained with neutral speech from all speakers. Finally, our model is trained using datasets of emotional speech from all speakers. In each stage, training samples of each speaker-emotion pair have equal probability to appear in mini-batches. Through this procedure, our model can synthesize speech for all targeted speakers and emotions. Our synthesized audio sets are available on our web page.

cs.CL↗

arXiv2020

Unigram-Normalized Perplexity as a Language Model Performance Measure with Different Vocabulary Sizes

Although Perplexity is a widely used performance metric for language models, the values are highly dependent upon the number of words in the corpus and is useful to compare performance of the same corpus only. In this paper, we propose a new metric that can be used to evaluate language model performance with different vocabulary sizes. The proposed unigram-normalized Perplexity actually presents the performance improvement of the language models from that of simple unigram model, and is robust on the vocabulary size. Both theoretical analysis and computational experiments are reported.

cs.CL↗

arXiv2020

Hierarchical GPT with Congruent Transformers for Multi-Sentence Language Models

We report a GPT-based multi-sentence language model for dialogue generation and document understanding. First, we propose a hierarchical GPT which consists of three blocks, i.e., a sentence encoding block, a sentence generating block, and a sentence decoding block. The sentence encoding and decoding blocks are basically the encoder-decoder blocks of the standard Transformers, which work on each sentence independently. The sentence generating block is inserted between the encoding and decoding blocks, and generates the next sentence embedding vector from the previous sentence embedding vectors. We believe it is the way human make conversation and understand paragraphs and documents. Since each sentence may consist of fewer words, the sentence encoding and decoding Transformers can use much smaller dimensional embedding vectors. Secondly, we note the attention in the Transformers utilizes the inner-product similarity measure. Therefore, to compare the two vectors in the same space, we set the transform matrices for queries and keys to be the same. Otherwise, the similarity concept is incongruent. We report experimental results to show that these two modifications increase the language model performance for tasks with multiple sentences.

cs.CL↗

arXiv2020

Semi-supervised Disentanglement with Independent Vector Variational Autoencoders

We aim to separate the generative factors of data into two latent vectors in a variational autoencoder. One vector captures class factors relevant to target classification tasks, while the other vector captures style factors relevant to the remaining information. To learn the discrete class features, we introduce supervision using a small amount of labeled data, which can simply yet effectively reduce the effort required for hyperparameter tuning performed in existing unsupervised methods. Furthermore, we introduce a learning objective to encourage statistical independence between the vectors. We show that (i) this vector independence term exists within the result obtained on decomposing the evidence lower bound with multiple latent vectors, and (ii) encouraging such independence along with reducing the total correlation within the vectors enhances disentanglement performance. Experiments conducted on several image datasets demonstrate that the disentanglement achieved via our method can improve classification performance and generation controllability.

cs.LG↗

arXiv2019

Emotional Voice Conversion using Multitask Learning with Text-to-speech

Voice conversion (VC) is a task to transform a person's voice to different style while conserving linguistic contents. Previous state-of-the-art on VC is based on sequence-to-sequence (seq2seq) model, which could mislead linguistic information. There was an attempt to overcome it by using textual supervision, it requires explicit alignment which loses the benefit of using seq2seq model. In this paper, a voice converter using multitask learning with text-to-speech (TTS) is presented. The embedding space of seq2seq-based TTS has abundant information on the text. The role of the decoder of TTS is to convert embedding space to speech, which is same to VC. In the proposed model, the whole network is trained to minimize loss of VC and TTS. VC is expected to capture more linguistic information and to preserve training stability by multitask learning. Experiments of VC were performed on a male Korean emotional text-speech dataset, and it is shown that multitask learning is helpful to keep linguistic contents in VC.

eess.AS↗

arXiv2019

Boundary integral equation method for resonances in gradient index cavities designed by conformal transformation optics

In the case of two-dimensional gradient index cavities designed by the conformal transformation optics, we propose a boundary integral equation method for the calculation of resonant mode functions by employing a fictitious space which is reciprocally equivalent to the physical space. Using the Green's function of the interior region of the uniform index cavity in the fictitious space, resonant mode functions and their far-field distributions in the physical space can be obtained. As a verification, resonant modes in limaçon-shaped transformation cavities were calculated and mode patterns and far-field intensity distributions were compared with those of the same modes obtained from the finite element method.

physics.optics↗

arXiv2019

A Fully Time-domain Neural Model for Subband-based Speech Synthesizer

This paper introduces a deep neural network model for subband-based speech synthesizer. The model benefits from the short bandwidth of the subband signals to reduce the complexity of the time-domain speech generator. We employed the multi-level wavelet analysis/synthesis to decompose/reconstruct the signal into subbands in time domain. Inspired from the WaveNet, a convolutional neural network (CNN) model predicts subband speech signals fully in time domain. Due to the short bandwidth of the subbands, a simple network architecture is enough to train the simple patterns of the subbands accurately. In the ground truth experiments with teacher-forcing, the subband synthesizer outperforms the fullband model significantly in terms of both subjective and objective measures. In addition, by conditioning the model on the phoneme sequence using a pronunciation dictionary, we have achieved the fully time-domain neural model for subband-based text-to-speech (TTS) synthesizer, which is nearly end-to-end. The generated speech of the subband TTS shows comparable quality as the fullband one with a slighter network architecture for each subband.

eess.AS↗

arXiv2019

Adjusting Pleasure-Arousal-Dominance for Continuous Emotional Text-to-speech Synthesizer

Emotion is not limited to discrete categories of happy, sad, angry, fear, disgust, surprise, and so on. Instead, each emotion category is projected into a set of nearly independent dimensions, named pleasure (or valence), arousal, and dominance, known as PAD. The value of each dimension varies from -1 to 1, such that the neutral emotion is in the center with all-zero values. Training an emotional continuous text-to-speech (TTS) synthesizer on the independent dimensions provides the possibility of emotional speech synthesis with unlimited emotion categories. Our end-to-end neural speech synthesizer is based on the well-known Tacotron. Empirically, we have found the optimum network architecture for injecting the 3D PADs. Moreover, the PAD values are adjusted for the speech synthesis purpose.

eess.AS↗

arXiv2018

Unpaired Speech Enhancement by Acoustic and Adversarial Supervision for Speech Recognition

Many speech enhancement methods try to learn the relationship between noisy and clean speech, obtained using an acoustic room simulator. We point out several limitations of enhancement methods relying on clean speech targets; the goal of this work is proposing an alternative learning algorithm, called acoustic and adversarial supervision (AAS). AAS makes the enhanced output both maximizing the likelihood of transcription on the pre-trained acoustic model and having general characteristics of clean speech, which improve generalization on unseen noisy speeches. We employ the connectionist temporal classification and the unpaired conditional boundary equilibrium generative adversarial network as the loss function of AAS. AAS is tested on two datasets including additive noise without and with reverberation, Librispeech + DEMAND and CHiME-4. By visualizing the enhanced speech with different loss combinations, we demonstrate the role of each supervision. AAS achieves a lower word error rate than other state-of-the-art methods using the clean speech target in both datasets.

cs.CL↗

arXiv2018

End-to-end Multimodal Emotion and Gender Recognition with Dynamic Joint Loss Weights

Multi-task learning is a method for improving the generalizability of multiple tasks. In order to perform multiple classification tasks with one neural network model, the losses of each task should be combined. Previous studies have mostly focused on multiple prediction tasks using joint loss with static weights for training models, choosing the weights between tasks without making sufficient considerations by setting them uniformly or empirically. In this study, we propose a method to calculate joint loss using dynamic weights to improve the total performance, instead of the individual performance, of tasks. We apply this method to design an end-to-end multimodal emotion and gender recognition model using audio and video data. This approach provides proper weights for the loss of each task when the training process ends. In our experiments, emotion and gender recognition with the proposed method yielded a lower joint loss, which is computed as the negative log-likelihood, than using static weights for joint loss. Moreover, our proposed model has better generalizability than other models. To the best of our knowledge, this research is the first to demonstrate the strength of using dynamic weights for joint loss for maximizing overall performance in emotion and gender recognition tasks.

cs.LG↗

arXiv2018

Voice Imitating Text-to-Speech Neural Networks

We propose a neural text-to-speech (TTS) model that can imitate a new speaker's voice using only a small amount of speech sample. We demonstrate voice imitation using only a 6-seconds long speech sample without any other information such as transcripts. Our model also enables voice imitation instantly without additional training of the model. We implemented the voice imitating TTS model by combining a speaker embedder network with a state-of-the-art TTS model, Tacotron. The speaker embedder network takes a new speaker's speech sample and returns a speaker embedding. The speaker embedding with a target sentence are fed to Tacotron, and speech is generated with the new speaker's voice. We show that the speaker embeddings extracted by the speaker embedder network can represent the latent structure in different voices. The generated speech samples from our model have comparable voice quality to the ones from existing multi-speaker TTS models.

cs.SD↗

arXiv2017

Emotional End-to-End Neural Speech Synthesizer

In this paper, we introduce an emotional speech synthesizer based on the recent end-to-end neural model, named Tacotron. Despite its benefits, we found that the original Tacotron suffers from the exposure bias problem and irregularity of the attention alignment. Later, we address the problem by utilization of context vector and residual connection at recurrent neural networks (RNNs). Our experiments showed that the model could successfully train and generate speech for given emotion labels.

cs.SD↗

arXiv2016

Deep CNNs along the Time Axis with Intermap Pooling for Robustness to Spectral Variations

Convolutional neural networks (CNNs) with convolutional and pooling operations along the frequency axis have been proposed to attain invariance to frequency shifts of features. However, this is inappropriate with regard to the fact that acoustic features vary in frequency. In this paper, we contend that convolution along the time axis is more effective. We also propose the addition of an intermap pooling (IMP) layer to deep CNNs. In this layer, filters in each group extract common but spectrally variant features, then the layer pools the feature maps of each group. As a result, the proposed IMP CNN can achieve insensitivity to spectral variations characteristic of different speakers and utterances. The effectiveness of the IMP CNN architecture is demonstrated on several LVCSR tasks. Even without speaker adaptation techniques, the architecture achieved a WER of 12.7% on the SWB part of the Hub5'2000 evaluation test set, which is competitive with other state-of-the-art methods.

cs.CL↗

arXiv2015

A Novel Analytic Approach to Model Line Edge Roughness using Stochastic Exposure Distribution in Electron-beam Lithography

The line edge roughness (LER) becomes a issue of e-beam lithography when feature size is reduced into nanometers. Therefore, minimizing the LER is a important method to increase the density of circuit patterns. One of the possible ways is through simulation. The stochastic exposure distributions in the resist is generated by the Monte Carlo simulation. In addition a resist development simulation needs to be carried out. Although there are several ways to simulate or estimate LER but none of them can reveal as much the inner relationship between LER and different parameters as theocratical analysing methods can do. In this paper, a new approach to analytically derive the LER based on the statistical exposure, is described. Our approach is based on analytic model of stochastic exposure distribution, deriving standard deviation of exposure and analyzing the variance of edge location after development. Even though it may not be a complete modeling of LER, it can still show some strong relationship between LER and some inner parameters.

cond-mat.mes-hall↗

arXiv2015

Exceptional points in coupled dissipative dynamical systems

We study the transient behavior in coupled dissipative dynamical systems based on the linear analysis around the steady state. We find that the transient time is minimized at a specific set of system parameters and show that at this parameter set, two eigenvalues and two eigenvectors of Jacobian matrix coalesce at the same time, this degenerate point is called the exceptional point. For the case of coupled limit cycle oscillators, we investigate the transient behavior into the amplitude death state, and clarify that the exceptional point is associated with a critical point of frequency locking, as well as the transition of the envelope oscillation.

nlin.CD↗

arXiv2014

Abnormal high-$Q$ modes of coupled stadium-shaped microcavities

It is well known that the strongly deformed microcavity with fully chaotic ray dynamics cannot support high-Q modes due to its fast chaotic diffusion to the critical line of refractive emission. Here, we investigate how the Q factor is modified when two chaotic cavities are coupled, and show that some modes, whose Q factor is about 10 times higher than that of the corresponding single cavity, can exist. These abnormal high-Q modes are the result of an optimal combination of coupling and cavity geometry. As an example, in the coupled stadium-shaped microcavities, the mode pattern extends over both cavities such that it follows a whispering-gallery-type mode at both ends, whereas a big coupling spot forms at the closest contact of the two microcavities. The pattern of such a 'rounded bow tie' mode allows the mode to have a high-Q factor. This mode pattern minimizes the leakage of light at both ends of the microcavities as the pattern at both ends is similar to whispering gallery mode.

physics.optics↗

arXiv2014

Quantum Goos-Hänchen shift and tunneling transmission at a curved step potential

We study the quantum Goos-Hänchen (GH) shift and the tunneling transmission at a curved step potential by investigating the time evolution of a wave packet. An initial wave packet is expanded in terms of the eigenmodes of a circular step potential. Its time evolution is then given by the interference of their simple eigenmode oscillations. We show that the GH shift along the step boundary can be explained by the energy-dependent phase loss upon reflection, which is defined by modifying the one-dimensional (1D) effective potential derived from the 2D circular system. We also demonstrate that the tunneling transmission of the wave packet is characterized by a free-space image distant from the boundary. The tunneling transmission exhibits a rather wide angle divergence and the direction of maximum tunneling is slightly rotated from the tangent at the incident point, which is consistent with the time delay of the tunneling wave packet computed in the 1D modified effective potential.

quant-ph↗

arXiv2013

Hierarchical Data Representation Model - Multi-layer NMF

In this paper, we propose a data representation model that demonstrates hierarchical feature learning using nsNMF. We extend unit algorithm into several layers. Experiments with document and image data successfully discovered feature hierarchies. We also prove that proposed method results in much better classification and reconstruction performance, especially for small number of features. feature hierarchies.

cs.LG↗