Searcharxiv⌕ Search

arXiv subjects

Chanwoo Kim

Publications and source records attributed to Chanwoo Kim.

At least 73 records · Page 4Linked to original sources
arXiv2023

Asymptotics of Helmholtz-Kirchhoff Point-Vortices in the Phase Space

A rigorous derivation of point vortex systems from kinetic equations has been a challenging open problem, due to singular layers in the inviscid limit, giving a large velocity gradient in the Boltzmann equations. In this paper, we derive the Helmholtz-Kirchhoff point-vortex system from the hydrodynamic limits of the Boltzmann equations. We construct Boltzmann solutions by the Hilbert-type expansion associated to the point vortices solutions of the 2D Navier-Stokes equations. We give a precise pointwise estimate for the solution of the Boltzmann equations with small Strouhal number and Knudsen number.

math.AP↗
arXiv2023

Learning to Estimate Shapley Values with Vision Transformers

Transformers have become a default architecture in computer vision, but understanding what drives their predictions remains a challenging problem. Current explanation approaches rely on attention values or input gradients, but these provide a limited view of a model's dependencies. Shapley values offer a theoretically sound alternative, but their computational cost makes them impractical for large, high-dimensional models. In this work, we aim to make Shapley values practical for vision transformers (ViTs). To do so, we first leverage an attention masking approach to evaluate ViTs with partial information, and we then develop a procedure to generate Shapley value explanations via a separate, learned explainer model. Our experiments compare Shapley values to many baseline methods (e.g., attention rollout, GradCAM, LRP), and we find that our approach provides more accurate explanations than existing methods for ViTs.

cs.CV↗
arXiv2022

Macro-block dropout for improved regularization in training end-to-end speech recognition models

This paper proposes a new regularization algorithm referred to as macro-block dropout. The overfitting issue has been a difficult problem in training large neural network models. The dropout technique has proven to be simple yet very effective for regularization by preventing complex co-adaptations during training. In our work, we define a macro-block that contains a large number of units from the input to a Recurrent Neural Network (RNN). Rather than applying dropout to each unit, we apply random dropout to each macro-block. This algorithm has the effect of applying different drop out rates for each layer even if we keep a constant average dropout rate, which has better regularization effects. In our experiments using Recurrent Neural Network-Transducer (RNN-T), this algorithm shows relatively 4.30 % and 6.13 % Word Error Rates (WERs) improvement over the conventional dropout on LibriSpeech test-clean and test-other. With an Attention-based Encoder-Decoder (AED) model, this algorithm shows relatively 4.36 % and 5.85 % WERs improvement over the conventional dropout on the same test sets.

cs.LG↗
arXiv2022

Into-TTS : Intonation Template Based Prosody Control System

Intonations play an important role in delivering the intention of a speaker. However, current end-to-end TTS systems often fail to model proper intonations. To alleviate this problem, we propose a novel, intuitive method to synthesize speech in different intonations using predefined intonation templates. Prior to TTS model training, speech data are grouped into intonation templates in an unsupervised manner. Two proposed modules are added to the end-to-end TTS framework: an intonation predictor and an intonation encoder. The intonation predictor recommends a suitable intonation template to the given text. The intonation encoder, attached to the text encoder output, synthesizes speech abiding the requested intonation template. Main contributions of our paper are: (a) an easy-to-use intonation control system covering a wide range of users; (b) better performance in wrapping speech in a requested intonation with improved objective and subjective evaluation; and (c) incorporating a pre-trained language model for intonation modelling. Audio samples are available at https://srtts.github.io/IntoTTS.

eess.AS↗
arXiv2022

An Empirical Study on L2 Accents of Cross-lingual Text-to-Speech Systems via Vowel Space

With the recent developments in cross-lingual Text-to-Speech (TTS) systems, L2 (second-language, or foreign) accent problems arise. Moreover, running a subjective evaluation for such cross-lingual TTS systems is troublesome. The vowel space analysis, which is often utilized to explore various aspects of language including L2 accents, is a great alternative analysis tool. In this study, we apply the vowel space analysis method to explore L2 accents of cross-lingual TTS systems. Through the vowel space analysis, we observe the three followings: a) a parallel architecture (Glow-TTS) is less L2-accented than an auto-regressive one (Tacotron); b) L2 accents are more dominant in non-shared vowels in a language pair; and c) L2 accents of cross-lingual TTS systems share some phenomena with those of human L2 learners. Our findings imply that it is necessary for TTS systems to handle each language pair differently, depending on their linguistic characteristics such as non-shared vowels. They also hint that we can further incorporate linguistics knowledge in developing cross-lingual TTS systems.

eess.AS↗
arXiv2022

Transformer-based Global 3D Hand Pose Estimation in Two Hands Manipulating Objects Scenarios

This report describes our 1st place solution to ECCV 2022 challenge on Human Body, Hands, and Activities (HBHA) from Egocentric and Multi-view Cameras (hand pose estimation). In this challenge, we aim to estimate global 3D hand poses from the input image where two hands and an object are interacting on the egocentric viewpoint. Our proposed method performs end-to-end multi-hand pose estimation via transformer architecture. In particular, our method robustly estimates hand poses in a scenario where two hands interact. Additionally, we propose an algorithm that considers hand scales to robustly estimate the absolute depth. The proposed algorithm works well even when the hand sizes are various for each person. Our method attains 14.4 mm (left) and 15.9 mm (right) errors for each hand in the test set.

cs.CV↗
arXiv2022

Hölder regularity of the Boltzmann equation past an obstacle

Regularity and singularity of the solutions according to the shape of domains is a challenging research theme in the Boltzmann theory (\cite{Kim11,GKTT1}). In this paper, we prove an Hölder regularity in $C^{0,\frac{1}{2}-}_{x,v}$ for the Boltzmann equation of the hard-sphere molecule, which undergoes the elastic reflection in the intermolecular collision and the contact with the boundary of a convex obstacle. In particular, this Hölder regularity result is a stark contrast to the case of other physical boundary conditions (such as the diffuse reflection boundary condition and in-flow boundary condition), for which the solutions of the Boltzmann equation develop discontinuity in a codimension 1 subset (\cite{Kim11}), and therefore the best possible regularity is BV, which has been proved in \cite{GKTT2}.

math.AP↗
arXiv2022

Exponential Mixing of Vlasov equations under the effect of Gravity and Boundary

In this paper, we study exponentially fast mixing induced/enhanced by gravity and stochastic boundary in the kinetic theory of Vlasov equations. We consider the Vlasov equations with and without a vertical magnetic field inside a horizontally-periodic 3D half-space equipped with a non-isothermal diffusive reflection boundary condition of bounded continuous boundary temperature at the bottom. We construct both stationary solutions and global-in-time dynamical solutions in $L^\infty$. We prove that moments of a dynamical fluctuation around the steady solutions decay exponentially fast in $L^\infty$. As a key of this proof, we establish a uniform bound of so-called residual measures independently of the bouncing number of stochastic characteristics, by constructing a continuous stationary outgoing boundary flux which is strictly positive almost everywhere.

math.AP↗
arXiv2022

Decision Attentive Regularization to Improve Simultaneous Speech Translation Systems

Simultaneous translation systems start producing the output while processing the partial source sentence in the incoming input stream. These systems need to decide when to read more input and when to write the output. These decisions depend on the structure of source/target language and the information contained in the partial input sequence. Hence, read/write decision policy remains the same across different input modalities, i.e., speech and text. This motivates us to leverage the text transcripts corresponding to the speech input for improving simultaneous speech-to-text translation (SimulST). We propose Decision Attentive Regularization (DAR) to improve the decision policy of SimulST systems by using the simultaneous text-to-text translation (SimulMT) task. We also extend several techniques from the offline speech translation domain to explore the role of SimulMT task in improving SimulST performance. Overall, we achieve 34.66% / 4.5 BLEU improvement over the baseline model across different latency regimes for the MuST-C English-German (EnDe) SimulST task.

cs.SD↗
arXiv2022

Vorticity convergence from Boltzmann to 2D incompressible Euler equations below Yudovich class

It is challenging to perform a multiscale analysis of mesoscopic systems exhibiting singularities at the macroscopic scale. In this paper, we study the hydrodynamic limit of the Boltzmann equations $$\mathrm{St} \partial_t F + v\cdot \nabla_x F = \frac{1}{\mathrm{Kn}} Q(F ,F ) $$ toward the singular solutions of 2D incompressible Euler equations whose vorticity is unbounded $$\partial_t u + u \cdot \nabla_x u + \nabla_x p = 0,\text{div }u =0.$$ We obtain a microscopic description of the singularity through the so-called kinetic vorticity and understand its behavior in the vicinity of the macroscopic singularity. As a consequence of our new analysis, we settle affirmatively an open problem of the hydrodynamic limit toward Lagrangian solutions of the 2D incompressible Euler equation whose vorticity is unbounded ($ω\in L^\mathfrak{p}$ for any fixed $1 \leq \mathfrak{p} < \infty$). Moreover, we prove the convergence of kinetic vorticities toward the vorticity of the Lagrangian solution of the Euler equation. In particular, we obtain the rate of convergence when the vorticity blows up moderately in $L^\mathfrak{p}$ as $\mathfrak{p} \rightarrow \infty$ (localized Yudovich class).

math.AP↗
arXiv2022

Lipschitz continuous solutions of the Vlasov-Maxwell systems with a conductor boundary condition

We consider relativistic plasma particles subjected to an external gravitation force in a $3$D half space whose boundary is a perfect conductor. When the mean free path is much bigger than the variation of electromagnetic fields, the collision effect is negligible. As an effective PDE, we study the relativistic Vlasov-Maxwell system and its local-in-time unique solvability in the space-time locally Lipschitz space, for several basic mesoscopic (kinetic) boundary conditions: the inflow, diffuse, and specular reflection boundary conditions. We construct weak solutions to these initial-boundary value problems and study their locally Lipschitz continuity with the aid of a weight function depending on the solutions themselves. Finally, we prove the uniqueness of a solution, by using regularity estimate and realizing the Gauss's law at the boundary within Lipschitz continuous space.

math.AP↗
arXiv2022

Two-Pass End-to-End ASR Model Compression

Speech recognition on smart devices is challenging owing to the small memory footprint. Hence small size ASR models are desirable. With the use of popular transducer-based models, it has become possible to practically deploy streaming speech recognition models on small devices [1]. Recently, the two-pass model [2] combining RNN-T and LAS modules has shown exceptional performance for streaming on-device speech recognition. In this work, we propose a simple and effective approach to reduce the size of the two-pass model for memory-constrained devices. We employ a popular knowledge distillation approach in three stages using the Teacher-Student training technique. In the first stage, we use a trained RNN-T model as a teacher model and perform knowledge distillation to train the student RNN-T model. The second stage uses the shared encoder and trains a LAS rescorer for student model using the trained RNN-T+LAS teacher model. Finally, we perform deep-finetuning for the student model with a shared RNN-T encoder, RNN-T decoder, and LAS rescorer. Our experimental results on standard LibriSpeech dataset show that our system can achieve a high compression rate of 55% without significant degradation in the WER compared to the two-pass teacher model.

eess.AS↗
arXiv2021

A comparison of streaming models and data augmentation methods for robust speech recognition

In this paper, we present a comparative study on the robustness of two different online streaming speech recognition models: Monotonic Chunkwise Attention (MoChA) and Recurrent Neural Network-Transducer (RNN-T). We explore three recently proposed data augmentation techniques, namely, multi-conditioned training using an acoustic simulator, Vocal Tract Length Perturbation (VTLP) for speaker variability, and SpecAugment. Experimental results show that unidirectional models are in general more sensitive to noisy examples in the training set. It is observed that the final performance of the model depends on the proportion of training examples processed by data augmentation techniques. MoChA models generally perform better than RNN-T models. However, we observe that training of MoChA models seems to be more sensitive to various factors such as the characteristics of training sets and the incorporation of additional augmentations techniques. On the other hand, RNN-T models perform better than MoChA models in terms of latency, inference time, and the stability of training. Additionally, RNN-T models are generally more robust against noise and reverberation. All these advantages make RNN-T models a better choice for streaming on-device speech recognition compared to MoChA models.

eess.AS↗
arXiv2021

Semi-supervised transfer learning for language expansion of end-to-end speech recognition models to low-resource languages

In this paper, we propose a three-stage training methodology to improve the speech recognition accuracy of low-resource languages. We explore and propose an effective combination of techniques such as transfer learning, encoder freezing, data augmentation using Text-To-Speech (TTS), and Semi-Supervised Learning (SSL). To improve the accuracy of a low-resource Italian ASR, we leverage a well-trained English model, unlabeled text corpus, and unlabeled audio corpus using transfer learning, TTS augmentation, and SSL respectively. In the first stage, we use transfer learning from a well-trained English model. This primarily helps in learning the acoustic information from a resource-rich language. This stage achieves around 24% relative Word Error Rate (WER) reduction over the baseline. In stage two, We utilize unlabeled text data via TTS data-augmentation to incorporate language information into the model. We also explore freezing the acoustic encoder at this stage. TTS data augmentation helps us further reduce the WER by ~ 21% relatively. Finally, In stage three we reduce the WER by another 4% relative by using SSL from unlabeled audio data. Overall, our two-pass speech recognition system with a Monotonic Chunkwise Attention (MoChA) in the first pass and a full-attention in the second pass achieves a WER reduction of ~ 42% relative to the baseline.

eess.AS↗
arXiv2021

A review of on-device fully neural end-to-end automatic speech recognition algorithms

In this paper, we review various end-to-end automatic speech recognition algorithms and their optimization techniques for on-device applications. Conventional speech recognition systems comprise a large number of discrete components such as an acoustic model, a language model, a pronunciation model, a text-normalizer, an inverse-text normalizer, a decoder based on a Weighted Finite State Transducer (WFST), and so on. To obtain sufficiently high speech recognition accuracy with such conventional speech recognition systems, a very large language model (up to 100 GB) is usually needed. Hence, the corresponding WFST size becomes enormous, which prohibits their on-device implementation. Recently, fully neural network end-to-end speech recognition algorithms have been proposed. Examples include speech recognition systems based on Connectionist Temporal Classification (CTC), Recurrent Neural Network Transducer (RNN-T), Attention-based Encoder-Decoder models (AED), Monotonic Chunk-wise Attention (MoChA), transformer-based speech recognition systems, and so on. These fully neural network-based systems require much smaller memory footprints compared to conventional algorithms, therefore their on-device implementation has become feasible. In this paper, we review such end-to-end speech recognition models. We extensively discuss their structures, performance, and advantages compared to conventional algorithms.

cs.LG↗
arXiv2021

Faster Re-translation Using Non-Autoregressive Model For Simultaneous Neural Machine Translation

Recently, simultaneous translation has gathered a lot of attention since it enables compelling applications such as subtitle translation for a live event or real-time video-call translation. Some of these translation applications allow editing of partial translation giving rise to re-translation approaches. The current re-translation approaches are based on autoregressive sequence generation models (ReTA), which generate tar-get tokens in the (partial) translation sequentially. The multiple re-translations with sequential generation inReTAmodelslead to an increased inference time gap between the incoming source input and the corresponding target output as the source input grows. Besides, due to the large number of inference operations involved, the ReTA models are not favorable for resource-constrained devices. In this work, we propose a faster re-translation system based on a non-autoregressive sequence generation model (FReTNA) to overcome the aforementioned limitations. We evaluate the proposed model on multiple translation tasks and our model reduces the inference times by several orders and achieves a competitive BLEUscore compared to the ReTA and streaming (Wait-k) models.The proposed model reduces the average computation time by a factor of 20 when compared to the ReTA model by incurring a small drop in the translation quality. It also outperforms the streaming-based Wait-k model both in terms of computation time (1.5 times lower) and translation quality.

cs.CL↗
arXiv2021

Streaming end-to-end speech recognition with jointly trained neural feature enhancement

In this paper, we present a streaming end-to-end speech recognition model based on Monotonic Chunkwise Attention (MoCha) jointly trained with enhancement layers. Even though the MoCha attention enables streaming speech recognition with recognition accuracy comparable to a full attention-based approach, training this model is sensitive to various factors such as the difficulty of training examples, hyper-parameters, and so on. Because of these issues, speech recognition accuracy of a MoCha-based model for clean speech drops significantly when a multi-style training approach is applied. Inspired by Curriculum Learning [1], we introduce two training strategies: Gradual Application of Enhanced Features (GAEF) and Gradual Reduction of Enhanced Loss (GREL). With GAEF, the model is initially trained using clean features. Subsequently, the portion of outputs from the enhancement layers gradually increases. With GREL, the portion of the Mean Squared Error (MSE) loss for the enhanced output gradually reduces as training proceeds. In experimental results on the LibriSpeech corpus and noisy far-field test sets, the proposed model with GAEF-GREL training strategies shows significantly better results than the conventional multi-style training approach.

cs.SD↗