Searcharxiv⌕ Search

arXiv subjects

Chanwoo Kim

Publications and source records attributed to Chanwoo Kim.

At least 55 records · Page 3Linked to original sources
arXiv2025

Leveraging 2D Masked Reconstruction for Domain Adaptation of 3D Pose Estimation

RGB-based 3D pose estimation methods have been successful with the development of deep learning and the emergence of high-quality 3D pose datasets. However, most existing methods do not operate well for testing images whose distribution is far from that of training data. However, most existing methods do not operate well for testing images whose distribution is far from that of training data. This problem might be alleviated by involving diverse data during training, however it is non-trivial to collect such diverse data with corresponding labels (i.e. 3D pose). In this paper, we introduced an unsupervised domain adaptation framework for 3D pose estimation that utilizes the unlabeled data in addition to labeled data via masked image modeling (MIM) framework. Foreground-centric reconstruction and attention regularization are further proposed to increase the effectiveness of unlabeled data usage. Experiments are conducted on the various datasets in human and hand pose estimation tasks, especially using the cross-domain scenario. We demonstrated the effectiveness of ours by achieving the state-of-the-art accuracy on all datasets.

cs.CV↗
arXiv2025

Learning-based Dynamic Robot-to-Human Handover

This paper presents a novel learning-based approach to dynamic robot-to-human handover, addressing the challenges of delivering objects to a moving receiver. We hypothesize that dynamic handover, where the robot adjusts to the receiver's movements, results in more efficient and comfortable interaction compared to static handover, where the receiver is assumed to be stationary. To validate this, we developed a nonparametric method for generating continuous handover motion, conditioned on the receiver's movements, and trained the model using a dataset of 1,000 human-to-human handover demonstrations. We integrated preference learning for improved handover effectiveness and applied impedance control to ensure user safety and adaptiveness. The approach was evaluated in both simulation and real-world settings, with user studies demonstrating that dynamic handover significantly reduces handover time and improves user comfort compared to static methods. Videos and demonstrations of our approach are available at https://zerotohero7886.github.io/dyn-r2h-handover .

cs.RO↗
arXiv2025

Wave-U-Mamba: An End-To-End Framework For High-Quality And Efficient Speech Super Resolution

Speech Super-Resolution (SSR) is a task of enhancing low-resolution speech signals by restoring missing high-frequency components. Conventional approaches typically reconstruct log-mel features, followed by a vocoder that generates high-resolution speech in the waveform domain. However, as mel features lack phase information, this can result in performance degradation during the reconstruction phase. Motivated by recent advances with Selective State Spaces Models (SSMs), we propose a method, referred to as Wave-U-Mamba that directly performs SSR in time domain. In our comparative study, including models such as WSRGlow, NU-Wave 2, and AudioSR, Wave-U-Mamba demonstrates superior performance, achieving the lowest Log-Spectral Distance (LSD) across various low-resolution sampling rates, ranging from 8 to 24 kHz. Additionally, subjective human evaluations, scored using Mean Opinion Score (MOS) reveal that our method produces SSR with natural and human-like quality. Furthermore, Wave-U-Mamba achieves these results while generating high-resolution speech over nine times faster than baseline models on a single A100 GPU, with parameter sizes less than 2\% of those in the baseline models.

eess.AS↗
arXiv2024

Physics Informed Distillation for Diffusion Models

Diffusion models have recently emerged as a potent tool in generative modeling. However, their inherent iterative nature often results in sluggish image generation due to the requirement for multiple model evaluations. Recent progress has unveiled the intrinsic link between diffusion models and Probability Flow Ordinary Differential Equations (ODEs), thus enabling us to conceptualize diffusion models as ODE systems. Simultaneously, Physics Informed Neural Networks (PINNs) have substantiated their effectiveness in solving intricate differential equations through implicit modeling of their solutions. Building upon these foundational insights, we introduce Physics Informed Distillation (PID), which employs a student model to represent the solution of the ODE system corresponding to the teacher diffusion model, akin to the principles employed in PINNs. Through experiments on CIFAR 10 and ImageNet 64x64, we observe that PID achieves performance comparable to recent distillation methods. Notably, it demonstrates predictable trends concerning method-specific hyperparameters and eliminates the need for synthetic dataset generation during the distillation process. Both of which contribute to its easy-to-use nature as a distillation approach for Diffusion Models. Our code and pre-trained checkpoint are publicly available at: https://github.com/pantheon5100/pid_diffusion.git.

cs.LG↗
arXiv2024

Stochastic Amortization: A Unified Approach to Accelerate Feature and Data Attribution

Many tasks in explainable machine learning, such as data valuation and feature attribution, perform expensive computation for each data point and are intractable for large datasets. These methods require efficient approximations, and although amortizing the process by learning a network to directly predict the desired output is a promising solution, training such models with exact labels is often infeasible. We therefore explore training amortized models with noisy labels, and we find that this is inexpensive and surprisingly effective. Through theoretical analysis of the label noise and experiments with various models and datasets, we show that this approach tolerates high noise levels and significantly accelerates several feature attribution and data valuation methods, often yielding an order of magnitude speedup over existing approaches.

cs.LG↗
arXiv2024

Asymptotic stability of 3D relativistic collisionless plasma states in ambient magnetic fields with a boundary

Motivated by the stellar wind ejected from the upper atmosphere (Corona) of a star, we explore a boundary problem of the two-species nonlinear relativistic Vlasov-Poisson systems in the 3D half space in the presence of a constant vertical magnetic field and strong background gravity. We allow species to have different mass and charge (as proton and electron, for example). As the main result, we construct stationary solutions and establish their nonlinear dynamical asymptotic stability in time and space.

math.AP↗
arXiv2024

AdaMER-CTC: Connectionist Temporal Classification with Adaptive Maximum Entropy Regularization for Automatic Speech Recognition

In Automatic Speech Recognition (ASR) systems, a recurring obstacle is the generation of narrowly focused output distributions. This phenomenon emerges as a side effect of Connectionist Temporal Classification (CTC), a robust sequence learning tool that utilizes dynamic programming for sequence mapping. While earlier efforts have tried to combine the CTC loss with an entropy maximization regularization term to mitigate this issue, they employed a constant weighting term on the regularization during the training, which we find may not be optimal. In this work, we introduce Adaptive Maximum Entropy Regularization (AdaMER), a technique that can modulate the impact of entropy regularization throughout the training process. This approach not only refines ASR model training but ensures that as training proceeds, predictions display the desired model confidence.

eess.AS↗
arXiv2024

Latent Filling: Latent Space Data Augmentation for Zero-shot Speech Synthesis

Previous works in zero-shot text-to-speech (ZS-TTS) have attempted to enhance its systems by enlarging the training data through crowd-sourcing or augmenting existing speech data. However, the use of low-quality data has led to a decline in the overall system performance. To avoid such degradation, instead of directly augmenting the input data, we propose a latent filling (LF) method that adopts simple but effective latent space data augmentation in the speaker embedding space of the ZS-TTS system. By incorporating a consistency loss, LF can be seamlessly integrated into existing ZS-TTS systems without the need for additional training stages. Experimental results show that LF significantly improves speaker similarity while preserving speech quality.

eess.AS↗
arXiv2024

Data-driven grapheme-to-phoneme representations for a lexicon-free text-to-speech

Grapheme-to-Phoneme (G2P) is an essential first step in any modern, high-quality Text-to-Speech (TTS) system. Most of the current G2P systems rely on carefully hand-crafted lexicons developed by experts. This poses a two-fold problem. Firstly, the lexicons are generated using a fixed phoneme set, usually, ARPABET or IPA, which might not be the most optimal way to represent phonemes for all languages. Secondly, the man-hours required to produce such an expert lexicon are very high. In this paper, we eliminate both of these issues by using recent advances in self-supervised learning to obtain data-driven phoneme representations instead of fixed representations. We compare our lexicon-free approach against strong baselines that utilize a well-crafted lexicon. Furthermore, we show that our data-driven lexicon-free method performs as good or even marginally better than the conventional rule-based or lexicon-based neural G2Ps in terms of Mean Opinion Score (MOS) while using no prior language lexicon or phoneme set, i.e. no linguistic expertise.

cs.CL↗
arXiv2024

Macroscopic estimate of the linear Boltzmann and Landau equations with Specular reflection boundary

In this short note, we prove an $L^6$-control of the macroscopic part of the linear Boltzmann and Landau equations. This result is an extension of the test function method of Esposito-Guo-Kim-Marra~\cite{EGKM}\cite{EGKM2} to the specular reflection boundary condition, in which we crucially used the Korn's inequality \cite{DV2} and the system of symmetric Poisson equations \cite{Bernou}.

math.AP↗
arXiv2023

Gradient Decay in the Boltzmann theory of Non-isothermal boundary

We consider the Boltzmann equation in convex domain with non-isothermal boundary of diffuse reflection. For both unsteady/steady problems, we construct solutions belong to $W^{1,p}_x$ for any $p<3$. We prove that the unsteady solution converges to the steady solution in the same Sobolev space exponentially fast as $t \rightarrow \infty$.

math.AP↗
arXiv2023

On the compression of shallow non-causal ASR models using knowledge distillation and tied-and-reduced decoder for low-latency on-device speech recognition

Recently, the cascaded two-pass architecture has emerged as a strong contender for on-device automatic speech recognition (ASR). A cascade of causal and shallow non-causal encoders coupled with a shared decoder enables operation in both streaming and look-ahead modes. In this paper, we propose shallow cascaded model by combining various model compression techniques such as knowledge distillation, shared decoder, and tied-and-reduced transducer network in order to reduce the model footprint. The shared decoder is changed into a tied-and-reduced network. The cascaded two-pass model is further compressed using knowledge distillation using a Kullback-Leibler divergence loss on the model posteriors. We demonstrate a 50% reduction in the size of a 41 M parameter cascaded teacher model with no noticeable degradation in ASR accuracy and a 30% reduction in latency

cs.SD↗
arXiv2023

Class-Wise Buffer Management for Incremental Object Detection: An Effective Buffer Training Strategy

Class incremental learning aims to solve a problem that arises when continuously adding unseen class instances to an existing model This approach has been extensively studied in the context of image classification; however its applicability to object detection is not well established yet. Existing frameworks using replay methods mainly collect replay data without considering the model being trained and tend to rely on randomness or the number of labels of each sample. Also, despite the effectiveness of the replay, it was not yet optimized for the object detection task. In this paper, we introduce an effective buffer training strategy (eBTS) that creates the optimized replay buffer on object detection. Our approach incorporates guarantee minimum and hierarchical sampling to establish the buffer customized to the trained model. %These methods can facilitate effective retrieval of prior knowledge. Furthermore, we use the circular experience replay training to optimally utilize the accumulated buffer data. Experiments on the MS COCO dataset demonstrate that our eBTS achieves state-of-the-art performance compared to the existing replay schemes.

cs.CV↗
arXiv2023

Nonlinear asymptotic stability of inhomogeneous steady solutions to boundary problems of Vlasov-Poisson equation

We consider an ensemble of mass collisionless particles, which interact mutually either by an attraction of Newton's law of gravitation or by an electrostatic repulsion of Coulomb's law, under a background downward gravity in a horizontally-periodic 3D half-space, whose inflow distribution at the boundary is prescribed. We investigate a nonlinear asymptotic stability of its generic steady states in the dynamical kinetic PDE theory of the Vlasov-Poisson equations. We construct Lipschitz continuous space-inhomogeneous steady states and establish exponentially fast asymptotic stability of these steady states with respect to small perturbation in a weighted Sobolev topology. In this proof, we crucially use the Lipschitz continuity in the velocity of the steady states. Moreover, we establish well-posedness and regularity estimates for both steady and dynamic problems.

math.AP↗
arXiv2023

Mitigating the Exposure Bias in Sentence-Level Grapheme-to-Phoneme (G2P) Transduction

Text-to-Text Transfer Transformer (T5) has recently been considered for the Grapheme-to-Phoneme (G2P) transduction. As a follow-up, a tokenizer-free byte-level model based on T5 referred to as ByT5, recently gave promising results on word-level G2P conversion by representing each input character with its corresponding UTF-8 encoding. Although it is generally understood that sentence-level or paragraph-level G2P can improve usability in real-world applications as it is better suited to perform on heteronyms and linking sounds between words, we find that using ByT5 for these scenarios is nontrivial. Since ByT5 operates on the character level, it requires longer decoding steps, which deteriorates the performance due to the exposure bias commonly observed in auto-regressive generation models. This paper shows that the performance of sentence-level and paragraph-level G2P can be improved by mitigating such exposure bias using our proposed loss-based sampling method.

cs.CL↗
arXiv2023

Dynamical Billiard and a long-time behavior of the Boltzmann equation in general 3D toroidal domains

Establishing global well-posedness and convergence toward equilibrium of the Boltzmann equation with specular reflection boundary condition has been one of the central questions in the subject of kinetic theory. Despite recent significant progress in this question when domains are strictly convex, as shown by Guo and Kim-Lee, the same question without the strict convexity of domains is still totally open in 3D. The major difficulty arises when a billiard map has an infinite number of bounces in a finite time interval or when the map fails to be Lipschitz continuous, both of which happen generically when the domain is non-convex. In this paper, we develop a new method to control a billiard map on a surface of revolution generated by revolving any planar analytic convex closed curve (e.g., typical shape of tokamak reactors' chamber). In particular, we classify and measure the size (to be small) of a pullback set (along the billiard trajectory) of the infinite-bouncing and singular-bouncing cases. As a consequence, we solve the open question affirmatively in such domains. To the best of our knowledge, this work is the first construction of global solutions to the hard-sphere Boltzmann equation in generic non-convex 3-dimensional domains. In Appendix, we introduce a novel method for constructive coercivity of a linearized collision operator $L$ when the specular boundary condition is imposed. In particular, this method works for a periodic cylindrical domain with an annulus cross-section.

math.AP↗
arXiv2023

Contrastive Corpus Attribution for Explaining Representations

Despite the widespread use of unsupervised models, very few methods are designed to explain them. Most explanation methods explain a scalar model output. However, unsupervised models output representation vectors, the elements of which are not good candidates to explain because they lack semantic meaning. To bridge this gap, recent works defined a scalar explanation output: a dot product-based similarity in the representation space to the sample being explained (i.e., an explicand). Although this enabled explanations of unsupervised models, the interpretation of this approach can still be opaque because similarity to the explicand's representation may not be meaningful to humans. To address this, we propose contrastive corpus similarity, a novel and semantically meaningful scalar explanation output based on a reference corpus and a contrasting foil set of samples. We demonstrate that contrastive corpus similarity is compatible with many post-hoc feature attribution methods to generate COntrastive COrpus Attributions (COCOA) and quantitatively verify that features important to the corpus are identified. We showcase the utility of COCOA in two ways: (i) we draw insights by explaining augmentations of the same image in a contrastive learning setting (SimCLR); and (ii) we perform zero-shot object localization by explaining the similarity of image representations to jointly learned text representations (CLIP).

cs.LG↗