arXiv subjects

Kyungmin Lee

Publications and source records attributed to Kyungmin Lee.

At least 73 records · Page 4Linked to original sources

arXiv2023

S-CLIP: Semi-supervised Vision-Language Learning using Few Specialist Captions

Vision-language models, such as contrastive language-image pre-training (CLIP), have demonstrated impressive results in natural image domains. However, these models often struggle when applied to specialized domains like remote sensing, and adapting to such domains is challenging due to the limited number of image-text pairs available for training. To address this, we propose S-CLIP, a semi-supervised learning method for training CLIP that utilizes additional unpaired images. S-CLIP employs two pseudo-labeling strategies specifically designed for contrastive learning and the language modality. The caption-level pseudo-label is given by a combination of captions of paired images, obtained by solving an optimal transport problem between unpaired and paired images. The keyword-level pseudo-label is given by a keyword in the caption of the nearest paired image, trained through partial label learning that assumes a candidate set of labels for supervision instead of the exact one. By combining these objectives, S-CLIP significantly enhances the training of CLIP using only a few image-text pairs, as demonstrated in various specialist domains, including remote sensing, fashion, scientific figures, and comics. For instance, S-CLIP improves CLIP by 10% for zero-shot classification and 4% for image-text retrieval on the remote sensing benchmark, matching the performance of supervised CLIP while using three times fewer image-text pairs.

cs.CV↗

arXiv2023

Collaborative Score Distillation for Consistent Visual Synthesis

Generative priors of large-scale text-to-image diffusion models enable a wide range of new generation and editing applications on diverse visual modalities. However, when adapting these priors to complex visual modalities, often represented as multiple images (e.g., video), achieving consistency across a set of images is challenging. In this paper, we address this challenge with a novel method, Collaborative Score Distillation (CSD). CSD is based on the Stein Variational Gradient Descent (SVGD). Specifically, we propose to consider multiple samples as "particles" in the SVGD update and combine their score functions to distill generative priors over a set of images synchronously. Thus, CSD facilitates seamless integration of information across 2D images, leading to a consistent visual synthesis across multiple samples. We show the effectiveness of CSD in a variety of tasks, encompassing the visual editing of panorama images, videos, and 3D scenes. Our results underline the competency of CSD as a versatile method for enhancing inter-sample consistency, thereby broadening the applicability of text-to-image diffusion models.

cs.CV↗

arXiv2023

Triangular lattice Hubbard model physics at intermediate temperatures

Moire systems offer an exciting playground to study many-body effects of strongly correlated electrons in regimes that are not easily accessible in conventional material settings. Motivated by a recent experiment on $\text{WSe}_2/\text{WS}_2$ moire bilayers [Y. Tang et al., Nature 579, 353-358 (2020)], which realizes a triangular superlattice with a small hopping (of approximately 10 Kelvin), with tunable density of holes, we explore the Hubbard model on the triangular lattice for intermediate temperatures $t \lesssim T < U$. Employing finite temperature Lanczos calculations, and closely following the fitting protocols used in the experiment, we recover the observed trends in the reported Curie-Weiss temperature $Θ$ with filling, using the reported interaction strength $U/t=20$. We focus on the large increase of $|Θ|$ on decreasing the density below half filling and the sign change of $Θ$ at higher fillings, which signals the onset of ferromagnetism. The increase in $|Θ|$ is also seen in the $t$-$J$ model (the low energy limit of the Hubbard model) in the intermediate temperature range, which we clarify is opposite to the trend in its high temperature limit. Differences between the low, intermediate and high temperature regimes are discussed. Our numerical calculations also capture the crossover between short-range antiferromagnetic to ferromagnetic order in the intermediate temperature regime, a result broadly consistent with the experimental findings. We find that this behavior is a finite-temperature remnant of the underlying zero temperature phase transition, which we explore with ground state density matrix renormalization group calculations. We provide evidence of ferromagnetism characterized by weak (but robust) correlations that explain the small $Θ$ seen in the experiment.

cond-mat.str-el↗

arXiv2023

Realization of Non-Hermitian Hopf Bundle Matter

Line excitations in topological phases are a subject of particular interest because their mutual linking structures encode robust topological information of matter. It has been recently shown that the linking and winding of complex eigenenergy strings can classify one-dimensional non-Hermitian topological matter. However, in higher dimensions, bundles of linked strings can emerge such that every string is mutually linked with all the other strings. Interestingly, despite being an unconventional topological structure, a non-Hermitian Hopf bundle has not been experimentally clarified. Here, we make the first attempt to explore the non-Hermitian Hopf bundle by visualizing the global linking structure of spinor strings in the momentum space of a two-dimensional electric circuit. By exploiting the flexibility of reconfigurable couplings between circuit nodes, we can study the non-Hermitian topological phase transition and gain insight into the intricate structure of the Hopf bundle. Furthermore, we find that the emergence of a higher-order skin effect in real space is accompanied by the linking of spinor strings in momentum space, revealing a bulk-boundary correspondence between the two domains. The proposed non-Hermitian Hopf bundle platform and visualization methodology pave the way to design new topologically robust non-Hermitian phases of matter.

physics.app-ph↗

arXiv2023

STUNT: Few-shot Tabular Learning with Self-generated Tasks from Unlabeled Tables

Learning with few labeled tabular samples is often an essential requirement for industrial machine learning applications as varieties of tabular data suffer from high annotation costs or have difficulties in collecting new samples for novel tasks. Despite the utter importance, such a problem is quite under-explored in the field of tabular learning, and existing few-shot learning schemes from other domains are not straightforward to apply, mainly due to the heterogeneous characteristics of tabular data. In this paper, we propose a simple yet effective framework for few-shot semi-supervised tabular learning, coined Self-generated Tasks from UNlabeled Tables (STUNT). Our key idea is to self-generate diverse few-shot tasks by treating randomly chosen columns as a target label. We then employ a meta-learning scheme to learn generalizable knowledge with the constructed tasks. Moreover, we introduce an unsupervised validation scheme for hyperparameter search (and early stopping) by generating a pseudo-validation set using STUNT from unlabeled data. Our experimental results demonstrate that our simple framework brings significant performance gain under various tabular few-shot learning benchmarks, compared to prior semi- and self-supervised baselines. Code is available at https://github.com/jaehyun513/STUNT.

cs.LG↗

arXiv2023

GCISG: Guided Causal Invariant Learning for Improved Syn-to-real Generalization

Training a deep learning model with artificially generated data can be an alternative when training data are scarce, yet it suffers from poor generalization performance due to a large domain gap. In this paper, we characterize the domain gap by using a causal framework for data generation. We assume that the real and synthetic data have common content variables but different style variables. Thus, a model trained on synthetic dataset might have poor generalization as the model learns the nuisance style variables. To that end, we propose causal invariance learning which encourages the model to learn a style-invariant representation that enhances the syn-to-real generalization. Furthermore, we propose a simple yet effective feature distillation method that prevents catastrophic forgetting of semantic knowledge of the real domain. In sum, we refer to our method as Guided Causal Invariant Syn-to-real Generalization that effectively improves the performance of syn-to-real generalization. We empirically verify the validity of proposed methods, and especially, our method achieves state-of-the-art on visual syn-to-real domain generalization tasks such as image classification and semantic segmentation.

cs.CV↗

arXiv2023

RenyiCL: Contrastive Representation Learning with Skew Renyi Divergence

Contrastive representation learning seeks to acquire useful representations by estimating the shared information between multiple views of data. Here, the choice of data augmentation is sensitive to the quality of learned representations: as harder the data augmentations are applied, the views share more task-relevant information, but also task-irrelevant one that can hinder the generalization capability of representation. Motivated by this, we present a new robust contrastive learning scheme, coined RényiCL, which can effectively manage harder augmentations by utilizing Rényi divergence. Our method is built upon the variational lower bound of Rényi divergence, but a naïve usage of a variational method is impractical due to the large variance. To tackle this challenge, we propose a novel contrastive objective that conducts variational estimation of a skew Rényi divergence and provide a theoretical guarantee on how variational estimation of skew divergence leads to stable training. We show that Rényi contrastive learning objectives perform innate hard negative sampling and easy positive sampling simultaneously so that it can selectively learn useful features and ignore nuisance features. Through experiments on ImageNet, we show that Rényi contrastive learning with stronger augmentations outperforms other self-supervised methods without extra regularization or computational overhead. Moreover, we also validate our method on other domains such as graph and tabular, showing empirical gain over other contrastive methods.

stat.ML↗

arXiv2022

Multimagnon dynamics and thermalization in the $S=1$ easy-axis ferromagnetic chain

Quasiparticles are physically motivated mathematical constructs for simplifying the seemingly complicated many-body description of solids. A complete understanding of their dynamics and the nature of the effective interactions between them provides rich information on real material properties at the microscopic level. In this work, we explore the dynamics and interactions of magnon quasiparticles in a ferromagnetic spin-1 Heisenberg chain with easy-axis onsite anisotropy, a model relevant for the explanation of recent terahertz optics experiments on NiNb$_2$O$_6$ [P. Chauhan et al., Phys. Rev. Lett. 124, 037203 (2020)],and nonequilibrium dynamics in ultracold atomic settings [W.C. Chung et al., Phys. Rev. Lett. 126, 163203 (2021)]. We build a picture for the properties of clouds of a few magnons with the help of exact diagonalization and density matrix renormalization group calculations supported by physically motivated Jastrow wavefunctions. We show how the binding energy of magnons effectively reduces with their number and explain how this energy scale is of direct relevance for dynamical magnetic susceptibility measurements. This understanding is used to make predictions for ultracold-atomic platforms which are ideally suited to study the thermalization of multimagnon states. We simulate the nonequilibrium dynamics of these chains using the matrix product state based time-evolution block decimation algorithm and explore the dependence of revivals and thermalization on magnon density and easy-axis onsite anisotropy (which controls the strength of effective magnon interactions). We observe behaviors akin to those reported for many-body quantum scars which we explain with an analytic approximation that is accurate in the limit of small anisotropy.

cond-mat.str-el↗

arXiv2022

Revealing non-Hermitian band structures of photonic Floquet media

Periodically driven systems, characterised by their inherent non-equilibrium dynamics, are ubiquitously found in both classical and quantum regimes. In the field of photonics, these Floquet systems have begun to provide insight into how time periodicity can extend the concept of spatially periodic photonic crystals and metamaterials to the time domain. However, despite the necessity arising from the presence of non-reciprocal coupling between states in a photonic Floquet medium, a unified non-Hermitian band structure description remains elusive. Here, we experimentally reveal the unique Bloch-Floquet and non-Bloch band structures of a photonic Floquet medium emulated in the microwave regime with a one-dimensional array of time-periodically driven resonators. Specifically, these non-Hermitian band structures are shown to be two measurable distinct subsets of complex eigenfrequency surfaces of the photonic Floquet medium defined in complex momentum space. In the Bloch-Floquet band structure, the driving-induced non-reciprocal coupling between oppositely signed frequency states leads to opening of momentum gaps along the real momentum axis, at the edges of which exceptional phase transitions occur. More interestingly, we show that the non-Bloch band structure defined in the complex Brillouin zone supplements the information on the morphology of complex eigenfrequency surfaces of the photonic Floquet medium. Our work paves the way for a comprehensive understanding of photonic Floquet media in complex energy-momentum space and could provide general guidelines for the study of non-equilibrium photonic phases of matter.

physics.app-ph↗

arXiv2021

Frustration-induced emergent Hilbert space fragmentation

Although most quantum systems thermalize locally on short time scales independent of initial conditions, recent developments have shown this is not always the case. Lattice geometry and quantum mechanics can conspire to produce constrained quantum dynamics and associated glassy behavior, a phenomenon that falls outside the rubric of the eigenstate thermalization hypothesis. Constraints "fragment" the many-body Hilbert space due to which some states remain insulated from others and the system fails to attain thermal equilibrium. Such fragmentation is a hallmark of geometrically frustrated magnets with low-energy "icelike manifolds" exhibiting a broad range of relaxation times for different initial states. Focusing on the highly frustrated kagome lattice, we demonstrate these phenomena in the Balents-Fisher-Girvin Hamiltonian (easy-axis regime), and a three-coloring model (easy-plane regime), both with constrained Hilbert spaces. We study their level statistics and relaxation dynamics to develop a coherent picture of fragmentation in various limits of the XXZ model on the kagome lattice.

cond-mat.str-el↗

arXiv2021

A forecast of the sensitivity on the measurement of the optical depth to reionization with the GroundBIRD experiment

We compute the expected sensitivity on measurements of optical depth to reionization for a ground-based experiment at Teide Observatory. We simulate polarized partial sky maps for the GroundBIRD experiment at the frequencies 145 and 220 GHz. We perform fits for the simulated maps with our pixel-based likelihood to extract the optical depth to reionization. The noise levels of polarization maps are estimated as 110 $μ\mathrm{K~arcmin}$ and 780 $ μ\mathrm{K~arcmin}$ for 145 and 220 GHz, respectively, by assuming a three-year observing campaign and sky coverages of 0.537 for 145 GHz and 0.462 for 220 GHz. Our sensitivities for the optical depth to reionization are found to be $σ_τ$=0.030 with the simulated GroundBIRD maps, and $σ_τ$=0.012 by combining with the simulated QUIJOTE maps at 11, 13, 17, 19, 30, and 40 GHz.

astro-ph.CO↗

arXiv2021

Sequential Routing Framework: Fully Capsule Network-based Speech Recognition

Capsule networks (CapsNets) have recently gotten attention as a novel neural architecture. This paper presents the sequential routing framework which we believe is the first method to adapt a CapsNet-only structure to sequence-to-sequence recognition. Input sequences are capsulized then sliced by a window size. Each slice is classified to a label at the corresponding time through iterative routing mechanisms. Afterwards, losses are computed by connectionist temporal classification (CTC). During routing, the required number of parameters can be controlled by the window size regardless of the length of sequences by sharing learnable weights across the slices. We additionally propose a sequential dynamic routing algorithm to replace traditional dynamic routing. The proposed technique can minimize decoding speed degradation caused by the routing iterations since it can operate in a non-iterative manner without dropping accuracy. The method achieves a 1.1% lower word error rate at 16.9% on the Wall Street Journal corpus compared to bidirectional long short-term memory-based CTC networks. On the TIMIT corpus, it attains a 0.7% lower phone error rate at 17.5% compared to convolutional neural network-based CTC networks (Zhang et al., 2016).

eess.AS↗

arXiv2020

Metal-to-insulator transition in Pt-doped TiSe$_2$ driven by emergent network of narrow transport channels

Metal-to-insulator transitions (MIT) can be driven by a number of different mechanisms, each resulting in a different type of insulator -- Change in chemical potential can induce a transition from a metal to a band insulator; strong correlations can drive a metal into a Mott insulator with an energy gap; an Anderson transition, on the other hand, due to disorder leads to a localized insulator without a gap in the spectrum. Here we report the discovery of an alternative route for MIT driven by the creation of a network of narrow channels. Transport data on Pt substituted for Ti in TiSe$_2$ shows a dramatic increase of resistivity by five orders of magnitude for few % of Pt substitution, with a power-law dependence of the temperature-dependent resistivity $ρ(T)$. Our scanning tunneling microscopy data show that Pt induces an irregular network of nanometer-thick domain walls (DWs) of charge density wave (CDW) order, which pull charge carriers out of the bulk and into the DWs. While the CDW domains are gapped, the charges confined to the narrow DWs interact strongly, with pseudogap-like suppression in the local density of states, even when they were weakly interacting in the bulk, and scatter at the DW network interconnects thereby generating the highly resistive state. Angle-resolved photoemission spectroscopy spectra exhibit pseudogap behavior corroborating the spatial coexistence of gapped domains and narrow domain walls with excess charge carriers.

cond-mat.str-el↗

arXiv2020

GroundBIRD : A CMB polarization experiment with MKID arrays

GroundBIRD is a ground-based experiment for the precise observation of the polarization of the cosmic microwave background (CMB). To achieve high sensitivity at large angular scale, we adopt three features in this experiment: fast rotation scanning, microwave kinetic inductance detector (MKID) and cold optics. The rotation scanning strategy has the advantage to suppress $1/f$ noise. It also provides a large sky coverage of 40\%, which corresponds to the large angular scales of $l \sim 6$. This allows us to constrain the tensor-to-scalar ratio by using low $l$ B-mode spectrum. The focal plane consists of 7 MKID arrays for two target frequencies, 145 GHz and 220 GHz band. There are 161 pixels in total, of which 138 are for 144 GHz and 23 are for 220 GHz. This array is currently under development and the prototype will soon be evaluated in telescope. The GroundBIRD telescope will observe the CMB at the Teide observatory. The telescope was moved from Japan to Tenerife and is now under test. We present the status and plan of the GroundBIRD experiment.

astro-ph.IM↗

arXiv2020

Applying GPGPU to Recurrent Neural Network Language Model based Fast Network Search in the Real-Time LVCSR

Recurrent Neural Network Language Models (RNNLMs) have started to be used in various fields of speech recognition due to their outstanding performance. However, the high computational complexity of RNNLMs has been a hurdle in applying the RNNLM to a real-time Large Vocabulary Continuous Speech Recognition (LVCSR). In order to accelerate the speed of RNNLM-based network searches during decoding, we apply the General Purpose Graphic Processing Units (GPGPUs). This paper proposes a novel method of applying GPGPUs to RNNLM-based graph traversals. We have achieved our goal by reducing redundant computations on CPUs and amount of transfer between GPGPUs and CPUs. The proposed approach was evaluated on both WSJ corpus and in-house data. Experiments shows that the proposed approach achieves the real-time speed in various circumstances while maintaining the Word Error Rate (WER) to be relatively 10% lower than that of n-gram models.

cs.CL↗

arXiv2020

Exact three-colored quantum scars from geometric frustration

Non-equilibrium properties of quantum materials present many intriguing properties, among them athermal behavior, which violates the eigenstate thermalization hypothesis. Such behavior has primarily been observed in disordered systems. More recently, experimental and theoretical evidence for athermal eigenstates, known as "quantum scars" has emerged in non-integrable disorder-free models in one dimension with constrained dynamics. In this work, we show the existence of quantum scar eigenstates and investigate their dynamical properties in many simple two-body Hamiltonians with "staggered" interactions, involving ferromagnetic and antiferromagnetic motifs, in arbitrary dimensions. These magnetic models include simple modifications of widely studied ones (e.g., the XXZ model) on a variety of frustrated and unfrustrated lattices. We demonstrate our ideas by focusing on the two dimensional frustrated spin-1/2 kagome antiferromagnet, which was previously shown to harbor a special exactly solvable point with "three-coloring" ground states in its phase diagram. For appropriately chosen initial product states -- for example, those which correspond to any state of valid three-colors -- we show the presence of robust quantum revivals, which survive the addition of anisotropic terms. We also suggest avenues for future experiments which may see this effect in real materials.

cond-mat.str-el↗

arXiv2020

Attention based on-device streaming speech recognition with large speech corpus

In this paper, we present a new on-device automatic speech recognition (ASR) system based on monotonic chunk-wise attention (MoChA) models trained with large (> 10K hours) corpus. We attained around 90% of a word recognition rate for general domain mainly by using joint training of connectionist temporal classifier (CTC) and cross entropy (CE) losses, minimum word error rate (MWER) training, layer-wise pre-training and data augmentation methods. In addition, we compressed our models by more than 3.4 times smaller using an iterative hyper low-rank approximation (LRA) method while minimizing the degradation in recognition accuracy. The memory footprint was further reduced with 8-bit quantization to bring down the final model size to lower than 39 MB. For on-demand adaptation, we fused the MoChA models with statistical n-gram models, and we could achieve a relatively 36% improvement on average in word error rate (WER) for target domains including the general domain.

eess.AS↗

arXiv2019

end-to-end training of a large vocabulary end-to-end speech recognition system

In this paper, we present an end-to-end training framework for building state-of-the-art end-to-end speech recognition systems. Our training system utilizes a cluster of Central Processing Units(CPUs) and Graphics Processing Units (GPUs). The entire data reading, large scale data augmentation, neural network parameter updates are all performed "on-the-fly". We use vocal tract length perturbation [1] and an acoustic simulator [2] for data augmentation. The processed features and labels are sent to the GPU cluster. The Horovod allreduce approach is employed to train neural network parameters. We evaluated the effectiveness of our system on the standard Librispeech corpus [3] and the 10,000-hr anonymized Bixby English dataset. Our end-to-end speech recognition system built using this training infrastructure showed a 2.44 % WER on test-clean of the LibriSpeech test set after applying shallow fusion with a Transformer language model (LM). For the proprietary English Bixby open domain test set, we obtained a WER of 7.92 % using a Bidirectional Full Attention (BFA) end-to-end model after applying shallow fusion with an RNN-LM. When the monotonic chunckwise attention (MoCha) based approach is employed for streaming speech recognition, we obtained a WER of 9.95 % on the same Bixby open domain test set.

eess.AS↗