arXiv subjects

Jongmin Kim

Publications and source records attributed to Jongmin Kim.

At least 37 records · Page 2Linked to original sources

arXiv2025

Thunder-LLM: Efficiently Adapting LLMs to Korean with Minimal Resources

Since state-of-the-art LLMs often underperform in languages other than English or Chinese, improving the capability of LLMs in new languages has become an essential task. Moreover, LLMs' entire end-to-end training process remains largely unknown to the public due to proprietary reasons, technical complexity, inconsistent documentation, and ethical considerations. The complete picture remains a closely guarded secret within the industry. This paper presents methods to adapt an existing English-based LLM to Korean in a low-budget scenario. We describe the entire end-to-end process: collecting Korean datasets, preprocessing the data, training the model, creating downstream benchmarks, and conducting evaluations. The evaluation results indicate that our method can effectively and cost-efficiently add new language capabilities to existing LLMs. Our new bilingual models, Thunder-LLM and Thunder-LLM-Ins, achieve superior Korean performance compared to state-of-the-art models while utilizing minimal data and computational resources. We share our comprehensive experience and make the code publicly available.

cs.CL↗

arXiv2025

NeuJeans: Private Neural Network Inference with Joint Optimization of Convolution and FHE Bootstrapping

Fully homomorphic encryption (FHE) is a promising cryptographic primitive for realizing private neural network inference (PI) services by allowing a client to fully offload the inference task to a cloud server while keeping the client data oblivious to the server. This work proposes NeuJeans, an FHE-based solution for the PI of deep convolutional neural networks (CNNs). NeuJeans tackles the critical problem of the enormous computational cost for the FHE evaluation of CNNs. We introduce a novel encoding method called Coefficients-in-Slot (CinS) encoding, which enables multiple convolutions in one HE multiplication without costly slot permutations. We further observe that CinS encoding is obtained by conducting the first several steps of the Discrete Fourier Transform (DFT) on a ciphertext in conventional Slot encoding. This property enables us to save the conversion between CinS and Slot encodings as bootstrapping a ciphertext starts with DFT. Exploiting this, we devise optimized execution flows for various two-dimensional convolution (conv2d) operations and apply them to end-to-end CNN implementations. NeuJeans accelerates the performance of conv2d-activation sequences by up to 5.68 times compared to state-of-the-art FHE-based PI work and performs the PI of a CNN at the scale of ImageNet within a mere few seconds.

cs.CR↗

arXiv2024

Duplex: A Device for Large Language Models with Mixture of Experts, Grouped Query Attention, and Continuous Batching

Large language models (LLMs) have emerged due to their capability to generate high-quality content across diverse contexts. To reduce their explosively increasing demands for computing resources, a mixture of experts (MoE) has emerged. The MoE layer enables exploiting a huge number of parameters with less computation. Applying state-of-the-art continuous batching increases throughput; however, it leads to frequent DRAM access in the MoE and attention layers. We observe that conventional computing devices have limitations when processing the MoE and attention layers, which dominate the total execution time and exhibit low arithmetic intensity (Op/B). Processing MoE layers only with devices targeting low-Op/B such as processing-in-memory (PIM) architectures is challenging due to the fluctuating Op/B in the MoE layer caused by continuous batching. To address these challenges, we propose Duplex, which comprises xPU tailored for high-Op/B and Logic-PIM to effectively perform low-Op/B operation within a single device. Duplex selects the most suitable processor based on the Op/B of each layer within LLMs. As the Op/B of the MoE layer is at least 1 and that of the attention layer has a value of 4-8 for grouped query attention, prior PIM architectures are not efficient, which place processing units inside DRAM dies and only target extremely low-Op/B (under one) operations. Based on recent trends, Logic-PIM adds more through-silicon vias (TSVs) to enable high-bandwidth communication between the DRAM die and the logic die and place powerful processing units on the logic die, which is best suited for handling low-Op/B operations ranging from few to a few dozens. To maximally utilize the xPU and Logic-PIM, we propose expert and attention co-processing.

cs.AR↗

arXiv2024

CiFHER: A Chiplet-Based FHE Accelerator with a Resizable Structure

Fully homomorphic encryption (FHE) is in the spotlight as a definitive solution for privacy, but the high computational overhead of FHE poses a challenge to its practical adoption. Although prior studies have attempted to design ASIC accelerators to mitigate the overhead, their designs require excessive chip resources (e.g., areas) to contain and process massive data for FHE operations. We propose CiFHER, a chiplet-based FHE accelerator with a resizable structure, to tackle the challenge with a cost-effective multi-chip module (MCM) design. First, we devise a flexible core architecture whose configuration is adjustable to conform to the global organization of chiplets and design constraints. Its distinctive feature is a composable functional unit providing varying computational throughput for the number-theoretic transform, the most dominant function in FHE. Then, we establish generalized data mapping methodologies to minimize the interconnect overhead when organizing the chips into the MCM package in a tiled manner, which becomes a significant bottleneck due to the packaging constraints. This study demonstrates that a CiFHER package composed of a number of compact chiplets provides performance comparable to state-of-the-art monolithic ASIC accelerators while significantly reducing the package-wide power consumption and manufacturing cost.

cs.AR↗

arXiv2024

Flattening a trapped atomic gas using a programmable optical potential in a feedback loop

We present a method for producing a flat, large-area Fermi gas of $^6$Li with a uniform area density. The method uses a programmable optical potential within a feedback loop to flatten the in-plane trapping potential for atoms. The optical potential is generated using a laser beam, whose intensity profile is adjusted by a spatial light modulator and optimized through measurements of the density distribution of the sample. The resulting planar sample exhibits a uniform area density within a region of about 480 $μ$m in diameter and the standard deviation of the trap bottom potential is estimated to be $\approx k_B \times$ 6.1 nK, which is less than 20$\%$ of the transverse confinement energy. We discuss a dimensional crossover toward 2D regime by reducing the number of atoms in the planar trap, including the effect of the spatial variation of the transverse trapping frequency in the large-area sample.

cond-mat.quant-gas↗

arXiv2024

Chaos-assisted Turbulence in Spinor Bose-Einstein Condensates

We present a turbulence-sustaining mechanism in a spinor Bose-Einstein condensate, which is based on the chaotic nature of internal spin dynamics. Magnetic driving induces a complete chaotic evolution of the local spin state, thereby continuously randomizing the spin texture of the condensate to maintain the turbulent state. We experimentally demonstrate the onset of turbulence in the driven condensate as the driving frequency changes and show that it is consistent with the regular-to-chaotic transition of the local spin dynamics. This chaos-assisted turbulence establishes the spin-driven spinor condensate as an intriguing platform for exploring quantum chaos and related superfluid turbulence phenomena.

cond-mat.quant-gas↗

arXiv2023

HyPHEN: A Hybrid Packing Method and Optimizations for Homomorphic Encryption-Based Neural Networks

Convolutional neural network (CNN) inference using fully homomorphic encryption (FHE) is a promising private inference (PI) solution due to the capability of FHE that enables offloading the whole computation process to the server while protecting the privacy of sensitive user data. Prior FHE-based CNN (HCNN) work has demonstrated the feasibility of constructing deep neural network architectures such as ResNet using FHE. Despite these advancements, HCNN still faces significant challenges in practicality due to the high computational and memory overhead. To overcome these limitations, we present HyPHEN, a deep HCNN construction that incorporates novel convolution algorithms (RAConv and CAConv), data packing methods (2D gap packing and PRCR scheme), and optimization techniques tailored to HCNN construction. Such enhancements enable HyPHEN to substantially reduce the memory footprint and the number of expensive homomorphic operations, such as ciphertext rotation and bootstrapping. As a result, HyPHEN brings the latency of HCNN CIFAR-10 inference down to a practical level at 1.4 seconds (ResNet-20) and demonstrates HCNN ImageNet inference for the first time at 14.7 seconds (ResNet-18).

cs.CR↗

arXiv2023

The relevance of structural variability in the time-domain for computational reflection anisotropy spectroscopy at solid-liquid interfaces

In electrochemistry, reactions and charge-transfer are to a large extent determined by the atomistic structure of the solid-liquid interface. Yet due to the presence of the liquid electrolyte, many surface-science methods cannot be applied here. Hence, the exact microscopic structure that is present under operating conditions often remains unknown. Reflection anisotropy spectroscopy (RAS) is one of the few techniques that allow for an in operando investigation of the structure of solid-liquid interfaces. However, an interpretation of RAS data on the atomistic scale can only be obtained by comparison to computational spectroscopy. While the number of computational RAS studies related to electrochemical systems is currently still limited, those studies so far have not taken into account the dynamic nature of the solid-liquid interface. In this work, we investigate the temporal evolution of the spectroscopic response of the Au(110) missing row reconstruction in contact with water by combining ab initio molecular dynamics with computational spectroscopy. Our results show significant changes in the time evolution of the RA spectra, in particular providing an explanation for the typically observed differences in intensity when comparing theory and experiment. Moreover, these findings point to the importance of structural surface/interface variability while at the same time emphasising the potential of RAS for probing these dynamic interfaces.

cond-mat.mtrl-sci↗

arXiv2023

Toward Practical Privacy-Preserving Convolutional Neural Networks Exploiting Fully Homomorphic Encryption

Incorporating fully homomorphic encryption (FHE) into the inference process of a convolutional neural network (CNN) draws enormous attention as a viable approach for achieving private inference (PI). FHE allows delegating the entire computation process to the server while ensuring the confidentiality of sensitive client-side data. However, practical FHE implementation of a CNN faces significant hurdles, primarily due to FHE's substantial computational and memory overhead. To address these challenges, we propose a set of optimizations, which includes GPU/ASIC acceleration, an efficient activation function, and an optimized packing scheme. We evaluate our method using the ResNet models on the CIFAR-10 and ImageNet datasets, achieving several orders of magnitude improvement compared to prior work and reducing the latency of the encrypted CNN inference to 1.4 seconds on an NVIDIA A100 GPU. We also show that the latency drops to a mere 0.03 seconds with a custom hardware design.

cs.CR↗

arXiv2023

Random spin textures in turbulent spinor Bose-Einstein condensates

We numerically investigate the stationary turbulent states of spin-1 Bose-Einstein condensates under continuous spin driving. We analyze the entanglement entropy and magnetization correlation function to demonstrate the isotropic nature of the intricate spin texture that is generated in the nonequilibrium steady state. We observe a $-7/3$ power-law behavior in the spin-dependent interaction energy spectrum. To gain further insight into the statistical properties of the spin texture, we introduce a spin state ensemble obtained through position projection, revealing its close resemblance to the Haar random ensemble for spin-1 systems. We also present the probability distribution of the spin vector magnitude in the turbulent condensate, which can be tested in experiments. Our numerical study highlights the characteristics of stationary turbulence in the spinor BEC system and confirms previous experimental findings by Hong et al. [Phys. Rev. A 108, 013318 (2023)].

cond-mat.quant-gas↗

arXiv2023

Spin-driven stationary turbulence in spinor Bose-Einstein condensates

We report the observation of stationary turbulence in antiferromagnetic spin-1 Bose-Einstein condensates driven by a radio-frequency magnetic field. The magnetic driving injects energy into the system by spin rotation and the energy is dissipated via dynamic instability, resulting in the emergence of an irregular spin texture in the condensate. Under continuous driving, the spinor condensate evolves into a nonequilibrium steady state with characteristic spin turbulence, while the low energy scale of spin excitations ensures that the sample's lifetime is minimally affected. When the driving strength is on par with the system's spin interaction energy and the quadratic Zeeman energy, remarkably, the stationary turbulent state exhibits spin-isotropic features in spin composition and spatial spin texture. We numerically show that ambient field fluctuations play a crucial role in sustaining the turbulent state within the system. These results open up new avenues for exploring quantum turbulence in spinor superfluid systems.

cond-mat.quant-gas↗

arXiv2023

MAGVLT: Masked Generative Vision-and-Language Transformer

While generative modeling on multimodal image-text data has been actively developed with large-scale paired datasets, there have been limited attempts to generate both image and text data by a single model rather than a generation of one fixed modality conditioned on the other modality. In this paper, we explore a unified generative vision-and-language (VL) model that can produce both images and text sequences. Especially, we propose a generative VL transformer based on the non-autoregressive mask prediction, named MAGVLT, and compare it with an autoregressive generative VL transformer (ARGVLT). In comparison to ARGVLT, the proposed MAGVLT enables bidirectional context encoding, fast decoding by parallel token predictions in an iterative refinement, and extended editing capabilities such as image and text infilling. For rigorous training of our MAGVLT with image-text pairs from scratch, we combine the image-to-text, text-to-image, and joint image-and-text mask prediction tasks. Moreover, we devise two additional tasks based on the step-unrolled mask prediction and the selective prediction on the mixture of two image-text pairs. Experimental results on various downstream generation tasks of VL benchmarks show that our MAGVLT outperforms ARGVLT by a large margin even with significant inference speedup. Particularly, MAGVLT achieves competitive results on both zero-shot image-to-text and text-to-image generation tasks from MS-COCO by one moderate-sized model (fewer than 500M parameters) even without the use of monomodal data and networks.

cs.CV↗

arXiv2022

ARK: Fully Homomorphic Encryption Accelerator with Runtime Data Generation and Inter-Operation Key Reuse

Homomorphic Encryption (HE) is one of the most promising post-quantum cryptographic schemes that enable privacy-preserving computation on servers. However, noise accumulates as we perform operations on HE-encrypted data, restricting the number of possible operations. Fully HE (FHE) removes this restriction by introducing the bootstrapping operation, which refreshes the data; however, FHE schemes are highly memory-bound. Bootstrapping, in particular, requires loading GBs of evaluation keys and plaintexts from off-chip memory, which makes FHE acceleration fundamentally bottlenecked by the off-chip memory bandwidth. In this paper, we propose ARK, an Accelerator for FHE with Runtime data generation and inter-operation Key reuse. ARK enables practical FHE workloads with a novel algorithm-architecture co-design to accelerate bootstrapping. We first eliminate the off-chip memory bandwidth bottleneck through runtime data generation and inter-operation key reuse. This approach enables ARK to fully exploit on-chip memory by substantially reducing the size of the working set. On top of such algorithmic enhancements, we build ARK microarchitecture that minimizes on-chip data movement through an efficient, alternating data distribution policy based on the data access patterns and a streamlined dataflow organization of the tailored functional units -- including base conversion, number-theoretic transform, and automorphism units. Overall, our co-design effectively handles the heavy computation and data movement overheads of FHE, drastically reducing the cost of HE operations, including bootstrapping.

cs.CR↗

arXiv2022

LECO: Learnable Episodic Count for Task-Specific Intrinsic Reward

Episodic count has been widely used to design a simple yet effective intrinsic motivation for reinforcement learning with a sparse reward. However, the use of episodic count in a high-dimensional state space as well as over a long episode time requires a thorough state compression and fast hashing, which hinders rigorous exploitation of it in such hard and complex exploration environments. Moreover, the interference from task-irrelevant observations in the episodic count may cause its intrinsic motivation to overlook task-related important changes of states, and the novelty in an episodic manner can lead to repeatedly revisit the familiar states across episodes. In order to resolve these issues, in this paper, we propose a learnable hash-based episodic count, which we name LECO, that efficiently performs as a task-specific intrinsic reward in hard exploration problems. In particular, the proposed intrinsic reward consists of the episodic novelty and the task-specific modulation where the former employs a vector quantized variational autoencoder to automatically obtain the discrete state codes for fast counting while the latter regulates the episodic novelty by learning a modulator to optimize the task-specific extrinsic reward. The proposed LECO specifically enables the automatic transition from exploration to exploitation during reinforcement learning. We experimentally show that in contrast to the previous exploration methods LECO successfully solves hard exploration problems and also scales to large state spaces through the most difficult tasks in MiniGrid and DMLab environments.

cs.LG↗

arXiv2022

The Interfacial Structure of InP(100) in Contact with HCl and H$_2$SO$_4$ studied by Reflection Anisotropy Spectroscopy

Indium phosphide and derived compound semiconductors are materials often involved in high-efficiency solar water splitting due to their versatile opto-electronic properties. Surface corrosion, however, typically deteriorates the performance of photoelectrochemical solar cells based on this material class. It has been reported that (photo)electrochemical surface functionalisation protects the surface by combining etching and controlled corrosion. Nevertheless, the overall involved process is not fully understood. Therefore, access to the electrochemical interface structure under operando conditions is crucial for a more detailed understanding. One approach for gaining structural insight is the use of operando reflection anisotropy spectroscopy. This technique allows the time-resolved investigation of the interfacial structure while applying potentials in the electrolyte. In this study, p-doped InP(100) surfaces are cycled between anodic and cathodic potentials in two different electrolytes, hydrochloric acid and sulphuric acid. For low, 10 mM electrolyte concentrations, we observe a reversible processes related to the reduction of a surface oxide phase in the cathodic potential range which is reformed near open-circuit potentials. Higher concentrations of 0.5 N, however, already lead to initial surface corrosion.

physics.chem-ph↗

arXiv2022

Resta-like preconditioning for self-consistent field iterations in the linearized augmented planewave method

Convergence in self-consistent-field cycles can be a major computational bottleneck of density-functional theory calculations. We propose a Resta-like preconditioning method for full-potential all-electron calculations in the linearized augmented planewave (LAPW) method to smoothly converge to self-consistency. We implemented this preconditioner in the exciting code and apply it to the two semiconducting systems of MoS$_2$ slabs and P-rich GaP(100) surfaces as well as the metallic system Au(111), containing a sufficiently large amount of vacuum. Our calculations demonstrate that the implemented scheme performs reliably as well as more efficiently regardless of system size, suppressing long-range charge sloshing. While the suitability of this preconditioning higher for semiconducting systems, the convergence for metals is only slightly decreased and thus still trustworthy to apply. Furthermore, a mixing algorithm with the preconditioner shows an improvement over that with the Kerker preconditioner for the investigated semiconducting systems.

cond-mat.mtrl-sci↗

arXiv2022

BTS: An Accelerator for Bootstrappable Fully Homomorphic Encryption

Homomorphic encryption (HE) enables the secure offloading of computations to the cloud by providing computation on encrypted data (ciphertexts). HE is based on noisy encryption schemes in which noise accumulates as more computations are applied to the data. The limited number of operations applicable to the data prevents practical applications from exploiting HE. Bootstrapping enables an unlimited number of operations or fully HE (FHE) by refreshing the ciphertext. Unfortunately, bootstrapping requires a significant amount of additional computation and memory bandwidth as well. Prior works have proposed hardware accelerators for computation primitives of FHE. However, to the best of our knowledge, this is the first to propose a hardware FHE accelerator that supports bootstrapping as a first-class citizen. In particular, we propose BTS - Bootstrappable, Technologydriven, Secure accelerator architecture for FHE. We identify the challenges of supporting bootstrapping in the accelerator and analyze the off-chip memory bandwidth and computation required. In particular, given the limitations of modern memory technology, we identify the HE parameter sets that are efficient for FHE acceleration. Based on the insights gained from our analysis, we propose BTS, which effectively exploits the parallelism innate in HE operations by arranging a massive number of processing elements in a grid. We present the design and microarchitecture of BTS, including a network-on-chip design that exploits a deterministic communication pattern. BTS shows 5,556x and 1,306x improved execution time on ResNet-20 and logistic regression over a CPU, with a chip area of 373.6mm^2 and up to 163.2W of power.

cs.CR↗

arXiv2022

Insights From the NeurIPS 2021 NetHack Challenge

In this report, we summarize the takeaways from the first NeurIPS 2021 NetHack Challenge. Participants were tasked with developing a program or agent that can win (i.e., 'ascend' in) the popular dungeon-crawler game of NetHack by interacting with the NetHack Learning Environment (NLE), a scalable, procedurally generated, and challenging Gym environment for reinforcement learning (RL). The challenge showcased community-driven progress in AI with many diverse approaches significantly beating the previously best results on NetHack. Furthermore, it served as a direct comparison between neural (e.g., deep RL) and symbolic AI, as well as hybrid systems, demonstrating that on NetHack symbolic bots currently outperform deep RL by a large margin. Lastly, no agent got close to winning the game, illustrating NetHack's suitability as a long-term benchmark for AI research.

cs.LG↗