arXiv subjects

Minjun Kim

Publications and source records attributed to Minjun Kim.

At least 37 records · Page 2Linked to original sources

arXiv2025

KORMo: Korean Open Reasoning Model for Everyone

This work presents the first large-scale investigation into constructing a fully open bilingual large language model (LLM) for a non-English language, specifically Korean, trained predominantly on synthetic data. We introduce KORMo-10B, a 10.8B-parameter model trained from scratch on a Korean-English corpus in which 68.74% of the Korean portion is synthetic. Through systematic experimentation, we demonstrate that synthetic data, when carefully curated with balanced linguistic coverage and diverse instruction styles, does not cause instability or degradation during large-scale pretraining. Furthermore, the model achieves performance comparable to that of contemporary open-weight multilingual baselines across a wide range of reasoning, knowledge, and instruction-following benchmarks. Our experiments reveal two key findings: (1) synthetic data can reliably sustain long-horizon pretraining without model collapse, and (2) bilingual instruction tuning enables near-native reasoning and discourse coherence in Korean. By fully releasing all components including data, code, training recipes, and logs, this work establishes a transparent framework for developing synthetic data-driven fully open models (FOMs) in low-resource settings and sets a reproducible precedent for future multilingual LLM research.

cs.CL↗

arXiv2025

The Iterative Chainlet Partitioning Algorithm for the Traveling Salesman Problem with Drone and Neural Acceleration

This study introduces the Iterative Chainlet Partitioning (ICP) algorithm and its neural acceleration for solving the Traveling Salesman Problem with Drone (TSP-D). The proposed ICP algorithm decomposes a TSP-D solution into smaller segments called chainlets, each optimized individually by a dynamic programming subroutine. The chainlet with the highest improvement is updated, and the procedure is repeated until no further improvement is possible. We show that the subroutine runs in quadratic time and the number of subroutine calls is bounded linearly in problem size for the first iteration and remains constant in subsequent iterations, ensuring algorithmic scalability. Empirical results show that ICP outperforms existing algorithms in both solution quality and computational time. Tested over 1,249 benchmark instances, ICP yields an average improvement of 2.6\% in solution quality over the previous state-of-the-art algorithm while reducing computational time by 91.3\%. The procedure is deterministic, ensuring reliability without requiring multiple runs. The subroutine is the computational bottleneck in the already efficient ICP algorithm. To reduce the necessity of subroutine calls, we integrate a graph neural network (GNN) to predict incremental improvements. We demonstrate that the resulting Neuro ICP (NICP) achieves substantial acceleration while maintaining solution quality. Compared to ICP, NICP reduces the total computational time by 28.6\%, while the objective function value increase is limited to 0.14\%. A transfer learning framework enables efficient extension to various operational constraints, making this a valuable foundation for developing efficient algorithms for truck-drone synchronized routing problems.

cs.NE↗

arXiv2025

Unifying Uniform and Binary-coding Quantization for Accurate Compression of Large Language Models

How can we quantize large language models while preserving accuracy? Quantization is essential for deploying large language models (LLMs) efficiently. Binary-coding quantization (BCQ) and uniform quantization (UQ) are promising quantization schemes that have strong expressiveness and optimizability, respectively. However, neither scheme leverages both advantages. In this paper, we propose UniQuanF (Unified Quantization with Flexible Mapping), an accurate quantization method for LLMs. UniQuanF harnesses both strong expressiveness and optimizability by unifying the flexible mapping technique in UQ and non-uniform quantization levels of BCQ. We propose unified initialization, and local and periodic mapping techniques to optimize the parameters in UniQuanF precisely. After optimization, our unification theorem removes computational and memory overhead, allowing us to utilize the superior accuracy of UniQuanF without extra deployment costs induced by the unification. Experimental results demonstrate that UniQuanF outperforms existing UQ and BCQ methods, achieving up to 4.60% higher accuracy on GSM8K benchmark.

cs.CL↗

arXiv2025

Context-Robust Knowledge Editing for Language Models

Knowledge editing (KE) methods offer an efficient way to modify knowledge in large language models. Current KE evaluations typically assess editing success by considering only the edited knowledge without any preceding contexts. In real-world applications, however, preceding contexts often trigger the retrieval of the original knowledge and undermine the intended edit. To address this issue, we develop CHED -- a benchmark designed to evaluate the context robustness of KE methods. Evaluations on CHED show that they often fail when preceding contexts are present. To mitigate this shortcoming, we introduce CoRE, a KE method designed to strengthen context robustness by minimizing context-sensitive variance in hidden states of the model for edited knowledge. This method not only improves the editing success rate in situations where a preceding context is present but also preserves the overall capabilities of the model. We provide an in-depth analysis of the differing impacts of preceding contexts when introduced as user utterances versus assistant responses, and we dissect attention-score patterns to assess how specific tokens influence editing success.

cs.CL↗

arXiv2025

Zero-shot Quantization: A Comprehensive Survey

Network quantization has proven to be a powerful approach to reduce the memory and computational demands of deep learning models for deployment on resource-constrained devices. However, traditional quantization methods often rely on access to training data, which is impractical in many real-world scenarios due to privacy, security, or regulatory constraints. Zero-shot Quantization (ZSQ) emerges as a promising solution, achieving quantization without requiring any real data. In this paper, we provide a comprehensive overview of ZSQ methods and their recent advancements. First, we provide a formal definition of the ZSQ problem and highlight the key challenges. Then, we categorize the existing ZSQ methods into classes based on data generation strategies, and analyze their motivations, core ideas, and key takeaways. Lastly, we suggest future research directions to address the remaining limitations and advance the field of ZSQ. To the best of our knowledge, this paper is the first in-depth survey on ZSQ.

cs.CV↗

arXiv2025

AugWard: Augmentation-Aware Representation Learning for Accurate Graph Classification

How can we accurately classify graphs? Graph classification is a pivotal task in data mining with applications in social network analysis, web analysis, drug discovery, molecular property prediction, etc. Graph neural networks have achieved the state-of-the-art performance in graph classification, but they consistently struggle with overfitting. To mitigate overfitting, researchers have introduced various representation learning methods utilizing graph augmentation. However, existing methods rely on simplistic use of graph augmentation, which loses augmentation-induced differences and limits the expressiveness of representations. In this paper, we propose AugWard (Augmentation-Aware Training with Graph Distance and Consistency Regularization), a novel graph representation learning framework that carefully considers the diversity introduced by graph augmentation. AugWard applies augmentation-aware training to predict the graph distance between the augmented graph and its original one, aligning the representation difference directly with graph distance at both feature and structure levels. Furthermore, AugWard employs consistency regularization to encourage the classifier to handle richer representations. Experimental results show that AugWard gives the state-of-the-art performance in supervised, semi-supervised graph classification, and transfer learning.

cs.LG↗

arXiv2025

DTA: Dual Temporal-channel-wise Attention for Spiking Neural Networks

Spiking Neural Networks (SNNs) present a more energy-efficient alternative to Artificial Neural Networks (ANNs) by harnessing spatio-temporal dynamics and event-driven spikes. Effective utilization of temporal information is crucial for SNNs, leading to the exploration of attention mechanisms to enhance this capability. Conventional attention operations either apply identical operation or employ non-identical operations across target dimensions. We identify that these approaches provide distinct perspectives on temporal information. To leverage the strengths of both operations, we propose a novel Dual Temporal-channel-wise Attention (DTA) mechanism that integrates both identical/non-identical attention strategies. To the best of our knowledge, this is the first attempt to concentrate on both the correlation and dependency of temporal-channel using both identical and non-identical attention operations. Experimental results demonstrate that the DTA mechanism achieves state-of-the-art performance on both static datasets (CIFAR10, CIFAR100, ImageNet-1k) and dynamic dataset (CIFAR10-DVS), elevating spike representation and capturing complex temporal-channel relationship. We open-source our code: https://github.com/MnJnKIM/DTA-SNN.

cs.CV↗

arXiv2025

Vessel segmentation for X-separation

$χ$-separation is an advanced quantitative susceptibility mapping (QSM) method that is designed to generate paramagnetic ($χ_{para}$) and diamagnetic ($|χ_{dia}|$) susceptibility maps, reflecting the distribution of iron and myelin in the brain. However, vessels have shown artifacts, interfering with the accurate quantification of iron and myelin in applications. To address this challenge, a new vessel segmentation method for $χ$-separation is developed. The method comprises three steps: 1) Seed generation from $\textit{R}_2^*$ and the product of $χ_{para}$ and $|χ_{dia}|$ maps; 2) Region growing, guided by vessel geometry, creating a vessel mask; 3) Refinement of the vessel mask by excluding non-vessel structures. The performance of the method was compared to conventional vessel segmentation methods both qualitatively and quantitatively. To demonstrate the utility of the method, it was tested in two applications: quantitative evaluation of a neural network-based $χ$-separation reconstruction method ($χ$-sepnet-$\textit{R}_2^*$) and population-averaged region of interest (ROI) analysis. The proposed method demonstrates superior performance to the conventional vessel segmentation methods, effectively excluding the non-vessel structures, achieving the highest Dice score coefficient. For the applications, applying vessel masks report notable improvements for the quantitative evaluation of $χ$-sepnet-$\textit{R}_2^*$ and statistically significant differences in population-averaged ROI analysis. These applications suggest excluding vessels when analyzing the $χ$-separation maps provide more accurate evaluations. The proposed method has the potential to facilitate various applications, offering reliable analysis through the generation of a high-quality vessel mask.

cs.CV↗

arXiv2024

FedCAR: Cross-client Adaptive Re-weighting for Generative Models in Federated Learning

Generative models trained on multi-institutional datasets can provide an enriched understanding through diverse data distributions. However, training the models on medical images is often challenging due to hospitals' reluctance to share data for privacy reasons. Federated learning(FL) has emerged as a privacy-preserving solution for training distributed datasets across data centers by aggregating model weights from multiple clients instead of sharing raw data. Previous research has explored the adaptation of FL to generative models, yet effective aggregation algorithms specifically tailored for generative models remain unexplored. We hereby propose a novel algorithm aimed at improving the performance of generative models within FL. Our approach adaptively re-weights the contribution of each client, resulting in well-trained shared parameters. In each round, the server side measures the distribution distance between fake images generated by clients instead of directly comparing the Fréchet Inception Distance per client, thereby enhancing efficiency of the learning. Experimental results on three public chest X-ray datasets show superior performance in medical image generation, outperforming both centralized learning and conventional FL algorithms. Our code is available at https://github.com/danny0628/FedCAR.

cs.CV↗

arXiv2024

VLR-Bench: Multilingual Benchmark Dataset for Vision-Language Retrieval Augmented Generation

We propose the VLR-Bench, a visual question answering (VQA) benchmark for evaluating vision language models (VLMs) based on retrieval augmented generation (RAG). Unlike existing evaluation datasets for external knowledge-based VQA, the proposed VLR-Bench includes five input passages. This allows testing of the ability to determine which passage is useful for answering a given query, a capability lacking in previous research. In this context, we constructed a dataset of 32,000 automatically generated instruction-following examples, which we denote as VLR-IF. This dataset is specifically designed to enhance the RAG capabilities of VLMs by enabling them to learn how to generate appropriate answers based on input passages. We evaluated the validity of the proposed benchmark and training data and verified its performance using the state-of-the-art Llama3-based VLM, the Llava-Llama-3 model. The proposed VLR-Bench and VLR-IF datasets are publicly available online.

cs.CV↗

arXiv2024

Effects of Network Connectivity and Functional Diversity Distribution on Human Collective Ideation

Human collective tasks in teams and organizations increasingly require participation of members with diverse backgrounds working in networked social environments. However, little is known about how network structure and the functional diversity of member backgrounds would interact with each other and affect collective processes. Here we conducted three sets of human-subject experiments which involved 617 university students who collaborated anonymously in a collective ideation task on a custom-made online social network platform. We found that spatially clustered collectives with assortative background distribution tended to explore more diverse ideas than in other conditions, whereas collectives with random background distribution consistently generated ideas with the highest utility. We also found that higher network connectivity may improve individuals' overall experience but may not improve the collective performance regarding idea generation, idea diversity, and final idea quality.

cs.SI↗

arXiv2024

A Supercomputing Based Distributed Cloud Marketplace

The once mythological 51% attack has moved beyond the hypothetical and now poses a legitimate widespread threat to blockchain technology. Current blockchains provide inferior throughput capacity when compared to that of centralized systems, creating an obvious vulnerability which allows the 51% attack to occur within decentralized systems. Despite recent advancements in blockchain which introduce interesting models that achieve high throughputs with enhanced security and privacy, no current networks have evolved to deploy the optimal solution of combining scalability, security, and distributed systems to create a legitimate supercomputing enterprise-grade developer sandbox. In this paper, we introduce an infinitely scalable, secure, and high throughput blockchain capable of amassing supercomputer speeds with off-the-shelf hardware, LuluChain. LuluChain simplifies the blockchain model to obtain greater functionality, speed, scalability, privacy, and flexibility, that works to combat the inflated pricing models set by the oligopolistic cloud computing market as it requires minimal computational work. By eliminating the need for timestamp synchronization and majority agreement among all participants, LuluChain opens the door to reliable trust, low-cost instant transactions, and flexible instant smart contracts. The supercomputing, high throughput distributed system is the ideal foundation for an essential distributed cloud marketplace.

cs.CR↗

arXiv2024

χ-sepnet: Deep neural network for magnetic susceptibility source separation

Magnetic susceptibility source separation ($χ$-separation), an advanced quantitative susceptibility mapping (QSM) method, enables the separate estimation of para- and diamagnetic susceptibility source distributions in the brain. The method utilizes reversible transverse relaxation (R2'=R2*-R2) to complement frequency shift information for estimating susceptibility source concentrations, requiring time-consuming data acquisition for R2 in addition R2*. To address this challenge, we develop a new deep learning network, $χ$-sepnet, and propose two deep learning-based susceptibility source separation pipelines, $χ$-sepnet-R2' for inputs with multi-echo GRE and multi-echo spin-echo, and $χ$-sepnet-R2* for input with multi-echo GRE only. $χ$-sepnet is trained using multiple head orientation data that provide streaking artifact-free labels, generating high-quality $χ$-separation maps. The evaluation of the pipelines encompasses both qualitative and quantitative assessments in healthy subjects, and visual inspection of lesion characteristics in multiple sclerosis patients. The susceptibility source-separated maps of the proposed pipelines delineate detailed brain structures with substantially reduced artifacts compared to those from conventional regularization-based reconstruction methods. In quantitative analysis, $χ$-sepnet-R2' achieves the best outcomes followed by $χ$-sepnet-R2*, outperforming the conventional methods. When the lesions of multiple sclerosis patients are assessed, both pipelines report identical lesion characteristics in most lesions ($χ$para: 99.6% and $χ$dia: 98.4% out of 250 lesions). The $χ$-sepnet-R2* pipeline, which only requires multi-echo GRE data, has demonstrated its potential to offer broad clinical and scientific applications, although further evaluations for various diseases and pathological conditions are necessary.

eess.IV↗

arXiv2024

In-vivo high-resolution χ-separation at 7T

A recently introduced quantitative susceptibility mapping (QSM) technique, $χ$-separation, offers the capability to separate paramagnetic ($χ_{\text{para}}$) and diamagnetic ($χ_{\text{dia}}$) susceptibility distribution within the brain. In-vivo high-resolution mapping of iron and myelin distribution, estimated by $χ$-separation, could provide a deeper understanding of brain substructures, assisting the investigation of their functions and alterations. This can be achieved using 7T MRI, which benefits from a high signal-to-noise ratio and susceptibility effects. However, applying $χ$-separation at 7T presents difficulties due to the requirement of an $R_2$ map, coupled with issues such as high specific absorption rate (SAR), large $B_1$ transmit field inhomogeneities, and prolonged scan time. To address these challenges, we developed a novel deep neural network, R2PRIMEnet7T, designed to convert a 7T $R_2^*$ map into a 3T $R_2'$ map. Building on this development, we present a new pipeline for $χ$-separation at 7T, enabling us to generate high-resolution $χ$-separation maps from multi-echo gradient-echo data. The proposed method is compared with alternative pipelines, such as an end-to-end network and linearly-scaled $R_2'$, and is validated against $χ$-separation maps at 3T, demonstrating its accuracy. The 7T $χ$-separation maps generated by the proposed method exhibit similar contrasts to those from 3T, while 7T high-resolution maps offer enhanced clarity and detail. Quantitative analysis confirms that the proposed method surpasses the alternative pipelines. The proposed method results well delineate the detailed brain structures associated with iron and myelin. This new pipeline holds promise for analyzing iron and myelin concentration changes in various neurodegenerative diseases through precise structural examination.

q-bio.QM↗

arXiv2024

Soft Segmented Randomization: Enhancing Domain Generalization in SAR-ATR for Synthetic-to-Measured

Synthetic aperture radar technology is crucial for high-resolution imaging under various conditions; however, the acquisition of real-world synthetic aperture radar data for deep learning-based automatic target recognition remains challenging due to high costs and data availability issues. To overcome these challenges, synthetic data generated through simulations have been employed, although discrepancies between synthetic and real data can degrade model performance. In this study, we introduce a novel framework, soft segmented randomization, designed to reduce domain discrepancy and improve the generalize ability of synthetic aperture radar automatic target recognition models. The soft segmented randomization framework applies a Gaussian mixture model to segment target and clutter regions softly, introducing randomized variations that align the synthetic data's statistical properties more closely with those of real-world data. Experimental results demonstrate that the proposed soft segmented randomization framework significantly enhances model performance on measured synthetic aperture radar data, making it a promising approach for robust automatic target recognition in scenarios with limited or no access to measured data.

cs.CV↗

arXiv2024

A human brain atlas of chi-separation for normative iron and myelin distributions

Iron and myelin are primary susceptibility sources in the human brain. These substances are essential for healthy brain, and their abnormalities are often related to various neurological disorders. Recently, an advanced susceptibility mapping technique, which is referred to as chi-separation, has been proposed, successfully disentangling paramagnetic iron from diamagnetic myelin. This method opened a potential for generating high resolution iron and myelin maps in the brain. Utilizing this technique, this study constructs a normative chi-separation atlas from 106 healthy human brains. The resulting atlas provides detailed anatomical structures associated with the distributions of iron and myelin, clearly delineating subcortical nuclei, thalamic nuclei, and white matter fiber bundles. Additionally, susceptibility values in a number of regions of interest are reported along with age-dependent changes. This atlas may have direct applications such as localization of subcortical structures for deep brain stimulation or high-intensity focused ultrasound and also serve as a valuable resource for future research.

eess.IV↗

arXiv2024

X-LLaVA: Optimizing Bilingual Large Vision-Language Alignment

The impressive development of large language models (LLMs) is expanding into the realm of large multimodal models (LMMs), which incorporate multiple types of data beyond text. However, the nature of multimodal models leads to significant expenses in the creation of training data. Furthermore, constructing multilingual data for LMMs presents its own set of challenges due to language diversity and complexity. Therefore, in this study, we propose two cost-effective methods to solve this problem: (1) vocabulary expansion and pretraining of multilingual LLM for specific languages, and (2) automatic and elaborate construction of multimodal datasets using GPT4-V. Based on015 these methods, we constructed a 91K English-Korean-Chinese multilingual, multimodal training dataset. Additionally, we developed a bilingual multimodal model that exhibits excellent performance in both Korean and English, surpassing existing approaches.

cs.CL↗

arXiv2024

BOK-VQA: Bilingual outside Knowledge-Based Visual Question Answering via Graph Representation Pretraining

The current research direction in generative models, such as the recently developed GPT4, aims to find relevant knowledge information for multimodal and multilingual inputs to provide answers. Under these research circumstances, the demand for multilingual evaluation of visual question answering (VQA) tasks, a representative task of multimodal systems, has increased. Accordingly, we propose a bilingual outside-knowledge VQA (BOK-VQA) dataset in this study that can be extended to multilingualism. The proposed data include 17K images, 17K question-answer pairs for both Korean and English and 280K instances of knowledge information related to question-answer content. We also present a framework that can effectively inject knowledge information into a VQA system by pretraining the knowledge information of BOK-VQA data in the form of graph embeddings. Finally, through in-depth analysis, we demonstrated the actual effect of the knowledge information contained in the constructed training data on VQA.

cs.CL↗