arXiv subjects

Wonguk Cho

Publications and source records attributed to Wonguk Cho.

9 recordsLinked to original sources

arXiv2026

MoNe: Modular Neural Memory for Efficient Long Context Inference

We present MoNe, a lightweight modular neural memory that attaches to any frozen pretrained Transformer to enable long-context inference without retraining. MoNe reads context in fixed-size segments via test-time learning of fast-weight neural memory networks with layer-localized gradient updates; at inference, the memory generates keys and values from the query tokens alone, with no context tokens re-read. This two-phase design decouples inference cost from context length, achieving $O(N)$ preprocessing and $O(1)$ query cost with peak GPU memory that does not grow with $N$. At 128K tokens, MoNe reduces both compute and peak GPU memory by approximately 80% compared to ICL with only 6.4% parameter overhead. MoNe generalizes to context lengths far beyond the backbone's native window, achieving strong performance on needle-in-a-haystack and word extraction benchmarks from RULER, where ICL degrades sharply.

cs.AI↗

arXiv2026

ReCo: Reweighting GRPO Against Distributional Concentration

Group Relative Policy Optimization (GRPO) has become a standard reinforcement learning method for post-training language models. Recent work shows that GRPO can reduce the base model's reasoning capacity and underperform it in Pass@k when k is large, indicating reduced coverage of reasoning paths. We find that this reduction is associated with GRPO concentrating on responses that the base model already generates with high probability. We trace this concentration to two mechanisms in the GRPO update. At the response level, high-probability responses dominate the group gradient through repeated occurrence. At the token level, GRPO's importance ratio scales gradients, further reinforcing tokens that become more likely under the current policy. We propose ReCo, a reweighting method that addresses both effects. Response contributions are normalized by their expected occurrence within the rollout group, and the token-level importance ratio is replaced with a variance-based ratio that gives larger update scale to non-saturated decision points where alternative token choices remain plausible. Across Qwen2.5-Math-1.5B/7B and Llama-3.1-8B-Instruct on five mathematical reasoning benchmarks, ReCo improves Pass@k for large values of k and is comparable to GRPO for small values of k.

cs.LG↗

arXiv2025

PiCa: Parameter-Efficient Fine-Tuning with Column Space Projection

Fine-tuning large foundation models is essential for building expert models tailored to specialized tasks and domains, but fully updating billions of parameters is computationally prohibitive. Reducing the number of trainable parameters using Parameter-Efficient Fine-Tuning (PEFT), such as Low-Rank Adaptation (LoRA), is therefore crucial not only to reduce training costs but also to mitigate storage, caching, and serving overheads during deployment. Prior works, such as Singular Vectors-guided Fine-Tuning (SVFT), have shown that exploiting the geometry of pre-trained weights based on Singular Value Decomposition (SVD) can significantly improve parameter-efficiency, but they lack a solid theoretical foundation. In this paper, we introduce Parameter-Efficient Fine-Tuning with Column Space Projection (PiCa), a novel theoretically grounded PEFT method. We prove that projecting gradients onto the principal column space of pre-trained weights provides an effective inductive bias for adaptation and further enhance parameter efficiency through a novel weight-sharing strategy. Across diverse NLP and vision tasks, PiCa consistently outperforms state-of-the-art baselines under comparable or smaller parameter budgets, demonstrating both theoretical rigor and practical effectiveness.

cs.LG↗

arXiv2024

Hollowed Net for On-Device Personalization of Text-to-Image Diffusion Models

Recent advancements in text-to-image diffusion models have enabled the personalization of these models to generate custom images from textual prompts. This paper presents an efficient LoRA-based personalization approach for on-device subject-driven generation, where pre-trained diffusion models are fine-tuned with user-specific data on resource-constrained devices. Our method, termed Hollowed Net, enhances memory efficiency during fine-tuning by modifying the architecture of a diffusion U-Net to temporarily remove a fraction of its deep layers, creating a hollowed structure. This approach directly addresses on-device memory constraints and substantially reduces GPU memory requirements for training, in contrast to previous methods that primarily focus on minimizing training steps and reducing the number of parameters to update. Additionally, the personalized Hollowed Net can be transferred back into the original U-Net, enabling inference without additional memory overhead. Quantitative and qualitative analyses demonstrate that our approach not only reduces training memory to levels as low as those required for inference but also maintains or improves personalization performance compared to existing methods.

cs.CV↗

arXiv2024

Overcoming Data Inequality across Domains with Semi-Supervised Domain Generalization

While there have been considerable advancements in machine learning driven by extensive datasets, a significant disparity still persists in the availability of data across various sources and populations. This inequality across domains poses challenges in modeling for those with limited data, which can lead to profound practical and ethical concerns. In this paper, we address a representative case of data inequality problem across domains termed Semi-Supervised Domain Generalization (SSDG), in which only one domain is labeled while the rest are unlabeled. We propose a novel algorithm, ProUD, which can effectively learn domain-invariant features via domain-aware prototypes along with progressive generalization via uncertainty-adaptive mixing of labeled and unlabeled domains. Our experiments on three different benchmark datasets demonstrate the effectiveness of ProUD, outperforming all baseline models including single domain generalization and semi-supervised learning. Source code will be released upon acceptance of the paper.

cs.LG↗

arXiv2023

Complementary Domain Adaptation and Generalization for Unsupervised Continual Domain Shift Learning

Continual domain shift poses a significant challenge in real-world applications, particularly in situations where labeled data is not available for new domains. The challenge of acquiring knowledge in this problem setting is referred to as unsupervised continual domain shift learning. Existing methods for domain adaptation and generalization have limitations in addressing this issue, as they focus either on adapting to a specific domain or generalizing to unseen domains, but not both. In this paper, we propose Complementary Domain Adaptation and Generalization (CoDAG), a simple yet effective learning framework that combines domain adaptation and generalization in a complementary manner to achieve three major goals of unsupervised continual domain shift learning: adapting to a current domain, generalizing to unseen domains, and preventing forgetting of previously seen domains. Our approach is model-agnostic, meaning that it is compatible with any existing domain adaptation and generalization algorithms. We evaluate CoDAG on several benchmark datasets and demonstrate that our model outperforms state-of-the-art models in all datasets and evaluation metrics, highlighting its effectiveness and robustness in handling unsupervised continual domain shift learning.

cs.LG↗

arXiv2022

Multiresolution community analysis of international trade networks

The international trade network is a complex system where multiple trade blocs with varying sizes coexist and overlap with each other. However, the resulting structures of community detection in trade networks are often inconsistent and fails to capture the complex landscape of international trade. To address these problems, we propose a multiresolution framework that aggregates all the configuration information from a range of resolutions. This allows us to consider trade communities of different sizes and illuminate the underlying hierarchical structure of trade networks and its constituting blocks. Furthermore, by measuring membership inconsistency (MeI) of each country and conducting multiple regression analysis with various economic and political indicators, we demonstrate that there exists a positive correlation between the external instability of countries and their structural inconsistency in terms of network topology.

physics.soc-ph↗

arXiv2021

Self-consistent gravity model for inferring node mass in flow networks

The gravity model, inspired by Newton's law of universal gravitation, has long served as a primary tool for interpreting trade flows between countries, using a country's economic `mass' as a key determinant. Despite its wide application, the definition of `mass' within this model remains ambiguous. It is often approximated using indicators like GDP, which may not accurately reflect a country's true trade potential. Here, we introduce a data-driven, self-consistent numerical approach that redefines `mass' from a static proxy to a dynamic attribute inferred directly from flow data. We infer mass distribution and interaction nature through our method, mirroring Newton's approach to understanding gravity. Our methodology accurately identifies predefined embeddings and reconstructs system attributes when applied to synthetic flow data, demonstrating its strong predictive power and adaptability. Further application to real-world trade networks yields critical insights, revealing the spatial spectrum of trade flows and the economic mass of countries, two key features unexplored in depth by existing models. Our methodology not only enables accurate reconstruction of the original flow but also allows for a deep understanding of the unique capabilities of each node within the network. This study marks a significant shift in the understanding and application of the gravity model, providing a more comprehensive tool for analyzing complex systems and uncovering new insights into various fields, including global trade, traffic engineering, epidemic disease prevention, and infrastructure design.

physics.data-an↗

arXiv2021

Impact of personal income on mortality by age: biological versus socio-economic effects

The influence of per capita income on life expectancy is well documented, mostly through studies of multinational samples. However, one expects fairly weak correlations at both ends of the life span, that is to say in early infancy and in age groups of elderly from 85 to 100 years. The reason is that at both ends mortality is largely controled by biological factors rather than by socio-economic conditions. In order to test this conjecture, we explore the influence of income on age groups, separately in France, the United States and South Korea. More precisely in each country we compare income and mortality data in as many regional subunits as possible. One noteworthy constatation is that, contrary to a common view, personal income is only weakly correlated with infant mortality (i.e. mortality under the age of one year). More broadly, we propose as a conjecture that the common pattern revealed by the analysis of the three countries is also valid in other developed countries.

physics.soc-ph↗