arXiv subjects

Dongseok Kim

Publications and source records attributed to Dongseok Kim.

At least 19 recordsLinked to original sources

arXiv2026

CUBE: Contrastive Understanding by Balanced Experiments

We introduce CUBE, Contrastive Understanding by Balanced Experiments, to estimate main effects and pairwise Banzhaf interactions over a fixed two-state probe space using shared balanced queries. Each design cancels constant and odd-order contamination from pairwise estimates, while randomization ensures unbiasedness and explicit alias probabilities yield exact mean squared errors without sparsity assumptions or order truncation. Matching risk bounds establish asymptotic minimax optimality within a specified class of nonadaptive, equal-weight, reversal-symmetric balanced contrast estimators, as admissible budgets grow while remaining small relative to the full probe space. Experiments reveal different interaction-specific risks despite identical output distributions and low-order effects, and different sign risks despite equal mean squared errors. Across evaluated tabular models, CUBE generally outperforms several sampling-based estimators, while Faith-Banzhaf-2 achieves the lowest mean error at the largest tested budgets.

cs.LG↗

arXiv2026

Motif 3: Technical Report

We introduce Motif 3, a decoder-only Mixture-of-Experts language model with 314 billion total parameters and 13.2 billion activated per token. Each sparse MoE layer contains 384 routed experts, with eight selected per token. This fine-grained sparsity provides substantial expert capacity while limiting computation. Motif 3 is built around Grouped Differential Latent Attention (GDLA), which integrates grouped differential attention with the compressed key-value representation of Multi-head Latent Attention. The architecture further incorporates modified manifold-constrained hyper-connections, Expert Specific PolyNorm activations, and multi-token prediction to improve optimization stability, expert specialization, and inference efficiency. We pretrain Motif 3 on approximately 12.5 trillion tokens spanning web documents, STEM, code, mathematics, multilingual content, and domain-specialized corpora. Expert-balancing and numerical-stabilization techniques support stable training at scale, while selective MXFP8 computation and communication, memory-efficient fused kernels, and window-aware context parallelism enable training with context lengths up to 256K tokens. Our post-training pipeline combines general supervised fine-tuning, six specialist teachers trained with reinforcement learning, a software-engineering teacher trained with supervised fine-tuning, and Multi-teacher On-Policy Distillation. The resulting unified model consolidates complementary capabilities in reasoning, coding, tool use, professional work, long-context understanding, calibrated abstention, and instruction following. Across a broad evaluation suite, Motif 3 demonstrates competitive performance against leading open weight models, including strong results on long-horizon agentic tasks, mathematical reasoning, scientific knowledge, and hallucination-sensitive evaluation.

cs.AI↗

arXiv2026

Motif-Video 2B: Technical Report

Training strong video generation models usually requires massive datasets, large parameter counts, and substantial compute. In this work, we ask whether strong text-to-video quality is possible at a much smaller budget: fewer than 10M clips and less than 100,000 H200 GPU hours. Our core claim is that part of the answer lies in how model capacity is organized, not only in how much of it is used. In video generation, prompt alignment, temporal consistency, and fine-detail recovery can interfere with one another when they are handled through the same pathway. Motif-Video 2B addresses this by separating these roles architecturally, rather than relying on scale alone. The model combines two key ideas. First, Shared Cross-Attention strengthens text control when video token sequences become long. Second, a three-part backbone separates early fusion, joint representation learning, and detail refinement. To make this design effective under a limited compute budget, we pair it with an efficient training recipe based on dynamic token routing and early-phase feature alignment to a frozen pretrained video encoder. Our analysis shows that later blocks develop clearer cross-frame attention structure than standard single-stream baselines. On VBench, Motif-Video~2B reaches 83.76\%, surpassing Wan2.1 14B while using 7$\times$ fewer parameters and substantially less training data. These results suggest that careful architectural specialization, combined with an efficiency-oriented training recipe, can narrow or exceed the quality gap typically associated with much larger video models.

cs.CV↗

arXiv2026

CLAPS: Aleatoric-Epistemic Scaling via Last-Layer Laplace for Conformal Regression

Conformal regression provides finite-sample marginal coverage, but it does not by itself determine how interval width should adapt across heterogeneous inputs. Existing locally adaptive methods mainly account for aleatoric noise, leaving uncertainty from weak training support less explicit. We propose Conformal Laplace-Aware Predictive Scaling (CLAPS), a split conformal regression method that uses heteroscedastic last-layer Laplace uncertainty as the local normalization scale. CLAPS combines learned input-dependent noise with last-layer epistemic uncertainty, while retaining validity through standard conformal calibration. We characterize this aleatoric--epistemic scale, derive its heteroscedastic last-layer precision, and show that it reduces to aleatoric local scaling as epistemic uncertainty contracts. Experiments show nominal-level coverage with competitive interval efficiency.

cs.LG↗

arXiv2026

$ϕ$-Table: A Statistical Explanation for Global SHAP

Global SHAP explanations are typically presented as feature-importance rankings, which identify variables that matter to a black-box model but do not indicate whether their effects admit clear directional summaries, how uncertain those summaries are, or how faithfully they represent the fitted response. This paper proposes the $ϕ$-table, a SHAP-based statistical explanation table for tabular black-box regression models. The procedure selects features by SHAP importance and fits a standardized linear surrogate to the fitted model response $f(X)$, reporting SHAP importance together with model-response coefficients, uncertainty summaries, surrogate fidelity, and bootstrap coefficient stability. The resulting coefficients are interpreted as projections of the fitted model response onto the SHAP-selected feature set. Across synthetic, semi-synthetic, and real-data experiments, the $ϕ$-table extends ranking-only SHAP into a statistical global explanation by exposing direction, uncertainty, fidelity, and stability as distinct components of fitted model behavior.

stat.ML↗

arXiv2026

How Prompts Move Language Model Behavior: Frames, Salience, and Construal as Semantic Control

Prompt engineering is widely used to shape large language model behavior, yet it is often treated as a practical heuristic rather than as a form of natural-language control. This paper develops a cognitive-semantic account in which prompts function as semantic conditions on how a fixed model interprets inputs, foregrounds information, and structures tasks. We formalize this account through three notions -- frame activation, salience control, and construal selection -- and study them in natural language inference, claim verification, and multi-hop question answering. Across these settings, prompts produce measurable changes in label judgments, evidence use, and answer-support organization, showing that prompt effects differ not only in magnitude but also in semantic direction. The paper therefore reframes prompting as the analysis of how instructions move model behavior, rather than only whether they improve performance.

cs.LG↗

arXiv2026

A Ridge Too Far: Correcting Over-Shrinkage via Negative Regularization

Conventional regularization is designed to control variance, but in small-data regression it can also aggravate underfitting when predictive signal is concentrated in weak directions of a restricted representation. We study a negative-capable ridge family that permits a feasible negative region whenever the estimator remains well posed, and show that negative regularization acts there as controlled anti-shrinkage by increasing effective complexity most strongly along weak eigendirections. Building on this mechanism, we formalize weak-spectrum underfitting, derive a sign-switch result under conservative baseline shrinkage, and study criterion-based automatic selection over the full negative-capable family. Synthetic and semi-synthetic experiments support the theory by verifying feasibility, spectral complexity increase, sign-switch behavior, and effective recovery of negative adjustments in the predicted regimes.

cs.LG↗

arXiv2026

Gaming and Cooperation in Federated Learning: What Can Happen and How to Monitor It

The success of federated learning (FL) ultimately depends on how strategic participants behave under partial observability, yet most formulations still treat FL as a static optimization problem. We instead view FL deployments as governed strategic systems and develop an analytical framework that separates welfare-improving behavior from metric gaming. Within this framework, we introduce indices that quantify manipulability, the price of gaming, and the price of cooperation, and we use them to study how rules, information disclosure, evaluation metrics, and aggregator-switching policies reshape incentives and cooperation patterns. We derive threshold conditions for deterring harmful gaming while preserving benign cooperation, and for triggering auto-switch rules when early-warning indicators become critical. Building on these results, we construct a design toolkit including a governance checklist and a simple audit-budget allocation algorithm with a provable performance guarantee. Simulations across diverse stylized environments and a federated learning case study consistently match the qualitative and quantitative patterns predicted by our framework. Taken together, our results provide design principles and operational guidelines for reducing metric gaming while sustaining stable, high-welfare cooperation in FL platforms.

cs.LG↗

arXiv2025

Motif-2-12.7B-Reasoning: A Practitioner's Guide to RL Training Recipes

We introduce Motif-2-12.7B-Reasoning, a 12.7B parameter language model designed to bridge the gap between open-weight systems and proprietary frontier models in complex reasoning and long-context understanding. Addressing the common challenges of model collapse and training instability in reasoning adaptation, we propose a comprehensive, reproducible training recipe spanning system, data, and algorithmic optimizations. Our approach combines memory-efficient infrastructure for 64K-token contexts using hybrid parallelism and kernel-level optimizations with a two-stage Supervised Fine-Tuning (SFT) curriculum that mitigates distribution mismatch through verified, aligned synthetic data. Furthermore, we detail a robust Reinforcement Learning Fine-Tuning (RLFT) pipeline that stabilizes training via difficulty-aware data filtering and mixed-policy trajectory reuse. Empirical results demonstrate that Motif-2-12.7B-Reasoning achieves performance comparable to models with significantly larger parameter counts across mathematics, coding, and agentic benchmarks, offering the community a competitive open model and a practical blueprint for scaling reasoning capabilities under realistic compute constraints.

cs.AI↗

arXiv2025

Development of an Agentic AI Model for NGS Downstream Analysis Targeting Researchers with Limited Biological Background

Next-Generation Sequencing (NGS) has become a cornerstone of genomic research, yet the complexity of downstream analysis-ranging from differential expression gene (DEG) identification to biological interpretations-remains a significant barrier for researchers lacking specialized computational and biological expertise. While recent studies have introduced AI agents for RNA-seq analysis, most focus on general workflows without offering tailored interpretations or guidance for novices. To address this gap, we developed an Agentic AI model designed to automate NGS downstream analysis, provide literature-backed interpretations, and autonomously recommend advanced analytical methods. Built on the Llama 3 70B Large Language Model (LLM) and a Retrieval-Augmented Generation (RAG) framework, the model is deployed as an interactive Streamlit web application. The system integrates standard bioinformatics tools (Biopython, GSEApy, gProfiler) to execute core analyses, including DEG identification, clustering, and pathway enrichment. Uniquely, the agent utilizes RAG to query PubMed via Entrez, synthesizing biological insights and validating hypotheses with current literature. In a case study using cancer-related dataset, the model successfully identified significant DEGs, visualized clinical correlations, and derived evidence-based insights (e.g., linking BRAF mutations to prognosis), subsequently executing advanced survival modeling upon user selection. This framework democratizes bioinformatics by enabling researchers with limited backgrounds to seamlessly transition from basic data processing to advanced hypothesis testing and validation.

q-bio.GN↗

arXiv2025

Motif 2 12.7B technical report

We introduce Motif-2-12.7B, a new open-weight foundation model that pushes the efficiency frontier of large language models by combining architectural innovation with system-level optimization. Designed for scalable language understanding and robust instruction generalization under constrained compute budgets, Motif-2-12.7B builds upon Motif-2.6B with the integration of Grouped Differential Attention (GDA), which improves representational efficiency by disentangling signal and noise-control attention pathways. The model is pre-trained on 5.5 trillion tokens spanning diverse linguistic, mathematical, scientific, and programming domains using a curriculum-driven data scheduler that gradually changes the data composition ratio. The training system leverages the MuonClip optimizer alongside custom high-performance kernels, including fused PolyNorm activations and the Parallel Muon algorithm, yielding significant throughput and memory efficiency gains in large-scale distributed environments. Post-training employs a three-stage supervised fine-tuning pipeline that successively enhances general instruction adherence, compositional understanding, and linguistic precision. Motif-2-12.7B demonstrates competitive performance across diverse benchmarks, showing that thoughtful architectural scaling and optimized training design can rival the capabilities of much larger models.

cs.CL↗

arXiv2025

Grouped Differential Attention

The self-attention mechanism, while foundational to modern Transformer architectures, suffers from a critical inefficiency: it frequently allocates substantial attention to redundant or noisy context. Differential Attention addressed this by using subtractive attention maps for signal and noise, but its required balanced head allocation imposes rigid constraints on representational flexibility and scalability. To overcome this, we propose Grouped Differential Attention (GDA), a novel approach that introduces unbalanced head allocation between signal-preserving and noise-control groups. GDA significantly enhances signal focus by strategically assigning more heads to signal extraction and fewer to noise-control, stabilizing the latter through controlled repetition (akin to GQA). This design achieves stronger signal fidelity with minimal computational overhead. We further extend this principle to group-differentiated growth, a scalable strategy that selectively replicates only the signal-focused heads, thereby ensuring efficient capacity expansion. Through large-scale pretraining and continual training experiments, we demonstrate that moderate imbalance ratios in GDA yield substantial improvements in generalization and stability compared to symmetric baselines. Our results collectively establish that ratio-aware head allocation and selective expansion offer an effective and practical path toward designing scalable, computation-efficient Transformer architectures.

cs.LG↗

arXiv2025

Expanding Foundational Language Capabilities in Open-Source LLMs through a Korean Case Study

We introduce Llama-3-Motif, a language model consisting of 102 billion parameters, specifically designed to enhance Korean capabilities while retaining strong performance in English. Developed on the Llama 3 architecture, Llama-3-Motif employs advanced training techniques, including LlamaPro and Masked Structure Growth, to effectively scale the model without altering its core Transformer architecture. Using the MoAI platform for efficient training across hyperscale GPU clusters, we optimized Llama-3-Motif using a carefully curated dataset that maintains a balanced ratio of Korean and English data. Llama-3-Motif shows decent performance on Korean-specific benchmarks, outperforming existing models and achieving results comparable to GPT-4.

cs.CL↗

arXiv2025

SCOPE: Stochastic and Counterbiased Option Placement for Evaluating Large Language Models

Large Language Models (LLMs) can achieve inflated scores on multiple-choice tasks by exploiting inherent biases in option positions or labels, rather than demonstrating genuine understanding. This study introduces SCOPE, an evaluation framework designed to measure and mitigate such selection bias in a dataset-independent manner. By repeatedly invoking a null prompt that lacks semantic content, SCOPE estimates each model's unique position-bias distribution. It then redistributes the answer slot according to the inverse-bias distribution, thereby equalizing the lucky-rate, the probability of selecting the correct answer by chance. Furthermore, it prevents semantically similar distractors from being placed adjacent to the answer, thereby blocking near-miss guesses based on superficial proximity cues. Across multiple benchmark experiments, SCOPE consistently outperformed existing debiasing methods in terms of stable performance improvements and showed clearer confidence distributions over correct options. This framework thus offers a new standard for enhancing the fairness and reliability of LLM evaluations.

cs.CL↗

arXiv2025

Motif 2.6B Technical Report

Recent advancements in Large Language Models (LLMs) have revolutionized artificial intelligence, yet developing an effective foundational LLM that balances high performance with computational efficiency remains challenging, especially for emerging research groups. To address this gap, we introduce Motif-2.6B, a 2.6-billion-parameter foundation model designed to democratize advanced LLM capabilities. Motif-2.6B incorporates several innovative architectural enhancements, including Differential Attention and PolyNorm activation functions, which improve long-context comprehension, reduce hallucination, and enhance in-context learning capabilities. We rigorously tested multiple novel architectural components through extensive experimentation to determine the optimal architecture for Motif-2.6B. Comprehensive evaluations demonstrate that Motif-2.6B consistently meets or exceeds the performance of similarly sized state-of-the-art models across diverse benchmarks, showcasing its effectiveness, scalability, and real-world applicability. Through detailed experiments and tailored techniques, Motif-2.6B significantly advances the landscape of efficient, scalable, and powerful foundational LLMs, offering valuable insights and a robust foundation for future research and deployment.

cs.LG↗

arXiv2015

A classification of transitive links and periodic links

We generalized the periodic links to \emph{transitive} links in a $3$-manifold $M$. We find a complete classification theorem of transitive links in a $3$-dimensional sphere $\mathbb{R}^3$. We study these links from several different aspects including polynomial invariants using the relation between link polynomials of a transitive link and its factor links.

math.GT↗

arXiv2015

The complete list of prime knots whose flat plumbing basket numbers are 6 or less

Flat plumbing basket surfaces of links were introduced to study the geometry of the complement of the links. These flat plumbing basket surface can be presented by a sequential presentation known as flat plumbing basket code first found by Furihata, Hirasawa and Kobayashi. The minimum number of flat plumbings to obtain a flat plumbing basket surfaces of a link is defined to be the flat plumbing basket number of the given link. In present article, we use these sequential presentations to find the complete classification theorem of prime knots whose flat plumbing basket number $6$ or less. As applications, this result improves the work of Hirose and Nakashima which finds the flat plumbing basket number of prime knots up to 9 crossings.

math.GT↗

arXiv2015

On the (n,k)-th Catalan numbers

In this paper, we generalize the Catalan number to the $(n,k)$-th Catalan numbers and find a combinatorial description that the $(n,k)$-th Catalan numbers is equal to the number of partitions of $n(k-1)+2$ polygon by $(k+1)$-gon where all vertices of all $(k+1)$-gons lie on the vertices of $n(k-1)+2$ polygon.

math.CO↗