Searcharxiv⌕ Search

arXiv subjects

Junseok Lee

Publications and source records attributed to Junseok Lee.

At least 19 recordsLinked to original sources
arXiv2026

Residual Sparsification via Output Importance for Compressing Mixture-of-Experts LLMs

Mixture-of-experts (MoE) architectures scale large language models efficiently, but they demand massive GPU memory. To cope with such demand, models are commonly compressed to reduce their memory footprint. Residual sparsification is a representative compression technique that decomposes each projection matrix of an expert into a shared base matrix and per-expert residual matrix, and then compresses the residuals. Existing sparsification methods compress each residual matrix independently by minimizing its compression error, thereby minimizing the error of each projection matrix. However, our analysis shows that this objective is misaligned with preserving model accuracy after compression. In an expert, the final output is produced through computations coupled across multiple projections and hidden representations. Therefore, even small errors in individual matrices can propagate through hidden representations and projection interactions, leading to large expert output errors and accuracy degradation. To address this misalignment, we propose PARSER, a new residual sparsification method that shifts the compression objective from minimizing isolated matrix errors to preserving the expert output error. PARSER achieves this by introducing output importance, which measures the actual contribution to the expert output error. Our experiments show that, compared with existing methods, PARSER narrows the accuracy gap to the uncompressed model by 1.41$\times$ on Qwen and 1.44$\times$ on DeepSeek, while achieving the same peak memory reduction. Our code is available at https://github.com/OSSS-KU/PARSER.

cs.AI↗
arXiv2026

Axion Isocurvature Perturbations Survive the Scaling Evolution of Axion Domain Walls

We revisit the evolution of axion domain walls seeded by inflationary fluctuations. In our previous work, we showed that such domain-wall networks retain superhorizon correlations even after entering the scaling regime. We extend our previous analysis to the case with large initial fluctuations, where many minima of the axion potential are already populated when the axion starts to oscillate. Although the conventional misalignment contribution can have suppressed long-wavelength isocurvature perturbations when many vacua are averaged over, axions produced by domain-wall collapse provide an additional contribution that can dominate when the walls enter the scaling regime before annihilation. In particular, the biased vacuum energy released during wall annihilation inherits the superhorizon correlations of the inflationary fluctuations and transfers them to the axion energy density. We find that sizable isocurvature perturbations can therefore survive even after the walls annihilate. We also discuss generic isocurvature constraints on dark matter produced by domain-wall collapse.

hep-ph↗
arXiv2026

NELSSA: A GPU-PNM Heterogeneous System for Mixed-Length LLM Serving via Length-based Request Placement

Modern LLMs and their agentic applications are broadening the range of serving workloads, spanning context lengths from a few hundred tokens to hundreds of thousands. As these requests frequently interleave within the same serving window, LLM serving systems must handle highly heterogeneous mixed-length workloads. Such mixed-length workloads expose fundamental inefficiencies in GPU-centric serving architectures, whose throughput depends on large, memory-constrained batches. In this paper, we present NELSSA, an LLM serving system that integrates GPUs with real-world Processing-near-Memory (PNM) accelerator devices to efficiently support mixed-length workloads. NELSSA employs length-based request placement to route short-context requests to GPUs and long-context requests to the PNM tier, incorporating runtime migration to accommodate dynamic context growth without recomputation. We prototype NELSSA as an end-to-end system, implementing device-level sparse attention on PNM, GPU decode kernels, and a host-side runtime that orchestrates scheduling and cross-tier memory movement over a CXL-enabled infrastructure with RPC and RDMA support. Across mixed-length LLM workloads, NELSSA improves decode throughput by up to 5.5x in tokens/sec and reduces P99 latency by up to 15x compared to GPU-only baselines. Our end-to-end prototype and experimental results suggest that integrated GPU-PNM serving, enabled by CXL-based disaggregation, is a promising system paradigm for scalable and flexible LLM infrastructures that support evolving workloads.

cs.AR↗
arXiv2026

Arithmetic Symmetry in Ideal Prouhet-Tarry-Escott Solutions

Motivated in part by anomaly cancellation for integral charge spectra in chiral gauge theory, we study the symmetric locus in the ideal degree-three Prouhet-Tarry-Escott problem. A symmetric integer solution is one whose entries are paired about a common center $c\in \frac12\mathbb Z$. This symmetry reduces the problem to a sum-of-two-squares equation, $x^2+y^2=u^2+v^2$, in integer variables, subject to the appropriate parity conditions. Thus the problem is governed by representations as sums of two squares. For the full symmetric locus, let $N_{\mathrm{sym}}(H)$ denote the number of nontrivial symmetric integer solutions of height at most $H$, counted with unordered multiset conventions and summed over the admissible centers. Then \begin{align*} N_{\mathrm{sym}}(H) = \frac{4\log 2}{3\pi^2}H^3\log H+O(H^3). \end{align*} The logarithmic enhancement comes from the second moment of the sum-of-two-squares representation function. In particular, the symmetric locus is larger than one would expect from the naive $H^3$ degree-weighted box-counting scale alone. This asymptotic identifies a large arithmetically structured subfamily of the ideal degree-three solution space, and suggests that paired anomaly-free integral charge spectra reflect a fundamental number-theoretic structure.

math.NT↗
arXiv2026

GuardMarkGS: Unified Ownership Tracing and Edit Deterrence for 3D Gaussian Splatting

3D Gaussian Splatting (3DGS) is becoming a practical representation for novel view synthesis, but its growing adoption, together with rapid advances in instruction-driven 3DGS editing, also exposes a dual copyright risk: once a 3DGS-based asset is released, it can be used without permission and manipulated through 3D editing. Existing protection methods address only one side of this problem. Watermarking can trace ownership after unauthorized use, but it cannot prevent malicious editing. Adversarial edit-deterrence methods can disrupt editing, but they do not provide evidence of ownership. To the best of our knowledge, we present the first unified protection framework for 3DGS that jointly optimizes ownership tracing and unauthorized editing deterrence. Our framework combines a scene-wide watermarking objective over all Gaussians with an adversarial objective for edit deterrence. The adversarial branch combines latent-anchor separation, denoising-trajectory diversion, and cross-attention diversion to divert the editing trajectory, while an update-saliency-motivated Gaussian selection strategy assigns stronger adversarial updates to mask-selected Gaussians, improving the balance among watermark recovery, edit deterrence, and rendering fidelity. Experiments on scenes from Mip-NeRF 360 and Instruct-NeRF2NeRF demonstrate that the proposed framework achieves a favorable balance among bit accuracy, edit deterrence, and rendering quality. These results suggest that practical copyright protection of 3DGS-based assets can be more effectively addressed by integrating ownership tracing and unauthorized editing deterrence into a single optimization framework.

cs.CV↗
arXiv2026

CD-FKD: Cross-Domain Feature Knowledge Distillation for Robust Single-Domain Generalization in Object Detection

Single-domain generalization is essential for object detection, particularly when training models on a single source domain and evaluating them on unseen target domains. Domain shifts, such as changes in weather, lighting, or scene conditions, pose significant challenges to the generalization ability of existing models. To address this, we propose Cross-Domain Feature Knowledge Distillation (CD-FKD), which enhances the generalization capability of the student network by leveraging both global and instance-wise feature distillation. The proposed method uses diversified data through downscaling and corruption to train the student network, whereas the teacher network receives the original source domain data. The student network mimics the features of the teacher through both global and instance-wise distillation, enabling it to extract object-centric features effectively, even for objects that are difficult to detect owing to corruption. Extensive experiments on challenging scenes demonstrate that CD-FKD outperforms state-of-the-art methods in both target domain generalization and source domain performance, validating its effectiveness in improving object detection robustness to domain shifts. This approach is valuable in real-world applications, like autonomous driving and surveillance, where robust object detection in diverse environments is crucial.

cs.CV↗
arXiv2026

Number Theory in Quantum Physics: Minicharged Particles and the Prouhet-Tarry-Escott Problem

In quantum gauge theories, anomaly cancellation severely restricts the allowed patterns of chiral charges. Here we show that, in a phenomenologically motivated framework for light minicharged particles, the anomaly cancellation conditions are equivalent to the degree $k=3$ Prouhet-Tarry-Escott problem in number theory. This correspondence immediately implies that the hidden sector must contain at least four minicharged states. For constructions based on minimal ideal solutions, the mass spectrum generically exhibits a near-degenerate doublet structure, so that the discovery of one minicharged particle would point to a partner state with the same minicharge and a nearby mass. Our results uncover an unexpected link between quantum consistency and number theory, with direct implications for model building and future searches.

hep-ph↗
arXiv2026

Habilis-$\beta$: A Fast-Motion and Long-Lasting On-Device Vision-Language-Action Model

We introduce Habilis-$\beta$, a fast-motion and long-lasting on-device vision-language-action (VLA) model designed for real-world deployment. Current VLA evaluation remains largely confined to single-trial success rates under curated resets, which fails to capture the fast-motion and long-lasting capabilities essential for practical operation. To address this, we introduce the Productivity-Reliability Plane (PRP), which evaluates performance through Tasks per Hour (TPH) and Mean Time Between Intervention (MTBI) under a continuous-run protocol that demands both high-speed execution and sustained robustness. Habilis-$\beta$ achieves high performance by integrating language-free pre-training on large-scale play data for robust interaction priors with post-training on cyclic task demonstrations that capture state drift across consecutive task iterations. The system further employs ESPADA for phase-adaptive motion shaping to accelerate free-space transit, utilizes rectified-flow distillation to enable high-frequency control on edge devices, and incorporates classifier-free guidance (CFG) as a deployment-time knob to dynamically balance instruction adherence and learned interaction priors. In 1-hour continuous-run evaluations, Habilis-$\beta$ achieves strong performance under the PRP metrics, compared to $\pi_{0.5}$ in both simulation and real-world environments. In simulation, Habilis-$\beta$ achieves 572.6 TPH and 39.2 s MTBI (vs. 120.5 TPH and 30.5 s for $\pi_{0.5}$), while in a real-world humanoid logistics workflow it achieves 124 TPH and 137.4 s MTBI (vs. 19 TPH and 46.1 s for $\pi_{0.5}$). Finally, Habilis-$\beta$ achieves the highest reported performance on the standard RoboTwin 2.0 leaderboard across representative tasks, validating its effectiveness in complex manipulation scenarios.

cs.RO↗
arXiv2026

LSR-Net: A Lightweight and Strong Robustness Network for Bearing Fault Diagnosis in Noise Environment

Rotating bearings play an important role in modern industries, but have a high probability of occurrence of defects because they operate at high speed, high load, and poor operating environments. Therefore, if a delay time occurs when a bearing is diagnosed with a defect, this may cause economic loss and loss of life. Moreover, since the vibration sensor from which the signal is collected is highly affected by the operating environment and surrounding noise, accurate defect diagnosis in a noisy environment is also important. In this paper, we propose a lightweight and strong robustness network (LSR-Net) that is accurate in a noisy environment and enables real-time fault diagnosis. To this end, first, a denoising and feature enhancement module (DFEM) was designed to create a 3-channel 2D matrix by giving several nonlinearity to the feature-map that passed through the denoising module (DM) block composed of convolution-based denoising (CD) blocks. Moreover, adaptive pruning was applied to DM to improve denoising ability when the power of noise is strong. Second, for lightweight model design, a convolution-based efficiency shuffle (CES) block was designed using group convolution (GConv), group pointwise convolution (GPConv) and channel split that can design the model while maintaining low parameters. In addition, the trade-off between the accuracy and model computational complexity that can occur due to the lightweight design of the model was supplemented using attention mechanisms and channel shuffle. In order to verify the defect diagnosis performance of the proposed model, performance verification was conducted in a noisy environment using a vibration signal. As a result, it was confirmed that the proposed model had the best anti-noise ability compared to the benchmark models, and the computational complexity of the model was also the lowest.

eess.SP↗
arXiv2026

FastSLM: Hierarchical Temporal Abstraction for Efficient Long-Form Speech Adaptation

Scaling Multimodal Large Language Models (MLLMs) to long-form speech is bottlenecked by the explosive growth of input tokens. Existing speech-language models project high-frame-rate acoustic features directly into the LLM input space, making long-context processing computationally prohibitive. Unlike images or videos, speech lacks spatial redundancy, making extreme token compression particularly challenging. To address this limitation, we propose FastSLM, a token-efficient architecture featuring the Hierarchical Temporal Abstractor (HTA), which progressively distills acoustic features across multiple temporal scales. HTA achieves an extreme compression rate of 1.67 tokens per second (97% reduction) while preserving essential linguistic information for downstream speech-language understanding. Experimental results demonstrate that FastSLM achieves competitive performance across diverse speech-language tasks while requiring substantially fewer speech tokens and FLOPs than existing speech-language models. The source code and model checkpoints are available at https://github.com/Lee-junseok1025/FastSLM.

eess.AS↗
arXiv2026

ASKD-Whisper: Adaptive Self-knowledge Distillation for Efficient and Low-Latency Automatic Speech Recognition

Knowledge distillation (KD) is one of the most effective paradigms for compressing large-scale foundation models into deployable architectures. In the context of Automatic Speech Recognition (ASR), previous studies have predominantly focused on forcing the student model to strictly mimic the predictive distribution of a massive teacher model. However, this static dependency often presents an inherent trade-off: while the student rapidly acquires basic linguistic representations, it simultaneously inherits the teacher's domain-specific blind spots and over-confident hallucinations, leading to a severe decline in out-of-distribution generalization capacity. To effectively mitigate this issue, we propose Adaptive Self-Knowledge Distillation (ASKD), a dynamic curriculum framework. ASKD systematically decays the dependency on the teacher's distribution as training progresses-thereby unlocking the student's independent reasoning capacity-and subsequently employs a self-knowledge distillation phase to act as a structural regularizer. By applying ASKD, we distill the massive Whisper architecture into a compact variant, ASKD-Whisper. In our comprehensive evaluations across diverse acoustic domains, ASKD-Whisper not only achieves a 5x speedup in inference latency but also outperforms its teacher model by yielding a 1.07% lower word error rate (WER). These results demonstrate that ASKD effectively prevents teacher-induced overfitting and establishes a new state-of-the-art for generalizable model compression.

cs.CL↗
arXiv2025

Disentangling Hyperedges through the Lens of Category Theory

Despite the promising results of disentangled representation learning in discovering latent patterns in graph-structured data, few studies have explored disentanglement for hypergraph-structured data. Integrating hyperedge disentanglement into hypergraph neural networks enables models to leverage hidden hyperedge semantics, such as unannotated relations between nodes, that are associated with labels. This paper presents an analysis of hyperedge disentanglement from a category-theoretical perspective and proposes a novel criterion for disentanglement derived from the naturality condition. Our proof-of-concept model experimentally showed the potential of the proposed criterion by successfully capturing functional relations of genes (nodes) in genetic pathways (hyperedges).

cs.LG↗
arXiv2025

Dynamical Prevention of Topological Defect Formation

Topological defects can have significant cosmological consequences, so their production must be examined carefully. It is usually assumed that topological defects are produced if the temperature becomes sufficiently high, but in reality their formation depends on the post-inflationary dynamics of a symmetry-breaking scalar. We analyze the dynamics of a symmetry-breaking scalar field in the early universe within models that provide an effective negative mass term at the origin, and show that the symmetry can remain broken so that topological defects are never formed. In particular, we demonstrate that nonthermally produced particles (such as the Standard Model Higgs) during preheating can generate such an effective negative mass term, allowing the scalar field to follow a time-dependent minimum even in renormalizable models with a quartic coupling. We also discuss the implications of this result for the Peccei-Quinn scalar in axion models.

hep-ph↗
arXiv2025

Oldie but Goodie: Re-illuminating Label Propagation on Graphs with Partially Observed Features

In real-world graphs, we often encounter missing feature situations where a few or the majority of node features, e.g., sensitive information, are missed. In such scenarios, directly utilizing Graph Neural Networks (GNNs) would yield sub-optimal results in downstream tasks such as node classification. Despite the emergence of a few GNN-based methods attempting to mitigate its missing situation, when only a few features are available, they rather perform worse than traditional structure-based models. To this end, we propose a novel framework that further illuminates the potential of classical Label Propagation (Oldie), taking advantage of Feature Propagation, especially when only a partial feature is available. Now called by GOODIE, it takes a hybrid approach to obtain embeddings from the Label Propagation branch and Feature Propagation branch. To do so, we first design a GNN-based decoder that enables the Label Propagation branch to output hidden embeddings that align with those of the FP branch. Then, GOODIE automatically captures the significance of structure and feature information thanks to the newly designed Structure-Feature Attention. Followed by a novel Pseudo-Label contrastive learning that differentiates the contribution of each positive pair within pseudo-labels originating from the LP branch, GOODIE outputs the final prediction for the unlabeled nodes. Through extensive experiments, we demonstrate that our proposed model, GOODIE, outperforms the existing state-of-the-art methods not only when only a few features are available but also in abundantly available situations. Source code of GOODIE is available at: https://github.com/SukwonYun/GOODIE.

cs.LG↗
arXiv2025

Resolving the QCD Axion Domain Wall Problem with a Light Axion

We propose two novel solutions to the domain wall problem of the QCD axion by introducing a massless or light axion that also couples to gluons. The first solution applies when the new axion forms strings after inflation. Due to its mixing with the QCD axion, domain walls of the QCD axion are bounded by these strings and confined into cosmologically safe string bundles. This scenario predicts the existence of such string bundles, which may survive until today and leave observable signatures, such as gravitational waves, cosmic birefringence, and CMB anisotropies. The simultaneous detection of the QCD axion and any of these cosmological signatures would serve as a smoking-gun signal. The second solution assumes a homogeneous initial condition for the new axion. If it is sufficiently light, its potential temporarily induces a bias in the QCD axion potential before the onset of oscillations, rendering the domain walls unstable. In both scenarios, the Peccei-Quinn mechanism remains effective, and the strong CP problem is not reintroduced. We identify the viable parameter regions and discuss the resulting dark matter abundance.

hep-ph↗
arXiv2025

Towards LLM-Centric Multimodal Fusion: A Survey on Integration Strategies and Techniques

The rapid progress of Multimodal Large Language Models(MLLMs) has transformed the AI landscape. These models combine pre-trained LLMs with various modality encoders. This integration requires a systematic understanding of how different modalities connect to the language backbone. Our survey presents an LLM-centric analysis of current approaches. We examine methods for transforming and aligning diverse modal inputs into the language embedding space. This addresses a significant gap in existing literature. We propose a classification framework for MLLMs based on three key dimensions. First, we examine architectural strategies for modality integration. This includes both the specific integration mechanisms and the fusion level. Second, we categorize representation learning techniques as either joint or coordinate representations. Third, we analyze training paradigms, including training strategies and objective functions. By examining 125 MLLMs developed between 2021 and 2025, we identify emerging patterns in the field. Our taxonomy provides researchers with a structured overview of current integration techniques. These insights aim to guide the development of more robust multimodal integration strategies for future models built on pre-trained foundations.

cs.CL↗
arXiv2025

Global Context-aware Representation Learning for Spatially Resolved Transcriptomics

Spatially Resolved Transcriptomics (SRT) is a cutting-edge technique that captures the spatial context of cells within tissues, enabling the study of complex biological networks. Recent graph-based methods leverage both gene expression and spatial information to identify relevant spatial domains. However, these approaches fall short in obtaining meaningful spot representations, especially for spots near spatial domain boundaries, as they heavily emphasize adjacent spots that have minimal feature differences from an anchor node. To address this, we propose Spotscape, a novel framework that introduces the Similarity Telescope module to capture global relationships between multiple spots. Additionally, we propose a similarity scaling strategy to regulate the distances between intra- and inter-slice spots, facilitating effective multi-slice integration. Extensive experiments demonstrate the superiority of Spotscape in various downstream tasks, including single-slice and multi-slice scenarios. Our code is available at the following link: https: //github.com/yunhak0/Spotscape.

cs.LG↗
arXiv2025

AbBiBench: A Benchmark for Antibody Binding Affinity Maturation and Design

We introduce AbBiBench (Antibody Binding Benchmarking), a benchmarking framework for antibody binding affinity maturation and design. Unlike previous strategies that evaluate antibodies in isolation, typically by comparing them to natural sequences with metrics such as amino acid recovery rate or structural RMSD, AbBiBench instead treats the antibody-antigen (Ab-Ag) complex as the fundamental unit. It evaluates an antibody design's binding potential by measuring how well a protein model scores the full Ab-Ag complex. We first curate, standardize, and share more than 184,500 experimental measurements of antibody mutants across 14 antibodies and 9 antigens-including influenza, lysozyme, HER2, VEGF, integrin, Ang2, and SARS-CoV-2-covering both heavy-chain and light-chain mutations. Using these datasets, we systematically compare 15 protein models including masked language models, autoregressive language models, inverse folding models, diffusion-based generative models, and geometric graph models by comparing the correlation between model likelihood and experimental affinity values. Additionally, to demonstrate AbBiBench's generative utility, we apply it to antibody F045-092 in order to introduce binding to influenza H1N1. We sample new antibody variants with the top-performing models, rank them by the structural integrity and biophysical properties of the Ab-Ag complex, and assess them with in vitro ELISA binding assays. Our findings show that structure-conditioned inverse folding models outperform others in both affinity correlation and generation tasks. Overall, AbBiBench provides a unified, biologically grounded evaluation framework to facilitate the development of more effective, function-aware antibody design models.

q-bio.QM↗