arXiv subjects

Jahyun Koo

Publications and source records attributed to Jahyun Koo.

At least 19 recordsLinked to original sources

arXiv2026

Carryover Drafting: Recycling Rejected States for Speculative Decoding

Speculative decoding accelerates LLM inference by verifying multiple drafted tokens in parallel, allowing a single target forward pass to accept several tokens. By construction, verification computes representations for both accepted and rejected tokens. Yet, conventional drafters retain only the representations of accepted tokens, leaving the substantial verifier computation spent on rejected tokens effectively wasted. We find that these discarded hidden states generated during target forward retain useful information about future tokens that can improve subsequent drafts. However, realizing this opportunity poses two distinct challenges. At inference, recycling overhead can increase drafting latency, diminishing the speedup gained from increased acceptance length. During training, standard parallel drafter training does not produce inference-aligned rejected states, while obtaining them through sequential rollouts would sacrifice parallelism across training positions. We introduce Carryover Drafting, which addresses both challenges. Carryover recycles rejected target hidden states as temporary KV context, allowing the drafter to selectively attend to them. It reuses the drafter's existing interface and adds only a single learned embedding to distinguish rejected states from committed context. The additional KV context is replaced each drafting round, keeping its length bounded by one proposal block. We introduce parallel draft--verify--draft training that exposes the drafter to inference-aligned rejected states while preserving parallelism across training positions. Experiments with DFlash and a DSpark-derived semi-autoregressive drafter across two target models show that this simple Carryover mechanism improves average acceptance length by 6.5--14.7% and end-to-end vLLM speedup by 7.9--14.4% over the corresponding baselines, with speedup gains reaching 28.8% on translation.

cs.LG↗

arXiv2026

MX-SAFE: Versatile Inference- and Training-Proof Microscaling Format with On-the-Fly Exponent and Mantissa Bit Allocation

As the demand for deep learning grows, cost reduction through quantization has become essential for both training and inference. In 2022, the Open Compute Project (OCP) consortium standardized narrow precision formats for deep learning, called the microscaling (MX) format. The MX format is a hardware-friendly dynamic quantization scheme that effectively reduces the data size by sharing an 8-bit exponent across multiple operands. The MX format can be categorized into two types with their own strengths: (i) MXINT which focuses on a high precision consisting only of mantissa bits and (ii) MXFP which focuses on a wider dynamic range by allowing local exponent bits. In this work, we present a versatile MXFP format, called MX-SAFE (MXSF in short), that adaptively uses two modes, i.e., a wider mantissa mode (FP8 E2M5) and a subnormal FP mode (FP5 E3M2), to support both training and direct-cast inference. Furthermore, we propose a tile-based block design to increase hardware efficiency by reducing the burden of re-quantization process during the training with the MXSF format. Owing to the use of the proposed MXSF format, 0.05%/11.1% and 3.55%/3.57% improvements in accuracy, on average, for inference/full-training compared to MXFP8 E2M5 and MXFP8 E4M3 are observed, respectively. Moreover, we present a training-inference accelerator that supports the MXSF format and it achieves similar accuracy to the BF16 baseline while using 24.9% less total energy consumption.

cs.AR↗

arXiv2025

Program Synthesis via Test-Time Transduction

We introduce transductive program synthesis, a new formulation of the program synthesis task that explicitly leverages test inputs during synthesis. While prior approaches to program synthesis--whether based on natural language descriptions or input-output examples--typically aim to generalize from training examples, they often struggle with robustness, especially in real-world settings where training examples are limited and test inputs involve various edge cases. To address this, we propose a novel framework that improves robustness by treating synthesis as an active learning over a finite hypothesis class defined by programs' outputs. We use an LLM to predict outputs for selected test inputs and eliminate inconsistent hypotheses, where the inputs are chosen via a greedy maximin algorithm to minimize the number of LLM queries required. We evaluate our approach on four benchmarks: Playgol, MBPP+, 1D-ARC, and programmatic world modeling on MiniGrid. We demonstrate that our method significantly improves program synthesis in both accuracy and efficiency. We release our code at https://github.com/klee972/SYNTRA.

cs.AI↗

arXiv2025

Black-Box Hallucination Detection via Consistency Under the Uncertain Expression

Despite the great advancement of Language modeling in recent days, Large Language Models (LLMs) such as GPT3 are notorious for generating non-factual responses, so-called "hallucination" problems. Existing methods for detecting and alleviating this hallucination problem require external resources or the internal state of LLMs, such as the output probability of each token. Given the LLM's restricted external API availability and the limited scope of external resources, there is an urgent demand to establish the Black-Box approach as the cornerstone for effective hallucination detection. In this work, we propose a simple black-box hallucination detection metric after the investigation of the behavior of LLMs under expression of uncertainty. Our comprehensive analysis reveals that LLMs generate consistent responses when they present factual responses while non-consistent responses vice versa. Based on the analysis, we propose an efficient black-box hallucination detection metric with the expression of uncertainty. The experiment demonstrates that our metric is more predictive of the factuality in model responses than baselines that use internal knowledge of LLMs.

cs.CL↗

arXiv2025

Materials and Design Strategies of Fully 3D Printed Biodegradable Wireless Devices for Biomedical Applications

Three-dimensional (3D) printing of bioelectronics offers a versatile platform for fabricating personalized and structurally integrated electronic systems within biological scaffolds. Biodegradable electronics, which naturally dissolve after their functional lifetime, minimize the long-term burden on both patients and healthcare providers by eliminating the need for surgical retrieval. In this study, we developed a library of 3D-printable, biodegradable electronic inks encompassing conductors, semiconductors, dielectrics, thereby enabling the direct printing of fully functional, multi-material, customizable electronic systems in a single integrated process. Especially, conjugated molecules were introduced to improve charge mobility, energy level alignment in semiconducting inks. This ink platform supports the fabrication of passive/active components and physical/chemical sensors making it suitable for complex biomedical applications. Versatility of this system was demonstrated through two representative applications: (i) wireless pressure sensor embedded within biodegradable scaffolds, (ii) wireless electrical stimulators that retain programmable electrical functionality in vivo and degrade post-implantation. This work establishes a foundation of modules for autonomous, biodegradable bioelectronic systems fabricated entirely via 3D printing, with implications for personalized diagnostics, therapeutic interfaces, and transient medical devices.

physics.med-ph↗

arXiv2025

SWITCH: Studying with Teacher for Knowledge Distillation of Large Language Models

Despite the success of Large Language Models (LLMs), they still face challenges related to high inference costs and memory requirements. To address these issues, Knowledge Distillation (KD) has emerged as a popular method for model compression, with student-generated outputs (SGOs) as training data being particularly notable for reducing the mismatch between training and inference. However, SGOs often produce noisy and biased sequences, which can lead to misguidance from the teacher model, especially in long sequences. To mitigate these challenges, we propose SWITCH (Studying WIth TeaCHer for Knowledge Distillation), a novel approach that strategically incorporates the teacher model during the student's sequence generation. SWITCH identifies discrepancies between the token probabilities of the teacher and student models, allowing the teacher to intervene selectively, particularly in long sequences that are more prone to teacher misguidance. Extensive experimental results across three model families and five instruction-following datasets show that SWITCH surpasses traditional KD methods, particularly excelling in the generation of long sequential data.

cs.CL↗

arXiv2025

LLMs can be easily Confused by Instructional Distractions

Despite the fact that large language models (LLMs) show exceptional skill in instruction following tasks, this strength can turn into a vulnerability when the models are required to disregard certain instructions. Instruction-following tasks typically involve a clear task description and input text containing the target data to be processed. However, when the input itself resembles an instruction, confusion may arise, even if there is explicit prompting to distinguish between the task instruction and the input. We refer to this phenomenon as instructional distraction. In this paper, we introduce a novel benchmark, named DIM-Bench, specifically designed to assess LLMs' performance under instructional distraction. The benchmark categorizes real-world instances of instructional distraction and evaluates LLMs across four instruction tasks: rewriting, proofreading, translation, and style transfer -- alongside five input tasks: reasoning, code generation, mathematical reasoning, bias detection, and question answering. Our experimental results reveal that even the most advanced LLMs are susceptible to instructional distraction, often failing to accurately follow user intent in such cases.

cs.CL↗

arXiv2024

Pomeranchuk Instability Induced by an Emergent Higher-Order van Hove Singularity on the Distorted Kagome Surface of Co$_3$Sn$_2$S$_2$

Materials hosting flat bands at the vicinity of the Fermi level promote exotic symmetry broken states. Common to many of these are van Hove singularities at saddle points of the dispersion or even higher-order van Hove singularities where the dispersion is flattened further. The band structure of kagome metals hosts both a flat band and two regular saddle points flanking a Dirac node. We investigate the kagome ferromagnetic metal Co$_3$Sn$_2$S$_2$ using scanning tunneling spectroscopy. We identify a new mechanism by which a triangular distortion on its kagome Co$_3$Sn surface termination considerably flattens the saddle point dispersion, and induces an isolated higher-order van Hove singularity (HOvHS) with algebraically divergent density of states pinned to the Fermi energy. The distortion-induced HOvHS precipitates a Pomeranchuk instability of the Fermi surface, resulting in the formation of a series of nematic electronic states. We visualize the nematic order across an energy shell of about 100 meV in both real-, reciprocal-, and momentum-spaces, as a cascade of wavefunction distributions which spontaneously break the remaining rotational symmetry of the underlying distorted kagome lattice, without generating any additional translational symmetry breaking. It signifies the spontaneous removal of a subset of saddle points from the Fermi energy to lower energies. By tracking the electronic wavefunction structure across the deformed Fermi surface we further identify a charge pumping-like evolution of the wavefunction center of mass. The mechanism we find for the generation of higher-order saddle points under a kagome distortion may be common to other kagome materials, and potentially other lattice structures, suggesting a generic new avenue for inducing unconventional electronic instabilities towards exotic states of matter.

cond-mat.str-el↗

arXiv2024

OPAL: Outlier-Preserved Microscaling Quantization Accelerator for Generative Large Language Models

To overcome the burden on the memory size and bandwidth due to ever-increasing size of large language models (LLMs), aggressive weight quantization has been recently studied, while lacking research on quantizing activations. In this paper, we present a hardware-software co-design method that results in an energy-efficient LLM accelerator, named OPAL, for generation tasks. First of all, a novel activation quantization method that leverages the microscaling data format while preserving several outliers per sub-tensor block (e.g., four out of 128 elements) is proposed. Second, on top of preserving outliers, mixed precision is utilized that sets 5-bit for inputs to sensitive layers in the decoder block of an LLM, while keeping inputs to less sensitive layers to 3-bit. Finally, we present the OPAL hardware architecture that consists of FP units for handling outliers and vectorized INT multipliers for dominant non-outlier related operations. In addition, OPAL uses log2-based approximation on softmax operations that only requires shift and subtraction to maximize power efficiency. As a result, we are able to improve the energy efficiency by 1.6~2.2x, and reduce the area by 2.4~3.1x with negligible accuracy loss, i.e., <1 perplexity increase.

cs.LG↗

arXiv2024

Charge transfer and Spin-Valley locking in 4Hb-TaS$_{2}$

4Hb-TaS$_2$ is a superconductor that exhibits unique characteristics such as time-reversal symmetry breaking, hidden magnetic memory, and topological edge modes. It is a naturally occurring heterostructure comprising of alternating layers of 1H-TaS$_2$ and 1T-TaS$_2$. The former is a well-known superconductor, while the latter is a correlated insulator with a possible non-trivial magnetic ground state. In this study, we use angle resolved photoemission spectroscopy to investigate the normal state electronic structure of this unconventional superconductor. Our findings reveal that the band structure of 4H-TaS$_2$ fundamentally differs from that of its constituent materials. Specifically, we observe a significant charge transfer from the 1T layers to the 1H layers that drives the 1T layers away from half-filling. In addition, we find a substantial reduction in inter-layer coupling in 4Hb-TaS$_2$ compared to the coupling in 2H-TaS$_2$ that results in a pronounced spin-valley locking within 4Hb-TaS$_2$

cond-mat.supr-con↗

arXiv2024

LifeTox: Unveiling Implicit Toxicity in Life Advice

As large language models become increasingly integrated into daily life, detecting implicit toxicity across diverse contexts is crucial. To this end, we introduce LifeTox, a dataset designed for identifying implicit toxicity within a broad range of advice-seeking scenarios. Unlike existing safety datasets, LifeTox comprises diverse contexts derived from personal experiences through open-ended questions. Experiments demonstrate that RoBERTa fine-tuned on LifeTox matches or surpasses the zero-shot performance of large language models in toxicity classification tasks. These results underscore the efficacy of LifeTox in addressing the complex challenges inherent in implicit toxicity. We open-sourced the dataset\footnote{\url{https://huggingface.co/datasets/mbkim/LifeTox}} and the LifeTox moderator family; 350M, 7B, and 13B.

cs.CL↗

arXiv2024

Theoretical investigation of the vertical dielectric screening dependence on defects for few-layered van der Waals materials

First-principle calculations were employed to analyze the effects induced by vacancies of molybdenum (Mo) and sulfur (S) on the dielectric properties of few-layered MoS2. We explored the combined effects of vacancies and dipole interactions on the dielectric properties of few-layered MoS2. In the presence of dielectric screening, we investigated uniformly distributed Mo and S vacancies, and then considered the case of concentrated vacancies. Our results show that the dielectric screening remarkably depends on the distribution of vacancies owing to the polarization induced by the vacancies and on the interlayer distances. This conclusion was validated for a wide range of wide-gap semiconductors with different positions and distributions of vacancies, providing an effective and reliable method for calculating and predicting electrostatic screening of dimensionally reduced materials. We further provided a method for engineering the dielectric constant by changing the interlayer distance, tuning the number of vacancies and the distribution of vacancies in few-layered van der Waals materials for their application in nanodevices and supercapacitors.

cond-mat.mtrl-sci↗

arXiv2022

LightNorm: Area and Energy-Efficient Batch Normalization Hardware for On-Device DNN Training

When training early-stage deep neural networks (DNNs), generating intermediate features via convolution or linear layers occupied most of the execution time. Accordingly, extensive research has been done to reduce the computational burden of the convolution or linear layers. In recent mobile-friendly DNNs, however, the relative number of operations involved in processing these layers has significantly reduced. As a result, the proportion of the execution time of other layers, such as batch normalization layers, has increased. Thus, in this work, we conduct a detailed analysis of the batch normalization layer to efficiently reduce the runtime overhead in the batch normalization process. Backed up by the thorough analysis, we present an extremely efficient batch normalization, named LightNorm, and its associated hardware module. In more detail, we fuse three approximation techniques that are i) low bit-precision, ii) range batch normalization, and iii) block floating point. All these approximate techniques are carefully utilized not only to maintain the statistics of intermediate feature maps, but also to minimize the off-chip memory accesses. By using the proposed LightNorm hardware, we can achieve significant area and energy savings during the DNN training without hurting the training accuracy. This makes the proposed hardware a great candidate for the on-device training.

cs.AR↗

arXiv2022

Field-linear anomalous Hall effect and Berry curvature induced by spin chirality in the kagome antiferromagnet Mn3Sn

During the past two decades, it has been established that a non-trivial electron wave-function topology generates an anomalous Hall effect (AHE), which shows itself as a Hall conductivity non-linear in magnetic field. Here, we report on an unprecedented case of field-linear AHE. In Mn$_3$Sn, a kagome magnet, the out-of-plane Hall response, which shows an abrupt jump, was discovered to be a case of AHE. We find now that the in-plane Hall response, which is perfectly linear in magnetic field, is set by the Berry curvature of the wavefunction. The amplitude of the Hall response and its concomitant Nernst signal exceed by far what is expected in the semiclassical picture. We argue that magnetic field induces out-of-plane spin canting and thereafter gives rise to nontrivial spin chirality on the kagome lattice. In band structure, we find that the spin chirality modifies the topology by gapping out Weyl nodal lines unknown before, accounting for the AHE observed. Our work reveals intriguing unification of real-space Berry phase from spin chirality and momentum-space Berry curvature.

cond-mat.mtrl-sci↗

arXiv2022

FlexBlock: A Flexible DNN Training Accelerator with Multi-Mode Block Floating Point Support

Training deep neural networks (DNNs) is a computationally expensive job, which can take weeks or months even with high performance GPUs. As a remedy for this challenge, community has started exploring the use of more efficient data representations in the training process, e.g., block floating point (BFP). However, prior work on BFP-based DNN accelerators rely on a specific BFP representation making them less versatile. This paper builds upon an algorithmic observation that we can accelerate the training by leveraging multiple BFP precisions without compromising the finally achieved accuracy. Backed up by this algorithmic opportunity, we develop a flexible DNN training accelerator, dubbed FlexBlock, which supports three different BFP precision modes, possibly different among activation, weight, and gradient tensors. While several prior works proposed such multi-precision support for DNN accelerators, not only do they focus only on the inference, but also their core utilization is suboptimal at a fixed precision and specific layer types when the training is considered. Instead, FlexBlock is designed in such a way that high core utilization is achievable for i) various layer types, and ii) three BFP precisions by mapping data in a hierarchical manner to its compute units. We evaluate the effectiveness of FlexBlock architecture using well-known DNNs on CIFAR, ImageNet and WMT14 datasets. As a result, training in FlexBlock significantly improves the training speed by 1.5~5.3x and the energy efficiency by 2.4~7.0x on average compared to other training accelerators and incurs marginal accuracy loss compared to full-precision training.

cs.LG↗

arXiv2022

Exchange-biased topological transverse thermoelectric effects in a Kagome ferrimagnet

Kagome metal TbMn6Sn6 was recently discovered to be a ferrimagnetic topological Dirac material by scanning tunneling microscopy/spectroscopy measurements. Here, we report the observation of large anomalous Nernst effect and anomalous thermal Hall effect in this compound. The anomalous transverse transport is consistent with the Berry curvature contribution from the massive Dirac gaps in the 3D momentum space as demonstrated by our first-principles calculations. Furthermore, the transverse thermoelectric transport exhibits asymmetry with respect to the applied magnetic field, i.e., an exchange-bias behavior. Together, these features place TbMn6Sn6 as a promising system for the outstanding thermoelectric performance based on anomalous Nernst effect.

cond-mat.mtrl-sci↗

arXiv2021

Visualization of Topological Boundary Modes Manifesting Topological Nodal-Point Superconductivity

The extension of the topological classification of band insulators to topological semimetals gave way to the topology classes of Dirac, Weyl, and nodal line semimetals with their unique Fermi arc and drum head boundary modes. Similarly, there are several suggestions to employ the classification of topological superconductors for topological nodal superconductors with Majorana boundary modes. Here, we show that the surface 1H termination of the transition metal dichalcogenide compound 4Hb-TaS$_2$, in which 1T-TaS$_2$ and 1H-TaS$_2$ layers are interleaved, has the phenomenology of a topological nodal point superconductor. We find in scanning tunneling spectroscopy a residual density of states within the superconducting gap. An exponentially decaying bound mode is imaged within the superconducting gap along the boundaries of the exposed 1H layer characteristic of a gapless Majorana edge mode. The anisotropic nature of the localization length of the edge mode aims towards topological nodal superconductivity. A zero-bias conductance peak is further imaged within fairly isotropic vortex cores. All our observations are accommodated by a theoretical model of a two-dimensional nodal Weyl-like superconducting state, which ensues from inter-orbital Cooper pairing. The observation of an intrinsic topological nodal superconductivity in a layered material will pave the way for further studies of Majorana edge modes and its applications in quantum information processing.

cond-mat.supr-con↗

arXiv2021

Quantum oscillations, Magnetic breakdown and thermal Hall effect in Co$_3$Sn$_2$S$_2$

Co$_3$Sn$_2$S$_2$ is a ferromagnetic semi-metal with Weyl nodes in its band structure and a large anomalous Hall effect below its Curie temperature of 177 K. We present a detailed study of its Fermi surface and examine the relevance of the anomalous transverse Wiedemann Franz law to it. We studied Shubnikov-de Haas oscillations along two orientations in single crystals with a mobility as high as $2.7\times$10$^3$ cm$^2$V$^{-1}$s$^{-1}$ subject to a magnetic field as large as $\sim$ 60 T. The angle dependence of the frequencies is in agreement with density functional theory (DFT) calculations and reveals two types of hole pockets (H1, H2) and two types of electron pockets (E1, E2). An additional unexpected frequency emerges at high magnetic field. We attribute it to magnetic breakdown between the hole pocket H2 and the electron pocket E2, since it is close to the sum of the E2 and H2 fundamental frequencies. By measuring the anomalous thermal and electrical Hall conductivities, we quantified the anomalous transverse Lorenz ratio, which is close to the Sommerfeld ratio ($L_0=\frac{π^2}{3}\frac{k_B^2}{e^2}$) below 100 K and deviates downwards at higher temperatures. This finite temperature deviation from the anomalous Wiedemann-Franz law is a source of information on the distance between the sources and sinks of the Berry curvature and the chemical potential.

cond-mat.mes-hall↗