arXiv subjects

Gyeonghun Kim

Publications and source records attributed to Gyeonghun Kim.

9 recordsLinked to original sources

arXiv2026

VinQA: Visual Elements Interleaved Long-form Answer Generation for Real-World Multimodal Document QA

Real-world documents combine text with tables, charts, photographs, and diagrams arranged in diverse layouts, yet existing research on multimodal large language models (MLLMs) for document QA predominantly produces text-only responses, underutilizing these visual elements. We introduce VinQA, a dataset for long-form answer generation where cited visual elements are explicitly interleaved with their supporting text and grounded in relevant document pages. To support this task, we study two encoding methods for feeding raw document page images into an MLLM, along with their visual-element citation mechanisms: (1) Page Encoding, which directly encodes full-page images with bounding boxes of visual elements and treats these boxed regions as citable units; and (2) Modality Encoding, which parses each page to extract text and crop visual elements, encodes them separately, and uses these cropped elements as citable units. In our experiments, we propose M-GroSE, a multimodal evaluation framework extending GroUSE to assess answers along four dimensions: completeness, answer relevancy, faithfulness, and unanswerability. We additionally report Visual Source F1 to directly measure visual citation accuracy. Although proprietary frontier models still achieve the best overall scores on the VinQA test split, fine-tuning open Qwen2.5-VL models on the training split substantially improves their performance and narrows this gap. Modality Encoding is initially more robust for complex documents with long text, many visual elements, and diverse citation requirements. After training on VinQA, however, Page Encoding reaches a comparable level, competing effectively even without the explicit parsing used in Modality Encoding. Finally, Visual G-Eval, an MLLM-based judge, confirms that fine-tuned models insert visual elements at semantically appropriate positions with faithful supporting text.

cs.CV↗

arXiv2026

Adaptive Capacity Allocation for Vision Language Action Fine-tuning

Vision language action models (VLAs) are increasingly used for Physical AI, but deploying a pre-trained VLA model to unseen environments, embodiments, or tasks still requires adaptation. Parameter-efficient fine-tuning (PEFT), especially LoRA, is common for VLA policies, yet the exposed capacity knob, the rank, does not transfer uniformly: robotics transfer exhibits a higher and task-varying intrinsic rank than language fine-tuning. Small ranks suffice for LLMs (e.g., $r \in \{4, 8\}$), while spectral analyses indicate VLAs may require much larger ranks (e.g., $r \approx 128$) or near-full rank, a mismatch that worsens in multi-task settings. We present LoRA-SP (Select-Prune), a rank-adaptive fine-tuning method that replaces fixed-rank updates with input- and layer-wise capacity. LoRA-SP uses an SVD-style parameterization with a small router whose nonnegative scores act as singular values over a shared vector bank. The active set is chosen by an energy target on the cumulative squared scores $E(k) \ge \eta$, providing a direct link to approximation error via our spectral analysis. During training, $\eta$ concentrates energy on a few directions and teaches the router to rely on fewer vectors while preserving accuracy. This yields compact adapters that reduce cross-task interference and improve generalization. On four real-robot manipulation tasks collected on an unseen AgileX PiPER arm, across two VLA backbones ($\pi_0$ and SmolVLA), LoRA-SP matches or exceeds full fine-tuning with far fewer trainable parameters, and improves multi-task success by up to 31.6% over standard LoRA while remaining robust to rank choice.

cs.RO↗

arXiv2023

TemporalWiki: A Lifelong Benchmark for Training and Evaluating Ever-Evolving Language Models

Language Models (LMs) become outdated as the world changes; they often fail to perform tasks requiring recent factual information which was absent or different during training, a phenomenon called temporal misalignment. This is especially a challenging problem because the research community still lacks a coherent dataset for assessing the adaptability of LMs to frequently-updated knowledge corpus such as Wikipedia. To this end, we introduce TemporalWiki, a lifelong benchmark for ever-evolving LMs that utilizes the difference between consecutive snapshots of English Wikipedia and English Wikidata for training and evaluation, respectively. The benchmark hence allows researchers to periodically track an LM's ability to retain previous knowledge and acquire updated/new knowledge at each point in time. We also find that training an LM on the diff data through continual learning methods achieves similar or better perplexity than on the entire snapshot in our benchmark with 12 times less computational cost, which verifies that factual knowledge in LMs can be safely updated with minimal training data via continual learning. The dataset and the code are available at https://github.com/joeljang/temporalwiki.

cs.CL↗

arXiv2023

Fresnel-type Solid Immersion Lens for efficient light collection from quantum defects in diamond

Quantum defects in diamonds have been studied as a promising resource for quantum science. The subtractive fabrication process for improving photon collection efficiency often require excessive milling time that can adversely affect the fabrication accuracy. We designed and fabricated a Fresnel-type solid immersion lens using the focused ion beam. For a 5.8 um-deep Nitrogen-vacancy (NV-) center, the milling time was highly reduced (1/3 compared to a hemispherical structure), while retaining high photon collection efficiency (> 2.24 compared to a flat surface). In numerical simulation, this benefit of the proposed structure is expected for a wide range of milling depths.

physics.optics↗

arXiv2022

External Knowledge Selection with Weighted Negative Sampling in Knowledge-grounded Task-oriented Dialogue Systems

Constructing a robust dialogue system on spoken conversations bring more challenge than written conversation. In this respect, DSTC10-Track2-Task2 is proposed, which aims to build a task-oriented dialogue (TOD) system incorporating unstructured external knowledge on a spoken conversation, extending DSTC9-Track1. This paper introduces our system containing four advanced methods: data construction, weighted negative sampling, post-training, and style transfer. We first automatically construct a large training data because DSTC10-Track2 does not release the official training set. For the knowledge selection task, we propose weighted negative sampling to train the model more fine-grained manner. We also employ post-training and style transfer for the response generation task to generate an appropriate response with a similar style to the target response. In the experiment, we investigate the effect of weighted negative sampling, post-training, and style transfer. Our model ranked 7 out of 16 teams in the objective evaluation and 6 in human evaluation.

cs.CL↗

arXiv2022

Wigner-molecularization-enabled dynamic nuclear field programming

Multielectron semiconductor quantum dots (QDs) provide a novel platform to study the role of Coulomb correlations in finite quantum systems and their impact on many-body energy spectra. An example is the formation of interaction-driven, spatially localized electron states of Wigner molecules (WMs). Although Wigner molecularization has been confirmed by real-space imaging and coherent spectroscopy, the open system dynamics of the strongly-correlated states with the environment are not yet well understood. Here, we demonstrate efficient control of spin transfer between an artificial three-electron WM and the nuclear environment in a GaAs double QD. A Landau-Zener sweep-based polarization sequence and low-lying anti-crossings of spin multiplet states enabled by Wigner molecularization are utilized. An efficient polarization rate of 2.58 $h \cdotp kHz \cdotp (g^* \cdotp μ_B)^{-1}$ per electron spin flip and, consequently, programmable nuclear polarization by controlled single-electron tunneling are achieved. Combined with coherent control of spin states, we achieve control of magnitude, polarity, and site dependence of the nuclear field. It is demonstrated that the same level of control cannot be achieved in the non-interacting regime. Thus, we confirm the multiplet spin structure of a WM, paving the way for active control of newly emerging correlated electron states for application in mesoscopic environment engineering.

cond-mat.mes-hall↗

arXiv2022

Towards Continual Knowledge Learning of Language Models

Large Language Models (LMs) are known to encode world knowledge in their parameters as they pretrain on a vast amount of web corpus, which is often utilized for performing knowledge-dependent downstream tasks such as question answering, fact-checking, and open dialogue. In real-world scenarios, the world knowledge stored in the LMs can quickly become outdated as the world changes, but it is non-trivial to avoid catastrophic forgetting and reliably acquire new knowledge while preserving invariant knowledge. To push the community towards better maintenance of ever-changing LMs, we formulate a new continual learning (CL) problem called Continual Knowledge Learning (CKL). We construct a new benchmark and metric to quantify the retention of time-invariant world knowledge, the update of outdated knowledge, and the acquisition of new knowledge. We adopt applicable recent methods from literature to create several strong baselines. Through extensive experiments, we find that CKL exhibits unique challenges that are not addressed in previous CL setups, where parameter expansion is necessary to reliably retain and learn knowledge simultaneously. By highlighting the critical causes of knowledge forgetting, we show that CKL is a challenging and important problem that helps us better understand and train ever-changing LMs. The benchmark datasets, evaluation script, and baseline code to reproduce our results are available at https://github.com/joeljang/continual-knowledge-learning.

cs.CL↗

arXiv2021

Single-shot readout of a driven hybrid qubit in a GaAs double quantum dot

We report a single-shot-based projective readout of a semiconductor hybrid qubit formed by three electrons in a GaAs double quantum dot. Voltage-controlled adiabatic transitions between the qubit operations and readout conditions allow high-fidelity mapping of quantum states. We show that a large ratio both in relaxation time vs. tunneling time (~ 50) and singlet-triplet splitting vs. thermal energy (~ 20) allow energy-selective tunneling-based spin-to-charge conversion with readout visibility ~ 92.6%. Combined with ac driving, we demonstrate high visibility coherent Rabi and Ramsey oscillations of a hybrid qubit in GaAs. Further, we discuss the generality of the method for use in other materials, including silicon.

quant-ph↗

arXiv2020

Deep learning enhanced individual nuclear-spin detection

The detection of nuclear spins using individual electron spins has enabled new opportunities in quantum sensing and quantum information processing. Proof-of-principle experiments have demonstrated atomic-scale imaging of nuclear-spin samples and controlled multi-qubit registers. However, to image more complex samples and to realize larger-scale quantum processors, computerized methods that efficiently and automatically characterize spin systems are required. Here, we realize a deep learning model for automatic identification of nuclear spins using the electron spin of single nitrogen-vacancy (NV) centers in diamond as a sensor. Based on neural network algorithms, we develop noise recovery procedures and training sequences for highly non-linear spectra. We apply these methods to experimentally demonstrate fast identification of 31 nuclear spins around a single NV center and accurately determine the hyperfine parameters. Our methods can be extended to larger spin systems and are applicable to a wide range of electron-nuclear interaction strengths. These results enable efficient imaging of complex spin samples and automatic characterization of large spin-qubit registers.

quant-ph↗