arXiv subjects

Junyong Choi

Publications and source records attributed to Junyong Choi.

8 recordsLinked to original sources

arXiv2026

Grid-Preserving Knowledge Distillation: Transferring Convolutional Inductive Bias to Vision Transformers under Data Scarcity

Vision Transformers demonstrate remarkable global modeling capacity but often underperform in data-scarce regimes. Distilling convolutional inductive biases from a CNN teacher provides an effective remedy while leaving the deployed model unchanged. However, general-purpose feature distillation transfers little in this setting. In CNN-to-CNN distillation, pooling, flattening, and logit-space projections remove the spatial grid that encodes locality and translation equivariance. Unlike a convolutional student, a ViT cannot readily reconstruct this structure on its own. In this paper, we propose iBKD, a distillation framework that preserves the spatial grid throughout the entire transfer process. Its core module, the Inductive Bias Attention Module, aggregates features from all student layers onto the teacher's grid using learned weights. It then enhances structural cues through channel and deformable spatial attention and injects them via convolutional cross-attention operating directly between spatial grids rather than token sets. The module is used only during training, leaving the deployed model as an unmodified ViT with no inference overhead. Across seven Transformer backbones and six data-scarce benchmarks, iBKD consistently outperforms both locality-guidance methods and general knowledge distillation baselines, with its advantage increasing as the amount of training data decreases.

cs.CV↗

arXiv2026

Self-stabilization of microcombs

Optical frequency combs form phase-locked spectral lines arranged on an equidistant grid fully defined by two degrees of freedom, i.e., the repetition rate and frequency offset. Stabilizing these parameters to a common frequency reference results in a coherent frequency ruler, central for modern precision metrology. However, extending this level of stability to chip-scale microcombs remains an outstanding challenge. Here, we demonstrate a self-stabilizing mechanism based on self-injection locking of a selected comb line via an external feedback loop. This process establishes a second anchor point in addition to the pump, thereby constraining the comb's frequency noise dynamics. We show that, with an appropriate choice between pump frequency noise and feedback strength, collective fluctuations of the repetition rate are strongly suppressed. The result is a microcomb exhibiting ultralow phase noise and dramatically reduced timing jitter. In a 100 GHz silicon nitride soliton microcomb, we achieve an unprecedented combination of high-conversion efficiency, sub-Hertz intrinsic linewidth across the entire C band, and an integrated timing jitter of 1 fs. This approach enables chip-scale microcombs with remarkable noise performance and fs-level pulse stability, surpassing conventional noise limits and opening new avenues for precision metrology at the chip scale.

physics.optics↗

arXiv2026

Atomic-referenced Hz-linewidth lasers via fiber interferometric stabilization

Narrow-linewidth lasers with absolute frequency anchoring are essential for precision metrology, coherent sensing, and emerging quantum technologies beyond laboratory environments. Optical cavities and interferometers provide exceptional short-term spectral purity but lack intrinsic absolute frequency references. Atomic transitions, in contrast, provide stable frequency anchors but offer limited discrimination sensitivity. Recent hybrid approaches have demonstrated the combination of compact optical resonators with atomic references, yet achieving the Hz-level regime remains challenging. Here, we present a hybrid architecture that enables simultaneous realization of Hz-level linewidth and atomic-referenced frequency stability. An external-cavity diode laser is first stabilized to a fiber interferometer to achieve Hz-level spectral purity, while the interferometer is subsequently anchored to an 87Rb D2 transition via modulation transfer spectroscopy to suppress long-term drift and define the laser frequency relative to the atomic transition. This dual-stabilization scheme realizes a compact atomic-referenced laser with a 3.4-Hz linewidth (1-rad integrated-phase method), a minimum fractional frequency stability of 3.4x10-14 at 0.56 s, and 9x10-13 at 100 s. This architecture establishes a practical and scalable route toward compact and field-deployable atomic-referenced narrow-linewidth lasers for precision metrology and quantum technologies.

physics.optics↗

arXiv2025

PLOT: Pseudo-Labeling via Object Tracking for Monocular 3D Object Detection

Monocular 3D object detection is crucial for scalable perception across fields like autonomous driving, robotics, and surveillance. However, progress is hindered by limited 3D annotations and the inherent ambiguity of single-image geometry. Existing methods often rely on strong geometric assumptions or carefully curated datasets, which limit their applicability to real-world scenarios. In this paper, we present PLOT (Pseudo-Labeling via Object Tracking), a framework that generates 3D annotations from monocular videos without auxiliary sensors or model retraining. PLOT tracks object and background trajectories to estimate camera motion and perform object association in pose-unknown settings. These trajectories provide point correspondences that align frame-wise pseudo-LiDARs, which are then fused via simple optimization into a unified object shape robust to occlusion and viewpoint shifts. Recognizing temporal coherence as a fundamental requirement for reliable shape fusion and video perception, we design a global object memory that preserves consistent object identities across frames. PLOT achieves robust annotation quality and strong generalization on both M3OD video benchmarks and in-the-wild videos, proving its effectiveness across diverse and unconstrained domains. Project page: https://plot-eccv.github.io.

cs.CV↗

arXiv2025

Optical frequency comb integration in radio telescopes: advancing signal generation and phase calibration

Very long baseline interferometry (VLBI) enables high-angular-resolution observations in astronomy and geodesy by synthesizing a virtual telescope with baselines spanning hundreds to thousands of kilometres. Achieving high instrumental phase stability in VLBI relies on the generation of high-quality, atomic-referenced RF local oscillator (LO) and RF-comb signals for the effective downconversion of celestial RF signals and precise phase calibration, respectively. As observing frequencies move into higher ranges with wider bandwidth, conventional electronic methods face significant challenges in maintaining the quality of these signals. Here, we demonstrate that an optical frequency comb (OFC) can be used as a versatile tool to generate and distribute low-noise and atomic-referenced RF-comb and RF-LO signals in the VLBI telescope. Hydrogen maser-stabilized optical pulses are transmitted over a timing-stabilized fibre link from the observatory building to the VLBI receiver system at the telescope, where photodetection converts them into the required RF-comb and RF-LO signals. In VLBI test observation, we successfully detected VLBI fringes and extracted the RF-combs characteristics in a format suitable for VLBI instrumental phase calibration. These results highlight the high potential of OFC-based technology for enhancing next-generation broadband VLBI measurements, advancing astrophysical research and facilitating intercontinental clock comparison.

astro-ph.IM↗

arXiv2023

ORC: Network Group-based Knowledge Distillation using Online Role Change

In knowledge distillation, since a single, omnipotent teacher network cannot solve all problems, multiple teacher-based knowledge distillations have been studied recently. However, sometimes their improvements are not as good as expected because some immature teachers may transfer the false knowledge to the student. In this paper, to overcome this limitation and take the efficacy of the multiple networks, we divide the multiple networks into teacher and student groups, respectively. That is, the student group is a set of immature networks that require learning the teacher's knowledge, while the teacher group consists of the selected networks that are capable of teaching successfully. We propose our online role change strategy where the top-ranked networks in the student group are able to promote to the teacher group at every iteration. After training the teacher group using the error samples of the student group to refine the teacher group's knowledge, we transfer the collaborative knowledge from the teacher group to the student group successfully. We verify the superiority of the proposed method on CIFAR-10, CIFAR-100, and ImageNet which achieves high performance. We further show the generality of our method with various backbone architectures such as ResNet, WRN, VGG, Mobilenet, and Shufflenet.

cs.LG↗

arXiv2023

itKD: Interchange Transfer-based Knowledge Distillation for 3D Object Detection

Point-cloud based 3D object detectors recently have achieved remarkable progress. However, most studies are limited to the development of network architectures for improving only their accuracy without consideration of the computational efficiency. In this paper, we first propose an autoencoder-style framework comprising channel-wise compression and decompression via interchange transfer-based knowledge distillation. To learn the map-view feature of a teacher network, the features from teacher and student networks are independently passed through the shared autoencoder; here, we use a compressed representation loss that binds the channel-wised compression knowledge from both student and teacher networks as a kind of regularization. The decompressed features are transferred in opposite directions to reduce the gap in the interchange reconstructions. Lastly, we present an head attention loss to match the 3D object detection information drawn by the multi-head self-attention mechanism. Through extensive experiments, we verify that our method can train the lightweight model that is well-aligned with the 3D point cloud detection task and we demonstrate its superiority using the well-known public datasets; e.g., Waymo and nuScenes.

cs.CV↗

arXiv2021

Densely Guided Knowledge Distillation using Multiple Teacher Assistants

With the success of deep neural networks, knowledge distillation which guides the learning of a small student network from a large teacher network is being actively studied for model compression and transfer learning. However, few studies have been performed to resolve the poor learning issue of the student network when the student and teacher model sizes significantly differ. In this paper, we propose a densely guided knowledge distillation using multiple teacher assistants that gradually decreases the model size to efficiently bridge the large gap between the teacher and student networks. To stimulate more efficient learning of the student network, we guide each teacher assistant to every other smaller teacher assistants iteratively. Specifically, when teaching a smaller teacher assistant at the next step, the existing larger teacher assistants from the previous step are used as well as the teacher network. Moreover, we design stochastic teaching where, for each mini-batch, a teacher or teacher assistants are randomly dropped. This acts as a regularizer to improve the efficiency of teaching of the student network. Thus, the student can always learn salient distilled knowledge from the multiple sources. We verified the effectiveness of the proposed method for a classification task using CIFAR-10, CIFAR-100, and ImageNet. We also achieved significant performance improvements with various backbone architectures such as ResNet, WideResNet, and VGG.

cs.CV↗