arXiv subjects

Haeyong Kang

Publications and source records attributed to Haeyong Kang.

18 recordsLinked to original sources

arXiv2026

Plasmonic polaron in self-intercalated 1T-TiS2

Electron-boson coupling is central to a comprehensive understanding of the diverse physical phenomena emerging from many-body interactions. Yet less attention has been paid to how plasmons, collective bosonic modes of electron density oscillation, interact with conduction electrons and how external parameters can tune this interaction. Here, we present a clear display of composite quasiparticles stemming from electron-plasmon coupling, known as the plasmonic polaron, in self-intercalated 1T-TiS2, by using angle-resolved photoemission spectroscopy (ARPES), high-resolution electron energy loss spectroscopy (HR-EELS) and first-principles calculations. The single particle spectral function exhibits a distinctive plasmon-loss satellite with the same characteristic energy scale determined by HR-EELS measurements. The bosonic energy scale of plasmonic polaron is tunable by controlling charge carrier density and temperature, distinguishing itself from conventional polarons arising from electron-phonon interactions. Furthermore, we find that the dielectric screening strongly affects the formation of the plasmonic polaron states. Our findings provide direct spectroscopic evidence of plasmonic polarons and establish self-intercalated layered materials as a promising platform for studying, controlling, and harnessing plasmonic interactions in quantum materials.

cond-mat.mtrl-sci↗

arXiv2025

Mixtures of SubExperts for Large Language Continual Learning

Enabling lifelong learning in LLMs demands resolving the stability-plasticity dilemma (i.e., models must incorporate new knowledge without overwriting prior representations) while maintaining scalability under bounded parameter growth. Existing PEFT methods fail to satisfy this triad; shared-parameter approaches suffer from catastrophic interference, while task-isolated expansions preclude knowledge transfer and scale linearly. We propose Mixtures of SubExperts (MoSEs), a modular and sparse framework that factorizes model capacity into reusable, compositional primitives. MoSEs augment transformer layers with lightweight SubExperts and a learned sub-routing function that dynamically selects and composes a sparse subset of modules conditioned on task inputs. This induces a structured decomposition of the parameter space where knowledge is localized yet accessible, mitigating interference while preserving reuse. Specifically, MoSEs balance the dilemma via three pillars: (i) stability by isolating knowledge within sparsely activated modules, (ii) plasticity through routing-driven recombination and selective expansion, and (iii) scalability via sublinear growth in effective capacity. Notably, the routing mechanism enables compositional generalization, allowing new tasks to be represented as combinations of previously acquired sub-functions. We empirically validate MoSEs on TRACE and SuperNI, showing reduced forgetting, improved forward transfer, and better parameter efficiency over strong PEFT baselines. MoSEs establish a new Pareto frontier, achieving state-of-the-art performance while maintaining strict parameter budgets. Our results suggest that modular sparsity and compositional routing are key inductive biases for building foundation models that continually learn without saturation.

cs.LG↗

arXiv2024

Continual Learning: Forget-free Winning Subnetworks for Video Representations

Inspired by the Lottery Ticket Hypothesis (LTH), which highlights the existence of efficient subnetworks within larger, dense networks, a high-performing Winning Subnetwork (WSN) in terms of task performance under appropriate sparsity conditions is considered for various continual learning tasks. It leverages pre-existing weights from dense networks to achieve efficient learning in Task Incremental Learning (TIL) and Task-agnostic Incremental Learning (TaIL) scenarios. In Few-Shot Class Incremental Learning (FSCIL), a variation of WSN referred to as the Soft subnetwork (SoftNet) is designed to prevent overfitting when the data samples are scarce. Furthermore, the sparse reuse of WSN weights is considered for Video Incremental Learning (VIL). The use of Fourier Subneural Operator (FSO) within WSN is considered. It enables compact encoding of videos and identifies reusable subnetworks across varying bandwidths. We have integrated FSO into different architectural frameworks for continual learning, including VIL, TIL, and FSCIL. Our comprehensive experiments demonstrate FSO's effectiveness, significantly improving task performance at various convolutional representational levels. Specifically, FSO enhances higher-layer performance in TIL and FSCIL and lower-layer performance in VIL.

cs.CV↗

arXiv2024

Soft-TransFormers for Continual Learning

Inspired by the Well-initialized Lottery Ticket Hypothesis (WLTH), we introduce Soft-TransFormers (Soft-TF), a continual learning framework that adapts a frozen pre-trained Transformer through task-specific soft subnetworks: real-valued multiplicative masks over the query, key, value, and output projections of selected self-attention layers. The masks are initialized at one, so optimization starts exactly at the pre-trained solution, and mask-space gradient descent is intrinsically biased toward modulating the backbone's dominant pathways; we prove that, under standard convex-Lipschitz assumptions, both the convergence rate and the parameter drift of mask-only fine-tuning are controlled by the distance from the pre-trained weights to a task-optimal configuration. This bounded drift yields two properties. Since the backbone and per-task masks are never overwritten, forgetting is structurally eliminated. And since every task subnetwork stays near the shared pre-trained solution, a wrong mask still evaluates a near-generalist function, so task-inference errors are largely harmless and class-incremental accuracy is decoupled from task-inference reliability. As a plug-in, Soft-TF couples with L2P, DualPrompt, HiDe-Prompt, and NoRGa, selecting masks by task-key matching, an entropy-gradient criterion, or a learned task-identity classifier. Across class-incremental learning benchmarks -- Split-CIFAR100, Split-ImageNet-R, CUB-200, and 5-Datasets -- Soft-TF consistently outperforms prompt-based, adapter-based, and LoRA-style baselines at comparable trainable-parameter budgets, while keeping inference cost identical to the unmodified backbone.

cs.LG↗

arXiv2024

Band-selective simulation of photoelectron intensity and converging Berry phase in trilayer graphene

Berry phase is one of the key elements to understand quantum-mechanical phenomena such as the Aharonov-Bohm effect and the unconventional Hall effect in graphene. The Berry phase in monolayer and bilayer graphene has been manifested by the anisotropic distribution of photoelectron intensity along a closed loop in the momentum space as well as its rotation by a characteristic angle upon rotating light polarization. Here we report the band-selective simulation of photoelectron intensity of trilayer graphene to understand its Berry phase within the tight-binding formalism. ABC- and ABA-stacked trilayer graphene show characteristic rotational angles of photoelectron intensity distribution, as predicted from their well-known Berry phases. Surprisingly, however, in ABA-stacked trilayer graphene, the rotational angle changes upon approaching toward the band touching point between the conduction and valence bands, which suggest that Berry phase changes as a function of binding energy. The binding energy-dependent Berry phase is attributed to the enhanced hybridization of the two electron bands of ABA-stacked trilayer graphene that converge at the band touching point, resulting in the converging Berry phase. These findings will provide an efficient way of tuning Berry phase and hence exotic phenomena stemming from the Berry phase.

cond-mat.mtrl-sci↗

arXiv2024

On the Perturbed States for Transformed Input-robust Reinforcement Learning

Reinforcement Learning (RL) agents demonstrating proficiency in a training environment exhibit vulnerability to adversarial perturbations in input observations during deployment. This underscores the importance of building a robust agent before its real-world deployment. To alleviate the challenging point, prior works focus on developing robust training-based procedures, encompassing efforts to fortify the deep neural network component's robustness or subject the agent to adversarial training against potent attacks. In this work, we propose a novel method referred to as Transformed Input-robust RL (TIRL), which explores another avenue to mitigate the impact of adversaries by employing input transformation-based defenses. Specifically, we introduce two principles for applying transformation-based defenses in learning robust RL agents: (1) autoencoder-styled denoising to reconstruct the original state and (2) bounded transformations (bit-depth reduction and vector quantization (VQ)) to achieve close transformed inputs. The transformations are applied to the state before feeding it into the policy network. Extensive experiments on multiple MuJoCo environments demonstrate that input transformation-based defenses, i.e., VQ, defend against several adversaries in the state observations. The official code is available at https://github.com/tunglm2203/tirl

cs.LG↗

arXiv2024

Progressive Fourier Neural Representation for Sequential Video Compilation

Neural Implicit Representation (NIR) has recently gained significant attention due to its remarkable ability to encode complex and high-dimensional data into representation space and easily reconstruct it through a trainable mapping function. However, NIR methods assume a one-to-one mapping between the target data and representation models regardless of data relevancy or similarity. This results in poor generalization over multiple complex data and limits their efficiency and scalability. Motivated by continual learning, this work investigates how to accumulate and transfer neural implicit representations for multiple complex video data over sequential encoding sessions. To overcome the limitation of NIR, we propose a novel method, Progressive Fourier Neural Representation (PFNR), that aims to find an adaptive and compact sub-module in Fourier space to encode videos in each training session. This sparsified neural encoding allows the neural network to hold free weights, enabling an improved adaptation for future videos. In addition, when learning a representation for a new video, PFNR transfers the representation of previous videos with frozen weights. This design allows the model to continuously accumulate high-quality neural representations for multiple videos while ensuring lossless decoding that perfectly preserves the learned representations for previous videos. We validate our PFNR method on the UVG8/17 and DAVIS50 video sequence benchmarks and achieve impressive performance gains over strong continual learning baselines. The PFNR code is available at https://github.com/ihaeyong/PFNR.git.

cs.CV↗

arXiv2023

Forget-free Continual Learning with Soft-Winning SubNetworks

Inspired by Regularized Lottery Ticket Hypothesis (RLTH), which states that competitive smooth (non-binary) subnetworks exist within a dense network in continual learning tasks, we investigate two proposed architecture-based continual learning methods which sequentially learn and select adaptive binary- (WSN) and non-binary Soft-Subnetworks (SoftNet) for each task. WSN and SoftNet jointly learn the regularized model weights and task-adaptive non-binary masks of subnetworks associated with each task whilst attempting to select a small set of weights to be activated (winning ticket) by reusing weights of the prior subnetworks. Our proposed WSN and SoftNet are inherently immune to catastrophic forgetting as each selected subnetwork model does not infringe upon other subnetworks in Task Incremental Learning (TIL). In TIL, binary masks spawned per winning ticket are encoded into one N-bit binary digit mask, then compressed using Huffman coding for a sub-linear increase in network capacity to the number of tasks. Surprisingly, in the inference step, SoftNet generated by injecting small noises to the backgrounds of acquired WSN (holding the foregrounds of WSN) provides excellent forward transfer power for future tasks in TIL. SoftNet shows its effectiveness over WSN in regularizing parameters to tackle the overfitting, to a few examples in Few-shot Class Incremental Learning (FSCIL).

cs.LG↗

arXiv2023

Maximum margin learning of t-SPNs for cell classification with filtered input

An algorithm based on a deep probabilistic architecture referred to as a tree-structured sum-product network (t-SPN) is considered for cell classification. The t-SPN is constructed such that the unnormalized probability is represented as conditional probabilities of a subset of most similar cell classes. The constructed t-SPN architecture is learned by maximizing the margin, which is the difference in the conditional probability between the true and the most competitive false label. To enhance the generalization ability of the architecture, L2-regularization (REG) is considered along with the maximum margin (MM) criterion in the learning process. To highlight cell features, this paper investigates the effectiveness of two generic high-pass filters: ideal high-pass filtering and the Laplacian of Gaussian (LOG) filtering. On both HEp-2 and Feulgen benchmark datasets, the t-SPN architecture learned based on the max-margin criterion with regularization produced the highest accuracy rate compared to other state-of-the-art algorithms that include convolutional neural network (CNN) based algorithms. The ideal high-pass filter was more effective on the HEp-2 dataset, which is based on immunofluorescence staining, while the LOG was more effective on the Feulgen dataset, which is based on Feulgen staining.

cs.LG↗

arXiv2023

Skew Class-balanced Re-weighting for Unbiased Scene Graph Generation

An unbiased scene graph generation (SGG) algorithm referred to as Skew Class-balanced Re-weighting (SCR) is proposed for considering the unbiased predicate prediction caused by the long-tailed distribution. The prior works focus mainly on alleviating the deteriorating performances of the minority predicate predictions, showing drastic dropping recall scores, i.e., losing the majority predicate performances. It has not yet correctly analyzed the trade-off between majority and minority predicate performances in the limited SGG datasets. In this paper, to alleviate the issue, the Skew Class-balanced Re-weighting (SCR) loss function is considered for the unbiased SGG models. Leveraged by the skewness of biased predicate predictions, the SCR estimates the target predicate weight coefficient and then re-weights more to the biased predicates for better trading-off between the majority predicates and the minority ones. Extensive experiments conducted on the standard Visual Genome dataset and Open Image V4 \& V6 show the performances and generality of the SCR with the traditional SGG models.

cs.LG↗

arXiv2023

On the Soft-Subnetwork for Few-shot Class Incremental Learning

Inspired by Regularized Lottery Ticket Hypothesis (RLTH), which hypothesizes that there exist smooth (non-binary) subnetworks within a dense network that achieve the competitive performance of the dense network, we propose a few-shot class incremental learning (FSCIL) method referred to as \emph{Soft-SubNetworks (SoftNet)}. Our objective is to learn a sequence of sessions incrementally, where each session only includes a few training instances per class while preserving the knowledge of the previously learned ones. SoftNet jointly learns the model weights and adaptive non-binary soft masks at a base training session in which each mask consists of the major and minor subnetwork; the former aims to minimize catastrophic forgetting during training, and the latter aims to avoid overfitting to a few samples in each new training session. We provide comprehensive empirical validations demonstrating that our SoftNet effectively tackles the few-shot incremental learning problem by surpassing the performance of state-of-the-art baselines over benchmark datasets.

cs.LG↗

arXiv2022

Learning Imbalanced Datasets with Maximum Margin Loss

A learning algorithm referred to as Maximum Margin (MM) is proposed for considering the class-imbalance data learning issue: the trained model tends to predict the majority of classes rather than the minority ones. That is, underfitting for minority classes seems to be one of the challenges of generalization. For a good generalization of the minority classes, we design a new Maximum Margin (MM) loss function, motivated by minimizing a margin-based generalization bound through the shifting decision bound. The theoretically-principled label-distribution-aware margin (LDAM) loss was successfully applied with prior strategies such as re-weighting or re-sampling along with the effective training schedule. However, they did not investigate the maximum margin loss function yet. In this study, we investigate the performances of two types of hard maximum margin-based decision boundary shift with LDAM's training schedule on artificially imbalanced CIFAR-10/100 for fair comparisons and effectiveness.

cs.LG↗

arXiv2021

Semantic Grouping Network for Video Captioning

This paper considers a video caption generating network referred to as Semantic Grouping Network (SGN) that attempts (1) to group video frames with discriminating word phrases of partially decoded caption and then (2) to decode those semantically aligned groups in predicting the next word. As consecutive frames are not likely to provide unique information, prior methods have focused on discarding or merging repetitive information based only on the input video. The SGN learns an algorithm to capture the most discriminating word phrases of the partially decoded caption and a mapping that associates each phrase to the relevant video frames - establishing this mapping allows semantically related frames to be clustered, which reduces redundancy. In contrast to the prior methods, the continuous feedback from decoded words enables the SGN to dynamically update the video representation that adapts to the partially decoded caption. Furthermore, a contrastive attention loss is proposed to facilitate accurate alignment between a word phrase and video frames without manual annotations. The SGN achieves state-of-the-art performances by outperforming runner-up methods by a margin of 2.1%p and 2.4%p in a CIDEr-D score on MSVD and MSR-VTT datasets, respectively. Extensive experiments demonstrate the effectiveness and interpretability of the SGN.

cs.CV↗

arXiv2020

SCNet: Training Inference Sample Consistency for Instance Segmentation

Cascaded architectures have brought significant performance improvement in object detection and instance segmentation. However, there are lingering issues regarding the disparity in the Intersection-over-Union (IoU) distribution of the samples between training and inference. This disparity can potentially exacerbate detection accuracy. This paper proposes an architecture referred to as Sample Consistency Network (SCNet) to ensure that the IoU distribution of the samples at training time is close to that at inference time. Furthermore, SCNet incorporates feature relay and utilizes global contextual information to further reinforce the reciprocal relationships among classifying, detecting, and segmenting sub-tasks. Extensive experiments on the standard COCO dataset reveal the effectiveness of the proposed method over multiple evaluation metrics, including box AP, mask AP, and inference speed. In particular, while running 38\% faster, the proposed SCNet improves the AP of the box and mask predictions by respectively 1.3 and 2.3 points compared to the strong Cascade Mask R-CNN baseline. Code is available at \url{https://github.com/thangvubk/SCNet}.

cs.CV↗

arXiv2017

Quantum Conductance Probing of Oxygen Vacancies in SrTiO3 Epitaxial Thin Film Using Graphene

The quantum Hall conductance in monolayer graphene on an epitaxial SrTiO3 (STO) thin film is studied to understand the role of oxygen vacancies in determining the dielectric properties of STO. As the gate voltage sweep range is gradually increased in our device, we observe systematic generation and annihilation of oxygen vacancies evidenced from the hysteretic conductance behavior in graphene. Furthermore, based on the experimentally observed linear scaling relation between the effective capacitance and the voltage sweep range, a simple model is constructed to manifest the relationship among the dielectric properties of STO with oxygen vacancies. The inherent quantum Hall conductance in graphene can be considered as a sensitive, robust, and non-invasive probe for understanding the electronic and ionic phenomena in complex transition metal oxides without impairing the oxide layer underneath.

cond-mat.mtrl-sci↗

arXiv2016

NMR Evidences for the coupling between conduction electrons and molecular degrees of freedom in the exotic member of the Bechgaard salt, (TMTSF)2FSO3

We performed Se and F-NMR measurements on single crystals of (TMTSF)2FSO3 to characterize the electronic structures of different phases in the Temperature-Pressure phase diagram, determined by precise transport measurements [Jo et al., Phys. Rev. B67, 014516 (2003)]. We claim that such varieties of electronic states in the refined phase diagram are caused by strong couplings of the conduction electrons with FSO3 anions, especially with the permanent electric dipoles on the anions. We suggest that as temperature decreases, the FSO3 anions form orientational ordering through two steps; first only the tetrahedrons form an orientational order leaving the orientations of the electronic dipoles in random (transition I); then the dipoles form a perfect orientational order at a lower temperature (transition II). In the intermediate temperature range between transitions I and II, we found an appreciable enhancement of homogeneous and inhomogeneous widths of 77Se-NMR spectrum. From the analysis of the angular dependence of the linewidth, we attributed these anomalies to the intramolecular charge disproportionation or imbalance and its slow dynamics caused by the coupling with the permanent electric dipole of anion. Results of 19F-NMR relaxation and lineshape measurements support this picture very well. Electronic structures at higher pressures up to 1.25 GPa are discussed on the basis of the results of the 77Se and 19F-NMR measurements.

cond-mat.str-el↗

arXiv2016

Voltage Scaling of Graphene Device on SrTiO3 Epitaxial Thin Film

Electrical transport in monolayer graphene on SrTiO3 (STO) thin film is examined in order to promote gate-voltage scaling using a high-k dielectric material. The atomically flat surface of thin STO layer epitaxially grown on Nb-doped STO single-crystal substrate offers good adhesion between the high-k film and graphene, resulting in nonhysteretic conductance as a function of gate voltage at all temperatures down to 2 K. The two-terminal conductance quantization under magnetic fields corresponding to quantum Hall states survives up to 200 K at a magnetic field of 14 T. In addition, the substantial shift of charge neutrality point in graphene seems to correlate with the temperature-dependent dielectric constant of the STO thin film, and its effective dielectric properties could be deduced from the universality of quantum phenomena in graphene. Our experimental data prove that the operating voltage reduction can be successfully realized due to the underlying high-k STO thin film, without any noticeable degradation of graphene device performance.

cond-mat.mtrl-sci↗

arXiv2004

Hot Spots and Transition from d-Wave to Another Pairing Symmetry in the Electron-Doped Cuprate Superconductors

We present a simple theoretical explanation for a transition from d-wave to another superconducting pairing observed in the electron-doped cuprates. The d_{x^2-y^2} pairing potential Delta, which has the maximal magnitude and opposite signs at the hot spots on the Fermi surface, becomes suppressed with the increase of electron doping, because the hot spots approach the Brillouin zone diagonals, where Delta vanishes. Then, the d_{x^2-y^2} pairing is replaced by either singlet s-wave or triplet p-wave pairing. We argue in favor of the latter and discuss experiments to uncover it.

cond-mat.supr-con↗