arXiv subjects

Dongkyu Lee

Publications and source records attributed to Dongkyu Lee.

At least 37 records · Page 2Linked to original sources

arXiv2024

Moiré flat bands and antiferroelectric domains in lattice relaxed twisted bilayer hexagonal boron nitride under perpendicular electric fields

Local interlayer charge polarization of twisted bilayer hexagonal boron nitride (t2BN) is calculated and parametrized as a function of twist angle and perpendicular electric fields through tight-binding calculations on lattice relaxed geometries Lattice relaxations tend to increase the bandwidth of the nearly flat bands, where widths smaller than 1 meV are expected for angle less than 1.08 degree for parallel BN/BN alignment, and for angle less than 1.5 degree for the antiparallel BN/NB alignment. Local interlayer charge polarization maxima of 2.6 pC/m corresponding are expected at the AB and BA stacking sites of BN/BN aligned t2BN in the long moire period limit for angle less than 1 degree, and evolves non-monotonically with a maximum of 3.5 pC/m at angle equal to 1.6 degree before reaching 2 pC/m for angle equal to 6 degree. The electrostatic potential maxima due to the t2BN are overall enhanced by 20 percentage with respect to the rigid system assuming potential modulation depths of up to 300 mV near its surface. In BN/BN aligned bilayers the relative areas of the AB or BA local stacking regions can be expanded or reduced through a vertical electric field depending on its sign.

cond-mat.mes-hall↗

arXiv2024

Optimal Qubit Mapping Search for Encoding Classical Data into Matrix Product State Representation with Minimal Loss

Matrix product state (MPS) offers a framework for encoding classical data into quantum states, enabling the efficient utilization of quantum resources for data representation and processing. This research paper investigates techniques to enhance the efficiency and accuracy of MPS representations specifically designed for encoding classical data. Based on the observations that MPS truncation error depends on the pattern of the classical data, we devised an algorithm that finds optimal qubit mapping for given classical data, thereby improving the efficiency and fidelity of the MPS representation. Furthermore, we evaluate the impact of the optimized MPS in the context of quantum classifiers, demonstrating their enhanced performance compared to the conventional mapping. This improvement confirms the efficacy of the proposed techniques for encoding classical data into quantum states. MPS representation combined with optimal qubit mapping can pave a new way for more efficient and accurate quantum data representation and processing.

quant-ph↗

arXiv2024

MATTER: Memory-Augmented Transformer Using Heterogeneous Knowledge Sources

Leveraging external knowledge is crucial for achieving high performance in knowledge-intensive tasks, such as question answering. The retrieve-and-read approach is widely adopted for integrating external knowledge into a language model. However, this approach suffers from increased computational cost and latency due to the long context length, which grows proportionally with the number of retrieved knowledge. Furthermore, existing retrieval-augmented models typically retrieve information from a single type of knowledge source, limiting their scalability to diverse knowledge sources with varying structures. In this work, we introduce an efficient memory-augmented transformer called MATTER, designed to retrieve relevant knowledge from multiple heterogeneous knowledge sources. Specifically, our model retrieves and reads from both unstructured sources (paragraphs) and semi-structured sources (QA pairs) in the form of fixed-length neural memories. We demonstrate that our model outperforms existing efficient retrieval-augmented models on popular QA benchmarks in terms of both accuracy and speed. Furthermore, MATTER achieves competitive results compared to conventional read-and-retrieve models while having 100x throughput during inference.

cs.CL↗

arXiv2022

Semi-Supervised Lifelong Language Learning

Lifelong learning aims to accumulate knowledge and alleviate catastrophic forgetting when learning tasks sequentially. However, existing lifelong language learning methods only focus on the supervised learning setting. Unlabeled data, which can be easily accessed in real-world scenarios, are underexplored. In this paper, we explore a novel setting, semi-supervised lifelong language learning (SSLL), where a model learns sequentially arriving language tasks with both labeled and unlabeled data. We propose an unlabeled data enhanced lifelong learner to explore SSLL. Specially, we dedicate task-specific modules to alleviate catastrophic forgetting and design two modules to exploit unlabeled data: (1) a virtual supervision enhanced task solver is constructed on a teacher-student framework to mine the underlying knowledge from unlabeled data; and (2) a backward augmented learner is built to encourage knowledge transfer from newly arrived unlabeled data to previous tasks. Experimental results on various language tasks demonstrate our model's effectiveness and superiority over competitive baselines under the new setting SSLL.

cs.CL↗

arXiv2022

Spin-orbit coupling-enhanced valley ordering of malleable bands in twisted bilayer graphene on WSe2

New phases of matter can be stabilized by a combination of diverging electronic density of states, strong interactions, and spin-orbit coupling. Recent experiments in magic-angle twisted bilayer graphene (TBG) have uncovered a wealth of novel phases as a result of interaction-driven spin-valley flavour polarization. In this work, we explore correlated phases appearing due to the combined effect of spin-orbit coupling-enhanced valley polarization and large density of states below half filling ($ν\lesssim 2$) of the moiré band in a TBG coupled to tungsten diselenide. We observe anomalous Hall effect, accompanied by a series of Lifshitz transitions, that are highly tunable with carrier density and magnetic field. Strikingly, the magnetization shows an abrupt sign change in the vicinity of half-filling, confirming its orbital nature. The coercive fields reported are about an order of magnitude higher than previous studies in graphene-based moiré systems, presumably aided by a Stoner instability favoured by the van Hove singularities in the malleable bands. While the Hall resistance is not quantized at zero magnetic fields, indicative of a ground state with partial valley polarization, perfect quantization and complete valley polarization are observed at finite fields. Our findings illustrate that singularities in the flat bands in the presence of spin-orbit coupling can stabilize ordered phases even at non-integer moiré band fillings.

cond-mat.mes-hall↗

arXiv2022

Hard Gate Knowledge Distillation -- Leverage Calibration for Robust and Reliable Language Model

In knowledge distillation, a student model is trained with supervisions from both knowledge from a teacher and observations drawn from a training data distribution. Knowledge of a teacher is considered a subject that holds inter-class relations which send a meaningful supervision to a student; hence, much effort has been put to find such knowledge to be distilled. In this paper, we explore a question that has been given little attention: "when to distill such knowledge." The question is answered in our work with the concept of model calibration; we view a teacher model not only as a source of knowledge but also as a gauge to detect miscalibration of a student. This simple and yet novel view leads to a hard gate knowledge distillation scheme that switches between learning from a teacher model and training data. We verify the gating mechanism in the context of natural language generation at both the token-level and the sentence-level. Empirical comparisons with strong baselines show that hard gate knowledge distillation not only improves model generalization, but also significantly lowers model calibration error.

cs.CL↗

arXiv2022

Adaptive Label Smoothing with Self-Knowledge in Natural Language Generation

Overconfidence has been shown to impair generalization and calibration of a neural network. Previous studies remedy this issue by adding a regularization term to a loss function, preventing a model from making a peaked distribution. Label smoothing smoothes target labels with a pre-defined prior label distribution; as a result, a model is learned to maximize the likelihood of predicting the soft label. Nonetheless, the amount of smoothing is the same in all samples and remains fixed in training. In other words, label smoothing does not reflect the change in probability distribution mapped by a model over the course of training. To address this issue, we propose a regularization scheme that brings dynamic nature into the smoothing parameter by taking model probability distribution into account, thereby varying the parameter per instance. A model in training self-regulates the extent of smoothing on the fly during forward propagation. Furthermore, inspired by recent work in bridging label smoothing and knowledge distillation, our work utilizes self-knowledge as a prior label distribution in softening target labels, and presents theoretical support for the regularization effect by knowledge distillation and the dynamic smoothing parameter. Our regularizer is validated comprehensively, and the result illustrates marked improvements in model generalization and calibration, enhancing robustness and trustworthiness of a model.

cs.LG↗

arXiv2022

Improving Meta-learning for Low-resource Text Classification and Generation via Memory Imitation

Building models of natural language processing (NLP) is challenging in low-resource scenarios where only limited data are available. Optimization-based meta-learning algorithms achieve promising results in low-resource scenarios by adapting a well-generalized model initialization to handle new tasks. Nonetheless, these approaches suffer from the memorization overfitting issue, where the model tends to memorize the meta-training tasks while ignoring support sets when adapting to new tasks. To address this issue, we propose a memory imitation meta-learning (MemIML) method that enhances the model's reliance on support sets for task adaptation. Specifically, we introduce a task-specific memory module to store support set information and construct an imitation module to force query sets to imitate the behaviors of some representative support-set samples stored in the memory. A theoretical analysis is provided to prove the effectiveness of our method, and empirical results also demonstrate that our method outperforms competitive baselines on both text classification and generation tasks.

cs.CL↗

arXiv2021

Emergence of broken-symmetry states at half-integer band fillings in twisted bilayer graphene

The dominance of Coulomb interactions over kinetic energy of electrons in narrow, non-trivial moiré bands of magic-angle twisted bilayer graphene (TBG) gives rise to a variety of correlated phases such as correlated insulators, superconductivity, orbital ferromagnetism, Chern insulators and nematicity. Most of these phases occur at or near an integer number of carriers per moiré unit cell. Experimental demonstration of ordered states at fractional moiré band-fillings at zero applied magnetic field $B$, is a challenging pursuit. In this letter, we report the observation of states near half-integer band-fillings of $ν\approx 0.5$ and $\pm3.5$ at $B\approx 0$ in TBG proximitized by tungsten diselenide (WSe$_2$) through magnetotransport and thermoelectricity measurements. A series of Lifshitz transitions due to the changes in the topology of the Fermi surface implies the evolution of van Hove singularities (VHSs) of the diverging density of states (DOS) at a discrete set of partial fillings of flat bands. Furthermore, at a band filling of $ν\approx-0.5$, a symmetry-broken Chern insulator emerges at high $B$, compatible with the band structure calculations within a translational symmetry-broken supercell with twice the area of the original TBG moiré cell. Our results are consistent with a spin/charge density wave ground state in TBG in the zero $B$-field limit.

cond-mat.mes-hall↗

arXiv2021

Enhancing Content Preservation in Text Style Transfer Using Reverse Attention and Conditional Layer Normalization

Text style transfer aims to alter the style (e.g., sentiment) of a sentence while preserving its content. A common approach is to map a given sentence to content representation that is free of style, and the content representation is fed to a decoder with a target style. Previous methods in filtering style completely remove tokens with style at the token level, which incurs the loss of content information. In this paper, we propose to enhance content preservation by implicitly removing the style information of each token with reverse attention, and thereby retain the content. Furthermore, we fuse content information when building the target style representation, making it dynamic with respect to the content. Our method creates not only style-independent content representation, but also content-dependent style representation in transferring style. Empirical results show that our method outperforms the state-of-the-art baselines by a large margin in terms of content preservation. In addition, it is also competitive in terms of style transfer accuracy and fluency.

cs.CL↗

arXiv2021

Learning from My Friends: Few-Shot Personalized Conversation Systems via Social Networks

Personalized conversation models (PCMs) generate responses according to speaker preferences. Existing personalized conversation tasks typically require models to extract speaker preferences from user descriptions or their conversation histories, which are scarce for newcomers and inactive users. In this paper, we propose a few-shot personalized conversation task with an auxiliary social network. The task requires models to generate personalized responses for a speaker given a few conversations from the speaker and a social network. Existing methods are mainly designed to incorporate descriptions or conversation histories. Those methods can hardly model speakers with so few conversations or connections between speakers. To better cater for newcomers with few resources, we propose a personalized conversation model (PCM) that learns to adapt to new speakers as well as enabling new speakers to learn from resource-rich speakers. Particularly, based on a meta-learning based PCM, we propose a task aggregator (TA) to collect other speakers' information from the social network. The TA provides prior knowledge of the new speaker in its meta-learning. Experimental results show our methods outperform all baselines in appropriateness, diversity, and consistency with speakers.

cs.CL↗

arXiv2021

Catalytic effect of plasma in lowering the reduction temperature of $Fe_{2}O_{3}$

Atmospheric pressure plasma (APP) generates highly reactive species that are useful for surface activations. We demonstrate a fast regeneration of iron oxides, that are popular catalysts in various industrial processes, using microwave-driven argon APP under ambient condition. The surface treatment of hematite powder by the APP with a small portion of hydrogen (0.5%) lowers the oxide's reduction temperature. A near-infrared laser is used for localized heating to control the surface temperature. Controlled experiments without plasma confirm the catalytic effect of the plasma. Raman, XRD, SEM, and XPS analyses show that the plasma treatment changed the chemical state of the hematite to that of magnetite without sintering.

physics.plasm-ph↗

arXiv2020

Restore from Restored: Single Image Denoising with Pseudo Clean Image

In this study, we propose a simple and effective fine-tuning algorithm called "restore-from-restored", which can greatly enhance the performance of fully pre-trained image denoising networks. Many supervised denoising approaches can produce satisfactory results using large external training datasets. However, these methods have limitations in using internal information available in a given test image. By contrast, recent self-supervised approaches can remove noise in the input image by utilizing information from the specific test input. However, such methods show relatively lower performance on known noise types such as Gaussian noise compared to supervised methods. Thus, to combine external and internal information, we fine-tune the fully pre-trained denoiser using pseudo training set at test time. By exploiting internal self-similar patches (i.e., patch-recurrence), the baseline network can be adapted to the given specific input image. We demonstrate that our method can be easily employed on top of the state-of-the-art denoising networks and further improve the performance on numerous denoising benchmark datasets including real noisy images.

cs.CV↗

arXiv2020

Response-Anticipated Memory for On-Demand Knowledge Integration in Response Generation

Neural conversation models are known to generate appropriate but non-informative responses in general. A scenario where informativeness can be significantly enhanced is Conversing by Reading (CbR), where conversations take place with respect to a given external document. In previous work, the external document is utilized by (1) creating a context-aware document memory that integrates information from the document and the conversational context, and then (2) generating responses referring to the memory. In this paper, we propose to create the document memory with some anticipated responses in mind. This is achieved using a teacher-student framework. The teacher is given the external document, the context, and the ground-truth response, and learns how to build a response-aware document memory from three sources of information. The student learns to construct a response-anticipated document memory from the first two sources, and the teacher's insight on memory creation. Empirical results show that our model outperforms the previous state-of-the-art for the CbR task.

cs.CL↗

arXiv2020

Multi-lane Detection Using Instance Segmentation and Attentive Voting

Autonomous driving is becoming one of the leading industrial research areas. Therefore many automobile companies are coming up with semi to fully autonomous driving solutions. Among these solutions, lane detection is one of the vital driver-assist features that play a crucial role in the decision-making process of the autonomous vehicle. A variety of solutions have been proposed to detect lanes on the road, which ranges from using hand-crafted features to the state-of-the-art end-to-end trainable deep learning architectures. Most of these architectures are trained in a traffic constrained environment. In this paper, we propose a novel solution to multi-lane detection, which outperforms state of the art methods in terms of both accuracy and speed. To achieve this, we also offer a dataset with a more intuitive labeling scheme as compared to other benchmark datasets. Using our approach, we are able to obtain a lane segmentation accuracy of 99.87% running at 54.53 fps (average).

cs.CV↗

arXiv2019

Exploiting symmetry-mismatch to control magnetism in a ferroelastic heterostructure

In the bulk, LaCoO3 (LCO) is a paramagnet, yet in tensile strained thin films at low temperature ferromagnetism (FM) is observed, and its origin remains unresolved. Polarized neutron reflectometry (PNR) is a powerful tool to determine the depth profiles of the structure and magnetization simultaneously and thus the evolution of the interfacial FM with strain can be accurately revealed. Here we quantitatively measured the distribution of atomic density and magnetization in LCO films by PNR and found that the LCO layers near the heterointerfaces exhibit a reduced magnetization but an enhanced atomic density, whereas the interior shows the opposite trend. We attribute the nonuniformity to the symmetry mismatch at the interface, which induces a structural distortion related to the ferroelasticity of LCO. This assertion is tested by systematic application of hydrostatic pressure during the PNR experiments. These results provide unique insights into mechanisms driving FM in strained LCO films while offering a tantalizing observation that tunable deformation of the CoO6 octahedra in combination with the ferroelastic order parameter.

cond-mat.mtrl-sci↗

arXiv2019

Nanoscale ferroelastic twins formed in strained LaCoO3 films

The coexistence and coupling of ferroelasticity and magnetic ordering in a single material offers a great opportunity to realize novel devices with multiple tuning knobs. Complex oxides are a particularly promising class of materials to find multiferroic interactions as they often possess rich phase diagrams and the interactions are very sensitive to external perturbations. Still, there are very few examples of these systems. Here we report the observation of twinning domains in ferroelastic LaCoO3 epitaxial thin films and their geometric control of structural symmetry that are intimately linked to the material electronic and magnetic states. A unidirectional structural modulation is achieved by selective choice of substrates possessing two-fold rotational symmetry. This modulation perturbs the crystal field splitting energy, leading to unexpected in plane anisotropy of orbital configuration and magnetization. These findings demonstrate the utilization of structural modulation to control multiferroic interactions and may enable a great potential for stimulation of exotic phenomena through artificial domain engineering.

cond-mat.mtrl-sci↗

arXiv2017

Stretching Epitaxial La0.6Sr0.4CoO3-δ for Fast Oxygen Reduction

The slow kinetics of the oxygen reduction reaction (ORR) is one of the key challenges in developing high performance energy devices, such as solid oxide fuel cells. Straining a film by growing on a lattice-mismatched substrate has been a conventional approach to enhance the ORR activity. However, due to the limited choice of electrolyte substrates to alter the degree of strain, a systematic study in various materials has been a challenge. Here, we explore the strain modulation of the ORR kinetics by growing epitaxial La0.6Sr0.4CoO3-δ (LSCO) films on yttria-stabilized zirconia substrates with the film thickness below and above the critical thickness for strain relaxation. Two orders of magnitude higher ORR kinetics is achieved in an ultra-thin film with ~0.8% tensile strain as compared to unstrained films. Time-of-flight secondary ion mass spectrometry depth profiling confirms that the Sr surface segregation is not responsible for the enhanced ORR in strained films. We attribute this enhancement of ORR kinetics to the increase in oxygen vacancy concentration in the tensile-strained LSCO film owing to the reduced activation barrier for oxygen surface exchange kinetics. Density functional theory calculations reveal an upshift of the oxygen 2p-band center relative to the Fermi level by tensile strain, indicating the origin of the enhanced ORR kinetics.

cond-mat.mtrl-sci↗