arXiv subjects

Yongjae Lee

Publications and source records attributed to Yongjae Lee.

68 records · Page 4Linked to original sources

arXiv2024

Ignore Me But Don't Replace Me: Utilizing Non-Linguistic Elements for Pretraining on the Cybersecurity Domain

Cybersecurity information is often technically complex and relayed through unstructured text, making automation of cyber threat intelligence highly challenging. For such text domains that involve high levels of expertise, pretraining on in-domain corpora has been a popular method for language models to obtain domain expertise. However, cybersecurity texts often contain non-linguistic elements (such as URLs and hash values) that could be unsuitable with the established pretraining methodologies. Previous work in other domains have removed or filtered such text as noise, but the effectiveness of these methods have not been investigated, especially in the cybersecurity domain. We propose different pretraining methodologies and evaluate their effectiveness through downstream tasks and probing tasks. Our proposed strategy (selective MLM and jointly training NLE token classification) outperforms the commonly taken approach of replacing non-linguistic elements (NLEs). We use our domain-customized methodology to train CyBERTuned, a cybersecurity domain language model that outperforms other cybersecurity PLMs on most tasks.

cs.CR↗

arXiv2024

Temporal Graph Networks for Graph Anomaly Detection in Financial Networks

This paper explores the utilization of Temporal Graph Networks (TGN) for financial anomaly detection, a pressing need in the era of fintech and digitized financial transactions. We present a comprehensive framework that leverages TGN, capable of capturing dynamic changes in edges within financial networks, for fraud detection. Our study compares TGN's performance against static Graph Neural Network (GNN) baselines, as well as cutting-edge hypergraph neural network baselines using DGraph dataset for a realistic financial context. Our results demonstrate that TGN significantly outperforms other models in terms of AUC metrics. This superior performance underlines TGN's potential as an effective tool for detecting financial fraud, showcasing its ability to adapt to the dynamic and complex nature of modern financial systems. We also experimented with various graph embedding modules within the TGN framework and compared the effectiveness of each module. In conclusion, we demonstrated that, even with variations within TGN, it is possible to achieve good performance in the anomaly detection task.

q-fin.ST↗

arXiv2024

Probing superconducting gap in CeH$_9$ under pressure

The recent discovery of superconductivity in hydrogen-rich compounds has garnered significant experimental and theoretical interest because of the record-setting critical temperatures. As the direct observation of the superconducting (SC) gap in these superhydrides is rare, the underlying mechanism behind its occurrence has yet to be settled down. Here, we report a successful synthesis of the $\textit{P6$_3$}$/$\textit{mmc}$ phase of CeH$_9$ that exhibits the SC transition with SC critical temperature of about 100 K at a pressure of about 100 GPa. The observation of the zero electrical resistance and the critical current demonstrates that the SC phase is realized in Ce-based superhydride. Quasiparticle scattering spectroscopy (QSS) reveals the Andreev reflection at zero bias voltage, a hallmark of superconductivity, in the differential conductance. The obtained SC gap-to-$\textit{T}$$_c$ ratio of 4.36 and temperature dependence of SC gap are consistent with the prediction from the Bardeen-Cooper-Schrieffer theory with a moderate coupling strength. The successful realization of QSS under Megabar conditions is expected to provide a desired route to the study of the mechanism of superconductivity as well as the establishment of the SC phase in superhydride high-$\textit{T}$$_c$ systems.

cond-mat.supr-con↗

arXiv2023

DRAINCLoG: Detecting Rogue Accounts with Illegally-obtained NFTs using Classifiers Learned on Graphs

As Non-Fungible Tokens (NFTs) continue to grow in popularity, NFT users have become targets of phishing attacks by cybercriminals, called \textit{NFT drainers}. Over the last year, \$100 million worth of NFTs were stolen by drainers, and their presence remains a serious threat to the NFT trading space. However, no work has yet comprehensively investigated the behaviors of drainers in the NFT ecosystem. In this paper, we present the first study on the trading behavior of NFT drainers and introduce the first dedicated NFT drainer detection system. We collect 127M NFT transaction data from the Ethereum blockchain and 1,135 drainer accounts from five sources for the year 2022. We find that drainers exhibit significantly different transactional and social contexts from those of regular users. With these insights, we design \textit{DRAINCLoG}, an automatic drainer detection system utilizing Graph Neural Networks. This system effectively captures the multifaceted web of interactions within the NFT space through two distinct graphs: the NFT-User graph for transaction contexts and the User graph for social contexts. Evaluations using real-world NFT transaction data underscore the robustness and precision of our model. Additionally, we analyze the security of \textit{DRAINCLoG} under a wide variety of evasion attacks.

cs.CR↗

arXiv2023

MF-NeRF: Memory Efficient NeRF with Mixed-Feature Hash Table

Neural radiance field (NeRF) has shown remarkable performance in generating photo-realistic novel views. Among recent NeRF related research, the approaches that involve the utilization of explicit structures like grids to manage features achieve exceptionally fast training by reducing the complexity of multilayer perceptron (MLP) networks. However, storing features in dense grids demands a substantial amount of memory space, resulting in a notable memory bottleneck within computer system. Consequently, it leads to a significant increase in training times without prior hyper-parameter tuning. To address this issue, in this work, we are the first to propose MF-NeRF, a memory-efficient NeRF framework that employs a Mixed-Feature hash table to improve memory efficiency and reduce training time while maintaining reconstruction quality. Specifically, we first design a mixed-feature hash encoding to adaptively mix part of multi-level feature grids and map it to a single hash table. Following that, in order to obtain the correct index of a grid point, we further develop an index transformation method that transforms indices of an arbitrary level grid to those of a canonical grid. Extensive experiments benchmarking with state-of-the-art Instant-NGP, TensoRF, and DVGO, indicate our MF-NeRF could achieve the fastest training time on the same GPU hardware with similar or even higher reconstruction quality.

cs.CV↗

arXiv2023

NFTs to MARS: Multi-Attention Recommender System for NFTs

Recommender systems have become essential tools for enhancing user experiences across various domains. While extensive research has been conducted on recommender systems for movies, music, and e-commerce, the rapidly growing and economically significant Non-Fungible Token (NFT) market remains underexplored. The unique characteristics and increasing prominence of the NFT market highlight the importance of developing tailored recommender systems to cater to its specific needs and unlock its full potential. In this paper, we examine the distinctive characteristics of NFTs and propose the first recommender system specifically designed to address NFT market challenges. In specific, we develop a Multi-Attention Recommender System for NFTs (NFT-MARS) with three key characteristics: (1) graph attention to handle sparse user-item interactions, (2) multi-modal attention to incorporate feature preference of users, and (3) multi-task learning to consider the dual nature of NFTs as both artwork and financial assets. We demonstrate the effectiveness of NFT-MARS compared to various baseline models using the actual transaction data of NFTs collected directly from blockchain for four of the most popular NFT collections. The source code and data are available at https://anonymous.4open.science/r/RecSys2023-93ED.

cs.IR↗

arXiv2023

DarkBERT: A Language Model for the Dark Side of the Internet

Recent research has suggested that there are clear differences in the language used in the Dark Web compared to that of the Surface Web. As studies on the Dark Web commonly require textual analysis of the domain, language models specific to the Dark Web may provide valuable insights to researchers. In this work, we introduce DarkBERT, a language model pretrained on Dark Web data. We describe the steps taken to filter and compile the text data used to train DarkBERT to combat the extreme lexical and structural diversity of the Dark Web that may be detrimental to building a proper representation of the domain. We evaluate DarkBERT and its vanilla counterpart along with other widely used language models to validate the benefits that a Dark Web domain specific model offers in various use cases. Our evaluations show that DarkBERT outperforms current language models and may serve as a valuable resource for future research on the Dark Web.

cs.CL↗

arXiv2022

Value Function Gradient Learning for Large-Scale Multistage Stochastic Programming Problems

A stagewise decomposition algorithm called value function gradient learning (VFGL) is proposed for large-scale multistage stochastic convex programs. VFGL finds the parameter values that best fit the gradient of the value function within a given parametric family. Widely used decomposition algorithms for multistage stochastic programming, such as stochastic dual dynamic programming (SDDP), approximate the value function by adding linear subgradient cuts at each iteration. Although this approach has been successful for linear problems, nonlinear problems may suffer from the increasing size of each subproblem as the iteration proceeds. On the other hand, VFGL has a fixed number of parameters; thus, the size of the subproblems remains constant throughout the iteration. Furthermore, VFGL can learn the parameters by means of stochastic gradient descent, which means that it can be easily parallelized and does not require a scenario tree approximation of the underlying uncertainties. VFGL was compared with a deterministic equivalent formulation of the multistage stochastic programming problem and SDDP approaches for three illustrative examples: production planning, hydrothermal generation, and the lifetime financial planning problem. Numerical examples show that VFGL generates high-quality solutions and is computationally efficient.

math.OC↗

arXiv2022

Shedding New Light on the Language of the Dark Web

The hidden nature and the limited accessibility of the Dark Web, combined with the lack of public datasets in this domain, make it difficult to study its inherent characteristics such as linguistic properties. Previous works on text classification of Dark Web domain have suggested that the use of deep neural models may be ineffective, potentially due to the linguistic differences between the Dark and Surface Webs. However, not much work has been done to uncover the linguistic characteristics of the Dark Web. This paper introduces CoDA, a publicly available Dark Web dataset consisting of 10000 web documents tailored towards text-based Dark Web analysis. By leveraging CoDA, we conduct a thorough linguistic analysis of the Dark Web and examine the textual differences between the Dark Web and the Surface Web. We also assess the performance of various methods of Dark Web page classification. Finally, we compare CoDA with an existing public Dark Web dataset and evaluate their suitability for various use cases.

cs.CL↗

arXiv2019

DEEP-BO for Hyperparameter Optimization of Deep Networks

The performance of deep neural networks (DNN) is very sensitive to the particular choice of hyper-parameters. To make it worse, the shape of the learning curve can be significantly affected when a technique like batchnorm is used. As a result, hyperparameter optimization of deep networks can be much more challenging than traditional machine learning models. In this work, we start from well known Bayesian Optimization solutions and provide enhancement strategies specifically designed for hyperparameter optimization of deep networks. The resulting algorithm is named as DEEP-BO (Diversified, Early-termination-Enabled, and Parallel Bayesian Optimization). When evaluated over six DNN benchmarks, DEEP-BO easily outperforms or shows comparable performance with some of the well-known solutions including GP-Hedge, Hyperband, BOHB, Median Stopping Rule, and Learning Curve Extrapolation. The code used is made publicly available at https://github.com/snu-adsl/DEEP-BO.

cs.LG↗

arXiv2018

Pressure-driven collapse of the relativistic electronic ground state in a honeycomb iridate

The electronic ground state in many iridate materials is described by a complex wave-function in which spin and orbital angular momenta are entangled due to relativistic spin-orbit coupling (SOC). Such a localized electronic state carries an effective total angular momentum of $J_{eff}=1/2$. In materials with an edge-sharing octahedral crystal structure, such as the honeycomb iridates Li2IrO3 and Na2IrO3, these $J_{eff}=1/2$ moments are expected to be coupled through a special bond-dependent magnetic interaction, which is a necessary condition for the realization of a Kitaev quantum spin liquid. However, this relativistic electron picture is challenged by an alternate description, in which itinerant electrons are confined to a benzene-like hexagon, keeping the system insulating despite the delocalized nature of the electrons. In this quasi-molecular orbital (QMO) picture, the honeycomb iridates are an unlikely choice for a Kitaev spin liquid. Here we show that the honeycomb iridate Li2IrO3 is best described by a $J_{eff}=1/2$ state at ambient pressure, but crosses over into a QMO state under the application of small (~ 0.1 GPa) hydrostatic pressure. This result illustrates that the physics of iridates is extremely rich due to a delicate balance between electronic bandwidth, spin-orbit coupling, crystal field, and electron correlation.

cond-mat.str-el↗

arXiv2004

Structural Distortion and Disappearance of Layer Ordering in LixCoO2x2H2O

The structure of a new member of the AxCoO2 x nH2O family (A=Li,Na,K) was determined by high-resolution synchrotron X-ray powder diffraction. Layered lithium cobalt oxide hydrate LixCoO2 x 2H2O crystallizes in a disordered monoclinic structure (space group C2/m) with lattice parameters a = 4.8915(2) A, b = 2.8239(1) A, c = 10.7033(9) A, beta = 112.386(4)deg. Unlike the superconducting sodium cobalt oxide hydrate, this material possesses a structure with lithium-water layers intercalated between disordered octahedral sheets of cobalt oxide. The interlayer spacing is slightly larger (~0.9%) due to the higher water content, and one of the two lithium sites extends into the water layer. This material shows no superconducting transition above 2K.

cond-mat.supr-con↗

arXiv2003

The role of the lattice in the $γ\to α$ phase transition of Ce: a high pressure neutron and x-ray diffraction study

The temperature and pressure dependence of the thermal displacements and lattice parameters were obtained across the $γ\to α$ phase transition of Ce using high-pressure, high-resolution neutron and synchrotron x-ray powder diffraction. The estimated vibrational entropy change per atom in the $γ\to α$ phase transition, $ΔS^{γ- α}_{\rm vib} \approx (0.75 \pm 0.15)$k$_{\rm B}$, is about half of the total entropy change. The bulk modulus follows a power-law pressure dependence which is well described using the framework of electron-phonon coupling. These results clearly demonstrate the importance of lattice vibrations, in addition to the spin and charge degrees of freedom, for a complete description of the $γ\to α$ phase transition in elemental Ce.

cond-mat.str-el↗

arXiv2003

Novel Synthesis and High Pressure Behavior of Na0.3CoO2 x 1.3 H2O and Related Phases

We have prepared powder samples of NaxCoO2 x yH2O using a new synthesis route. Superconductivity was observed in Na0.3CoO2 x 1.3H2O between 4 and 5K as indicated by the magnetic susceptibility. The bulk compressibilities of Na0.3CoO2 x 1.3H2O, Na0.3CoO2 x 0.6H2O and Na0.3CoO2 were determined using a diamond anvil cell and synchrotron powder diffraction. Chemical changes occurring under pressure when using different pressure transmitting media are discussed and further transport measurements are advocated.

cond-mat.supr-con↗