arXiv subjects

Taeyoung Kim

Publications and source records attributed to Taeyoung Kim.

36 records · Page 2Linked to original sources

arXiv2024

Mixed-Session Conversation with Egocentric Memory

Recently introduced dialogue systems have demonstrated high usability. However, they still fall short of reflecting real-world conversation scenarios. Current dialogue systems exhibit an inability to replicate the dynamic, continuous, long-term interactions involving multiple partners. This shortfall arises because there have been limited efforts to account for both aspects of real-world dialogues: deeply layered interactions over the long-term dialogue and widely expanded conversation networks involving multiple participants. As the effort to incorporate these aspects combined, we introduce Mixed-Session Conversation, a dialogue system designed to construct conversations with various partners in a multi-session dialogue setup. We propose a new dataset called MiSC to implement this system. The dialogue episodes of MiSC consist of 6 consecutive sessions, with four speakers (one main speaker and three partners) appearing in each episode. Also, we propose a new dialogue model with a novel memory management mechanism, called Egocentric Memory Enhanced Mixed-Session Conversation Agent (EMMA). EMMA collects and retains memories from the main speaker's perspective during conversations with partners, enabling seamless continuity in subsequent interactions. Extensive human evaluations validate that the dialogues in MiSC demonstrate a seamless conversational flow, even when conversation partners change in each session. EMMA trained with MiSC is also evaluated to maintain high memorability without contradiction throughout the entire conversation.

cs.CL↗

arXiv2024

An Infinite-Width Analysis on the Jacobian-Regularised Training of a Neural Network

The recent theoretical analysis of deep neural networks in their infinite-width limits has deepened our understanding of initialisation, feature learning, and training of those networks, and brought new practical techniques for finding appropriate hyperparameters, learning network weights, and performing inference. In this paper, we broaden this line of research by showing that this infinite-width analysis can be extended to the Jacobian of a deep neural network. We show that a multilayer perceptron (MLP) and its Jacobian at initialisation jointly converge to a Gaussian process (GP) as the widths of the MLP's hidden layers go to infinity and characterise this GP. We also prove that in the infinite-width limit, the evolution of the MLP under the so-called robust training (i.e., training with a regulariser on the Jacobian) is described by a linear first-order ordinary differential equation that is determined by a variant of the Neural Tangent Kernel. We experimentally show the relevance of our theoretical claims to wide finite networks, and empirically analyse the properties of kernel regression solution to obtain an insight into Jacobian regularisation.

cs.LG↗

arXiv2024

Approximating Numerical Fluxes Using Fourier Neural Operators for Hyperbolic Conservation Laws

Traditionally, classical numerical schemes have been employed to solve partial differential equations (PDEs) using computational methods. Recently, neural network-based methods have emerged. Despite these advancements, neural network-based methods, such as physics-informed neural networks (PINNs) and neural operators, exhibit deficiencies in robustness and generalization. To address these issues, numerous studies have integrated classical numerical frameworks with machine learning techniques, incorporating neural networks into parts of traditional numerical methods. In this study, we focus on hyperbolic conservation laws by replacing traditional numerical fluxes with neural operators. To this end, we developed loss functions inspired by established numerical schemes related to conservation laws and approximated numerical fluxes using Fourier neural operators (FNOs). Our experiments demonstrated that our approach combines the strengths of both traditional numerical schemes and FNOs, outperforming standard FNO methods in several respects. For instance, we demonstrate that our method is robust, has resolution invariance, and is feasible as a data-driven method. In particular, our method can make continuous predictions over time and exhibits superior generalization capabilities with out-of-distribution (OOD) samples, which are challenges that existing neural operator methods encounter.

math.NA↗

arXiv2024

Cluster-based Sampling in Hindsight Experience Replay for Robotic Tasks (Student Abstract)

In multi-goal reinforcement learning with a sparse binary reward, training agents is particularly challenging, due to a lack of successful experiences. To solve this problem, hindsight experience replay (HER) generates successful experiences even from unsuccessful ones. However, generating successful experiences from uniformly sampled ones is not an efficient process. In this paper, the impact of exploiting the property of achieved goals in generating successful experiences is investigated and a novel cluster-based sampling strategy is proposed. The proposed sampling strategy groups episodes with different achieved goals by using a cluster model and samples experiences in the manner of HER to create the training batch. The proposed method is validated by experiments with three robotic control tasks of the OpenAI Gym. The results of experiments demonstrate that the proposed method is substantially sample efficient and achieves better performance than baseline approaches.

cs.RO↗

arXiv2023

Virtual Action Actor-Critic Framework for Exploration (Student Abstract)

Efficient exploration for an agent is challenging in reinforcement learning (RL). In this paper, a novel actor-critic framework namely virtual action actor-critic (VAAC), is proposed to address the challenge of efficient exploration in RL. This work is inspired by humans' ability to imagine the potential outcomes of their actions without actually taking them. In order to emulate this ability, VAAC introduces a new actor called virtual actor (VA), alongside the conventional actor-critic framework. Unlike the conventional actor, the VA takes the virtual action to anticipate the next state without interacting with the environment. With the virtual policy following a Gaussian distribution, the VA is trained to maximize the anticipated novelty of the subsequent state resulting from a virtual action. If any next state resulting from available actions does not exhibit high anticipated novelty, training the VA leads to an increase in the virtual policy entropy. Hence, high virtual policy entropy represents that there is no room for exploration. The proposed VAAC aims to maximize a modified Q function, which combines cumulative rewards and the negative sum of virtual policy entropy. Experimental results show that the VAAC improves the exploration performance compared to existing algorithms.

cs.LG↗

arXiv2023

Enhanced Transformer Architecture for Natural Language Processing

Transformer is a state-of-the-art model in the field of natural language processing (NLP). Current NLP models primarily increase the number of transformers to improve processing performance. However, this technique requires a lot of training resources such as computing capacity. In this paper, a novel structure of Transformer is proposed. It is featured by full layer normalization, weighted residual connection, positional encoding exploiting reinforcement learning, and zero masked self-attention. The proposed Transformer model, which is called Enhanced Transformer, is validated by the bilingual evaluation understudy (BLEU) score obtained with the Multi30k translation dataset. As a result, the Enhanced Transformer achieves 202.96% higher BLEU score as compared to the original transformer with the translation dataset.

cs.CL↗

arXiv2023

SmartMark: Software Watermarking Scheme for Smart Contracts

Smart contracts are self-executing programs on a blockchain to ensure immutable and transparent agreements without the involvement of intermediaries. Despite the growing popularity of smart contracts for many blockchain platforms like Ethereum, smart contract developers cannot prevent copying their smart contracts from competitors due to the absence of technical means available. However, applying existing software watermarking techniques is challenging because of the unique properties of smart contracts, such as a code size constraint, non-free execution cost, and no support for dynamic allocation under a virtual machine environment. This paper introduces a novel software watermarking scheme, dubbed SmartMark, aiming to protect the piracy of smart contracts. SmartMark builds the control flow graph of a target contract runtime bytecode and locates a series of bytes randomly selected from a collection of opcodes to represent a watermark. We implement a full-fledged prototype for Ethereum, applying SmartMark to 27,824 unique smart contract bytecodes. Our empirical results demonstrate that SmartMark can effectively embed a watermark into smart contracts and verify its presence, meeting the requirements of credibility and imperceptibility while incurring a slight performance degradation. Furthermore, our security analysis shows that SmartMark is resilient against foreseeable watermarking corruption attacks; e.g., a large number of dummy opcodes are needed to disable a watermark effectively, resulting in producing illegitimate smart contract clones that are not economical.

cs.SE↗

arXiv2023

Road Redesign Technique Achieving Enhanced Road Safety by Inpainting with a Diffusion Model

Road infrastructure can affect the occurrence of road accidents. Therefore, identifying roadway features with high accident probability is crucial. Here, we introduce image inpainting that can assist authorities in achieving safe roadway design with minimal intervention in the current roadway structure. Image inpainting is based on inpainting safe roadway elements in a roadway image, replacing accident-prone (AP) features by using a diffusion model. After object-level segmentation, the AP features identified by the properties of accident hotspots are masked by a human operator and safe roadway elements are inpainted. With only an average time of 2 min for image inpainting, the likelihood of an image being classified as an accident hotspot drops by an average of 11.85%. In addition, safe urban spaces can be designed considering human factors of commuters such as gaze saliency. Considering this, we introduce saliency enhancement that suggests chrominance alteration for a safe road view.

cs.CV↗

arXiv2022

Off-Policy Reinforcement Learning with Loss Function Weighted by Temporal Difference Error

Training agents via off-policy deep reinforcement learning (RL) requires a large memory, named replay memory, that stores past experiences used for learning. These experiences are sampled, uniformly or non-uniformly, to create the batches used for training. When calculating the loss function, off-policy algorithms assume that all samples are of the same importance. In this paper, we hypothesize that training can be enhanced by assigning different importance for each experience based on their temporal-difference (TD) error directly in the training objective. We propose a novel method that introduces a weighting factor for each experience when calculating the loss function at the learning stage. In addition to improving convergence speed when used with uniform sampling, the method can be combined with prioritization methods for non-uniform sampling. Combining the proposed method with prioritization methods improves sampling efficiency while increasing the performance of TD-based off-policy RL algorithms. The effectiveness of the proposed method is demonstrated by experiments in six environments of the OpenAI Gym suite. The experimental results demonstrate that the proposed method achieves a 33%~76% reduction of convergence speed in three environments and an 11% increase in returns and a 3%~10% increase in success rate for other three environments.

cs.LG↗

arXiv2022

Reinforcement Learning for Predicting Traffic Accidents

As the demand for autonomous driving increases, it is paramount to ensure safety. Early accident prediction using deep learning methods for driving safety has recently gained much attention. In this task, early accident prediction and a point prediction of where the drivers should look are determined, with the dashcam video as input. We propose to exploit the double actors and regularized critics (DARC) method, for the first time, on this accident forecasting platform. We derive inspiration from DARC since it is currently a state-of-the-art reinforcement learning (RL) model on continuous action space suitable for accident anticipation. Results show that by utilizing DARC, we can make predictions 5\% earlier on average while improving in multiple metrics of precision compared to existing methods. The results imply that using our RL-based problem formulation could significantly increase the safety of autonomous driving.

cs.AI↗

arXiv2022

Kick-motion Training with DQN in AI Soccer Environment

This paper presents a technique to train a robot to perform kick-motion in AI soccer by using reinforcement learning (RL). In RL, an agent interacts with an environment and learns to choose an action in a state at each step. When training RL algorithms, a problem called the curse of dimensionality (COD) can occur if the dimension of the state is high and the number of training data is low. The COD often causes degraded performance of RL models. In the situation of the robot kicking the ball, as the ball approaches the robot, the robot chooses the action based on the information obtained from the soccer field. In order not to suffer COD, the training data, which are experiences in the case of RL, should be collected evenly from all areas of the soccer field over (theoretically infinite) time. In this paper, we attempt to use the relative coordinate system (RCS) as the state for training kick-motion of robot agent, instead of using the absolute coordinate system (ACS). Using the RCS eliminates the necessity for the agent to know all the (state) information of entire soccer field and reduces the dimension of the state that the agent needs to know to perform kick-motion, and consequently alleviates COD. The training based on the RCS is performed with the widely used Deep Q-network (DQN) and tested in the AI Soccer environment implemented with Webots simulation software.

cs.RO↗

arXiv2022

Bounding the Rademacher Complexity of Fourier neural operators

A Fourier neural operator (FNO) is one of the physics-inspired machine learning methods. In particular, it is a neural operator. In recent times, several types of neural operators have been developed, e.g., deep operator networks, Graph neural operator (GNO), and Multiwavelet-based operator (MWTO). Compared with other models, the FNO is computationally efficient and can learn nonlinear operators between function spaces independent of a certain finite basis. In this study, we investigated the bounding of the Rademacher complexity of the FNO based on specific group norms. Using capacity based on these norms, we bound the generalization error of the model. In addition, we investigated the correlation between the empirical generalization error and the proposed capacity of FNO. From the perspective of our result, we inferred that the type of group norms determines the information about the weights and architecture of the FNO model stored in the capacity. And then, we confirmed these inferences through experiments. Based on this fact, we gained insight into the impact of the number of modes used in the FNO model on the generalization error. And we got experimental results that followed our insights.

cs.LG↗

arXiv2021

Planar and 3-dimensional damage free etching of $β$-Ga2O3 using atomic gallium flux

In-situ etching using Ga flux in an ultra-high vacuum environment like MBE is introduced as a method to make high aspect ratio 3 dimensional structures in $β$-Ga2O3. Etching of $β$-Ga2O3 due to excess Ga adatoms on the epilayer surface had been viewed as non-ideal for epitaxial growth especially since it results in plateauing and lowering of growth rate. In this study, we use this well-known reaction from epitaxial growth to intentionally etch $β$-Ga2O3. We demonstrate etch rate ranging from 2.9 nm/min to 30 nm/min with the highest reported etch rate being only limited by the highest Ga flux used. Patterned in-situ etching is also demonstrated and used to study the effect of fin orientation on the sidewall profiles and dopant (Si) segregation on the etched surface. Using in-situ Ga etching, we also demonstrate 150 nm wide fins and 200 nm wide nano pillars with high aspect ratio. This new etching method could enable future development of highly scaled vertical and lateral 3D devices in $β$-Ga2O3.

cond-mat.mtrl-sci↗

arXiv2021

Two-stage training algorithm for AI robot soccer

In multi-agent reinforcement learning, the cooperative learning behavior of agents is very important. In the field of heterogeneous multi-agent reinforcement learning, cooperative behavior among different types of agents in a group is pursued. Learning a joint-action set during centralized training is an attractive way to obtain such cooperative behavior, however, this method brings limited learning performance with heterogeneous agents. To improve the learning performance of heterogeneous agents during centralized training, two-stage heterogeneous centralized training which allows the training of multiple roles of heterogeneous agents is proposed. During training, two training processes are conducted in a series. One of the two stages is to attempt training each agent according to its role, aiming at the maximization of individual role rewards. The other is for training the agents as a whole to make them learn cooperative behaviors while attempting to maximize shared collective rewards, e.g., team rewards. Because these two training processes are conducted in a series in every timestep, agents can learn how to maximize role rewards and team rewards simultaneously. The proposed method is applied to 5 versus 5 AI robot soccer for validation. Simulation results show that the proposed method can train the robots of the robot soccer team effectively, achieving higher role rewards and higher team rewards as compared to other approaches that can be used to solve problems of training cooperative multi-agent.

cs.AI↗

arXiv2019

Real-time 3-D Mapping with Estimating Acoustic Materials

This paper proposes a real-time system integrating an acoustic material estimation from visual appearance and an on-the-fly mapping in the 3-dimension. The proposed method estimates the acoustic materials of surroundings in indoor scenes and incorporates them to a 3-D occupancy map, as a robot moves around the environment. To estimate the acoustic material from the visual cue, we apply the state-of-the-art semantic segmentation CNN network based on the assumption that the visual appearance and the acoustic materials have a strong association. Furthermore, we introduce an update policy to handle the material estimations during the online mapping process. As a result, our environment map with acoustic material can be used for sound-related robotics applications, such as sound source localization taking into account various acoustic propagation (e.g., reflection).

cs.RO↗

arXiv2018

Advanced Quantizer Designs for FDD-Based FD-MIMO Systems Using Uniform Planar Arrays

Massive multiple-input multiple-output (MIMO) systems, which utilize a large number of antennas at the base station, are expected to enhance network throughput by enabling improved multiuser MIMO techniques. To deploy many antennas in reasonable form factors, base stations are expected to employ antenna arrays in both horizontal and vertical dimensions, which is known as full-dimension (FD) MIMO. The most popular two-dimensional array is the uniform planar array (UPA), where antennas are placed in a grid pattern. To exploit the full benefit of massive MIMO in frequency division duplexing (FDD), the downlink channel state information (CSI) should be estimated, quantized, and fed back from the receiver to the transmitter. However, it is difficult to accurately quantize the channel in a computationally efficient manner due to the high dimensionality of the massive MIMO channel. In this paper, we develop both narrowband and wideband CSI quantizers for FD-MIMO taking the properties of realistic channels and the UPA into consideration. To improve quantization quality, we focus on not only quantizing dominant radio paths in the channel, but also combining the quantized beams. We also develop a hierarchical beam search approach, which scans both vertical and horizontal domains jointly with moderate computational complexity. Numerical simulations verify that the performance of the proposed quantizers is better than that of previous CSI quantization techniques.

cs.IT↗

arXiv2015

Exploiting the Preferred Domain of FDD Massive MIMO Systems with Uniform Planar Arrays

Massive multiple-input multiple-output (MIMO) systems hold the potential to be an enabling technology for 5G cellular. Uniform planar array (UPA) antenna structures are a focus of much commercial discussion because of their ability to enable a large number of antennas in a relatively small area. With UPA antenna structures, the base station can control the beam direction in both the horizontal and vertical domains simultaneously. However, channel conditions may dictate that one dimension requires higher channel state information (CSI) accuracy than the other. We propose the use of an additional one bit of feedback information sent from the user to the base station to indicate the preferred domain on top of the feedback overhead of CSI quantization in frequency division duplexing (FDD) massive MIMO systems. Combined with variable-rate CSI quantization schemes, the numerical studies show that the additional one bit of feedback can increase the quality of CSI significantly for UPA antenna structures.

cs.IT↗

arXiv2014

Trellis-Extended Codebooks and Successive Phase Adjustment: A Path from LTE-Advanced to FDD Massive MIMO Systems

It is of great interest to develop efficient ways to acquire accurate channel state information (CSI) for frequency division duplexing (FDD) massive multiple-input multiple-output (MIMO) systems for backward compatibility. It is theoretically well known that the codebook size for CSI quantization should be increased as the number of transmit antennas becomes larger, and 3GPP long term evolution (LTE) and LTE-Advanced codebooks also follow this trend. Thus, in massive MIMO, it is hard to apply the conventional approach of using pre-defined vector-quantized codebooks for CSI quantization mainly because of codeword search complexity. In this paper, we propose a trellis-extended codebook (TEC) that can be easily harmonized with current wireless standards such as LTE or LTE-Advanced by extending standardized codebooks designed for 2, 4, or 8 antennas with trellis structures. TEC exploits a Viterbi decoder and convolutional encoder in channel coding as the CSI quantizer and the CSI reconstructer, respectively. By quantizing multiple channel entries simultaneously using standardized codebooks in a state transition of trellis search, TEC can achieve fractional bits per channel entry quantization to have a practical feedback overhead. Thus, TEC can solve both the complexity and the feedback overhead issues of CSI quantization in massive MIMO systems. We also develop trellis-extended successive phase adjustment (TE-SPA) which works as a differential codebook of TEC. This is similar to the dual codebook concept of LTE-Advanced. TE-SPA can reduce CSI quantization error even with lower feedback overhead in temporally correlated channels. Numerical results verify the effectiveness of the proposed schemes in FDD massive MIMO systems.

cs.IT↗