arXiv subjects

Bumsoo Park

Publications and source records attributed to Bumsoo Park.

5 recordsLinked to original sources

arXiv2026

Identifiable Token Correspondence for World Models

Token-based transformer world models have shown strong performance in visual reinforcement learning, but often suffer from temporal inconsistency in long-horizon rollouts, including object duplication, disappearance, and transmutation. A key reason is that most existing approaches treat next-frame prediction purely as a token generation problem, without considering the persistence of tokens across time. We introduce Identifiable Token Correspondence (ITC), a decoding step for token-based transformer world models that formulates next-frame prediction as a structured assignment problem with latent token correspondence variables: each next-frame token is explained either by copying a token from the previous frame or by generating a new one. ITC leaves the transformer architecture and training procedure unchanged and can be added on top of existing backbones. Our experiments show state-of-the-art performance on 4 challenging benchmarks. The proposed method achieves a return of 72.5% and a score of 35.6% on the Craftax-classic benchmark, significantly surpassing the previous best of 67.4% and 27.9%. We release our source code on https://github.com/snu-mllab/Identifiable-Token-Correspondence.

cs.LG↗

arXiv2024

Learning to Better Search with Language Models via Guided Reinforced Self-Training

While language models have shown remarkable performance across diverse tasks, they still encounter challenges in complex reasoning scenarios. Recent research suggests that language models trained on linearized search traces toward solutions, rather than solely on the final solutions, exhibit improved generalization, despite the search traces being potentially noisy or suboptimal. However, relying on such imperfect traces can result in inefficient use of test-time compute. To address this, we propose guided reinforced self-training (Guided-ReST), a fine-tuning algorithm designed to improve the model's capability for effective search during inference. The key insight behind Guided-ReST is that optimal solutions can serve as valuable step-by-step landmarks to guide the model's search process. Based on this insight, we introduce a novel data generation method that seamlessly incorporates optimal solutions into the model's search procedure, enabling the generation of high-quality search traces. By fine-tuning the model on these search traces, we effectively distill improved search strategies into the model. Our method significantly enhances the search capabilities of language models on arithmetic reasoning and code self-repair tasks, including Countdown, CodeContests, and CodeForces. We release the source code at https://github.com/snu-mllab/guided-rest.

cs.AI↗

arXiv2023

Discovering Hierarchical Achievements in Reinforcement Learning via Contrastive Learning

Discovering achievements with a hierarchical structure in procedurally generated environments presents a significant challenge. This requires an agent to possess a broad range of abilities, including generalization and long-term reasoning. Many prior methods have been built upon model-based or hierarchical approaches, with the belief that an explicit module for long-term planning would be advantageous for learning hierarchical dependencies. However, these methods demand an excessive number of environment interactions or large model sizes, limiting their practicality. In this work, we demonstrate that proximal policy optimization (PPO), a simple yet versatile model-free algorithm, outperforms previous methods when optimized with recent implementation practices. Moreover, we find that the PPO agent can predict the next achievement to be unlocked to some extent, albeit with limited confidence. Based on this observation, we introduce a novel contrastive learning method, called achievement distillation, which strengthens the agent's ability to predict the next achievement. Our method exhibits a strong capacity for discovering hierarchical achievements and shows state-of-the-art performance on the challenging Crafter environment in a sample-efficient manner while utilizing fewer model parameters.

cs.LG↗

arXiv2023

Control-Oriented Modeling and Layer-to-Layer Spatial Control of Powder Bed Fusion Processes

Powder Bed Fusion (PBF) is an important Additive Manufacturing (AM) process that is seeing widespread utilization. However, due to inherent process variability, it is still very costly and time consuming to certify the process and the part. This has led researchers to conduct numerous studies in process modeling, in-situ monitoring and feedback control to better understand the PBF process and decrease variations, thereby making the process more repeatable. In this study, we develop a layer-to-layer, spatial, control-oriented thermal PBF model. This model enables a framework for capturing spatially-driven thermal effects and constructing layer-to-layer spatial controllers that do not suffer from inherent temporal delays. Further, this framework is amenable to voxel-level monitoring and characterization efforts. System output controllability is analyzed and output controllability conditions are determined. A spatial Iterative Learning Controller (ILC), constructed using the spatial modeling framework, is implemented in two experiments, one where the path and part geometry are layer-invariant and another where the path and part geometry change each layer. The results illustrate the ability of the controller to thermally regulate the entire part, even at corners that tend to overheat and even as the path and part geometry change each layer.

eess.SY↗

arXiv2022

An empirical model for feedforward control of laser powder bed fusion

While considerable progress has recently been made in real-time melt pool monitoring for laser powder bed fusion (LPBF), results in in-situ melt pool control are relatively sparse, a major reason being lack of suitable control-oriented models. This study demonstrates an empirical control-oriented model of geometry-dependent melt pool behavior, and subsequent melt pool regulation with a model-based feedforward controller for laser power. First, it shows that the melt pool "footprint" exponentially increases when the scan lines become shorter. The empirical model of this behavior is developed and validated on different geometries at different laser power levels. Second, the developed model is used to design a feedforward controller for obtaining optimal laser power profiles. This controller is then validated experimentally and is demonstrated to suppress the in-layer geometry-related melt pool signal deviations, for different part geometries.

eess.SY↗