arXiv subjects

Hyungjoon Soh

Publications and source records attributed to Hyungjoon Soh.

12 recordsLinked to original sources

arXiv2026

Multiplicative learning from observation-prediction ratios

Additive parameter updates, as used in gradient descent and its adaptive extensions, underpin most modern machine-learning optimization. Yet, such additive schemes often demand numerous iterations and intricate learning-rate schedules to cope with scale and curvature of loss functions. Here we introduce Expectation Reflection (ER), a multiplicative learning paradigm that updates parameters based on the ratio of observed to predicted outputs, rather than their differences. ER eliminates the need for ad hoc loss functions or learning-rate tuning while maintaining internal consistency. Extending ER to multilayer networks, we demonstrate its efficacy in image classification, achieving optimal weight determination in a single iteration. We further show that ER can be interpreted as a modified gradient descent incorporating an inverse target-propagation mapping. Together, these results position ER as a fast and scalable alternative to conventional optimization methods for neural-network training.

cs.LG↗

arXiv2026

Variational Garrote for Sparse Inverse Problems

Sparse regularization plays a central role in solving inverse problems arising from incomplete or corrupted measurements. Different regularizers correspond to different prior assumptions about the structure of the unknown signal, and reconstruction performance depends on how well these priors match the intrinsic sparsity of the data. This work investigates the effect of sparsity priors in inverse problems by comparing conventional L1 regularization with the Variational Garrote (VG), a probabilistic method that approximates L0 sparsity through variational binary gating variables. A unified experimental framework is constructed across multiple reconstruction tasks including signal resampling, signal denoising, and sparse-view computed tomography. To enable consistent comparison across models with different parameterizations, regularization strength is swept across wide ranges and reconstruction behavior is analyzed through train-generalization error curves. Experiments reveal characteristic bias-variance tradeoff patterns across tasks and demonstrate that VG frequently achieves lower minimum generalization error and improved stability in strongly underdetermined regimes where accurate support recovery is critical. These results suggest that sparsity priors closer to spike-and-slab structure can provide advantages when the underlying coefficient distribution is strongly sparse. The study highlights the importance of prior-data alignment in sparse inverse problems and provides empirical insights into the behavior of variational L0-type methods across different information bottlenecks.

stat.ML↗

arXiv2025

Stochastic Clock Attention for Aligning Continuous and Ordered Sequences

We formulate an attention mechanism for continuous and ordered sequences that explicitly functions as an alignment model, which serves as the core of many sequence-to-sequence tasks. Standard scaled dot-product attention relies on positional encodings and masks but does not enforce continuity or monotonicity, which are crucial for frame-synchronous targets. We propose learned nonnegative \emph{clocks} to source and target and model attention as the meeting probability of these clocks; a path-integral derivation yields a closed-form, Gaussian-like scoring rule with an intrinsic bias toward causal, smooth, near-diagonal alignments, without external positional regularizers. The framework supports two complementary regimes: normalized clocks for parallel decoding when a global length is available, and unnormalized clocks for autoregressive decoding -- both nearly-parameter-free, drop-in replacements. In a Transformer text-to-speech testbed, this construction produces more stable alignments and improved robustness to global time-scaling while matching or improving accuracy over scaled dot-product baselines. We hypothesize applicability to other continuous targets, including video and temporal signal modeling.

cs.LG↗

arXiv2025

Variational Garrote for Statistical Physics-based Sparse and Robust Variable Selection

Selecting key variables from high-dimensional data is increasingly important in the era of big data. Sparse regression serves as a powerful tool for this purpose by promoting model simplicity and explainability. In this work, we revisit a valuable yet underutilized method, the statistical physics-based Variational Garrote (VG), which introduces explicit feature selection spin variables and leverages variational inference to derive a tractable loss function. We enhance VG by incorporating modern automatic differentiation techniques, enabling scalable and efficient optimization. We evaluate VG on both fully controllable synthetic datasets and complex real-world datasets. Our results demonstrate that VG performs especially well in highly sparse regimes, offering more consistent and robust variable selection than Ridge and LASSO regression across varying levels of sparsity. We also uncover a sharp transition: as superfluous variables are admitted, generalization degrades abruptly and the uncertainty of the selection variables increases. This transition point provides a practical signal for estimating the correct number of relevant variables, an insight we successfully apply to identify key predictors in real-world data. We expect that VG offers strong potential for sparse modeling across a wide range of applications, including compressed sensing and model pruning in machine learning.

cs.LG↗

arXiv2025

Unified Cross-modal Translation of Score Images, Symbolic Music, and Performance Audio

Music exists in various modalities, such as score images, symbolic scores, MIDI, and audio. Translations between each modality are established as core tasks of music information retrieval, such as automatic music transcription (audio-to-MIDI) and optical music recognition (score image to symbolic score). However, most past work on multimodal translation trains specialized models on individual translation tasks. In this paper, we propose a unified approach, where we train a general-purpose model on many translation tasks simultaneously. Two key factors make this unified approach viable: a new large-scale dataset and the tokenization of each modality. Firstly, we propose a new dataset that consists of more than 1,300 hours of paired audio-score image data collected from YouTube videos, which is an order of magnitude larger than any existing music modal translation datasets. Secondly, our unified tokenization framework discretizes score images, audio, MIDI, and MusicXML into a sequence of tokens, enabling a single encoder-decoder Transformer to tackle multiple cross-modal translation as one coherent sequence-to-sequence task. Experimental results confirm that our unified multitask model improves upon single-task baselines in several key areas, notably reducing the symbol error rate for optical music recognition from 24.58% to a state-of-the-art 13.67%, while similarly substantial improvements are observed across the other translation tasks. Notably, our approach achieves the first successful score-image-conditioned audio generation, marking a significant breakthrough in cross-modal music generation.

cs.SD↗

arXiv2024

Improving Demand Forecasting in Open Systems with Cartogram-Enhanced Deep Learning

Predicting temporal patterns across various domains poses significant challenges due to their nuanced and often nonlinear trajectories. To address this challenge, prediction frameworks have been continuously refined, employing data-driven statistical methods, mathematical models, and machine learning. Recently, as one of the challenging systems, shared transport systems such as public bicycles have gained prominence due to urban constraints and environmental concerns. Predicting rental and return patterns at bicycle stations remains a formidable task due to the system's openness and imbalanced usage patterns across stations. In this study, we propose a deep learning framework to predict rental and return patterns by leveraging cartogram approaches. The cartogram approach facilitates the prediction of demand for newly installed stations with no training data as well as long-period prediction, which has not been achieved before. We apply this method to public bicycle rental-and-return data in Seoul, South Korea, employing a spatial-temporal convolutional graph attention network. Our improved architecture incorporates batch attention and modified node feature updates for better prediction accuracy across different time scales. We demonstrate the effectiveness of our framework in predicting temporal patterns and its potential applications.

cs.LG↗

arXiv2023

Mirror descent of Hopfield model

Mirror descent is an elegant optimization technique that leverages a dual space of parametric models to perform gradient descent. While originally developed for convex optimization, it has increasingly been applied in the field of machine learning. In this study, we propose a novel approach for utilizing mirror descent to initialize the parameters of neural networks. Specifically, we demonstrate that by using the Hopfield model as a prototype for neural networks, mirror descent can effectively train the model with significantly improved performance compared to traditional gradient descent methods that rely on random parameter initialization. Our findings highlight the potential of mirror descent as a promising initialization technique for enhancing the optimization of machine learning models.

cs.LG↗

arXiv2020

KorNLI and KorSTS: New Benchmark Datasets for Korean Natural Language Understanding

Natural language inference (NLI) and semantic textual similarity (STS) are key tasks in natural language understanding (NLU). Although several benchmark datasets for those tasks have been released in English and a few other languages, there are no publicly available NLI or STS datasets in the Korean language. Motivated by this, we construct and release new datasets for Korean NLI and STS, dubbed KorNLI and KorSTS, respectively. Following previous approaches, we machine-translate existing English training sets and manually translate development and test sets into Korean. To accelerate research on Korean NLU, we also establish baselines on KorNLI and KorSTS. Our datasets are publicly available at https://github.com/kakaobrain/KorNLUDatasets.

cs.CL↗

arXiv2019

Passive Tracer Dynamics in Slow-Bond Problem

Asymptotic Kardar-Parisi-Zhang (KPZ) properties are investigated in the totally asymmetric simple exclusion process (TASEP) with a localized geometric defect. In particular, we focus on the universal nature of nonequilibrium steady states of the modified TASEP. Since the original TASEP belongs to the KPZ universality class, it is mathematically and physically a quite interesting question whether the localized columnar defect, the slow bond (SB), is really always relevant to the KPZ universality or not. However, it is numerically controversial to address the possibility of the non-queued SB phase in the weak-strength SB limit. Based on the detailed statistical analysis of KPZ-type growing interfaces, we present a comprehensive view of the non-queue SB phase, compared to finite-size crossover effects that reported in our earlier work [Soh {\it et al.}, Phys. Rev. E {\bf 95}, 042123 (2017)]. Moreover, we employ two types of passive tracer dynamics as the probe of the SB dynamics. Finally, we provide intuitive arguments for additional clues to resolve the controversy of the SB problem.

cond-mat.stat-mech↗

arXiv2018

Jamming and condensation in one-dimensional driven flow

We revisit the slow-bond (SB) problem of the one-dimensional (1D) totally asymmetric simple exclusion process (TASEP) with modified hopping rates. In the original SB problem, it turns out that a local defect is always relevant to the system as jamming, so that phase separation occurs in the 1D TASEP. However, crossover scaling behaviors are also observed as finite-size effects. In order to check if the SB can be irrelevant to the system with particle interaction, we employ the condensation concept in the zero-range process. The hopping rate in the modified TASEP depends on the interaction parameter and the distance up to the nearest particle in the moving direction, besides the SB factor. In particular, we focus on the interplay of jamming and condensation in the current-density relation of 1D driven flow. Based on mean-field calculations, we present the fundamental diagram and the phase diagram of the modified SB problem, which are numerically checked. Finally, we discuss how the condensation of holes suppresses the jamming of particles and vice versa, where the partially-condensed phase is the most interesting, compared to that in the original SB problem.

cond-mat.stat-mech↗

arXiv2017

Emergence of Long-Term Memory in Popularity

Popularity describes the dynamics of mass attention, and is a part of a broader class of population dynamics in ecology and social science literature. Studying accurate model of popularity is important for quantifying spreading of novelty, memes, and influences in human society. Although logistic equation and similar class of nonlinear differential equation formulates traditional population dynamics well, part of the deviation in long-term prediction is stated, yet fully understood. Recently, several studies hinted a long-term memory effect on popularity whose response function follows a power-law, especially that appears on online mass media such as YouTube, Twitter, or Amazon book sales. Here, we investigate the ranking of most popular music, \textit{the Billboard Hot 100 chart}, which is one of the largest popularity dataset spanning several decades. Using a popularity model that comprises logistic growth and a power-law decaying long-term memory, we showed that rank history is mainly characterized by initial popularity and memory strength. With this framework, we investigated temporal development of long-term memory on the whole popularity dynamics. As a result, abrupt emergence of long-term memory and broad initial popularity is illustrated, which was not clearly detected by time-independent measures. We emphasize not only development of the mass media, but also the difference of spreading and accumulated popularity affect dynamics significantly when the popularity has long-term memory.

physics.soc-ph↗

arXiv2017

Effects of a local defect on one-dimensional nonlinear surface growth

The slow-bond problem is a long-standing question about the minimal strength $ε_\mathrm{c}$ of a local defect with global effects on the Kardar--Parisi--Zhang (KPZ) universality class. A consensus on the issue has been delayed due to the discrepancy between various analytical predictions claiming $ε_\mathrm{c} = 0$ and numerical observations claiming $ε_\mathrm{c} > 0$. We revisit the problem via finite-size scaling analyses of the slow-bond effects, which are tested for different boundary conditions through extensive Monte Carlo simulations. Our results provide evidence that the previously reported nonzero $ε_\mathrm{c}$ is an artifact of a crossover phenomenon, which logarithmically converges to zero as the system size goes to infinity.

cond-mat.stat-mech↗