arXiv subjects

Gilhan Kim

Publications and source records attributed to Gilhan Kim.

6 recordsLinked to original sources

arXiv2026

Boltzmann Attention: Learnable Ising Couplings for Cooperative Attention

Attention mechanisms are central to modern sequence models, yet standard attention computes relevance primarily through individual query--key similarities. Although softmax normalization introduces competition among positions, a standard attention layer does not explicitly parameterize learnable interactions between attention decisions. This limits its ability to directly model cooperative or antagonistic co-attention structure within the attention mechanism itself. We propose Boltzmann attention, an energy-based generalization in which attention patterns are governed by an interacting Ising model. The method augments the usual data-dependent local fields with learnable pairwise couplings, allowing the model to represent inter-position correlations beyond those captured by softmax or sigmoid attention. Experiments on character-level language modeling and synthetic bracket matching show that Boltzmann attention consistently improves over standard softmax attention within a standard Transformer architecture, with the advantage becoming more pronounced as sequence length increases. A four-way ablation confirms that the improvement arises from the learnable pairwise couplings. These results suggest that explicit inter-position interactions provide a principled enhancement for attention-based sequence modeling. Moreover, the Ising formulation opens a natural path toward quantum-computing-based sampling strategies: we demonstrate that diabatic quantum annealing provides a practical training method while maintaining competitive performance with exact Boltzmann computation.

cs.LG↗

arXiv2026

Information-Geometric Decomposition of Generalization Error in Unsupervised Learning

We decompose the Kullback--Leibler generalization error (GE) -- the expected KL divergence from the data distribution to the trained model -- of unsupervised learning into three non-negative components: model error, data bias, and variance. The decomposition is exact for any e-flat model class and follows from two identities of information geometry: the generalized Pythagorean theorem and a dual e-mixture variance identity. As an analytically tractable demonstration, we apply the framework to $\epsilon$-PCA, a regularized principal component analysis in which the empirical covariance is truncated at rank $N_K$ and discarded directions are pinned at a fixed noise floor $\epsilon$. Although rank-constrained $\epsilon$-PCA is not itself e-flat, it admits a technical reformulation with the same total GE on isotropic Gaussian data, under which each component of the decomposition takes closed form. The optimal rank emerges as the cutoff $\lambda_{\mathrm{cut}}^{*} = \epsilon$ -- the model retains exactly those empirical eigenvalues exceeding the noise floor -- with the cutoff reflecting a marginal-rate balance between model-error gain and data-bias cost. A boundary comparison further yields a three-regime phase diagram -- retain-all, interior, and collapse -- separated by the lower Marchenko--Pastur edge and an analytically computable collapse threshold $\epsilon_{*}(\alpha)$, where $\alpha$ is the dimension-to-sample-size ratio. All claims are verified numerically.

stat.ML↗

arXiv2026

Multi-Mode Quantum Annealing for Generative Representation Learning with Boltzmann Priors

Energy-based models provide a natural bridge between statistical physics and machine learning by representing data through structured energy landscapes. Boltzmann machines are a particularly compelling class of such models for capturing complex interactions among latent variables, but their use in modern generative learning has been limited by the classical intractability of sampling from general (non-restricted) Boltzmann distributions. Here we develop a quantum-annealing-based framework that enables variational autoencoders with general Boltzmann priors. The framework employs three complementary annealing modes tailored to different stages of learning and deployment: diabatic quantum annealing provides unbiased Boltzmann samples for efficient training, slower annealing concentrates samples near low-energy configurations of the learned prior for unconditional generation, and conditional annealing with external fields steers the learned energy landscape toward attribute-specific regions for conditional generation and semantic editing. Using up to 2000 qubits on a D-Wave Advantage2 processor, we demonstrate stable training and high-quality generation on MNIST, Fashion-MNIST, and CelebA, achieving faster convergence and lower reconstruction loss than a Gaussian-prior VAE with the same encoder-decoder architecture. Beyond generation, the learned energy function provides out-of-distribution detection signals that add discriminative power beyond reconstruction loss. We demonstrate that these scores separate in-distribution samples from held-out digit classes in one-class MNIST experiments and improve the detection of market regime shifts in financial data. These results establish quantum annealing as a practical and controllable physical mechanism for energy-based representation learning and generative modeling beyond the reach of tractable classical approaches.

quant-ph↗

arXiv2025

Diabatic quantum annealing for training energy-based generative models

Energy-based generative models, such as restricted Boltzmann machines (RBMs), require unbiased Boltzmann samples for effective training. Classical Markov chain Monte Carlo methods, however, converge slowly and yield correlated samples, making large-scale training difficult. We address this bottleneck by applying the analytic relation between annealing schedules and effective inverse temperature in diabatic quantum annealing. By implementing this prescription on a quantum annealer, we obtain temperature-controlled Boltzmann samples that enable RBM training with faster convergence and lower validation error than classical sampling. We further identify a systematic temperature misalignment intrinsic to analog quantum computers and propose an analytical rescaling method that mitigates this hardware noise, thereby enhancing the practicality of quantum annealers as Boltzmann samplers. In our method, the model's connectivity is set directly by the qubit connectivity, transforming the computational complexity inherent in classical sampling into a requirement on quantum hardware. This shift allows the approach to extend naturally from RBMs to fully connected Boltzmann machines, opening opportunities inaccessible to classical training methods.

quant-ph↗

arXiv2024

Boltzmann Sampling by Diabatic Quantum Annealing

Boltzmann sampling is a central component of many computational frameworks, including numerous algorithms in machine learning. Although quantum annealers have been investigated as potential fast Boltzmann samplers, their dependence on environmental noise makes precise control of the effective temperature difficult, introducing uncertainty into the sampling process. As an alternative, we propose diabatic quantum annealing -- a faster, purely unitary process -- as a controllable Boltzmann sampler in which the effective temperature is determined by the annealing rate. Using the ferromagnetic Ising model and the Sherrington--Kirkpatrick model as test cases, we demonstrate that this method achieves rapid and accurate sampling in the high-temperature regime.

cond-mat.stat-mech↗

arXiv2023

Tradeoff of generalization error in unsupervised learning

Finding the optimal model complexity that minimizes the generalization error (GE) is a key issue of machine learning. For the conventional supervised learning, this task typically involves the bias-variance tradeoff: lowering the bias by making the model more complex entails an increase in the variance. Meanwhile, little has been studied about whether the same tradeoff exists for unsupervised learning. In this study, we propose that unsupervised learning generally exhibits a two-component tradeoff of the GE, namely the model error and the data error -- using a more complex model reduces the model error at the cost of the data error, with the data error playing a more significant role for a smaller training dataset. This is corroborated by training the restricted Boltzmann machine to generate the configurations of the two-dimensional Ising model at a given temperature and the totally asymmetric simple exclusion process with given entry and exit rates. Our results also indicate that the optimal model tends to be more complex when the data to be learned are more complex.

cond-mat.stat-mech↗