arXiv subjects

Naoki Kiyohara

Publications and source records attributed to Naoki Kiyohara.

5 recordsLinked to original sources

arXiv2026

Interdomain Attention: Beyond Token-Level Key-Value Memory

Transformers and deep state space models (SSMs) sit at opposite ends of a basic design choice: attention routes each query through a growing key-value (KV) cache by content-based matching at quadratic cost, while deep SSMs compress context into a fixed-size recurrent state that is not directly addressed by query-key matching. We propose Interdomain Attention, which integrates an SSM into an attention module through kernel methods: an attention kernel is approximated by a finite feature map, the resulting key features and values are projected onto a shared set of basis functions maintained by a single SSM recurrence, and each query attends to the compressed coefficients through its own feature map, recovering query-conditioned attention over a fixed-size state. The scalable layer is a learned relaxation of this derivation, and we validate its components through ablations. In a 125M to 1.3B autoregressive language-modeling study on FineWeb-Edu at matched recurrent-state budget, Interdomain Attention improves on an SSM token mixer at every scale, surpasses a same-recipe softmax baseline at 1.3B on validation perplexity and on the eight-task commonsense suite, and inherits the length-flat behavior of its fixed-state core out to 3.5x the training context. Ablations indicate that the query-conditioned projection is the main source of the gain.

cs.LG↗

arXiv2025

Neural Stochastic Flows: Solver-Free Modelling and Inference for SDE Solutions

Stochastic differential equations (SDEs) are well suited to modelling noisy and irregularly sampled time series found in finance, physics, and machine learning. Traditional approaches require costly numerical solvers to sample between arbitrary time points. We introduce Neural Stochastic Flows (NSFs) and their latent variants, which directly learn (latent) SDE transition laws using conditional normalising flows with architectural constraints that preserve properties inherited from stochastic flows. This enables one-shot sampling between arbitrary states and yields up to two orders of magnitude speed-ups at large time gaps. Experiments on synthetic SDE simulations and on real-world tracking and video data show that NSFs maintain distributional accuracy comparable to numerical approaches while dramatically reducing computation for arbitrary time-point sampling.

cs.LG↗

arXiv2025

Recurrent Memory for Online Interdomain Gaussian Processes

We propose a novel online Gaussian process (GP) model that is capable of capturing long-term memory in sequential data in an online learning setting. Our model, Online HiPPO Sparse Variational Gaussian Process (OHSVGP), leverages the HiPPO (High-order Polynomial Projection Operators) framework, which is popularized in the RNN domain due to its long-range memory modeling capabilities. We interpret the HiPPO time-varying orthogonal projections as inducing variables with time-dependent orthogonal polynomial basis functions, which allows the SVGP inducing variables to memorize the process history. We show that the HiPPO framework fits naturally into the interdomain GP framework and demonstrate that the kernel matrices can also be updated online in a recurrence form based on the ODE evolution of HiPPO. We evaluate OHSVGP with online prediction for 1D time series, continual learning in discriminative GP model for data with multidimensional inputs, and deep generative modeling with sparse Gaussian process variational autoencoder, showing that it outperforms existing online GP methods in terms of predictive performance, long-term memory preservation, and computational efficiency.

cs.LG↗

arXiv2019

Anomalous thermal Hall effect in the topological antiferromagnetic state

The anomalous Hall effect (AHE), a Hall signal occurring without an external magnetic field, is one of the most significant phenomena. However, understanding the AHE mechanism has been challenging and largely restricted to ferromagnetic metals. Here, we investigate the recently discovered AHE in the chiral antiferromagnet Mn3Sn by measuring a thermal analog of the AHE, known as an anomalous thermal Hall effect (ATHE). The amplitude of the ATHE scales with the anomalous Hall conductivity of Mn3Sn over a wide temperature range, demonstrating that the AHE of Mn3Sn arises from a dissipationless intrinsic mechanism associated with the Berry curvature. Moreover, we find that the dissipationless AHE is significantly stabilized by shifting the Fermi level toward the magnetic Weyl points. Thus, in Mn3Sn, the Berry curvature emerging from the proposed magnetic Weyl fermion state is a key factor for the observed AHE and ATHE.

cond-mat.str-el↗

arXiv2016

Giant Anomalous Hall Effect in the Chiral Antiferromagnet Mn3Ge

The external field control of antiferromagnetism is a significant subject both for basic science and technological applications. As a useful macroscopic response to detect magnetic states, the anomalous Hall effect (AHE) is known for ferromagnets, but it has never been observed in antiferromagnets until the recent discovery in Mn3Sn. Here we report another example of the AHE in a related antiferromagnet, namely, in the hexagonal chiral antiferromagnet Mn3Ge. Our single-crystal study reveals that Mn3Ge exhibits a giant anomalous Hall conductivity $|σ_{xz}|$ ~ 60 per ohm per cm at room temperature and approximately 380 per ohm per cm at 5 K in zero field, reaching nearly half of the value expected for the quantum Hall effect per atomic layer with Chern number of unity. Our detailed analyses on the anisotropic Hall conductivity indicate that in comparison with the in-plane-field components $|σ_{xz}|$ and $|σ_{zy}|$, which are very large and nearly comparable in size, we find $|σ_{yx}|$ obtained in the field along the c axis is found to be much smaller. The anomalous Hall effect shows a sign reversal with the rotation of a small magnetic field less than 0.1 T. The soft response of the AHE to magnetic field should be useful for applications, for example, to develop switching and memory devices based on antiferromagnets.

cond-mat.str-el↗