arXiv subjects

Hisato Komatsu

Publications and source records attributed to Hisato Komatsu.

17 recordsLinked to original sources

arXiv2026

Symmetric solution of the Bellman optimality equation for repeated harmony game

In social dilemma games, additional rewards or punishments have been studied as means of promoting cooperation. Therefore, it is important to investigate the ideal situation, in which such an additional payoff would change the game. In this study, we investigated the symmetric solution of the Bellman optimality equation for a repeated harmony game. The calculations showed that three types of symmetric solutions exist. One of them corresponds to the trivial All-C strategy, and another to the Win-stay Lose-shift strategy of the prisoners dilemma game. The nontrivial behavior of the strategy corresponding to the last solution is also discussed in detail. In addition, we numerically investigated which strategy the agents actually learn by the reinforcement learning algorithm.

cs.GT↗

arXiv2025

Application of linear regression and quasi-Newton methods to the deep reinforcement learning in continuous action cases

The linear regression (LR) method offers the advantage that optimal parameters can be calculated relatively easily, although its representation capability is limited than that of the deep learning technique. To improve deep reinforcement learning, the Least Squares Deep Q Network (LS-DQN) method was proposed by Levine et al., which combines Deep Q Network (DQN) with LR method. However, the LS-DQN method assumes that the actions are discrete. In this study, we propose the Double Least Squares Deep Deterministic Policy Gradient (DLS-DDPG) method to address this limitation. This method combines the LR method with the Deep Deterministic Policy Gradient (DDPG) technique, one of the representative deep reinforcement learning algorithms for continuous action cases. For the LR update of the critic network, DLS-DDPG uses an algorithm similar to the Fitted Q iteration, the method which LS-DQN adopted. In addition, we calculated the optimal action using the quasi-Newton method and used it as both the agent's action and the training data for the LR update of the actor network. Numerical experiments conducted in MuJoCo environments showed that the proposed method improved performance at least in some tasks, although there are difficulties such as the inability to make the regularization terms small.

cs.LG↗

arXiv2024

Multi-agent reinforcement learning using echo-state network and its application to pedestrian dynamics

In recent years, simulations of pedestrians using the multi-agent reinforcement learning (MARL) have been studied. This study considered the roads on a grid-world environment, and implemented pedestrians as MARL agents using an echo-state network and the least squares policy iteration method. Under this environment, the ability of these agents to learn to move forward by avoiding other agents was investigated. Specifically, we considered two types of tasks: the choice between a narrow direct route and a broad detour, and the bidirectional pedestrian flow in a corridor. The simulations results indicated that the learning was successful when the density of the agents was not that high.

cs.MA↗

arXiv2023

Transition between the stick and slip states in a simplified model of magnetic friction

We introduce a simplified model of magnetic friction, and investigate its behavior using both numerical and analytical methods. When resistance coefficient $γ$ is large, the movement of the system obeys the thermally activated process. In contrast, when $γ$ is sufficiently small, the slip and stick states behave as separate metastable states, and the lattice velocity depends on the probability that the slip state appears. We evaluate the velocities in both cases using several approximations and compare the results with those of numerical simulations.

cond-mat.stat-mech↗

arXiv2023

Analysis of the finite-size effect of the long-range Ising model under Glauber dynamics

We considered a long-range Ising model under Glauber dynamics and calculated the difference from the mean-field approximation in a finite-size system using perturbation theory. To deal with the BBGKY hierarchy, we assumed that certain types of extensive properties have a Gaussian distribution, which turned out to be equivalent to the Kirkwood superposition approximation within the range of first-order perturbation. After several calculations, ordinary differential equations that describe the time development of a two-body correlation were derived. This discussion is the generalization of our previous study which developed a similar consideration on the infinite-range Ising model. The results of the calculation fit those of the numerical simulations for the case in which the decay of the interaction was sufficiently slow; however, they exhibited different behaviors when the decay became rapid.

cond-mat.stat-mech↗

arXiv2022

Phase diagram of the dipolar Ising ferromagnet on a kagome lattice

We study the field--temperature phase diagram of the two-dimensional dipolar Ising ferromagnet on a kagome lattice with a specific ratio between the exchange and dipolar constants, $δ= 1$. Using the stochastic cutoff (SCO) $O(N)$ Monte Carlo method, we calculated order parameters for stripe and bubble phases and other thermodynamical quantities. We find two kinds of stripe phases at low fields, where the arrangement of the branch spins neighboring the stripe frame varies, and two bubble phases at high fields, in which three-spin domains (bubbles) form a regular triangular lattice but the triangular array of bubbles changes on a kagome lattice. We also find that with increasing the field, there exist a disordered phase between the stripe and bubble phases and between the two bubble phases. We discuss the details of the features of these phases and phase transitions.

cond-mat.stat-mech↗

arXiv2021

Analysis of finite-size effect of infinite-range Ising model under Glauber dynamics

We consider an infinite-range Ising model under the Glauber dynamics and determine the finite-size effect on the distribution of two spin variables as a perturbation of $O \left( 1/N \right)$. Based on several considerations, ordinary differential equations are derived for describing the time development of both a two-body correlation and the autocorrelation function of magnetization. The results of the calculation fit the simulation results, unless the perturbation theory breaks down because of critical phenomena or magnetization reversal.

cond-mat.stat-mech↗

arXiv2021

Relaxation process of magnetic friction under sudden changes in velocity

Although there have been many studies of statistical mechanical models of magnetic friction, most of these have focused on the behavior in the steady state. In this study, we prepare a system composed of a chain and a lattice of Ising spins that interact with each other, and investigate the relaxation of the system when the relative velocity $v$ changes suddenly. The situation where $v$ is given is realized by attaching the chain to a spring, the other end of which moves with a constant velocity $v$. Numerical simulation finds that, when the spring constant has a moderate value, the relaxation of the frictional force is divided into two processes, which are a sudden change and a slow relaxation. This behavior is also observed on regular solid surfaces, although caused by different factors than our model. More specifically, the slow relaxation process is caused by relaxation of the magnetic structure in our model, but is caused by creep deformation in regular solid surfaces.

cond-mat.stat-mech↗

arXiv2021

Phase diagram of the two-dimensional dipolar Heisenberg model with the Dzyaloshinskii-Moriya interaction and the Ising anisotropy

We study phase transitions in the two-dimensional Heisenberg model with the Dzyaloshinskii-Moriya interaction, the Ising anisotropy ($η$), and the dipolar interaction under zero and finite magnetic fields ($H$). For three typical strengths (zero, weak, and strong) of the dipolar interaction, we present the $H$-$η$ phase diagrams by estimating order parameters for skyrmion-lattice and helical phases and in-plane magnetization by using a Monte Carlo method with an $O(N)$ algorithm. We find in the phase diagrams three types of skyrmion-lattice phases, i.e., two square lattices and a triangular lattice, helical phases with diagonal and vertical (or horizontal) stripes, canted ferromagnetic phase and polarized ferromagnetic phase. The effect of the dipolar interaction varies the types of the skyrmion and helical phases in a complex manner. The dipolar interaction also expands the regions of the ordered phases accompanying shifts of the phase boundaries to the positive $H$ and $η$ directions, and causes increase of the density of skyrmions and shortening of the pitch length (stripe width) of helical structures. We discuss the details of the features of the phase transitions.

cond-mat.stat-mech↗

arXiv2020

A model of magnetic friction with the infinite-range interaction

We investigate a model of magnetic friction with the infinite-range interaction by mean field analysis and a numerical simulation, and compare its behavior with that of the short-range model that we considered previously [H.~Komatsu, Phys.\ Rev.\ E.\ \textbf{100}, 052130 (2019)]. This infinite-range model always obeys the Stokes law when the temperature is higher than the critical value, $T_c$, whereas it shows a crossover or transition from the Dieterich--Ruina law to the Stokes law when the temperature is lower than $T_c$. Considering that the short-range model in our previous study shows a crossover or transition irrespective of whether the temperature is above or below the equilibrium transition temperature, the behavior in the high-temperature state is the major difference between these two models.

cond-mat.stat-mech↗

arXiv2019

A model of magnetic friction obeying the Dieterich--Ruina law in the steady state

We propose a model of magnetic friction and investigate the relation between the frictional force and the relative velocity of surfaces in the steady state. The model comprises two square lattices adjacent to each other, the upper of which is subjected to an external force, and the magnetic interaction acts as a kind of "potential barrier" that prevents the upper lattice from moving. We consider two surface types for the upper lattice: smooth and rough. The behavior of this model is classified into two domains, which we refer to as domains I and II. In domain II, the external force is dominant compared with other forces, whereas in the domain I, the the velocity of the lattice is suppressed by the magnetic interaction and obeys the Dieterich--Ruina law. This characteristic property can be observed regardless of whether the surface is smooth or rough.

cond-mat.stat-mech↗

arXiv2019

Anisotropy-temperature phase diagram for the two-dimensional dipolar Heisenberg model with and without magnetic field

We investigate phase transitions in the two-dimensional dipolar Heisenberg model with uniaxial anisotropy with a specific ratio between the exchange and dipolar constants, $δ=1$. We obtain the $η$--$T$ (anisotropy vs. temperature) phase diagrams for typical values of magnetic field by a Monte Carlo method with an $O(N)$ algorithm. We find that at lower fields, the $η$--$T$ phase diagram consists of the planar ferromagnetic (F), (perpendicular) stripe-ordered (SO), and paramagnetic (P) phases, and is characterized by the triple point. In the SO phase realized at larger $η$ and smaller $T$, the SO pattern changes depending on the field. On the other hand, we find that at higher fields, the SO phase does not exist, while the planer F phase robustly remains. We study the properties of the phase boundaries by employing finite-size-scaling analyses. We find that the slope of the spin-reorientation-transition line is positive with and without field, i.e., $\frac{dη}{dT}>0$, which implies that the planar F phase changes to the SO phase with lowering temperature. In the phase diagrams we observe a characteristic shape of the P--planer F phase-transition line, whose maximum point of $η$ is located at an intermediate temperature. This structure leads to the temperature-induced reentrant transition associated with P and planar F phases, which appears in successive phase transitions with lowering temperature: P $\rightarrow$ planar F $\rightarrow$ P $\rightarrow$ SO phase at lower fields and P $\rightarrow$ planar F $\rightarrow$ P phases at higher fields.

cond-mat.stat-mech↗

arXiv2018

Temperature-field phase diagram of the two-dimensional dipolar Ising ferromagnet

We study field-induced phase transitions in the two-dimensional dipolar Ising ferromagnet with a specific ratio between the exchange and dipolar constants, $δ=1$, which exhibits a stripe-ordered phase with the width of one lattice unit at low temperatures without magnetic field. By using a mean-field (MF) approximation and a Monte Caro (MC) method with the stochastic-cutoff algorithm, which is an $O(N)$ simulation method, we show the temperature-field phase diagrams. In the MC study the orientational order and the structure factor are evaluated. Second-order transition points are determined by a finite-size-scaling analysis and first-order transition points are identified by the analysis of the energy histogram. Although both the MF and MC phase diagrams consist of wide regions of several stripe-ordered phases and narrow regions between them characterized by complicated stripe patterns, they show qualitative and quantitative differences in possible phases and phase boundaries. In the MF phase diagram, three main stripe-ordered phases exhibit a nesting structure, while in the MC phase diagram, two main stripe-ordered phases are located separately, which causes a characteristic field-induced reentrant transition of the orientational order.

cond-mat.stat-mech↗

arXiv2018

A "mean-field approximation" on the phase transitions of three-dimensional Lennard-Jones model

It is difficult to derive the solid-fluid transition theoretically from microscopic models, although this phenomenon itself has been investigated for a long time. We previously constructed an exactly-solvable model with the solid-fluid transition. This model resembles the infinite-range (or mean-field) model in spin systems in some points, hence it can be called a "mean-field model" of the solid-fluid transition. In the present paper, we construct a "mean-field approximation" of the solid-fluid transition by using the "mean-field model" introduced in our previous study, and tries to describe the phase transitions of the three-dimensional Lennard-Jones model as an example. This approximation succeeds in describing the phase diagram which contains three (gas, liquid, and fcc-solid) phase, at least qualitatively.

cond-mat.stat-mech↗

arXiv2017

Generalization of exactly-solvable model to exhibit solid-fluid phase transition in crystal structures with two particles in a primitive cell

In our previous paper [H. K., J.Stat.Mech.(2015) P08020], we investigated an interacting-particle model with infinite-range cosine potentials, and derived the partition function which shows solid-fluid phase transition by exact calculation. However, we could treat only simple lattice structures in which more than one stable point exist in a primitive cell such as the triangular or face-centered cubic lattice. In the present paper, we generalize our previous scheme to more complicated lattice structures with two particles in a primitive cell. Generalization to more complicated lattice structures is straightforward.

cond-mat.stat-mech↗

arXiv2015

Electronic structure of helicoidal graphene: massless Dirac particles on a curved surface with a screw symmetry

Massless Dirac particles on the helicoid are theoretically investigated. With its possible application being helical graphene, we explore how the peculiarities of Dirac particles appear on the curved, screw-symmetric surface. The zweibein is used to derive the massless Dirac equation on the helicoid, as well as general curved surfaces. Bound states of massless Dirac electrons are shown to be absent on the helicoid, and then the scattering probabilities and the phase shifts on the surface are obtained from numerically calculated wave functions. We find the local density of states and the phase shifts idiosyncratic especially around the axis of the helicoid. Bound states of massive Dirac electrons on the surface are also shown to be absent as an extension of the above result on massless Dirac electrons.

cond-mat.mes-hall↗

arXiv2015

Solid-fluid transition of two- or three-dimensional systems with infinite-range interaction

It is difficult to derive the solid--fluid transition from microscopic models. We introduce particle systems whose potentials do not decay with distance and calculate their partition function exactly using a method similar to that for lattice systems with infinite-range interaction. In particular, we investigate the behaviors of examples among these models, which become a triangular, body-centered cubic, face-centered cubic, or simple cubic lattice in low-temperature phase. The transitions of the first three examples are of the first order, and that of the last example is of the second order. Note that we define the solid phase as that whose order parameter, or Fourier component of the density, becomes nonzero, and the models we considered obey the ideal-gas law even in the solid phase.

cond-mat.stat-mech↗