arXiv subjects

Masahito Ueda

Publications and source records attributed to Masahito Ueda.

At least 55 records · Page 3Linked to original sources

arXiv2022

Stochastic Neural Networks with Infinite Width are Deterministic

This work theoretically studies stochastic neural networks, a main type of neural network in use. We prove that as the width of an optimized stochastic neural network tends to infinity, its predictive variance on the training set decreases to zero. Our theory justifies the common intuition that adding stochasticity to the model can help regularize the model by introducing an averaging effect. Two common examples that our theory can be relevant to are neural networks with dropout and Bayesian latent variable models in a special limit. Our result thus helps better understand how stochasticity affects the learning of neural networks and potentially design better architectures for practical problems.

cs.LG↗

arXiv2022

Universal properties of dissipative Tomonaga-Luttinger liquids: Case study of a non-Hermitian XXZ spin chain

We demonstrate the universal properties of dissipative Tomonaga-Luttinger (TL) liquids by calculating correlation functions and performing finite-size scaling analysis of a non-Hermitian XXZ spin chain as a prototypical model in one-dimensional open quantum many-body systems. Our analytic calculation is based on effective field theory with bosonization, finite-size scaling approach in conformal field theory, and the Bethe-ansatz solution. Our numerical analysis is based on the density-matrix renormalization group generalized to non-Hermitian systems (NH-DMRG). We uncover that the model in the massless regime with weak dissipation belongs to the universality class characterized by the complex-valued TL parameter, which is related to the complex generalization of the $c=1$ conformal field theory. As the dissipation strength increases, the values of the TL parameter obtained by the NH-DMRG begin to deviate from those obtained by the Bethe-ansatz analysis, indicating that the model becomes massive for strong dissipation. Our results can be tested with the two-component Bose-Hubbard system of ultracold atoms subject to two-body loss.

cond-mat.quant-gas↗

arXiv2022

Convergent and Efficient Deep Q Network Algorithm

Despite the empirical success of the deep Q network (DQN) reinforcement learning algorithm and its variants, DQN is still not well understood and it does not guarantee convergence. In this work, we show that DQN can indeed diverge and cease to operate in realistic settings. Although there exist gradient-based convergent methods, we show that they actually have inherent problems in learning dynamics which cause them to fail even in simple tasks. To overcome these problems, we propose a convergent DQN algorithm (C-DQN) that is guaranteed to converge and can work with large discount factors (0.9998). It learns robustly in difficult settings and can learn several difficult games in the Atari 2600 benchmark that DQN fails to solve. Our codes have been publicly released and can be used to reproduce our results.

cs.LG↗

arXiv2022

Eigenstate Thermalization in Long-Range Interacting Systems

Motivated by recent ion experiments on tunable long-range interacting quantum systems [B.Neyenhuis et al., Sci.Adv.3, e1700672 (2017, https://doi.org/10.1126/sciadv.1700672 )], we test the strong eigenstate thermalization hypothesis (ETH) for systems with power-law interactions $\sim 1/r^α$. We numerically demonstrate that the strong ETH typically holds at least for systems with $α\geq 0.6$, which include Coulomb, monopole-dipole, and dipole-dipole interactions. Compared with short-range interacting systems, the eigenstate expectation value of a generic local observable is shown to deviate significantly from its microcanonical ensemble average for long-range interacting systems. We find that Srednicki's ansatz breaks down for $α\lesssim 1.0$ at least for relatively large system sizes.

cond-mat.stat-mech↗

arXiv2022

Vortex lattices in binary Bose-Einstein condensates: Collective modes, quantum fluctuations, and intercomponent entanglement

We study binary Bose-Einstein condensates subject to synthetic magnetic fields in mutually parallel or antiparallel directions. Within the mean-field theory, the two types of fields have been shown to give the same vortex-lattice phase diagram. We develop an improved effective field theory to study properties of collective modes and ground-state intercomponent entanglement. Here, we point out the importance of introducing renormalized coupling constants for coarse-grained densities. We show that the low-energy excitation spectra for the two kindsof fields are related to each other by suitable rescaling using the renormalized constants. By calculating the entanglement entropy, we find that for an intercomponent repulsion (attraction), the two components are more strongly entangled in the case of parallel (antiparallel) fields, in qualitative agreement with recent studies for a quantum (spin) Hall regime. We also find that the entanglement spectrum exhibits an anomalous square-root dispersion relation, which leads to a subleading logarithmic term in the entanglement entropy. All of these are confirmed by numerical calculations based on the Bogoliubov theory with the lowest-Landau-level approximation. Finally, we investigate the effects of quantum fluctuations on the phase diagrams by calculating the correction to the ground-state energy due to zero-point fluctuations in the Bogoliubov theory. We find that the boundaries between rhombic-, square-, and rectangular-lattice phases shift appreciably with a decrease in the filling factor.

cond-mat.quant-gas↗

arXiv2022

Strength of Minibatch Noise in SGD

The noise in stochastic gradient descent (SGD), caused by minibatch sampling, is poorly understood despite its practical importance in deep learning. This work presents the first systematic study of the SGD noise and fluctuations close to a local minimum. We first analyze the SGD noise in linear regression in detail and then derive a general formula for approximating SGD noise in different types of minima. For application, our results (1) provide insight into the stability of training a neural network, (2) suggest that a large learning rate can help generalization by introducing an implicit regularization, (3) explain why the linear learning rate-batchsize scaling law fails at a large learning rate or at a small batchsize and (4) can provide an understanding of how discrete-time nature of SGD affects the recently discovered power-law phenomenon of SGD.

cs.LG↗

arXiv2022

Power-law escape rate of SGD

Stochastic gradient descent (SGD) undergoes complicated multiplicative noise for the mean-square loss. We use this property of SGD noise to derive a stochastic differential equation (SDE) with simpler additive noise by performing a random time change. Using this formalism, we show that the log loss barrier $Δ\log L=\log[L(θ^s)/L(θ^*)]$ between a local minimum $θ^*$ and a saddle $θ^s$ determines the escape rate of SGD from the local minimum, contrary to the previous results borrowing from physics that the linear loss barrier $ΔL=L(θ^s)-L(θ^*)$ decides the escape rate. Our escape-rate formula strongly depends on the typical magnitude $h^*$ and the number $n$ of the outlier eigenvalues of the Hessian. This result explains an empirical fact that SGD prefers flat minima with low effective dimensions, giving an insight into implicit biases of SGD.

cs.LG↗

arXiv2022

Interplay between depth of neural networks and locality of target functions

It has been recognized that heavily overparameterized deep neural networks (DNNs) exhibit surprisingly good generalization performance in various machine-learning tasks. Although benefits of depth have been investigated from different perspectives such as the approximation theory and the statistical learning theory, existing theories do not adequately explain the empirical success of overparameterized DNNs. In this work, we report a remarkable interplay between depth and locality of a target function. We introduce $k$-local and $k$-global functions, and find that depth is beneficial for learning local functions but detrimental to learning global functions. This interplay is not properly captured by the neural tangent kernel, which describes an infinitely wide neural network within the lazy learning regime.

cs.LG↗

arXiv2021

Physical Origin of the Universal Three-body Parameter in Atomic Efimov Physics

We address the microscopic origin of the universal three-body parameter that fixes the spectrum of three-atom systems in the Efimov regime. We identify it with the van der Waals two-body correlation, which causes the three-atom system to deform when the three atoms come within the distance of the van der Waals length, effectively preventing them from coming closer due to the kinetic-energy cost associated with this three-body deformation. This deformation mechanism explains the universal ratio of the scattering length at the triatomic resonance to the van der Waals length observed in several experiments and confirmed by numerical calculations.

cond-mat.quant-gas↗

arXiv2021

Liouvillian Skin Effect: Slowing Down of Relaxation Processes without Gap Closing

It is highly nontrivial to what extent we can deduce the relaxation behavior of a quantum dissipative system from the spectral gap of the Liouvillian that governs the time evolution of the density matrix. We investigate the relaxation processes of a quantum dissipative system that exhibits the Liouvillian skin effect, which means that the eigenmodes of the Liouvillian are localized exponentially close to the boundary of the system, and find that the timescale for the system to reach a steady state depends not only on the Liouvillian gap $Δ$ but also on the localization length $ξ$ of the eigenmodes. In particular, we show that the longest relaxation time $τ$ that is maximized over initial states and local observables is given by $τ\sim Δ^{-1}(1+L/ξ)$ with $L$ being the system size. This implies that the longest relaxation time can diverge for $L \to \infty$ without gap closing.

cond-mat.stat-mech↗

arXiv2021

Collective Excitations and Nonequilibrium Phase Transition in Dissipative Fermionic Superfluids

We predict a new mechanism to induce collective excitations and a nonequilibrium phase transition of fermionic superfluids via a sudden switch-on of two-body loss, for which we extend the BCS theory to fully incorporate a change in particle number. We find that a sudden switch-on of dissipation induces an amplitude oscillation of the superfluid order parameter accompanied by a chirped phase rotation as a consequence of particle loss. We demonstrate that when dissipation is introduced to one of the two superfluids coupled via a Josephson junction, it gives rise to a nonequilibrium dynamical phase transition characterized by the vanishing dc Josephson current. The dissipation-induced collective modes and nonequilibrium phase transition can be realized with ultracold fermionic atoms subject to inelastic collisions.

cond-mat.quant-gas↗

arXiv2021

LaProp: Separating Momentum and Adaptivity in Adam

We identity a by-far-unrecognized problem of Adam-style optimizers which results from unnecessary coupling between momentum and adaptivity. The coupling leads to instability and divergence when the momentum and adaptivity parameters are mismatched. In this work, we propose a method, Laprop, which decouples momentum and adaptivity in the Adam-style methods. We show that the decoupling leads to greater flexibility in the hyperparameters and allows for a straightforward interpolation between the signed gradient methods and the adaptive gradient methods. We experimentally show that Laprop has consistently improved speed and stability over Adam on a variety of tasks. We also bound the regret of Laprop on a convex problem and show that our bound differs from that of Adam by a key factor, which demonstrates its advantage.

cs.LG↗

arXiv2021

Noise and Fluctuation of Finite Learning Rate Stochastic Gradient Descent

In the vanishing learning rate regime, stochastic gradient descent (SGD) is now relatively well understood. In this work, we propose to study the basic properties of SGD and its variants in the non-vanishing learning rate regime. The focus is on deriving exactly solvable results and discussing their implications. The main contributions of this work are to derive the stationary distribution for discrete-time SGD in a quadratic loss function with and without momentum; in particular, one implication of our result is that the fluctuation caused by discrete-time dynamics takes a distorted shape and is dramatically larger than a continuous-time theory could predict. Examples of applications of the proposed theory considered in this work include the approximation error of variants of SGD, the effect of minibatch noise, the optimal Bayesian inference, the escape rate from a sharp minimum, and the stationary covariance of a few second-order methods including damped Newton's method, natural gradient descent, and Adam.

stat.ML↗

arXiv2021

$η$ Pairing of Light-Emitting Fermions: Nonequilibrium Pairing Mechanism at High Temperatures

Strongly interacting fermionic atoms are shown to develop $η$-pairing superfluid correlations in a nonequilibrium steady state in the presence of spontaneous emission of light from atoms. On the basis of the Hubbard model subject to spontaneous decay between internal spin states, we show that prohibition of radiative decay due to the Pauli exclusion principle and destructive interference between doublon-decay processes lead to nonequilibrium $η$ pairing. Because of the non-thermal nature of the steady state, pair correlations arise even from a completely uncorrelated infinite-temperature initial state, allowing coherent atom pairs to be formed at high temperatures. Experimental implementation with fermionic atoms in an optical lattice is discussed.

cond-mat.quant-gas↗

arXiv2021

Exact Liouvillian Spectrum of a One-Dimensional Dissipative Hubbard Model

A one-dimensional dissipative Hubbard model with two-body loss is shown to be exactly solvable. We obtain an exact eigenspectrum of a Liouvillian superoperator by employing a non-Hermitian extension of the Bethe-ansatz method. We find steady states, the Liouvillian gap, and an exceptional point that is accompanied by the divergence of the correlation length. A dissipative version of spin-charge separation induced by the quantum Zeno effect is also demonstrated. Our result presents a new class of exactly solvable Liouvillians of open quantum many-body systems, which can be tested with ultracold atoms subject to inelastic collisions.

cond-mat.quant-gas↗

arXiv2021

Intercomponent entanglement entropy and spectrum in binary Bose-Einstein condensates

We study the entanglement entropy and spectrum between components in binary Bose-Einstein condensates in $d$ spatial dimensions. We employ effective field theory to show that the entanglement spectrum exhibits an anomalous square-root dispersion relation in the presence of an intercomponent tunneling (a Rabi coupling) and a gapped dispersion relation in its absence. These spectral features are associated with the emergence of long-range interactions in terms of the superfluid velocity and the particle density in the entanglement Hamiltonian. Our results demonstrate that unusual long-range interactions can be emulated in a subsystem of multicomponent BECs that have only short-range interactions. We also find that for a finite Rabi coupling the entanglement entropy exhibits a volume-law scaling with subleading logarithmic corrections originating from the Nambu-Goldstone mode and the symmetry restoration for a finite volume.

cond-mat.quant-gas↗

arXiv2021

Tachyonic and Plasma Instabilities of $η$-Pairing States Coupled to Electromagnetic Fields

Cooper pairs featuring a nonzero center-of-mass crystal momentum ${\boldsymbol Q}=(π,π, \dots)$ and an off-diagonal long-range order ($η$-pairing states) constitute exact eigenstates of a Hubbard model [C. N. Yang, Phys. Rev. Lett. 63, 2144 (1989)]. Here we show that the $η$-pairing states are rendered dynamically unstable via coupling to dynamical electromagnetic fields. The instability is caused by "tachyonic" electromagnetic fields and unstable plasma modes for attractive and repulsive interactions, respectively. The typical time scale of the growth of the instability is of the order of femtoseconds for electron systems in solids, which places a strict bound on the lifetime of the $η$-pairing states. The decay of the $η$-pairing states leads to enhanced light emission with frequencies shifted from the Hubbard interaction strength for the repulsive case and unattenuated electromagnetic penetration for the attractive case.

cond-mat.supr-con↗

arXiv2021

Is deeper better? It depends on locality of relevant features

It has been recognized that a heavily overparameterized artificial neural network exhibits surprisingly good generalization performance in various machine-learning tasks. Recent theoretical studies have made attempts to unveil the mystery of the overparameterization. In most of those previous works, the overparameterization is achieved by increasing the width of the network, while the effect of increasing the depth has remained less well understood. In this work, we investigate the effect of increasing the depth within an overparameterized regime. To gain an insight into the advantage of depth, we introduce local and global labels as abstract but simple classification rules. It turns out that the locality of the relevant feature for a given classification rule plays a key role; our experimental results suggest that deeper is better for local labels, whereas shallower is better for global labels. We also compare the results of finite networks with those of the neural tangent kernel (NTK), which is equivalent to an infinitely wide network with a proper initialization and an infinitesimal learning rate. It is shown that the NTK does not correctly capture the depth dependence of the generalization performance, which indicates the importance of the feature learning rather than the lazy learning.

cs.LG↗