arXiv subjects

Pei Yang

Publications and source records attributed to Pei Yang.

43 records · Page 3Linked to original sources

arXiv2019

Densifying Assumed-sparse Tensors: Improving Memory Efficiency and MPI Collective Performance during Tensor Accumulation for Parallelized Training of Neural Machine Translation Models

Neural machine translation - using neural networks to translate human language - is an area of active research exploring new neuron types and network topologies with the goal of dramatically improving machine translation performance. Current state-of-the-art approaches, such as the multi-head attention-based transformer, require very large translation corpuses and many epochs to produce models of reasonable quality. Recent attempts to parallelize the official TensorFlow "Transformer" model across multiple nodes have hit roadblocks due to excessive memory use and resulting out of memory errors when performing MPI collectives. This paper describes modifications made to the Horovod MPI-based distributed training framework to reduce memory usage for transformer models by converting assumed-sparse tensors to dense tensors, and subsequently replacing sparse gradient gather with dense gradient reduction. The result is a dramatic increase in scale-out capability, with CPU-only scaling tests achieving 91% weak scaling efficiency up to 1200 MPI processes (300 nodes), and up to 65% strong scaling efficiency up to 400 MPI processes (200 nodes) using the Stampede2 supercomputer.

cs.LG↗

arXiv2019

Strong Decays of observed $Λ_c$ Baryons in the $^3P_0$ Model

The strong decay widths and some important branching ratios of possible Okubo-Zweig-Iizuka(OZI)-allowed strong decay channels of $Λ_c(2595)^+$, $Λ_c(2625)^+$, $Λ_c(2765)^+$ ($Σ_c(2765)^+$), $Λ_c(2860)^+$, $Λ_c(2880)^+$ and $Λ_c(2940)^+$ are computed in a $^{3}P_{0}$ model, and possible assignments of these $Λ_c$ are given. (1), $Λ_c(2595)^+$ and $Λ_c(2625)^+$ are possibly the $1P$-wave charmed baryons $Λ_{c1}(\frac{1}{2}^-)$ and $Λ_{c1}(\frac{3}{2}^-)$, respectively. (2), $Λ_c(2765)^+$ ($Σ_c(2765)^+$) seems impossibly the $1P$-wave $Λ_{c}$, it could be the $2S$-wave or $1D$-wave charmed baryon. So far, the experimental information has not been sufficient for its identification. (3), $Λ_c(2860)^+$ seems impossibly $2S$-wave charmed baryon, it may be the $P$-wave $\tildeΛ_{c2}^{ }(\frac{3}{2}^-)$ or $\tildeΛ_{c2}^{ }(\frac{5}{2}^-)$, it could also be the $D$-wave $\checkΛ_{c1}^{2}(\frac{1}{2}^+)$ or $\checkΛ_{c1}^{2}(\frac{3}{2}^+)$. If the hypothesis that $Λ_c(2860)^+$ has $J^P={3\over 2}^+$ is true, $Λ_c(2860)^+$ is possibly the $D$-wave $\checkΛ_{c1}^{2}(\frac{3}{2}^+)$ which has a predicted branching ratio $R=Γ(Σ_c(2520)π)/Γ(Σ_c(2455)π)=2.8$. (4), $Λ_c(2880)^+$ is impossibly a $1P$-wave or $2S$-wave charmed baryon, it may be a $D$-wave $\checkΛ_{c3}^{2}(\frac{5}{2}^+)$ with $Γ_{total}=1.3$ MeV. The predicted branching ratio $R=Γ(Σ_c(2520)π)/Γ(Σ_c(2455)π)=0.35$, which is consistent with experiment. (5), $Λ_c(2940)^+$ is the $P$-wave $\tildeΛ_{c2}^{ }(\frac{3}{2}^-)$ or $\tildeΛ_{c2}^{ }(\frac{5}{2}^-)$, it is also possibly the $D$-wave $\checkΛ_{c3}^{2}(\frac{5}{2}^+)$ or $\checkΛ_{c3}^{2}(\frac{7}{2}^+)$.

hep-ph↗

arXiv2018

Identification of the newly observed $Σ_b(6097)^\pm$ baryons from their strong decays

Two bottom $Σ_b(6097)^\pm$ baryons were observed in the final states $Λ_b^0π^-$ and $Λ_b^0π^+$ in $pp$ collision by LHCb collaboration, whose masses and widths were measured. In a $^{3}P_{0}$ model, the strong decay widths of two ground $S$-wave and seven excited $P$-wave $Σ_b$ baryons have been systematically computed. Numerical results indicate that the newly observed $Σ_b(6097)^\pm$ are very possibly $Σ_{b2}^1({3\over 2}^-)$ with $J^P={3\over 2}^-$ or $Σ_{b2}^1({5\over 2}^-)$ with $J^P={5\over 2}^-$. The predicted decay widths of $Σ_b(6097)^\pm$ are consistent with experimental measurement from LHCb. In particular, it may be possible to distinguish these two assignments through ratios $Γ({Σ_b(6097)^\pm\to Σ_b^\pmπ^0})/Γ({Σ_b(6097)^\pm\to Σ_b^{*\pm}π^0})$, which can be measured by experiments in the future. In the meantime, our results support the assignments that $Σ_b^\pm$ and $Σ_b^{*\pm}$ are the ground $S$-wave $Σ_b$ baryons with $J^P={1\over 2}^+$ and $J^P={3\over 2}^+$, respectively.

hep-ph↗

arXiv2018

Role of dimensional crossover on spin-orbit torque efficiency in magnetic insulator thin films

Magnetic insulators (MIs) attract tremendous interest for spintronic applications due to low Gilbert damping and absence of Ohmic loss. Magnetic order of MIs can be manipulated and even switched by spin-orbit torques (SOTs) generated through spin Hall effect and Rashba-Edelstein effect in heavy metal/MI bilayers. SOTs on MIs are more intriguing than magnetic metals since SOTs cannot be transferred to MIs through direct injection of electron spins. Understanding of SOTs on MIs remains elusive, especially how SOTs scale with the film thickness. Here, we observe the critical role of dimensionality on the SOT efficiency by systematically studying the MI layer thickness dependent SOT efficiency in tungsten/thulium iron garnet (W/TmIG) bilayers. We first show that the TmIG thin film evolves from two-dimensional to three-dimensional magnetic phase transitions as the thickness increases, due to the suppression of long-wavelength thermal fluctuation. Then, we report the significant enhancement of the measured SOT efficiency as the thickness increases. We attribute this effect to the increase of the magnetic moment density in concert with the suppression of thermal fluctuations. At last, we demonstrate the current-induced SOT switching in the W/TmIG bilayers with a TmIG thickness up to 15 nm. The switching current density is comparable with those of heavy metal/ferromagnetic metal cases. Our findings shed light on the understanding of SOTs in MIs, which is important for the future development of ultrathin MI-based low-power spintronics.

cond-mat.mes-hall↗

arXiv2017

An Effective Way to Improve YouTube-8M Classification Accuracy in Google Cloud Platform

Large-scale datasets have played a significant role in progress of neural network and deep learning areas. YouTube-8M is such a benchmark dataset for general multi-label video classification. It was created from over 7 million YouTube videos (450,000 hours of video) and includes video labels from a vocabulary of 4716 classes (3.4 labels/video on average). It also comes with pre-extracted audio & visual features from every second of video (3.2 billion feature vectors in total). Google cloud recently released the datasets and organized 'Google Cloud & YouTube-8M Video Understanding Challenge' on Kaggle. Competitors are challenged to develop classification algorithms that assign video-level labels using the new and improved Youtube-8M V2 dataset. Inspired by the competition, we started exploration of audio understanding and classification using deep learning algorithms and ensemble methods. We built several baseline predictions according to the benchmark paper and public github tensorflow code. Furthermore, we improved global prediction accuracy (GAP) from base level 77% to 80.7% through approaches of ensemble.

stat.ML↗

arXiv2016

Multi-agent Reinforcement Learning with Sparse Interactions by Negotiation and Knowledge Transfer

Reinforcement learning has significant applications for multi-agent systems, especially in unknown dynamic environments. However, most multi-agent reinforcement learning (MARL) algorithms suffer from such problems as exponential computation complexity in the joint state-action space, which makes it difficult to scale up to realistic multi-agent problems. In this paper, a novel algorithm named negotiation-based MARL with sparse interactions (NegoSI) is presented. In contrast to traditional sparse-interaction based MARL algorithms, NegoSI adopts the equilibrium concept and makes it possible for agents to select the non-strict Equilibrium Dominating Strategy Profile (non-strict EDSP) or Meta equilibrium for their joint actions. The presented NegoSI algorithm consists of four parts: the equilibrium-based framework for sparse interactions, the negotiation for the equilibrium set, the minimum variance method for selecting one joint action and the knowledge transfer of local Q-values. In this integrated algorithm, three techniques, i.e., unshared value functions, equilibrium solutions and sparse interactions are adopted to achieve privacy protection, better coordination and lower computational complexity, respectively. To evaluate the performance of the presented NegoSI algorithm, two groups of experiments are carried out regarding three criteria: steps of each episode (SEE), rewards of each episode (REE) and average runtime (AR). The first group of experiments is conducted using six grid world games and shows fast convergence and high scalability of the presented algorithm. Then in the second group of experiments NegoSI is applied to an intelligent warehouse problem and simulated results demonstrate the effectiveness of the presented NegoSI algorithm compared with other state-of-the-art MARL algorithms.

cs.MA↗

arXiv2014

High order maximum principle preserving finite volume method for convection dominated problems

In this paper, we investigate the application of the maximum principle preserving (MPP) parametrized flux limiters to the high order finite volume scheme with Runge-Kutta time discretization for solving convection dominated problems. Such flux limiter was originally proposed in [Xu, Math. Comp., 2013] and further developed in [Xiong et. al., J. Comp. Phys., 2013] for finite difference WENO schemes with Runge-Kutta time discretization for convection equations. The main idea is to limit the temporal integrated high order numerical flux toward a first order MPP monotone flux. In this paper, we generalize such flux limiter to high order finite volume methods solving convection-dominated problems, which is easy to implement and introduces little computational overhead. More importantly, for the first time in the finite volume setting, we provide a general proof that the proposed flux limiter maintains high order accuracy of the original WENO scheme for linear advection problems without any additional time step restriction. For general nonlinear convection-dominated problems, we prove that the proposed flux limiter introduces up to $O(Δx^3+Δt^3)$ modification to the high order temporal integrated flux in the original WENO scheme without extra time step constraint. We also numerically investigate the preservation of up to ninth order accuracy of the proposed flux limiter in a general setting. The advantage of the proposed method is demonstrated through various numerical experiments.

math.NA↗