arXiv subjects

Luping Shi

Publications and source records attributed to Luping Shi.

23 records · Page 2Linked to original sources

arXiv2018

Training and Inference with Integers in Deep Neural Networks

Researches on deep neural networks with discrete parameters and their deployment in embedded systems have been active and promising topics. Although previous works have successfully reduced precision in inference, transferring both training and inference processes to low-bitwidth integers has not been demonstrated simultaneously. In this work, we develop a new method termed as "WAGE" to discretize both training and inference, where weights (W), activations (A), gradients (G) and errors (E) among layers are shifted and linearly constrained to low-bitwidth integers. To perform pure discrete dataflow for fixed-point devices, we further replace batch normalization by a constant scaling layer and simplify other components that are arduous for integer implementation. Improved accuracies can be obtained on multiple datasets, which indicates that WAGE somehow acts as a type of regularization. Empirically, we demonstrate the potential to deploy training in hardware systems such as integer-based deep learning accelerators and neuromorphic chips with comparable accuracy and higher energy efficiency, which is crucial to future AI applications in variable scenarios with transfer and continual learning demands.

cs.LG↗

arXiv2017

Spatio-Temporal Backpropagation for Training High-performance Spiking Neural Networks

Compared with artificial neural networks (ANNs), spiking neural networks (SNNs) are promising to explore the brain-like behaviors since the spikes could encode more spatio-temporal information. Although pre-training from ANN or direct training based on backpropagation (BP) makes the supervised training of SNNs possible, these methods only exploit the networks' spatial domain information which leads to the performance bottleneck and requires many complicated training skills. Another fundamental issue is that the spike activity is naturally non-differentiable which causes great difficulties in training SNNs. To this end, we build an iterative LIF model that is more friendly for gradient descent training. By simultaneously considering the layer-by-layer spatial domain (SD) and the timing-dependent temporal domain (TD) in the training phase, as well as an approximated derivative for the spike activity, we propose a spatio-temporal backpropagation (STBP) training framework without using any complicated technology. We achieve the best performance of multi-layered perceptron (MLP) compared with existing state-of-the-art algorithms over the static MNIST and the dynamic N-MNIST dataset as well as a custom object detection dataset. This work provides a new perspective to explore the high-performance SNNs for future brain-like computing paradigm with rich spatio-temporal dynamics.

cs.NE↗

arXiv2015

Single neutral substitutional 3d-transition metal in GeTe and GeSb2Te4 by the screened exchange functional

Fe doped GST has shown experimentally the ability to alter its magnetic properties by phase change. In this work, we use screened exchange hybrid functional to study the single neutral substitutional 3d transition metal (TM) in crystalline GeTe and GeSb2Te4. By curing the problem of local density functional (LDA) such as over delocalization of the 3d states, we find that Fe on Ge/Sb site has its majority d states fully occupied while its minority d states are empty, which is different than previous predicted electronic configuration by LDA. From early transition metal Cr to heavier Ni, the majority 3d states are gradually populated until fully occupied and then the minority 3d states begin to be filled. In order to study the magnetic contrast, we use lower symmetry crystalline GeTe and GeSb2Te4 as the amorphous phases, respectively, which has been proposed to model the medium range disordering. We find that only Co substitution in r-GeSb2Te4 and s-GeSb2Te4 shows magnetic contrast. The experimental magnetic contrast for Fe doped GST may be due to additional TM-TM interaction, which is not included in our model. It can also be possible that these lower symmetry crystalline models are not sufficient to characterize the magnetic properties of real 3d TM doped amorphous GST.

cond-mat.mtrl-sci↗

arXiv2015

The electronic structures of TiO2/Ti4O7, Ta2O5/TaO2 interfaces and the interfacial effects of dopants

We study the electronic structures of TiO2/Ti4O7 and Ta2O5/TaO2 interfaces using the screened exchange (sX-LDA) functional. Both of these bilayer structures are useful infrastructures for high performance RRAMs. We find that the system Fermi energies of both interfaces are just above the conduction band edge of the corresponding stoichiometric oxides. According to the charge transition levels of the oxygen vacancies in Ta2O5 and TiO2, the oxygen vacancies are stabilized at the -2 charged state. We propose to introduce interfacial dopants to shift the system Fermi energies downward so that the +2 charged oxygen vacancy can be stable, which is important for the controlled switching under the electrical field. Several dipole models are presented to account for the ability of Fermi level shift due to the interfacial dopants.

cond-mat.mtrl-sci↗

arXiv2015

The role of single oxygen or metal induced defect and correlated multiple defects in the formation of conducting filaments

We study the dependence of the formation energies of oxygen and metal induced defects in Ta2O5, TaO2, TaO, TiO2 and Ti4O7 on the chemical potential of electron and atomic constitutes. In the study of single defect, metal induced defects are found to be preferable to oxygen induced defects. This is against the experimental fact of the dominant role of oxygen induced defects in the RS process. A simple multiple defects picture without correlated atomic rearrangement does not cure this problem. The problem is resolved under the correlated multiple defect picture where the multiple defects result in correlated atomic rearrangement and the final products show certain atomic ordering.

cond-mat.mtrl-sci↗