arXiv subjects

Linfeng Zhang

Publications and source records attributed to Linfeng Zhang.

321 records · Page 18Linked to original sources

arXiv2019

SCAN: A Scalable Neural Networks Framework Towards Compact and Efficient Models

Remarkable achievements have been attained by deep neural networks in various applications. However, the increasing depth and width of such models also lead to explosive growth in both storage and computation, which has restricted the deployment of deep neural networks on resource-limited edge devices. To address this problem, we propose the so-called SCAN framework for networks training and inference, which is orthogonal and complementary to existing acceleration and compression methods. The proposed SCAN firstly divides neural networks into multiple sections according to their depth and constructs shallow classifiers upon the intermediate features of different sections. Moreover, attention modules and knowledge distillation are utilized to enhance the accuracy of shallow classifiers. Based on this architecture, we further propose a threshold controlled scalable inference mechanism to approach human-like sample-specific inference. Experimental results show that SCAN can be easily equipped on various neural networks without any adjustment on hyper-parameters or neural networks architectures, yielding significant performance gain on CIFAR100 and ImageNet. Codes will be released on github soon.

cs.LG↗

arXiv2019

Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillation

Convolutional neural networks have been widely deployed in various application scenarios. In order to extend the applications' boundaries to some accuracy-crucial domains, researchers have been investigating approaches to boost accuracy through either deeper or wider network structures, which brings with them the exponential increment of the computational and storage cost, delaying the responding time. In this paper, we propose a general training framework named self distillation, which notably enhances the performance (accuracy) of convolutional neural networks through shrinking the size of the network rather than aggrandizing it. Different from traditional knowledge distillation - a knowledge transformation methodology among networks, which forces student neural networks to approximate the softmax layer outputs of pre-trained teacher neural networks, the proposed self distillation framework distills knowledge within network itself. The networks are firstly divided into several sections. Then the knowledge in the deeper portion of the networks is squeezed into the shallow ones. Experiments further prove the generalization of the proposed self distillation framework: enhancement of accuracy at average level is 2.65%, varying from 0.61% in ResNeXt as minimum to 4.07% in VGG19 as maximum. In addition, it can also provide flexibility of depth-wise scalable inference on resource-limited edge devices.Our codes will be released on github soon.

cs.LG↗

arXiv2019

Deep learning inter-atomic potential model for accurate irradiation damage simulations

We propose a hybrid scheme that interpolates smoothly the Ziegler-Biersack-Littmark (ZBL) screened nuclear repulsion potential with a newly developed deep learning potential energy model. The resulting DP-ZBL model can not only provide overall good performance on the predictions of near-equilibrium material properties but also capture the right physics when atoms are extremely close to each other, an event that frequently happens in computational simulations of irradiation damage events. We applied this scheme to the simulation of the irradiation damage processes in the face-centered-cubic aluminium system, and found better descriptions in terms of the defect formation energy, evolution of collision cascades, displacement threshold energy, and residual point defects, than the widely-adopted ZBL modified embedded atom method potentials and its variants. Our work provides a reliable and feasible scheme to accurately simulate the irradiation damage processes and opens up new opportunities to solve the predicament of lacking accurate potentials for enormous newly-discovered materials in the irradiation effect field.

physics.comp-ph↗

arXiv2019

StructADMM: A Systematic, High-Efficiency Framework of Structured Weight Pruning for DNNs

Weight pruning methods of DNNs have been demonstrated to achieve a good model pruning rate without loss of accuracy, thereby alleviating the significant computation/storage requirements of large-scale DNNs. Structured weight pruning methods have been proposed to overcome the limitation of irregular network structure and demonstrated actual GPU acceleration. However, in prior work the pruning rate (degree of sparsity) and GPU acceleration are limited (to less than 50%) when accuracy needs to be maintained. In this work,we overcome these limitations by proposing a unified, systematic framework of structured weight pruning for DNNs. It is a framework that can be used to induce different types of structured sparsity, such as filter-wise, channel-wise, and shape-wise sparsity, as well non-structured sparsity. The proposed framework incorporates stochastic gradient descent with ADMM, and can be understood as a dynamic regularization method in which the regularization target is analytically updated in each iteration. Without loss of accuracy on the AlexNet model, we achieve 2.58X and 3.65X average measured speedup on two GPUs, clearly outperforming the prior work. The average speedups reach 3.15X and 8.52X when allowing a moderate ac-curacy loss of 2%. In this case the model compression for convolutional layers is 15.0X, corresponding to 11.93X measured CPU speedup. Our experiments on ResNet model and on other data sets like UCF101 and CIFAR-10 demonstrate the consistently higher performance of our framework.

cs.NE↗

arXiv2019

Active Learning of Uniformly Accurate Inter-atomic Potentials for Materials Simulation

An active learning procedure called Deep Potential Generator (DP-GEN) is proposed for the construction of accurate and transferable machine learning-based models of the potential energy surface (PES) for the molecular modeling of materials. This procedure consists of three main components: exploration, generation of accurate reference data, and training. Application to the sample systems of Al, Mg and Al-Mg alloys demonstrates that DP-GEN can produce uniformly accurate PES models with a minimal number of reference data.

physics.comp-ph↗

arXiv2018

End-to-end Symmetry Preserving Inter-atomic Potential Energy Model for Finite and Extended Systems

Machine learning models are changing the paradigm of molecular modeling, which is a fundamental tool for material science, chemistry, and computational biology. Of particular interest is the inter-atomic potential energy surface (PES). Here we develop Deep Potential - Smooth Edition (DeepPot-SE), an end-to-end machine learning-based PES model, which is able to efficiently represent the PES for a wide variety of systems with the accuracy of ab initio quantum mechanics models. By construction, DeepPot-SE is extensive and continuously differentiable, scales linearly with system size, and preserves all the natural symmetries of the system. Further, we show that DeepPot-SE describes finite and extended systems including organic molecules, metals, semiconductors, and insulators with high fidelity.

physics.comp-ph↗

arXiv2018

Solving Many-Electron Schrödinger Equation Using Deep Neural Networks

We introduce a new family of trial wave-functions based on deep neural networks to solve the many-electron Schrödinger equation. The Pauli exclusion principle is dealt with explicitly to ensure that the trial wave-functions are physical. The optimal trial wave-function is obtained through variational Monte Carlo and the computational cost scales quadratically with the number of electrons. The algorithm does not make use of any prior knowledge such as atomic orbitals. Yet it is able to represent accurately the ground-states of the tested systems, including He, H2, Be, B, LiH, and a chain of 10 hydrogen atoms. This opens up new possibilities for solving large-scale many-electron Schrödinger equation.

physics.comp-ph↗

arXiv2018

Monge-Ampère Flow for Generative Modeling

We present a deep generative model, named Monge-Ampère flow, which builds on continuous-time gradient flow arising from the Monge-Ampère equation in optimal transport theory. The generative map from the latent space to the data space follows a dynamical system, where a learnable potential function guides a compressible fluid to flow towards the target density distribution. Training of the model amounts to solving an optimal control problem. The Monge-Ampère flow has tractable likelihoods and supports efficient sampling and inference. One can easily impose symmetry constraints in the generative model by designing suitable scalar potential functions. We apply the approach to unsupervised density estimation of the MNIST dataset and variational calculation of the two-dimensional Ising model at the critical point. This approach brings insights and techniques from Monge-Ampère equation, optimal transport, and fluid dynamics into reversible flow-based generative models.

cs.LG↗

arXiv2018

Adaptive coupling of a deep neural network potential to a classical force field

An adaptive modeling method (AMM) that couples a deep neural network potential and a classical force field is introduced to address the accuracy-efficiency dilemma faced by the molecular simulation community. The AMM simulated system is decomposed into three types of regions. The first type captures the important phenomena in the system and requires high accuracy, for which we use the Deep Potential Molecular Dynamics (DeePMD) model in this work. The DeePMD model is trained to accurately reproduce the statistical properties of the ab initio molecular dynamics. The second type does not require high accuracy and a classical force field is used to describe it in an efficient way. The third type is used for a smooth transition between the first and the second types of regions. By using a force interpolation scheme and imposing a thermodynamics force in the transition region, we make the DeePMD region embedded in the AMM simulated system as if it were embedded in a system that is fully described by the accurate potential. A representative example of the liquid water system is used to show the feasibility and promise of this method.

physics.chem-ph↗

arXiv2018

DeePCG: constructing coarse-grained models via deep neural networks

We introduce a general framework for constructing coarse-grained potential models without ad hoc approximations such as limiting the potential to two- and/or three-body contributions. The scheme, called Deep Coarse-Grained Potential (abbreviated DeePCG), exploits a carefully crafted neural network to construct a many-body coarse-grained potential. The network is trained with full atomistic data in a way that preserves the natural symmetries of the system. The resulting model is very accurate and can be used to sample the configurations of the coarse-grained variables in a much faster way than with the original atomistic model. As an application we consider liquid water and use the oxygen coordinates as the coarse-grained variables, starting from a full atomistic simulation of this system at the ab-initio molecular dynamics level. We found that the two-body, three-body and higher order oxygen correlation functions produced by the coarse-grained and full atomistic models agree very well with each other, illustrating the effectiveness of the DeePCG model on a rather challenging task.

physics.chem-ph↗

arXiv2018

Reinforced dynamics for enhanced sampling in large atomic and molecular systems

A new approach for efficiently exploring the configuration space and computing the free energy of large atomic and molecular systems is proposed, motivated by an analogy with reinforcement learning. There are two major components in this new approach. Like metadynamics, it allows for an efficient exploration of the configuration space by adding an adaptively computed biasing potential to the original dynamics. Like deep reinforcement learning, this biasing potential is trained on the fly using deep neural networks, with data collected judiciously from the exploration and an uncertainty indicator from the neural network model playing the role of the reward function. Parameterization using neural networks makes it feasible to handle cases with a large set of collective variables. This has the potential advantage that selecting precisely the right set of collective variables has now become less critical for capturing the structural transformations of the system. The method is illustrated by studying the full-atom, explicit solvent models of alanine dipeptide and tripeptide, as well as the system of a polyalanine-10 molecule with 20 collective variables.

physics.chem-ph↗

arXiv2017

DeePMD-kit: A deep learning package for many-body potential energy representation and molecular dynamics

Recent developments in many-body potential energy representation via deep learning have brought new hopes to addressing the accuracy-versus-efficiency dilemma in molecular simulations. Here we describe DeePMD-kit, a package written in Python/C++ that has been designed to minimize the effort required to build deep learning based representation of potential energy and force field and to perform molecular dynamics. Potential applications of DeePMD-kit span from finite molecules to extended systems and from metallic systems to chemically bonded systems. DeePMD-kit is interfaced with TensorFlow, one of the most popular deep learning frameworks, making the training process highly automatic and efficient. On the other end, DeePMD-kit is interfaced with high-performance classical molecular dynamics and quantum (path-integral) molecular dynamics packages, i.e., LAMMPS and the i-PI, respectively. Thus, upon training, the potential energy and force field models can be used to perform efficient molecular simulations for different purposes. As an example of the many potential applications of the package, we use DeePMD-kit to learn the interatomic potential energy and forces of a water model using data obtained from density functional theory. We demonstrate that the resulted molecular dynamics model reproduces accurately the structural information contained in the original model.

physics.comp-ph↗

arXiv2017

Deep Potential Molecular Dynamics: a scalable model with the accuracy of quantum mechanics

We introduce a scheme for molecular simulations, the Deep Potential Molecular Dynamics (DeePMD) method, based on a many-body potential and interatomic forces generated by a carefully crafted deep neural network trained with ab initio data. The neural network model preserves all the natural symmetries in the problem. It is "first principle-based" in the sense that there are no ad hoc components aside from the network model. We show that the proposed scheme provides an efficient and accurate protocol in a variety of systems, including bulk materials and molecules. In all these cases, DeePMD gives results that are essentially indistinguishable from the original data, at a cost that scales linearly with system size.

physics.comp-ph↗

arXiv2017

Deep Potential: a general representation of a many-body potential energy surface

We present a simple, yet general, end-to-end deep neural network representation of the potential energy surface for atomic and molecular systems. This methodology, which we call Deep Potential, is "first-principle" based, in the sense that no ad hoc approximations or empirical fitting functions are required. The neural network structure naturally respects the underlying symmetries of the systems. When tested on a wide variety of examples, Deep Potential is able to reproduce the original model, whether empirical or quantum mechanics based, within chemical accuracy. The computational cost of this new model is not substantially larger than that of empirical force fields. In addition, the method has promising scalability properties. This brings us one step closer to being able to carry out molecular simulations with accuracy comparable to that of quantum mechanics models and computational cost comparable to that of empirical potentials.

physics.comp-ph↗

arXiv2015

Global $α$-decay study based on the mass table of the relativistic continuum Hartree-Bogoliubov theory

The $α$-decay energies ($Q_α$) are systematically investigated with the nuclear masses for $10 \leq Z \leq 120$ isotopes obtained by the relativistic continuum Hartree-Bogoliubov (RCHB) theory with the covariant density functional PC-PK1, and compared with available experimental values. It is found that the $α$-decay energies deduced from the RCHB results present similar pattern as those from available experiments. Owing to the large predicted $Q_α$ values ($\geq$ 4 MeV), many undiscovered heavy nuclei in the proton-rich side and super-heavy nuclei may have large possibilities for $α$-decay. The influence of nuclear shell structure on $α$-decay energies is also analysed.

nucl-th↗