arXiv subjects

Linfeng Zhang

Publications and source records attributed to Linfeng Zhang.

At least 271 records · Page 15Linked to original sources

arXiv2023

Uni-QSAR: an Auto-ML Tool for Molecular Property Prediction

Recently deep learning based quantitative structure-activity relationship (QSAR) models has shown surpassing performance than traditional methods for property prediction tasks in drug discovery. However, most DL based QSAR models are restricted to limited labeled data to achieve better performance, and also are sensitive to model scale and hyper-parameters. In this paper, we propose Uni-QSAR, a powerful Auto-ML tool for molecule property prediction tasks. Uni-QSAR combines molecular representation learning (MRL) of 1D sequential tokens, 2D topology graphs, and 3D conformers with pretraining models to leverage rich representation from large-scale unlabeled data. Without any manual fine-tuning or model selection, Uni-QSAR outperforms SOTA in 21/22 tasks of the Therapeutic Data Commons (TDC) benchmark under designed parallel workflow, with an average performance improvement of 6.09\%. Furthermore, we demonstrate the practical usefulness of Uni-QSAR in drug discovery domains.

q-bio.BM↗

arXiv2023

Hybrid Auxiliary Field Quantum Monte Carlo for Molecular Systems

We propose a quantum Monte Carlo approach to solve the ground state many-body Schrodinger equation for the electronic ground state. The method combines optimization from variational Monte Carlo and propagation from auxiliary field quantum Monte Carlo, in a way that significantly alleviates the sign problem. In application to molecular systems, we obtain highly accurate results for configurations dominated by either dynamic or static electronic correlation.

physics.comp-ph↗

arXiv2023

DeePMD-kit v2: A software package for Deep Potential models

DeePMD-kit is a powerful open-source software package that facilitates molecular dynamics simulations using machine learning potentials (MLP) known as Deep Potential (DP) models. This package, which was released in 2017, has been widely used in the fields of physics, chemistry, biology, and material science for studying atomistic systems. The current version of DeePMD-kit offers numerous advanced features such as DeepPot-SE, attention-based and hybrid descriptors, the ability to fit tensile properties, type embedding, model deviation, Deep Potential - Range Correction (DPRc), Deep Potential Long Range (DPLR), GPU support for customized operators, model compression, non-von Neumann molecular dynamics (NVNMD), and improved usability, including documentation, compiled binary packages, graphical user interfaces (GUI), and application programming interfaces (API). This article presents an overview of the current major version of the DeePMD-kit package, highlighting its features and technical details. Additionally, the article benchmarks the accuracy and efficiency of different models and discusses ongoing developments.

physics.chem-ph↗

arXiv2023

Modeling the High-Pressure Solid and Liquid Phases of Tin from Deep Potentials with ab initio Accuracy

Constructing an accurate atomistic model for the high-pressure phases of tin (Sn) is challenging because properties of Sn are sensitive to pressures. We develop machine-learning-based deep potentials for Sn with pressures ranging from 0 to 50 GPa and temperatures ranging from 0 to 2000 K. In particular, we find the deep potential, which is obtained by training the ab initio data from density functional theory calculations with the state-of-the-art SCAN exchange-correlation functional, is suitable to characterize high-pressure phases of Sn. We systematically validate several structural and elastic properties of the α (diamond structure), β, bct, and bcc structures of Sn, as well as the structural and dynamic properties of liquid Sn. The thermodynamics integration method is further utilized to compute the free energies of the α, β, bct, and liquid phases, from which the deep potential successfully predicts the phase diagram of Sn including the existence of the triple-point that qualitatively agrees with the experiment.

cond-mat.mtrl-sci↗

arXiv2023

Modular development of deep potential for complex solid solutions

The multicomponent oxide solid solution is a versatile platform to tune the delicate balance between competing spin, charge, orbital, and lattice degrees of freedom for materials design and discovery. The development of compositionally complex oxides with superior functional properties has been largely empirical and serendipitous, in part due to the exceedingly complex chemistry and structure of solid solutions that span a range of length scales. The classical molecular dynamics (MD), as a powerful statistical method to investigate materials properties over large spatial and temporal scales, often plays a secondary role in computer-aided materials discovery because of the limited availability and accuracy of classical force fields. Here, we introduce the strategy of ``modular developing deep potential" (ModDP) that enables a systematic development and improvement of deep neural network-based model potential, termed as deep potential, for complex solid solutions with minimum human intervention. The converged training database associated with an end-member material is treated as an independent module and is reused to train the deep potential of solid solutions via a concurrent learning procedure. We apply ModDP to obtain classical force fields of two technologically important solid solutions, Pb$_x$Sr$_{1-x}$TiO$_3$ and Hf$_x$Zr$_{1-x}$O$_2$. For both materials systems, a single model potential is capable of predicting various properties of solid solutions including temperature-driven and composition-driven phase transitions over a wide range of compositions. In particular, the deep potential of Pb$_x$Sr$_{1-x}$TiO$_3$ reproduces a few known topological textures such as polar vortex lattice and electric dipole waves in PbTiO$_3$/SrTiO$_3$ superlattices, paving the way for MD investigations on the dynamics of topological structures in response to external stimuli.

cond-mat.mtrl-sci↗

arXiv2023

Entangled X-ray Photon Pair Generation by Free Electron Lasers

Einstein, Podolsky and Rosen's prediction on incompleteness of quantum mechanics was overturned by experimental tests on Bell's inequality that confirmed the existence of quantum entanglement. In X-ray optics, entangled photon pairs can be generated by X-ray parametric down conversion (XPDC), which has certain wavelength window. Meanwhile, free electron laser (FEL) has successfully lased at X-ray frequencies recently. However, FEL is usually seen as a classical light source, and its quantum effects are considered minor corrections to the classical theory. Here we investigate entangled X-ray photon pair emissions in FEL. We establish a theory for coherently amplified entangled photon pair emission from microbunched electron pulses in the undulator. We numerically demonstrate the properties of entangled emission, and provide a scheme to generate highly entangled X-ray photon pairs, which is of great importance in X-ray quantum optics. Our work shows a unique advantage of FELs over synchrotrons in entangled X-ray photon pair generation.

quant-ph↗

arXiv2023

Autoencoders as Cross-Modal Teachers: Can Pretrained 2D Image Transformers Help 3D Representation Learning?

The success of deep learning heavily relies on large-scale data with comprehensive labels, which is more expensive and time-consuming to fetch in 3D compared to 2D images or natural languages. This promotes the potential of utilizing models pretrained with data more than 3D as teachers for cross-modal knowledge transferring. In this paper, we revisit masked modeling in a unified fashion of knowledge distillation, and we show that foundational Transformers pretrained with 2D images or natural languages can help self-supervised 3D representation learning through training Autoencoders as Cross-Modal Teachers (ACT). The pretrained Transformers are transferred as cross-modal 3D teachers using discrete variational autoencoding self-supervision, during which the Transformers are frozen with prompt tuning for better knowledge inheritance. The latent features encoded by the 3D teachers are used as the target of masked point modeling, wherein the dark knowledge is distilled to the 3D Transformer students as foundational geometry understanding. Our ACT pretrained 3D learner achieves state-of-the-art generalization capacity across various downstream benchmarks, e.g., 88.21% overall accuracy on ScanObjectNN. Codes have been released at https://github.com/RunpeiDong/ACT.

cs.CV↗

arXiv2022

Atomistic modelling of all dislocations and twins in HCP and BCC Ti

Ti exhibits complex plastic deformation controlled by active dislocation and twinning systems. Understandings on dislocation cores and twin interfaces are currently not complete or quantitative, despite extensive experimental and simulation studies. Here, we determine all the core and twin interface properties in both HCP and BCC Ti using a Deep Potential (DP) and DFT. We determine the core structures, critical resolved shear stresses and mobilities of , , dislocations in HCP and <111>/2 dislocations in BCC Ti. The slip consists of slow core migration on pyramidal-I planes and fast migration on prism-planes, and is kinetically limited by cross-slips among them. This behaviour is consistent with "locking-unlocking" phenomena in TEM and is likely an intrinsic property. Large-scale DFT calculations provide a peek at the screw core and glide behaviour, which is further quantified using DP-Ti. The screw is unstable on pyramidal-II planes. The mixed is nearly sessile on pyramidal-I planes, consistent with observations of long dislocations in this orientation. The edge and mixed are unstable against a pyramidal-to-basal (PB) transition and become sessile at high temperatures, corroborate the difficulties in -axis compression of Ti. Finally, in BCC Ti, the <111>/2 screw has a degenerate core with average glide on {112} planes; the <111>/2 edge and mixed dislocations have non-dissociated cores on {110} planes. This work paints a self-consistent, complete picture on all dislocations in Ti, rationalises previous experimental observations and points to future HRTEM examinations of unusual dislocations such as the mixed and PB transformed cores.

cond-mat.mtrl-sci↗

arXiv2022

Structured Knowledge Distillation Towards Efficient and Compact Multi-View 3D Detection

Detecting 3D objects from multi-view images is a fundamental problem in 3D computer vision. Recently, significant breakthrough has been made in multi-view 3D detection tasks. However, the unprecedented detection performance of these vision BEV (bird's-eye-view) detection models is accompanied with enormous parameters and computation, which make them unaffordable on edge devices. To address this problem, in this paper, we propose a structured knowledge distillation framework, aiming to improve the efficiency of modern vision-only BEV detection models. The proposed framework mainly includes: (a) spatial-temporal distillation which distills teacher knowledge of information fusion from different timestamps and views, (b) BEV response distillation which distills teacher response to different pillars, and (c) weight-inheriting which solves the problem of inconsistent inputs between students and teacher in modern transformer architectures. Experimental results show that our method leads to an average improvement of 2.16 mAP and 2.27 NDS on the nuScenes benchmark, outperforming multiple baselines by a large margin.

cs.CV↗

arXiv2022

Machine learning assisted coarse-grained molecular dynamics modeling of meso-scale interfacial fluids

A hallmark of meso-scale interfacial fluids is the multi-faceted, scale-dependent interfacial energy, which often manifests different characteristics across the molecular and continuum scale. The multi-scale nature imposes a challenge to construct reliable coarse-grained (CG) models, where the CG potential function needs to faithfully encode the many-body interactions arising from the unresolved atomistic interactions and account for the heterogeneous density distributions across the interface. We construct the CG models of both single- and two-component of polymeric fluid systems based on the recently developed deep coarse-grained potential (DeePCG) scheme, where each polymer molecule is modeled as a CG particle. By only using the training samples of the instantaneous force under the thermal equilibrium state, the constructed CG models can accurately reproduce both the probability density function of the void formation in bulk and the spectrum of the capillary wave across the fluid interface. More importantly, the CG models accurately predict the volume-to-area scaling transition for the apolar solvation energy, illustrating the effectiveness to probe the meso-scale collective behaviors encoded with molecular-level fidelity.

physics.comp-ph↗

arXiv2022

Classical and Machine Learning Interatomic Potentials for BCC Vanadium

BCC transition metals (TMs) exhibit complex temperature and strain-rate dependent plastic deformation behaviour controlled by individual crystal lattice defects. Classical empirical and semi-empirical interatomic potentials have limited capability in modelling defect properties such as the screw dislocation core structures and Peierls barriers in the BCC structure. Machine learning (ML) potentials, trained on DFT-based datasets, have shown some successes in reproducing dislocation core properties. However, in group VB TMs, the most widely-used DFT functionals produce erroneous shear moduli C44 which are undesirably transferred to machine-learning interatomic potentials, leaving current ML approaches unsuitable for this important class of metals and alloys. Here, we develop two interatomic potentials for BCC vanadium (V) based on (i) an extension of the partial electron density and screening parameter in the classical semi-empirical modified embedded-atom method (XMEAM-V) and (ii) a recent hybrid descriptor in the ML Deep Potential framework (DP-HYB-V). We describe distinct features in these two disparate approaches, including their dataset generation, training procedure, weakness and strength in modelling lattice and defect properties in BCC V. Both XMEAM-V and DP-HYB-V reproduce a broad range of defect properties relevant to plastic deformation and fracture. In particular, XMEAM-V reproduces nearly all mechanical and thermodynamic properties at DFT accuracies and with C44 near experimental value. XMEAM-V also naturally exhibits the anomalous slip at 77 K widely observed in group VB and VIB TMs and outperforms all existing, publically available interatomic potentials for V. The XMEAM thus provides a practical path to developing accurate and efficient interatomic potentials for nonmagnetic BCC TMs and possibly multi-principal element TM alloys.

cond-mat.mtrl-sci↗

arXiv2022

DeePKS+ABACUS as a Bridge between Expensive Quantum Mechanical Models and Machine Learning Potentials

Recently, the development of machine learning (ML) potentials has made it possible to perform large-scale and long-time molecular simulations with the accuracy of quantum mechanical (QM) models. However, for high-level QM methods, such as density functional theory (DFT) at the meta-GGA level and/or with exact exchange, quantum Monte Carlo, etc., generating a sufficient amount of data for training a ML potential has remained computationally challenging due to their high cost. In this work, we demonstrate that this issue can be largely alleviated with Deep Kohn-Sham (DeePKS), a ML-based DFT model. DeePKS employs a computationally efficient neural network-based functional model to construct a correction term added upon a cheap DFT model. Upon training, DeePKS offers closely-matched energies and forces compared with high-level QM method, but the number of training data required is orders of magnitude less than that required for training a reliable ML potential. As such, DeePKS can serve as a bridge between expensive QM models and ML potentials: one can generate a decent amount of high-accuracy QM data to train a DeePKS model, and then use the DeePKS model to label a much larger amount of configurations to train a ML potential. This scheme for periodic systems is implemented in a DFT package ABACUS, which is open-source and ready for use in various applications.

physics.chem-ph↗

arXiv2022

DP Compress: a Model Compression Scheme for Generating Efficient Deep Potential Models

Machine-learning-based interatomic potential energy surface (PES) models are revolutionizing the field of molecular modeling. However, although much faster than electronic structure schemes, these models suffer from costly computations via deep neural networks to predict the energy and atomic forces, resulting in lower running efficiency as compared to the typical empirical force fields. Herein, we report a model compression scheme for boosting the performance of the Deep Potential (DP) model, a deep learning based PES model. This scheme, we call DP Compress, is an efficient post-processing step after the training of DP models (DP Train). DP Compress combines several DP-specific compression techniques, which typically speed up DP-based molecular dynamics simulations by an order of magnitude faster, and consume an order of magnitude less memory. We demonstrate that DP Compress is sufficiently accurate by testing a variety of physical properties of Cu, H2O, and Al-Cu-Mg systems. DP Compress applies to both CPU and GPU machines and is publicly available online.

physics.comp-ph↗

arXiv2022

Contrastive Deep Supervision

The success of deep learning is usually accompanied by the growth in neural network depth. However, the traditional training method only supervises the neural network at its last layer and propagates the supervision layer-by-layer, which leads to hardship in optimizing the intermediate layers. Recently, deep supervision has been proposed to add auxiliary classifiers to the intermediate layers of deep neural networks. By optimizing these auxiliary classifiers with the supervised task loss, the supervision can be applied to the shallow layers directly. However, deep supervision conflicts with the well-known observation that the shallow layers learn low-level features instead of task-biased high-level semantic features. To address this issue, this paper proposes a novel training framework named Contrastive Deep Supervision, which supervises the intermediate layers with augmentation-based contrastive learning. Experimental results on nine popular datasets with eleven models demonstrate its effects on general image classification, fine-grained image classification and object detection in supervised learning, semi-supervised learning and knowledge distillation. Codes have been released in Github.

cs.CV↗

arXiv2022

Viscosity in water from first-principles and deep-neural-network simulations

We report on an extensive study of the viscosity of liquid water at near-ambient conditions, performed within the Green-Kubo theory of linear response and equilibrium ab initio molecular dynamics (AIMD), based on density-functional theory (DFT). In order to cope with the long simulation times necessary to achieve an acceptable statistical accuracy, our ab initio approach is enhanced with deep-neural-network potentials (NNP). This approach is first validated against AIMD results, obtained by using the Perdew-Burke-Ernzerhof (PBE) exchange-correlation functional and paying careful attention to crucial, yet often overlooked, aspects of the statistical data analysis. Then, we train a second NNP to a dataset generated from the Strongly Constrained and Appropriately Normed (SCAN) functional. Once the error resulting from the imperfect prediction of the melting line is offset by referring the simulated temperature to the theoretical melting one, our SCAN predictions of the shear viscosity of water are in very good agreement with experiments.

cond-mat.mtrl-sci↗

arXiv2022

Finding the Task-Optimal Low-Bit Sub-Distribution in Deep Neural Networks

Quantized neural networks typically require smaller memory footprints and lower computation complexity, which is crucial for efficient deployment. However, quantization inevitably leads to a distribution divergence from the original network, which generally degrades the performance. To tackle this issue, massive efforts have been made, but most existing approaches lack statistical considerations and depend on several manual configurations. In this paper, we present an adaptive-mapping quantization method to learn an optimal latent sub-distribution that is inherent within models and smoothly approximated with a concrete Gaussian Mixture (GM). In particular, the network weights are projected in compliance with the GM-approximated sub-distribution. This sub-distribution evolves along with the weight update in a co-tuning schema guided by the direct task-objective optimization. Sufficient experiments on image classification and object detection over various modern architectures demonstrate the effectiveness, generalization property, and transferability of the proposed method. Besides, an efficient deployment flow for the mobile CPU is developed, achieving up to 7.46$\times$ inference acceleration on an octa-core ARM CPU. Our codes have been publicly released at \url{https://github.com/RunpeiDong/DGMS}.

cs.CV↗

arXiv2022

Region-aware Knowledge Distillation for Efficient Image-to-Image Translation

Recent progress in image-to-image translation has witnessed the success of generative adversarial networks (GANs). However, GANs usually contain a huge number of parameters, which lead to intolerant memory and computation consumption and limit their deployment on edge devices. To address this issue, knowledge distillation is proposed to transfer the knowledge from a cumbersome teacher model to an efficient student model. However, most previous knowledge distillation methods are designed for image classification and lead to limited performance in image-to-image translation. In this paper, we propose Region-aware Knowledge Distillation ReKo to compress image-to-image translation models. Firstly, ReKo adaptively finds the crucial regions in the images with an attention module. Then, patch-wise contrastive learning is adopted to maximize the mutual information between students and teachers in these crucial regions. Experiments with eight comparison methods on nine datasets demonstrate the substantial effectiveness of ReKo on both paired and unpaired image-to-image translation. For instance, our 7.08X compressed and 6.80X accelerated CycleGAN student outperforms its teacher by 1.33 and 1.04 FID scores on Horse to Zebra and Zebra to Horse, respectively. Codes will be released on GitHub.

cs.CV↗

arXiv2022

PointDistiller: Structured Knowledge Distillation Towards Efficient and Compact 3D Detection

The remarkable breakthroughs in point cloud representation learning have boosted their usage in real-world applications such as self-driving cars and virtual reality. However, these applications usually have an urgent requirement for not only accurate but also efficient 3D object detection. Recently, knowledge distillation has been proposed as an effective model compression technique, which transfers the knowledge from an over-parameterized teacher to a lightweight student and achieves consistent effectiveness in 2D vision. However, due to point clouds' sparsity and irregularity, directly applying previous image-based knowledge distillation methods to point cloud detectors usually leads to unsatisfactory performance. To fill the gap, this paper proposes PointDistiller, a structured knowledge distillation framework for point clouds-based 3D detection. Concretely, PointDistiller includes local distillation which extracts and distills the local geometric structure of point clouds with dynamic graph convolution and reweighted learning strategy, which highlights student learning on the crucial points or voxels to improve knowledge distillation efficiency. Extensive experiments on both voxels-based and raw points-based detectors have demonstrated the effectiveness of our method over seven previous knowledge distillation methods. For instance, our 4X compressed PointPillars student achieves 2.8 and 3.4 mAP improvements on BEV and 3D object detection, outperforming its teacher by 0.9 and 1.8 mAP, respectively. Codes have been released at https://github.com/RunpeiDong/PointDistiller.

cs.CV↗