Searcharxiv⌕ Search

arXiv subjects

Linfeng Zhang

Publications and source records attributed to Linfeng Zhang.

At least 253 records · Page 14Linked to original sources
arXiv2024

Uni-Mol Docking V2: Towards Realistic and Accurate Binding Pose Prediction

In recent years, machine learning (ML) methods have emerged as promising alternatives for molecular docking, offering the potential for high accuracy without incurring prohibitive computational costs. However, recent studies have indicated that these ML models may overfit to quantitative metrics while neglecting the physical constraints inherent in the problem. In this work, we present Uni-Mol Docking V2, which demonstrates a remarkable improvement in performance, accurately predicting the binding poses of 77+% of ligands in the PoseBusters benchmark with an RMSD value of less than 2.0 Å, and 75+% passing all quality checks. This represents a significant increase from the 62% achieved by the previous Uni-Mol Docking model. Notably, our Uni-Mol Docking approach generates chemically accurate predictions, circumventing issues such as chirality inversions and steric clashes that have plagued previous ML models. Furthermore, we observe enhanced performance in terms of high-quality predictions (RMSD values of less than 1.0 Å and 1.5 Å) and physical soundness when Uni-Mol Docking is combined with more physics-based methods like Uni-Dock. Our results represent a significant advancement in the application of artificial intelligence for scientific research, adopting a holistic approach to ligand docking that is well-suited for industrial applications in virtual screening and drug design. The code, data and service for Uni-Mol Docking are publicly available for use and further development in https://github.com/dptech-corp/Uni-Mol.

q-bio.BM↗
arXiv2024

Machine-Learning-Based Interatomic Potentials for Group IIB to VIA Semiconductors: Towards a Universal Model

Rapid advancements in machine-learning methods have led to the emergence of machine-learning-based interatomic potentials as a new cutting-edge tool for simulating large systems with ab initio accuracy. Still, the community awaits universal inter-atomic models that can be applied to a wide range of materials without tuning neural network parameters. We develop a unified deep-learning inter-atomic potential (the DPA-Semi model) for 19 semiconductors ranging from group IIB to VIA, including Si, Ge, SiC, BAs, BN, AlN, AlP, AlAs, InP, InAs, InSb, GaN, GaP, GaAs, CdTe, InTe, CdSe, ZnS, and CdS. In addition, independent deep potential models for each semiconductor are prepared for detailed comparison. The training data are obtained by performing density functional theory calculations with numerical atomic orbitals basis sets to reduce the computational costs. We systematically compare various properties of the solid and liquid phases of semiconductors between different machine-learning models. We conclude that the DPA-Semi model achieves GGA exchange-correlation functional quality accuracy and can be regarded as a pre-trained model towards a universal model to study group IIB to VIA semiconductors.

cond-mat.mtrl-sci↗
arXiv2024

Dflow, a Python framework for constructing cloud-native AI-for-Science workflows

In the AI-for-science era, scientific computing scenarios such as concurrent learning and high-throughput computing demand a new generation of infrastructure that supports scalable computing resources and automated workflow management on both cloud and high-performance supercomputers. Here we introduce Dflow, an open-source Python toolkit designed for scientists to construct workflows with simple programming interfaces. It enables complex process control and task scheduling across a distributed, heterogeneous infrastructure, leveraging containers and Kubernetes for flexibility. Dflow is highly observable and can scale to thousands of concurrent nodes per workflow, enhancing the efficiency of complex scientific computing tasks. The basic unit in Dflow, known as an Operation (OP), is reusable and independent of the underlying infrastructure or context. Dozens of workflow projects have been developed based on Dflow, spanning a wide range of projects. We anticipate that the reusability of Dflow and its components will encourage more scientists to publish their workflows and OP components. These components, in turn, can be adapted and reused in various contexts, fostering greater collaboration and innovation in the scientific community.

cs.DC↗
arXiv2024

An Extendable Cloud-Native Alloy Property Explorer

The ability to rapidly evaluate materials properties through atomistic simulation approaches is the foundation of many new artificial intelligence-based approaches to materials identification and design. This depends on the availability of accurate descriptions of atomic bonding through various forms of interatomic potentials. We present an efficient, robust platform for calculating materials properties, i.e., APEX, the Alloy Property Explorer. APEX enables the rapid evolution of interatomic potential development and optimization, which is of particular importance in fine-tuning new classes of general AI-based foundation models to forms that are readily applicable to impacting materials development. APEX is an open-source, extendable, and cloud-native platform for material property calculations using a range of atomistic simulation methodologies that effectively manages diverse computational resources and is built upon user-friendly features including automatic results visualization, web-based platforms and NoSQL database client. It is designed for expert and non-specialist users, lowers the barrier to entry for interdisciplinary research within the "AI for Materials" framework. We describe the foundation and use of APEX, as well as provide an example of its application to properties of titanium for a wide-range of bonding descriptions.

cond-mat.mtrl-sci↗
arXiv2023

Single-image based deep learning for precise atomic defects identification

Defect engineering has been profoundly employed to confer desirable functionality to materials that pristine lattices inherently lack. Although single atomic-resolution scanning transmission electron microscopy (STEM) images are widely accessible for defect engineering, harnessing atomic-scale images containing various defects through traditional image analysis methods is hindered by random noise and human bias. Yet the rise of deep learning (DL) offering an alternative approach, its widespread application is primarily restricted by the need for large amounts of training data with labeled ground truth. In this study, we propose a two-stage method to address the problems of high annotation cost and image noise in the detection of atomic defects in monolayer 2D materials. In the first stage, to tackle the issue of data scarcity, we employ a two-state transformation network based on U-GAT-IT for adding realistic noise to simulated images with pre-located ground truth labels, thereby infinitely expanding the training dataset. In the second stage, atomic defects in monolayer 2D materials are effectively detected with high accuracy using U-Net models trained with the data generated in the first stage, avoiding random noise and human bias issues. In both stages, we utilize segmented unit-cell-level images to simplify the model's task and enhance its accuracy. Our results demonstrate that not only sulfur vacancies, we are also able to visualize oxygen dopants in monolayer MoS2, which are usually overwhelmed by random background noise. As the training was based on a few segmented unit-cell-level realistic images, this method can be readily extended to other 2D materials. Therefore, our results outline novel ways to train the model with minimized datasets, offering great opportunities to fully exploit the power of machine learning (ML) applicable to a broad materials science community.

cond-mat.mtrl-sci↗
arXiv2023

Universal interatomic potential for perovskite oxides

With their celebrated structural and chemical flexibility, perovskite oxides have served as a highly adaptable material platform for exploring emergent phenomena arising from the interplay between different degrees of freedom. Molecular dynamics (MD) simulations leveraging classical force fields, commonly depicted as parameterized analytical functions, have made significant contributions in elucidating the atomistic dynamics and structural properties of crystalline solids including perovskite oxides. However, the force fields currently available for solids are rather specific and offer limited transferability, making it time-consuming to use MD to study new materials systems since a new force field must be parameterized and tested first. The lack of a generalized force field applicable to a broad spectrum of solid materials hinders the facile deployment of MD in computer-aided materials discovery (CAMD). Here, by utilizing a deep-neural network with a self-attention scheme, we have developed a unified force field that enables MD simulations of perovskite oxides involving 14 metal elements and conceivably their solid solutions with arbitrary compositions. Notably, isobaric-isothermal ensemble MD simulations with this model potential accurately predict the experimental phase transition sequences for several markedly different ferroelectric oxides, including a 6-element ternary solid solution, Pb(In$_{1/2}$Nb$_{1/2}$)O$_3$--Pb(Mg$_{1/3}$Nb$_{2/3}$)O$_3$--PbTiO$_3$. We believe the universal interatomic potential along with the training database, proposed regression tests, and the auto-testing workflow, all released publicly, will pave the way for a systematic improvement and extension of a unified force field for solids, potentially heralding a new era in CAMD.

cond-mat.mtrl-sci↗
arXiv2023

Deep Variational Free Energy Approach to Dense Hydrogen

We developed a deep generative model-based variational free energy approach to the equations of state of dense hydrogen. We employ a normalizing flow network to model the proton Boltzmann distribution and a fermionic neural network to model the electron wave function at given proton positions. By jointly optimizing the two neural networks we reached a comparable variational free energy to the previous coupled electron-ion Monte Carlo calculation. The predicted equation of state of dense hydrogen under planetary conditions is denser than the findings of ab initio molecular dynamics calculation and empirical chemical model. Moreover, direct access to the entropy and free energy of dense hydrogen opens new opportunities in planetary modeling and high-pressure physics research.

cond-mat.str-el↗
arXiv2023

DPA-1: Pretraining of Attention-based Deep Potential Model for Molecular Simulation

Machine learning assisted modeling of the inter-atomic potential energy surface (PES) is revolutionizing the field of molecular simulation. With the accumulation of high-quality electronic structure data, a model that can be pretrained on all available data and finetuned on downstream tasks with a small additional effort would bring the field to a new stage. Here we propose DPA-1, a Deep Potential model with a novel attention mechanism, which is highly effective for representing the conformation and chemical spaces of atomic systems and learning the PES. We tested DPA-1 on a number of systems and observed superior performance compared with existing benchmarks. When pretrained on large-scale datasets containing 56 elements, DPA-1 can be successfully applied to various downstream tasks with a great improvement of sample efficiency. Surprisingly, for different elements, the learned type embedding parameters form a $spiral$ in the latent space and have a natural correspondence with their positions on the periodic table, showing interesting interpretability of the pretrained DPA-1 model.

physics.chem-ph↗
arXiv2023

Ada3D : Exploiting the Spatial Redundancy with Adaptive Inference for Efficient 3D Object Detection

Voxel-based methods have achieved state-of-the-art performance for 3D object detection in autonomous driving. However, their significant computational and memory costs pose a challenge for their application to resource-constrained vehicles. One reason for this high resource consumption is the presence of a large number of redundant background points in Lidar point clouds, resulting in spatial redundancy in both 3D voxel and dense BEV map representations. To address this issue, we propose an adaptive inference framework called Ada3D, which focuses on exploiting the input-level spatial redundancy. Ada3D adaptively filters the redundant input, guided by a lightweight importance predictor and the unique properties of the Lidar point cloud. Additionally, we utilize the BEV features' intrinsic sparsity by introducing the Sparsity Preserving Batch Normalization. With Ada3D, we achieve 40% reduction for 3D voxels and decrease the density of 2D BEV feature maps from 100% to 20% without sacrificing accuracy. Ada3D reduces the model computational and memory cost by 5x, and achieves 1.52x/1.45x end-to-end GPU latency and 1.5x/4.5x GPU peak memory optimization for the 3D and 2D backbone respectively.

cs.CV↗
arXiv2023

Incident-Specific Cyber Insurance

In the current market practice, many cyber insurance products offer a coverage bundle for losses arising from various types of incidents, such as data breaches and ransomware attacks, and the coverage for each incident type comes with a separate limit and deductible. Although this gives prospective cyber insurance buyers more flexibility in customizing the coverage and better manages the risk exposures of sellers, it complicates the decision-making process in determining the optimal amount of risks to retain and transfer for both parties. This paper aims to build an economic foundation for these incident-specific cyber insurance products with a focus on how incident-specific indemnities should be designed for achieving Pareto optimality for both the insurance seller and buyer. Real data on cyber incidents is used to illustrate the feasibility of this approach. Several implementation improvement methods for practicality are also discussed.

q-fin.RM↗
arXiv2023

Tracking Berry curvature effect in molecular dynamics by ultrafast magnetic x-ray scattering

The spin-dependent Berry force is a genuine effect of Berry curvature in molecular dynamics, which can dramatically result in spatial spin separation and change of reaction pathways. However, the way to probe the effect of Berry force remains challenging, because the time-reversal (TR) symmetry required for opposite Berry forces conflicts with TR symmetry breaking spin alignment needed to observe the effect, and the net effect could be transient for a molecular wave packet. We demonstrate that in molecular photodissociation, the dissociation rates can be different for molecules with opposite initial spin directions due to Berry force. We showcase that the spatially separated spin density, which is transiently induced by Berry force as the molecular wave packet passes through conical intersection, can be reconstructed from the circular dichroism (CD) of ultrafast non-resonant magnetic x-ray scattering using free electron lasers.

physics.atom-ph↗
arXiv2023

Highly Accurate Quantum Chemical Property Prediction with Uni-Mol+

Recent developments in deep learning have made remarkable progress in speeding up the prediction of quantum chemical (QC) properties by removing the need for expensive electronic structure calculations like density functional theory. However, previous methods learned from 1D SMILES sequences or 2D molecular graphs failed to achieve high accuracy as QC properties primarily depend on the 3D equilibrium conformations optimized by electronic structure methods, far different from the sequence-type and graph-type data. In this paper, we propose a novel approach called Uni-Mol+ to tackle this challenge. Uni-Mol+ first generates a raw 3D molecule conformation from inexpensive methods such as RDKit. Then, the raw conformation is iteratively updated to its target DFT equilibrium conformation using neural networks, and the learned conformation will be used to predict the QC properties. To effectively learn this update process towards the equilibrium conformation, we introduce a two-track Transformer model backbone and train it with the QC property prediction task. We also design a novel approach to guide the model's training process. Our extensive benchmarking results demonstrate that the proposed Uni-Mol+ significantly improves the accuracy of QC property prediction in various datasets. We have made the code and model publicly available at \url{https://github.com/dptech-corp/Uni-Mol}.

physics.chem-ph↗
arXiv2023

DeePKS Model for Halide Perovskites with the Accuracy of Hybrid Functional

Accurate prediction for the electronic structure properties of halide perovskites plays a significant role in the design of highly efficient and stable solar cells. While density functional theory (DFT) within the generalized gradient approximation (GGA) offers reliable prediction in terms of lattice constants and potential energy surface for halide perovskites, it severely underestimates the band gap due to the lack of non-local exact exchange term, which exists in computationally expensive hybrid functionals. In this work, a universal Deep Kohn-Sham (DeePKS) model based on neural network is trained so as to enable electronic structure calculations with the accuracy of hybrid functional HSE06 and the efficiency comparable to GGA functional, for a plethora of halide perovskites, i.e., ABX$_3$ (A=FA, MA, Cs; B=Sn, Pb; X=Cl, Br, I). Forces, band gaps, and density of states (DOS) predicted by our DeePKS model for all aforementioned perovskites are in good agreement with the HSE06 results, with significantly improved efficiency. In addition, even though the spin-orbit coupling (SOC) effect has not been taken into consideration during the training process, DeePKS+SOC offers highly consistent band gap and DOS as compared to HSE06+SOC for Pb-containing systems. We believe such DeePKS model can be readily applied for an accurate yet efficient prediction of various properties for the family of halide perovskites.

physics.chem-ph↗
arXiv2023

$m^\ast$ of two-dimensional electron gas: a neural canonical transformation study

The quasiparticle effective mass $m^\ast$ of interacting electrons is a fundamental quantity in the Fermi liquid theory. However, the precise value of the effective mass of uniform electron gas is still elusive after decades of research. The newly developed neural canonical transformation approach [Xie et al., J. Mach. Learn. 1, (2022)] offers a principled way to extract the effective mass of electron gas by directly calculating the thermal entropy at low temperature. The approach models a variational many-electron density matrix using two generative neural networks: an autoregressive model for momentum occupation and a normalizing flow for electron coordinates. Our calculation reveals a suppression of effective mass in the two-dimensional spin-polarized electron gas, which is more pronounced than previous reports in the low-density strong-coupling region. This prediction calls for verification in two-dimensional electron gas experiments.

cond-mat.stat-mech↗
arXiv2023

Revisiting Data Augmentation in Model Compression: An Empirical and Comprehensive Study

The excellent performance of deep neural networks is usually accompanied by a large number of parameters and computations, which have limited their usage on the resource-limited edge devices. To address this issue, abundant methods such as pruning, quantization and knowledge distillation have been proposed to compress neural networks and achieved significant breakthroughs. However, most of these compression methods focus on the architecture or the training method of neural networks but ignore the influence from data augmentation. In this paper, we revisit the usage of data augmentation in model compression and give a comprehensive study on the relation between model sizes and their optimal data augmentation policy. To sum up, we mainly have the following three observations: (A) Models in different sizes prefer data augmentation with different magnitudes. Hence, in iterative pruning, data augmentation with varying magnitudes leads to better performance than data augmentation with a consistent magnitude. (B) Data augmentation with a high magnitude may significantly improve the performance of large models but harm the performance of small models. Fortunately, small models can still benefit from strong data augmentations by firstly learning them with "additional parameters" and then discard these "additional parameters" during inference. (C) The prediction of a pre-trained large model can be utilized to measure the difficulty of data augmentation. Thus it can be utilized as a criterion to design better data augmentation policies. We hope this paper may promote more research on the usage of data augmentation in model compression.

cs.CV↗
arXiv2023

Thermal Conductivity of Water at Extreme Conditions

Measuring the thermal conductivity ($κ$) of water at extreme conditions is a challenging task and few experimental data are available. We predict $κ$ for temperatures and pressures relevant to the conditions of the Earth mantle, between 1,000 and 2,000 K and up to 22 GPa. We employ close to equilibrium molecular dynamics simulations and a deep neural network potential fitted to density functional theory data. We then interpret our results by computing the equation of state of water on a fine grid of points and using a simple model for $κ$. We find that the thermal conductivity is weakly dependent on temperature and monotonically increases with pressure with an approximate square-root behavior. In addition we show how the increase of $κ$ at high pressure, relative to ambient conditions, is related to the corresponding increase in the sound velocity. Although the relationships between the thermal conductivity, pressure and sound velocity established here are not rigorous, they are sufficiently accurate to allow for a robust estimate of the thermal conductivity of water in a broad range of temperatures and pressures, where experiments are still difficult to perform.

cond-mat.mtrl-sci↗
arXiv2023

CORSD: Class-Oriented Relational Self Distillation

Knowledge distillation conducts an effective model compression method while holding some limitations:(1) the feature based distillation methods only focus on distilling the feature map but are lack of transferring the relation of data examples; (2) the relational distillation methods are either limited to the handcrafted functions for relation extraction, such as L2 norm, or weak in inter- and intra- class relation modeling. Besides, the feature divergence of heterogeneous teacher-student architectures may lead to inaccurate relational knowledge transferring. In this work, we propose a novel training framework named Class-Oriented Relational Self Distillation (CORSD) to address the limitations. The trainable relation networks are designed to extract relation of structured data input, and they enable the whole model to better classify samples by transferring the relational knowledge from the deepest layer of the model to shallow layers. Besides, auxiliary classifiers are proposed to make relation networks capture class-oriented relation that benefits classification task. Experiments demonstrate that CORSD achieves remarkable improvements. Compared to baseline, 3.8%, 1.5% and 4.5% averaged accuracy boost can be observed on CIFAR100, ImageNet and CUB-200-2011, respectively.

cs.CV↗
arXiv2023

Imperfectly coordinated water molecules pave the way for homogeneous ice nucleation

Water freezing is ubiquitous on Earth, affecting many areas from biology to climate science and aviation technology. Probing the atomic structure in the homogeneous ice nucleation process from scratch is of great value but still experimentally unachievable. Theoretical simulations have found that ice originates from the low-mobile region with increasing abundance and persistence of tetrahedrally coordinated water molecules. However, a detailed microscopic picture of how the disordered hydrogen-bond network rearranges itself into an ordered network is still unclear. In this work, we use a deep neural network (DNN) model to "learn" the interatomic potential energy from quantum mechanical data, thereby allowing for large-scale and long molecular dynamics (MD) simulations with ab initio accuracy. The nucleation mechanism and dynamics at atomic resolution, represented by a total of 36 $μ$s-long MD trajectories, are deeply affected by the structural and dynamical heterogeneity in supercooled water. We find that imperfectly coordinated (IC) water molecules with high mobility pave the way for hydrogen-bond network rearrangement, leading to the growth or shrinkage of the ice nucleus. The hydrogen-bond network formed by perfectly coordinated (PC) molecules stabilizes the nucleus, thus preventing it from vanishing and growing. Consequently, ice is born through competition and cooperation between IC and PC molecules. We anticipate that our picture of the microscopic mechanism of ice nucleation will provide new insights into many properties of water and other relevant materials.

cond-mat.mtrl-sci↗