arXiv subjects

Jianjun Hu

Publications and source records attributed to Jianjun Hu.

At least 37 records · Page 2Linked to original sources

arXiv2023

Towards Quantitative Evaluation of Crystal Structure Prediction Performance

Crystal structure prediction (CSP) is now increasingly used in the discovery of novel materials with applications in diverse industries. However, despite decades of developments, the problem is far from being solved. With the progress of deep learning, search algorithms, and surrogate energy models, there is a great opportunity for breakthroughs in this area. However, the evaluation of CSP algorithms primarily relies on manual structural and formation energy comparisons. The lack of a set of well-defined quantitative performance metrics for CSP algorithms make it difficult to evaluate the status of the field and identify the strengths and weaknesses of different CSP algorithms. Here, we analyze the quality evaluation issue in CSP and propose a set of quantitative structure similarity metrics, which when combined can be used to automatically determine the quality of the predicted crystal structures compared to the ground truths. Our CSP performance metrics can be then utilized to evaluate the large set of existing and emerging CSP algorithms, thereby alleviating the burden of manual inspection on a case-by-case basis. The related open-source code can be accessed freely at https://github.com/usccolumbia/CSPBenchMetrics

cond-mat.mtrl-sci↗

arXiv2023

MD-HIT: Machine learning for materials property prediction with dataset redundancy control

Materials datasets are usually featured by the existence of many redundant (highly similar) materials due to the tinkering material design practice over the history of materials research. For example, the materials project database has many perovskite cubic structure materials similar to SrTiO$_3$. This sample redundancy within the dataset makes the random splitting of machine learning model evaluation to fail so that the ML models tend to achieve over-estimated predictive performance which is misleading for the materials science community. This issue is well known in the field of bioinformatics for protein function prediction, in which a redundancy reduction procedure (CD-Hit) is always applied to reduce the sample redundancy by ensuring no pair of samples has a sequence similarity greater than a given threshold. This paper surveys the overestimated ML performance in the literature for both composition based and structure based material property prediction. We then propose a material dataset redundancy reduction algorithm called MD-HIT and evaluate it with several composition and structure based distance threshold sfor reducing data set sample redundancy. We show that with this control, the predicted performance tends to better reflect their true prediction capability. Our MD-hit code can be freely accessed at https://github.com/usccolumbia/MD-HIT

cond-mat.mtrl-sci↗

arXiv2023

AlphaCrystal: Contact map based crystal structure prediction using deep learning

Crystal structure prediction is one of the major unsolved problems in materials science. Traditionally, this problem is formulated as a global optimization problem for which global search algorithms are combined with first principle free energy calculations to predict the ground-state crystal structure given only a material composition or a chemical system. These ab initio algorithms usually cannot exploit a large amount of implicit physicochemical rules or geometric constraints (deep knowledge) of atom configurations embodied in a large number of known crystal structures. Inspired by the deep learning enabled breakthrough in protein structure prediction, herein we propose AlphaCrystal, a crystal structure prediction algorithm that combines a deep residual neural network model that learns deep knowledge to guide predicting the atomic contact map of a target crystal material followed by reconstructing its 3D crystal structure using genetic algorithms. Based on the experiments of a selected set of benchmark crystal materials, we show that our AlphaCrystal algorithm can predict structures close to the ground truth structures. It can also speed up the crystal structure prediction process by predicting and exploiting the predicted contact map so that it has the potential to handle relatively large systems. We believe that our deep learning based ab initio crystal structure prediction method that learns from existing material structures can be used to scale up current crystal structure prediction practice. To our knowledge, AlphaCrystal is the first neural network based algorithm for crystal structure contact map prediction and the first method for directly reconstructing crystal structures from materials composition, which can be further optimized by DFT calculations.

cond-mat.mtrl-sci↗

arXiv2023

Global mapping of structures and properties of crystal materials

Understanding material composition-structure-function relationships is of critical importance for the design and discovery of novel functional materials. While most such studies focus on individual materials, we conducted a global mapping study of all known materials deposited in the Material Project database to investigate their distributions in the space of a set of seven compositional, structural, physical, and neural latent descriptors. These two-dimensional materials maps along with their density maps allow us to illustrate the distribution of the patterns and clusters of different shapes, which indicates the propensity of these materials and the tinkering history of existing materials. We then overlap the material properties such as composition prototypes and piezoelectric properties over the background materials maps to study the relationships of how material compositions and structures affect their physical properties. We also use these maps to study the spatial distributions of properties of known inorganic materials, in particular those of local vicinities in structural space such as structural density and functional diversity. These maps provide a uniquely comprehensive overview of materials and space and thus reveal previously undescribed fundamental properties. Our methodology can be easily extended by other researchers to generate their own global material maps with different background maps and overlap properties for both distribution understanding and cluster-based new material discovery. The source code for feature generation and generated maps are available at https://github.com/usccolumbia/matglobalmapping

cond-mat.mtrl-sci↗

arXiv2023

Discovery of 2D materials using Transformer Network based Generative Design

Two-dimensional (2D) materials have wide applications in superconductors, quantum, and topological materials. However, their rational design is not well established, and currently less than 6,000 experimentally synthesized 2D materials have been reported. Recently, deep learning, data-mining, and density functional theory (DFT)-based high-throughput calculations are widely performed to discover potential new materials for diverse applications. Here we propose a generative material design pipeline, namely material transformer generator(MTG), for large-scale discovery of hypothetical 2D materials. We train two 2D materials composition generators using self-learning neural language models based on Transformers with and without transfer learning. The models are then used to generate a large number of candidate 2D compositions, which are fed to known 2D materials templates for crystal structure prediction. Next, we performed DFT computations to study their thermodynamic stability based on energy-above-hull and formation energy. We report four new DFT-verified stable 2D materials with zero e-above-hull energies, including NiCl$_4$, IrSBr, CuBr$_3$, and CoBrCl. Our work thus demonstrates the potential of our MTG generative materials design pipeline in the discovery of novel 2D materials and other functional materials.

cond-mat.mtrl-sci↗

arXiv2022

Physics Guided Deep Learning for Generative Design of Crystal Materials with Symmetry Constraints

Discovering new materials is a challenging task in materials science crucial to the progress of human society. Conventional approaches based on experiments and simulations are labor-intensive or costly with success heavily depending on experts' heuristic knowledge. Here, we propose a deep learning based Physics Guided Crystal Generative Model (PGCGM) for efficient crystal material design with high structural diversity and symmetry. Our model increases the generation validity by more than 700\% compared to FTCP, one of the latest structure generators and by more than 45\% compared to our previous CubicGAN model. Density Functional Theory (DFT) calculations are used to validate the generated structures with 1,869 materials out of 2,000 are successfully optimized and deposited into the Carolina Materials Database \url{www.carolinamatdb.org}, of which 39.6\% have negative formation energy and 5.3\% have energy-above-hull less than 0.25 eV/atom, indicating their thermodynamic stability and potential synthesizability.

cond-mat.mtrl-sci↗

arXiv2022

Composition based oxidation state prediction of materials using deep learning

Oxidation states are the charges of atoms after their ionic approximation of their bonds, which have been widely used in charge-neutrality verification, crystal structure determination, and reaction estimation. Currently only heuristic rules exist for guessing the oxidation states of a given compound with many exceptions. Recent work has developed machine learning models based on heuristic structural features for predicting the oxidation states of metal ions. However, composition based oxidation state prediction still remains elusive so far, which is more important in new material discovery for which the structures are not even available. This work proposes a novel deep learning based BERT transformer language model BERTOS for predicting the oxidation states of all elements of inorganic compounds given only their chemical composition. Our model achieves 96.82\% accuracy for all-element oxidation states prediction benchmarked on the cleaned ICSD dataset and achieves 97.61\% accuracy for oxide materials. We also demonstrate how it can be used to conduct large-scale screening of hypothetical material compositions for materials discovery.

cond-mat.mtrl-sci↗

arXiv2022

Materials Property Prediction with Uncertainty Quantification: A Benchmark Study

Uncertainty quantification (UQ) has increasing importance in building robust high-performance and generalizable materials property prediction models. It can also be used in active learning to train better models by focusing on getting new training data from uncertain regions. There are several categories of UQ methods each considering different types of uncertainty sources. Here we conduct a comprehensive evaluation on the UQ methods for graph neural network based materials property prediction and evaluate how they truly reflect the uncertainty that we want in error bound estimation or active learning. Our experimental results over four crystal materials datasets (including formation energy, adsorption energy, total energy, and band gap properties) show that the popular ensemble methods for uncertainty estimation is NOT the best choice for UQ in materials property prediction. For the convenience of the community, all the source code and data sets can be accessed freely at \url{https://github.com/usccolumbia/materialsUQ}.

cond-mat.mtrl-sci↗

arXiv2022

Probabilistic Generative Transformer Language models for Generative Design of Molecules

Self-supervised neural language models have recently found wide applications in generative design of organic molecules and protein sequences as well as representation learning for downstream structure classification and functional prediction. However, most of the existing deep learning models for molecule design usually require a big dataset and have a black-box architecture, which makes it difficult to interpret their design logic. Here we propose Generative Molecular Transformer (GMTransformer), a probabilistic neural network model for generative design of molecules. Our model is built on the blank filling language model originally developed for text processing, which has demonstrated unique advantages in learning the "molecules grammars" with high-quality generation, interpretability, and data efficiency. Benchmarked on the MOSES datasets, our models achieve high novelty and Scaf compared to other baselines. The probabilistic generation steps have the potential in tinkering molecule design due to their capability of recommending how to modify existing molecules with explanation, guided by the learned implicit molecule chemistry. The source code and datasets can be accessed freely at https://github.com/usccolumbia/GMTransformer

cond-mat.mtrl-sci↗

arXiv2022

Designing novel protein structures using sequence generator and AlphaFold2

Protein structures and functions are determined by a contiguous arrangement of amino acid sequences. Designing novel protein sequences and structures with desired geometry and functions is a complex task with large state spaces. Here we develop a novel protein design pipeline consisting of two deep learning algorithms, ProteinSolver and AlphaFold2. ProteinSolver is a deep graph neural network that generates amino acid sequences such that the forces between interacting amino acids are favorable and compatible with the fold while AlphaFold2 is a deep learning algorithm that predicts the protein structures from protein sequences. We present forty de novo designed binding sites of the PTP1B and P53 proteins with high precision, out of which thirty proteins are novel. Using ProteinSolver and AlphaFold2 in conjunction, we can trim the exploration of the large protein conformation space, thus expanding the ability to find novel and diverse de novo protein designs.

physics.chem-ph↗

arXiv2022

Materials Transformers Language Models for Generative Materials Design: a benchmark study

Pre-trained transformer language models on large unlabeled corpus have produced state-of-the-art results in natural language processing, organic molecule design, and protein sequence generation. However, no such models have been applied to learn the composition patterns of inorganic materials. Here we train a series of seven modern transformer language models (GPT, GPT-2, GPT-Neo, GPT-J, BLMM, BART, and RoBERTa) using the expanded formulas from material deposited in the ICSD, OQMD, and Materials Projects databases. Six different datasets with/out non-charge-neutral or balanced electronegativity samples are used to benchmark the performances and uncover the generation biases of modern transformer models for the generative design of materials compositions. Our extensive experiments showed that the causal language models based materials transformers can generate chemically valid materials compositions with as high as 97.54\% to be charge neutral and 91.40\% to be electronegativity balanced, which has more than 6 times higher enrichment compared to a baseline pseudo-random sampling algorithm. These models also demonstrate high novelty and their potential in new materials discovery has been proved by their capability to recover the leave-out materials. We also find that the properties of the generated samples can be tailored by training the models with selected training sets such as high-bandgap materials. Our experiments also showed that different models each have their own preference in terms of the properties of the generated samples and their running time complexity varies a lot. We have applied our materials transformer models to discover a set of new materials as validated using DFT calculations.

cond-mat.mtrl-sci↗

arXiv2022

Crystal Transformer: Self-learning neural language model for Generative and Tinkering Design of Materials

Self-supervised neural language models have recently achieved unprecedented success, from natural language processing to learning the languages of biological sequences and organic molecules. These models have demonstrated superior performance in the generation, structure classification, and functional predictions for proteins and molecules with learned representations. However, most of the masking-based pre-trained language models are not designed for generative design, and their black-box nature makes it difficult to interpret their design logic. Here we propose BLMM Crystal Transformer, a neural network based probabilistic generative model for generative and tinkering design of inorganic materials. Our model is built on the blank filling language model for text generation and has demonstrated unique advantages in learning the "materials grammars" together with high-quality generation, interpretability, and data efficiency. It can generate chemically valid materials compositions with as high as 89.7\% charge neutrality and 84.8\% balanced electronegativity, which are more than 4 and 8 times higher compared to a pseudo random sampling baseline. The probabilistic generation process of BLMM allows it to recommend tinkering operations based on learned materials chemistry and makes it useful for materials doping. Combined with the TCSP crysal structure prediction algorithm, We have applied our model to discover a set of new materials as validated using DFT calculations. Our work thus brings the unsupervised transformer language models based generative artificial intelligence to inorganic materials. A user-friendly web app has been developed for computational materials doping and can be accessed freely at \url{www.materialsatlas.org/blmtinker}.

cond-mat.mtrl-sci↗

arXiv2022

Genetic programming-based learning of carbon interatomic potential for materials discovery

Efficient and accurate interatomic potential functions are critical to computational study of materials while searching for structures with desired properties. Traditionally, potential functions or energy landscapes are designed by experts based on theoretical or heuristic knowledge. Here, we propose a new approach to leverage strongly typed parallel genetic programming (GP) for potential function discovery. We use a multi-objective evolutionary algorithm with NSGA-III selection to optimize individual age, fitness, and complexity through symbolic regression. With a DFT dataset of 863 unique carbon allotrope configurations drawn from 858 carbon structures, the generated potentials are able to predict total energies within $\pm 7.70$ eV at low computational cost while generalizing well across multiple carbon structures. Our code is open source and available at \url{http://www.github.com/usccolumbia/mlpotential

cond-mat.mtrl-sci↗

arXiv2022

DeepXRD, a Deep Learning Model for Predicting of XRD spectrum from Materials Composition

One of the long-standing problems in materials science is how to predict a material's structure and then its properties given only its composition. Experimental characterization of crystal structures has been widely used for structure determination, which is however too expensive for high-throughput screening. At the same time, directly predicting crystal structures from compositions remains a challenging unsolved problem. Herein we propose a deep learning algorithm for predicting the XRD spectrum given only the composition of a material, which can then be used to infer key structural features for downstream structural analysis such as crystal system or space group classification or crystal lattice parameter determination or materials property predictions. Benchmark studies on two datasets show that our DeepXRD algorithm can achieve good performance for XRD prediction as evaluated over our test sets. It can thus be used in high-throughput screening in the huge materials composition space for new materials discovery.

cond-mat.mtrl-sci↗

arXiv2021

Semi-supervised teacher-student deep neural network for materials discovery

Data driven generative machine learning models have recently emerged as one of the most promising approaches for new materials discovery. While the generator models can generate millions of candidates, it is critical to train fast and accurate machine learning models to filter out stable, synthesizable materials with desired properties. However, such efforts to build supervised regression or classification screening models have been severely hindered by the lack of unstable or unsynthesizable samples, which usually are not collected and deposited in materials databases such as ICSD and Materials Project (MP). At the same time, there are a significant amount of unlabelled data available in these databases. Here we propose a semi-supervised deep neural network (TSDNN) model for high-performance formation energy and synthesizability prediction, which is achieved via its unique teacher-student dual network architecture and its effective exploitation of the large amount of unlabeled data. For formation energy based stability screening, our semi-supervised classifier achieves an absolute 10.3\% accuracy improvement compared to the baseline CGCNN regression model. For synthesizability prediction, our model significantly increases the baseline PU learning's true positive rate from 87.9\% to 97.9\% using 1/49 model parameters. To further prove the effectiveness of our models, we combined our TSDNN-energy and TSDNN-synthesizability models with our CubicGAN generator to discover novel stable cubic structures. Out of 1000 recommended candidate samples by our models, 512 of them have negative formation energies as validated by our DFT formation energy calculations. Our experimental results show that our semi-supervised deep neural networks can significantly improve the screening accuracy in large-scale generative materials design.

cond-mat.mtrl-sci↗

arXiv2021

Physics guided deep learning generative models for crystal materials discovery

Deep learning based generative models such as deepfake have been able to generate amazing images and videos. However, these models may need significant transformation when applied to generate crystal materials structures in which the building blocks, the physical atoms are very different from the pixels. Naively transferred generative models tend to generate a large portion of physically infeasible crystal structures that are not stable or synthesizable. Herein we show that by exploiting and adding physically oriented data augmentation, loss function terms, and post processing, our deep adversarial network (GAN) based generative models can now generate crystal structures with higher physical feasibility and expand our previous models which can only create cubic structures.

cond-mat.mtrl-sci↗

arXiv2021

TCSP: a Template based crystal structure prediction algorithm and web server for materials discovery

Fast and accurate crystal structure prediction (CSP) algorithms and web servers are highly desirable for exploring and discovering new materials out of the infinite design space. However, currently, the computationally expensive first principle calculation based crystal structure prediction algorithms are applicable to relatively small systems and are out of reach of most materials researchers due to the requirement of high computing resources or the software cost related to ab initio code such as VASP. Several computational teams have used an element substitution approach for generating or predicting new structures, but usually in an ad hoc way. Here we develop a template based crystal structure prediction algorithm (TCSP) and its companion web server, which makes this tool to be accessible to all materials researchers. Our algorithm uses elemental/chemical similarity and oxidation states to guide the selection of template structures and then rank them based on the substitution compatibility and can return multiple predictions with ranking scores in a few minutes. Benchmark study on the ~98,290 formulas of the Materials Project database using leave-one-out evaluation shows that our algorithm can achieve high accuracy (for 13,145 target structures, TCSP predicted their structures with RMSD < 0.1) for a large portion of the formulas. We have also used TCSP to discover new materials of the Ga-B-N system showing its potential for high-throughput materials discovery. Our user-friendly web app TCSP can be accessed freely at \url{www.materialsatlas.org/crystalstructure} on our MaterialsAtlas.org web app platform.

cond-mat.mtrl-sci↗

arXiv2021

Predicting Lattice Phonon Vibrational Frequencies Using Deep Graph Neural Networks

Lattice vibration frequencies are related to many important materials properties such as thermal and electrical conductivity as well as superconductivity. However, computational calculation of vibration frequencies using density functional theory (DFT) methods is too computationally demanding for a large number of samples in materials screening. Here we propose a deep graph neural network-based algorithm for predicting crystal vibration frequencies from crystal structures with high accuracy. Our algorithm addresses the variable dimension of vibration frequency spectrum using the zero padding scheme. Benchmark studies on two data sets with 15,000 and 35,552 samples show that the aggregated $R^2$ scores of the prediction reaches 0.554 and 0.724 respectively. Our work demonstrates the capability of deep graph neural networks to learn to predict phonon spectrum properties of crystal structures in addition to phonon density of states (DOS) and electronic DOS in which the output dimension is constant.

cond-mat.mtrl-sci↗