arXiv subjects

Jianjun Hu

Publications and source records attributed to Jianjun Hu.

At least 55 records · Page 3Linked to original sources

arXiv2021

Scalable deeper graph neural networks for high-performance materials property prediction

Machine learning (ML) based materials discovery has emerged as one of the most promising approaches for breakthroughs in materials science. While heuristic knowledge based descriptors have been combined with ML algorithms to achieve good performance, the complexity of the physicochemical mechanisms makes it urgently needed to exploit representation learning from either compositions or structures for building highly effective materials machine learning models. Among these methods, the graph neural networks have shown the best performance by its capability to learn high-level features from crystal structures. However, all these models suffer from their inability to scale up the models due to the over-smoothing issue of their message-passing GNN architecture. Here we propose a novel graph attention neural network model DeeperGATGNN with differentiable group normalization and skip-connections, which allows to train very deep graph neural network models (e.g. 30 layers compared to 3-9 layers in previous works). Through systematic benchmark studies over six benchmark datasets for energy and band gap predictions, we show that our scalable DeeperGATGNN model needs little costly hyper-parameter tuning for different datasets and achieves the state-of-the-art prediction performances over five properties out of six with up to 10\% improvement. Our work shows that to deal with the high complexity of mapping the crystal materials structures to their properties, large-scale very deep graph neural networks are needed to achieve robust performances.

cond-mat.mtrl-sci↗

arXiv2021

MaterialsAtlas.org: A Materials Informatics Web App Platform for Materials Discovery and Survey of State-of-the-Art

The availability and easy access of large scale experimental and computational materials data have enabled the emergence of accelerated development of algorithms and models for materials property prediction, structure prediction, and generative design of materials. However, lack of user-friendly materials informatics web servers has severely constrained the wide adoption of such tools in the daily practice of materials screening, tinkering, and design space exploration by materials scientists. Herein we first survey current materials informatics web apps and then propose and develop MaterialsAtlas.org, a web based materials informatics toolbox for materials discovery, which includes a variety of routinely needed tools for exploratory materials discovery, including materials composition and structure check (e.g. for neutrality, electronegativity balance, dynamic stability, Pauling rules), materials property prediction (e.g. band gap, elastic moduli, hardness, thermal conductivity), and search for hypothetical materials. These user-friendly tools can be freely accessed at \url{www.materialsatlas.org}. We argue that such materials informatics apps should be widely developed by the community to speed up the materials discovery processes.

cond-mat.mtrl-sci↗

arXiv2021

Big Data driven Product Design: A Survey

With the improvement of living standards, user requirements of modern products are becoming increasingly more diversified and personalized. Traditional product design methods can no longer satisfy the market needs due to their strong subjectivity, small survey scope, poor real-time data, and lack of visual display, which calls for the development of big data driven product design methodology. Big data in the product lifecycle contains valuable information for guiding product design, such as customer preferences, market demands, product evaluation, and visual display: online product reviews reflect customer evaluations and requirements; product images contain information of shape,color, and texture which can inspire designers to get initial design schemes more quickly or even directly generate new product images. How to efficiently collect product design related data and exploit them effectively during the whole product design process is thus critical to modern product design. This paper aims to conduct a comprehensive survey on big data driven product design. It will help researchers and practitioners to comprehend the latest development of relevant studies and applications centered on how big data can be processed, analyzed, and exploited in aiding product design. We first introduce several representative traditional product design methods and highlight their limitations. Then we discuss current and potential applications of textual data, image data, audio data, and video data in product design cycles. Finally, major deficiencies of existing data driven product design studies and future research directions are summarized. We believe that this study can draw increasing attention to modern data driven product design.

cs.HC↗

arXiv2021

Crystal structure prediction using age-fitness multi-objective genetic algorithm and coordination number constraints

Crystal structure prediction (CSP) has emerged as one of the most important approaches for discovering new materials. CSP algorithms based on evolutionary algorithms and particle swarm optimization have discovered a great number of new materials. However, these algorithms based on ab initio calculation of free energy are inefficient. Moreover, they have severe limitations in terms of scalability. We recently proposed a promising crystal structure prediction method based on atomic contact maps, using global optimization algorithms to search for the Wyckoff positions by maximizing the match between the contact map of the predicted structure and the contact map of the true crystal structure. However, our previous contact map based CSP algorithms have two major limitations: (1) the loss of search capability due to getting trapped in local optima; (2) it only uses the connection of atoms in the unit cell to predict the crystal structure, ignoring the chemical environment outside the unit cell, which may lead to unreasonable coordination environments. Herein we propose a novel multi-objective genetic algorithms for contact map-based crystal structure prediction by optimizing three objectives, including contact map match accuracy, the individual age, and the coordination number match. Furthermore, we assign the age values to all the individuals of the GA and try to minimize the age aiming to avoid the premature convergence problem. Our experimental results show that compared to our previous CMCrystal algorithm, our multi-objective crystal structure prediction algorithm (CMCrystalMOO) can reconstruct the crystal structure with higher quality and alleviate the problem of premature convergence.

cond-mat.mtrl-sci↗

arXiv2021

Composition based crystal materials symmetry prediction using machine learning with enhanced descriptors

Geometric information such as the space groups and crystal systems plays an important role in the properties of crystal materials. Prediction of crystal system and space group thus has wide applications in crystal material property estimation and structure prediction. Previous works on experimental X-ray diffraction (XRD) and density functional theory (DFT) based structure determination methods achieved outstanding performance, but they are not applicable for large-scale screening of materials compositions. There are also machine learning models using Magpie descriptors for composition based material space group determination, but their prediction accuracy only ranges between 0.638 and 0.907 in different kinds of crystals. Herein, we report an improved machine learning model for predicting the crystal system and space group of materials using only the formula information. Benchmark study on a dataset downloaded from Materials Project Database shows that our random forest models based on our new descriptor set, achieve significant performance improvements compared with previous work with accuracy scores ranging between 0.712 and 0.961 in terms of space group classification. Our model also shows large performance improvement for crystal system prediction. Trained models and source code are freely available at \url{https://github.com/Yuxinya/SG_predict}

cond-mat.mtrl-sci↗

arXiv2021

Crystal structure prediction of materials with high symmetry using differential evolution

Crystal structure determines properties of materials. With the crystal structure of a chemical substance, many physical and chemical properties can be predicted by first-principles calculations or machine learning models. Since it is relatively easy to generate a hypothetical chemically valid formula, crystal structure prediction becomes an important method for discovering new materials. In our previous work, we proposed a contact map-based crystal structure prediction method, which uses global optimization algorithms such as genetic algorithms to maximize the match between the contact map of the predicted structure and the contact map of the real crystal structure to search for the coordinates at the Wyckoff Positions(WP). However, when predicting the crystal structure with high symmetry, we found that the global optimization algorithm has difficulty to find an effective combination of WPs that satisfies the chemical formula, which is mainly caused by the inconsistency between the dimensionality of the contact map of the predicted crystal structure and the dimensionality of the contact map of the target crystal structure. This makes it challenging to predict the crystal structures of high-symmetry crystals. In order to solve this problem, here we propose to use PyXtal to generate and filter random crystal structures with given symmetry constraints based on the information such as chemical formulas and space groups. With contact map as the optimization goal, we use differential evolution algorithms to search for non-special coordinates at the Wyckoff positions to realize the structure prediction of high-symmetry crystal materials. Our experimental results show that our proposed algorithm CMCrystalHS can effectively solve the problem of inconsistent contact map dimensions and predict the crystal structures with high symmetry.

cond-mat.mtrl-sci↗

arXiv2021

SoundCLR: Contrastive Learning of Representations For Improved Environmental Sound Classification

Environmental Sound Classification (ESC) is a challenging field of research in non-speech audio processing. Most of current research in ESC focuses on designing deep models with special architectures tailored for specific audio datasets, which usually cannot exploit the intrinsic patterns in the data. However recent studies have surprisingly shown that transfer learning from models trained on ImageNet is a very effective technique in ESC. Herein, we propose SoundCLR, a supervised contrastive learning method for effective environment sound classification with state-of-the-art performance, which works by learning representations that disentangle the samples of each class from those of other classes. Our deep network models are trained by combining a contrastive loss that contributes to a better probability output by the classification layer with a cross-entropy loss on the output of the classifier layer to map the samples to their respective 1-hot encoded labels. Due to the comparatively small sizes of the available environmental sound datasets, we propose and exploit a transfer learning and strong data augmentation pipeline and apply the augmentations on both the sound signals and their log-mel spectrograms before inputting them to the model. Our experiments show that our masking based augmentation technique on the log-mel spectrograms can significantly improve the recognition performance. Our extensive benchmark experiments show that our hybrid deep network models trained with combined contrastive and cross-entropy loss achieved the state-of-the-art performance on three benchmark datasets ESC-10, ESC-50, and US8K with validation accuracies of 99.75\%, 93.4\%, and 86.49\% respectively. The ensemble version of our models also outperforms other top ensemble methods. The code is available at https://github.com/alireza-nasiri/SoundCLR.

cs.SD↗

arXiv2021

Active learning based generative design for the discovery of wide bandgap materials

Active learning has been increasingly applied to screening functional materials from existing materials databases with desired properties. However, the number of known materials deposited in the popular materials databases such as ICSD and Materials Project is extremely limited and consists of just a tiny portion of the vast chemical design space. Herein we present an active generative inverse design method that combines active learning with a deep variational autoencoder neural network and a generative adversarial deep neural network model to discover new materials with a target property in the whole chemical design space. The application of this method has allowed us to discover new thermodynamically stable materials with high band gap (SrYF$_5$) and semiconductors with specified band gap ranges (SrClF$_3$, CaClF$_5$, YCl$_3$, SrC$_2$F$_3$, AlSCl, As$_2$O$_3$), all of which are verified by the first principle DFT calculations. Our experiments show that while active learning itself may sample chemically infeasible candidates, these samples help to train effective screening models for filtering out materials with desired properties from the hypothetical materials created by the generative model. The experiments show the effectiveness of our active generative inverse design approach.

cond-mat.mtrl-sci↗

arXiv2021

NODE-SELECT: A Graph Neural Network Based On A Selective Propagation Technique

While there exists a wide variety of graph neural networks (GNN) for node classification, only a minority of them adopt mechanisms that effectively target noise propagation during the message-passing procedure. Additionally, a very important challenge that significantly affects graph neural networks is the issue of scalability which limits their application to larger graphs. In this paper we propose our method named NODE-SELECT: an efficient graph neural network that uses subsetting layers which only allow the best sharing-fitting nodes to propagate their information. By having a selection mechanism within each layer which we stack in parallel, our proposed method NODE-SELECT is able to both reduce the amount noise propagated and adapt the restrictive sharing concept observed in real world graphs. Our NODE-SELECT significantly outperformed existing GNN frameworks in noise experiments and matched state-of-the art results in experiments without noise over different benchmark datasets.

cs.LG↗

arXiv2021

Contact Map based Crystal Structure Prediction using Global Optimization

Crystal structure prediction is now playing an increasingly important role in discovery of new materials. Global optimization methods such as genetic algorithms (GA) and particle swarm optimization (PSO) have been combined with first principle free energy calculations to predict crystal structures given composition or only a chemical system. While these approaches can exploit certain crystal patterns such as symmetry and periodicity in their search process, they usually do not exploit the large amount of implicit rules and constraints of atom configurations embodied in the large number of known crystal structures. They currently can only handle crystal structure prediction of relatively small systems. Inspired by the knowledge-rich protein structure prediction approach, herein we explore whether known geometric constraints such as the atomic contact map of a target crystal material can help predict its structure given its space group information. We propose a global optimization based algorithm, CMCrystal, for crystal structure reconstruction based on atomic contact maps. Based on extensive experiments using six global optimization algorithms, we show that it is viable to reconstruct the crystal structure given the atomic contact map for some crystal materials but more constraints are needed for other target materials to achieve successful reconstruction. This implies that atomic interaction information learned from existing materials can be used to improve crystal structure prediction.

cond-mat.mtrl-sci↗

arXiv2021

High-throughput discovery of novel cubic crystal materials using deep generative neural networks

High-throughput screening has become one of the major strategies for the discovery of novel functional materials. However, its effectiveness is severely limited by the lack of quantity and diversity of known materials deposited in the current materials repositories such as ICSD and OQMD. Recent progress in machine learning and especially deep learning have enabled a generative strategy that learns implicit chemical rules for creating chemically valid hypothetical materials with new compositions and structures. However, current materials generative models have difficulty in generating structurally diverse, chemically valid, and stable materials. Here we propose CubicGAN, a generative adversarial network (GAN) based deep neural network model for large scale generation of novel cubic crystal structures. When trained on 375,749 ternary crystal materials from the OQMD database, we show that our model is able to not only rediscover most of the currently known cubic materials but also generate hypothetical materials of new structure prototypes. A total of 506 such new materials (all of them are either ternary or quarternary) have been verified by DFT based phonon dispersion stability check, several of which have been found to potentially have exceptional functional properties. Considering the importance of cubic materials in wide applications such as solar cells and lithium batteries, our GAN model provides a promising approach to significantly expand the current repository of materials, enabling the discovery of new functional materials via screening. The new crystal structures finally verified by DFT are freely accessible at our Carolina Materials Database http://www.carolinamatdb.org.

cond-mat.mtrl-sci↗

arXiv2020

ComplexFace: a Multi-Representation Approach for Image Classification with Small Dataset

State-of-the-art face recognition algorithms are able to achieve good performance when sufficient training images are provided. Unfortunately, the number of facial images is limited in some real face recognition applications. In this paper, we propose ComplexFace, a novel and effective algorithm for face recognition with limited samples using complex number based data augmentation. The algorithm first generates new representations from original samples and then fuse both into complex numbers, which avoids the difficulty of weight setting in other fusion approaches. A test sample can then be expressed by the linear combination of all the training samples, which mapped the sample to the new representation space for classification by the kernel function. The collaborative representation based classifier is then built to make predictions. Extensive experiments on the Georgia Tech (GT) face database and the ORL face database show that our algorithm significantly outperforms existing methods: the average errors of previous approaches ranging from 31.66% to 41.75% are reduced to 14.54% over the GT database; the average errors of previous approaches ranging from 5.21% to 10.99% are reduced to 1.67% over the ORL database. In other words, our algorithm has decreased the average errors by up to 84.80% on the ORL database.

cs.CV↗

arXiv2020

Computational discovery of new 2D materials using deep learning generative models

Two dimensional (2D) materials have emerged as promising functional materials with many applications such as semiconductors and photovoltaics because of their unique optoelectronic properties. While several thousand 2D materials have been screened in existing materials databases, discovering new 2D materials remains to be challenging. Herein we propose a deep learning generative model for composition generation combined with random forest based 2D materials classifier to discover new hypothetical 2D materials. Furthermore, a template based element substitution structure prediction approach is developed to predict the crystal structures of a subset of the newly predicted hypothetical formulas, which allows us to confirm their structure stability using DFT calculations. So far, we have discovered 267,489 new potential 2D materials compositions and confirmed twelve 2D/layered materials by DFT formation energy calculation. Our results show that generative machine learning models provide an effective way to explore the vast chemical design space for new 2D materials discovery.

cond-mat.mtrl-sci↗

arXiv2020

MLatticeABC: Generic Lattice Constant Prediction of Crystal Materials using Machine Learning

Lattice constants such as unit cell edge lengths and plane angles are important parameters of the periodic structures of crystal materials. Predicting crystal lattice constants has wide applications in crystal structure prediction and materials property prediction. Previous work has used machine learning models such as neural networks and support vector machines combined with composition features for lattice constant prediction and has achieved a maximum performance for cubic structures with an average $R^2$ of 0.82. Other models tailored for special materials family of a fixed form such as ABX3 perovskites can achieve much higher performance due to the homogeneity of the structures. However, these models trained with small datasets are usually not applicable to generic lattice parameter prediction of materials with diverse compositions. Herein, we report MLatticeABC, a random forest machine learning model with a new descriptor set for lattice unit cell edge length ($a,b,c$) prediction which achieves an R2 score of 0.979 for lattice parameter $a$ of cubic crystals and significant performance improvement for other crystal systems as well. Source code and trained models can be freely accessed at https://github.com/usccolumbia/MLatticeABC

cond-mat.mtrl-sci↗

arXiv2020

Distance Matrix based Crystal Structure Prediction using Evolutionary Algorithms

Crystal structure prediction (CSP) for inorganic materials is one of the central and most challenging problems in materials science and computational chemistry. This problem can be formulated as a global optimization problem in which global search algorithms such as genetic algorithms (GA) and particle swarm optimization have been combined with first principle free energy calculations to predict crystal structures given only a material composition or only a chemical system. These DFT based ab initio CSP algorithms are computationally demanding and can only be used to predict crystal structures of relatively small systems. The vast coordinate space plus the expensive DFT free energy calculations limits their efficiency and effectiveness. On the other hand, a similar structure prediction problem has been intensively investigated in parallel in the protein structure prediction community of bioinformatics, in which the dominating predictors are knowledge based approaches including homology modeling and threading that exploit known protein structures. Herein we explore whether known geometric constraints such as the pairwise atomic distances of a target crystal material can help predict/reconstruct its structure given its space group and lattice information. We propose DMCrystal, a genetic algorithm based crystal structure reconstruction algorithm based on predicted atomic pairwise distances. Based on extensive experiments, we show that the predicted distance matrix can dramatically help to reconstruct the crystal structure and usually achieves much better performance than CMCrystal, an atomic contact map based crystal structure prediction algorithm. This implies that knowledge of atomic interaction information learned from existing materials can be used to significantly improve the crystal structure prediction in terms of both speed and quality.

cond-mat.mtrl-sci↗

arXiv2020

A Survey on Machine Reading Comprehension: Tasks, Evaluation Metrics and Benchmark Datasets

Machine Reading Comprehension (MRC) is a challenging Natural Language Processing(NLP) research field with wide real-world applications. The great progress of this field in recent years is mainly due to the emergence of large-scale datasets and deep learning. At present, a lot of MRC models have already surpassed human performance on various benchmark datasets despite the obvious giant gap between existing MRC models and genuine human-level reading comprehension. This shows the need for improving existing datasets, evaluation metrics, and models to move current MRC models toward "real" understanding. To address the current lack of comprehensive survey of existing MRC tasks, evaluation metrics, and datasets, herein, (1) we analyze 57 MRC tasks and datasets and propose a more precise classification method of MRC tasks with 4 different attributes; (2) we summarized 9 evaluation metrics of MRC tasks, 7 attributes and 10 characteristics of MRC datasets; (3) We also discuss key open issues in MRC research and highlighted future research directions. In addition, we have collected, organized, and published our data on the companion website(https://mrc-datasets.github.io/) where MRC researchers could directly access each MRC dataset, papers, baseline projects, and the leaderboard.

cs.CL↗

arXiv2020

Inverse Design of Composite Metal Oxide Optical Materials based on Deep Transfer Learning

Optical materials with special optical properties are widely used in a broad span of technologies, from computer displays to solar energy utilization leading to large dataset accumulated from years of extensive materials synthesis and optical characterization. Previously, machine learning models have been developed to predict the optical absorption spectrum from a materials characterization image or vice versa. Herein we propose TLOpt, a transfer learning based inverse optical materials design algorithm for suggesting material compositions with a desired target light absorption spectrum. Our approach is based on the combination of a deep neural network model and global optimization algorithms including a genetic algorithm and Bayesian optimization. A transfer learning strategy is employed to solve the small dataset issue in training the neural network predictor of optical absorption spectrum using the Magpie materials composition descriptor. Our extensive experiments show that our algorithm can inverse design the materials composition with stoichiometry with high accuracy.

cond-mat.mtrl-sci↗

arXiv2020

Lattice Thermal Conductivity Prediction using Symbolic Regression and Machine Learning

Prediction models of lattice thermal conductivity have wide applications in the discovery of thermoelectrics, thermal barrier coatings, and thermal management of semiconductors. kL is notoriously difficult to predict. While classic models such as the Debye-Callaway model and the Slack model have been used to approximate the kL of inorganic compounds, their accuracy is far from being satisfactory. Herein, we propose a genetic programming based Symbolic Regression approach for explicit kL models and compare it with Multi-Layer Perceptron neural networks and a Random Forest Regressor using a hybrid cross-validation approach including both K-Fold CV and holdout validation. Four formulae have been discovered by our symbolic regression approach that outperform the Slack formula as evaluated on our dataset. Through the analysis of our models' performance and the formulae generated, we found that the trained formulae successfully reproduce the correct physical law that governs the lattice thermal conductivity of materials. We also identified that extrapolation prediction remains to be a key issue in both symbolic regression and regular machine learning methods and find the distribution of the samples place a key role in training a prediction model with high generalization capability.

cond-mat.mtrl-sci↗