AtomSets as a hierarchical transfer learning framework for small and large materials datasets

Chi Chen,Shyue Ping Ong

doi:10.1038/s41524-021-00639-w

Abstract

Predicting properties from a material’s composition or structure is of great interest for materials design. Deep learning has recently garnered considerable interest in materials predictive tasks with low model errors when dealing with large materials data. However, deep learning models suffer in the small data regime that is common in materials science. Here we develop the AtomSets framework, which utilizes universal compositional and structural descriptors extracted from pre-trained graph network deep learning models with standard multi-layer perceptrons to achieve consistently high model accuracy for both small compositional data (<400) and large structural data (>130,000). The AtomSets models show lower errors than the graph network models at small data limits and other non-deep-learning models at large data limits. They also transfer better in a simulated materials discovery process where the targeted materials have property values out of the training data limits. The models require minimal domain knowledge inputs and are free from feature engineering. The presented AtomSets model framework can potentially accelerate machine learning-assisted materials design and discovery with less data restriction.

Highlights

Machine learning (ML) has garnered substantial interest as an effective method for developing surrogate models for materials property predictions in recent years.[1,2] a critical bottleneck is that materials datasets are often small and inhomogeneous, making it challenging to train reliable models.While large density functional theory (DFT) databases such as the Materials Project,[3] Open Quantum Materials Database,[4] and AFLOWLIB5 have ~O(106) relaxed structures and computed energies, data on other computed properties such as band gaps, elastic constants, dielectric constants, etc. tend to be several times or even orders of magnitude fewer.[2]
The hierarchical MatErials Graph Networks (MEGNet) features provide a cascade of descriptors that capture both short-ranged interactions at early graph convolution (GC) (e.g., V0, V1) and long-ranged interactions at later GC (e.g., V2, V3)
We can explain this part by drawing an analogy to convolutional neural networks (CNN) in facial recognition, where the early feature maps capture generic features such as lines and shapes and the later feature maps form human faces.[28]

Summary

Introduction

Machine learning (ML) has garnered substantial interest as an effective method for developing surrogate models for materials property predictions in recent years.[1,2] a critical bottleneck is that materials datasets are often small and inhomogeneous, making it challenging to train reliable models.While large density functional theory (DFT) databases such as the Materials Project,[3] Open Quantum Materials Database,[4] and AFLOWLIB5 have ~O(106) relaxed structures and computed energies, data on other computed properties such as band gaps, elastic constants, dielectric constants, etc. tend to be several times or even orders of magnitude fewer.[2]. Machine learning (ML) has garnered substantial interest as an effective method for developing surrogate models for materials property predictions in recent years.[1,2] a critical bottleneck is that materials datasets are often small and inhomogeneous, making it challenging to train reliable models. Tend to be several times or even orders of magnitude fewer.[2] In general, deep learning models based on neural networks tend to require much more data to train, resulting in lower performance in small datasets relative to non-deep learning models. Most TL studies were performed on the same property.[9,10,11] For example, Hutchinson et al.[9] developed three TL approaches that reduced the model errors in predicting experimental band gaps by including DFT band gaps. Jha et al.[10] trained models on the formation energies in the large OQMD database and demonstrated that transferring the model weights from

Methods

Results

Conclusion