Abstract
We present a benchmark test suite and an automated machine learning procedure for evaluating supervised machine learning (ML) models for predicting properties of inorganic bulk materials. The test suite, Matbench, is a set of 13 ML tasks that range in size from 312 to 132k samples and contain data from 10 density functional theory-derived and experimental sources. Tasks include predicting optical, thermal, electronic, thermodynamic, tensile, and elastic properties given a material’s composition and/or crystal structure. The reference algorithm, Automatminer, is a highly-extensible, fully automated ML pipeline for predicting materials properties from materials primitives (such as composition and crystal structure) without user intervention or hyperparameter tuning. We test Automatminer on the Matbench test suite and compare its predictive power with state-of-the-art crystal graph neural networks and a traditional descriptor-based Random Forest model. We find Automatminer achieves the best performance on 8 of 13 tasks in the benchmark. We also show our test suite is capable of exposing predictive advantages of each algorithm—namely, that crystal graph methods appear to outperform traditional machine learning methods given ~104 or greater data points. We encourage evaluating materials ML algorithms on the Matbench benchmark and comparing them against the latest version of Automatminer.
Highlights
New functional materials are vital for making fundamental advances across scientific domains, including computing and energy conversion
We test Automatminer on the test suite in order to establish baseline performance, and we present a comparison of Automatminer with published machine learning (ML) methods
The Matbench test suite v0.1 contains 13 supervised ML tasks following provides a high-level overview of each stage
Summary
New functional materials are vital for making fundamental advances across scientific domains, including computing and energy conversion. The test suite, is a collection of 13 materials sciencespecific data mining tasks curated to reflect the diversity of modern materials data Containing both traditional small materials datasets of only a few hundred samples and large datasets of >105 samples from simulation-derived databases, Matbench provides a consistent nested cross-validation[18] (NCV) method for estimating regression and classification errors on a range of mechanical, electronic, and thermodynamic material properties. In contrast to other published models that are trained to predict a specific property, Automatminer is capable of predicting any materials property given materials primitives (e.g., chemical composition) as input when provided with a suitable training dataset It does this by performing a procedure similar to a human researcher: by generating descriptors using Matminer’s library[19] of published abundant properties such as DFT-GGA21 formation energies. Materials-specific featurizations, performing feature reduction and data preprocessing, and determining the best machine learning Automatminer reference algorithm model by internally testing various possibilities on validation data. Automatminer can create persistent end-toend pipelines containing all internal training data, configuration, and the best-found model—allowing the final models to be further inspected, shared, and reproduced
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
