Chemist versus Machine: Traditional Knowledge versus Machine Learning Techniques

Abstract

In the past, traditional chemical heuristics have been very important for the discovery of new materials. Machine learning approaches have started to replace those chemical heuristics in recent years, and they offer new opportunities for materials science. Both approaches are very strongly interconnected.Classical chemical heuristics typically rely on less data than machine learning approaches. There are two different types of machine learning approaches in materials science: one relies on features inspired by classical chemical heuristics, the other one purely relies on relationships within the analyzed data.The growing amount of data offers an opportunity to test traditional chemical heuristics. In combination with machine learning techniques, also new, more data-driven chemical heuristics should be developed. Chemical heuristics have been fundamental to the advancement of chemistry and materials science. These heuristics are typically established by scientists using knowledge and creativity to extract patterns from limited datasets. Machine learning offers opportunities to perfect this approach using computers and larger datasets. Here, we discuss the relationships between traditional heuristics and machine learning approaches. We show how traditional rules can be challenged by large-scale statistical assessment and how traditional concepts commonly used as features are feeding the machine learning techniques. We stress the waste involved in relearning chemical rules and the challenges in terms of data size requirements for purely data-driven approaches. Our view is that heuristic and machine learning approaches are at their best when they work together. Chemical heuristics have been fundamental to the advancement of chemistry and materials science. These heuristics are typically established by scientists using knowledge and creativity to extract patterns from limited datasets. Machine learning offers opportunities to perfect this approach using computers and larger datasets. Here, we discuss the relationships between traditional heuristics and machine learning approaches. We show how traditional rules can be challenged by large-scale statistical assessment and how traditional concepts commonly used as features are feeding the machine learning techniques. We stress the waste involved in relearning chemical rules and the challenges in terms of data size requirements for purely data-driven approaches. Our view is that heuristic and machine learning approaches are at their best when they work together. Data science (see Glossary), artificial intelligence, and machine learning are nowadays present in all fields of science and technology, including chemistry and materials science. The impact of these techniques is expected to be very large, leading to a new path towards scientific discovery (sometimes called the fourth paradigm in science) [1.Agrawal A. Choudhary A. Perspective: materials informatics and big data: realization of the “fourth paradigm” of science in materials science.APL Mater. 2016; 4053208Crossref Scopus (425) Google Scholar,2.Hey T. et al.The Fourth Paradigm: Data-Intensive Scientific Discovery. Microsoft Research, 2009Google Scholar]. We will not review here all progress and future directions in the use of machine learning in materials science; we refer interested readers to recent reviews on this topic (e.g., [3.Schmidt J. et al.Recent advances and applications of machine learning in solid-state materials science.NPJ Comput. Mater. 2019; 5: 1-36Crossref Scopus (594) Google Scholar, 4.Butler K.T. et al.Machine learning for molecular and materials science.Nature. 2018; 559: 547-555Crossref PubMed Scopus (1218) Google Scholar, 5.Deringer V.L. et al.Machine learning interatomic potentials as emerging tools for materials science.Adv. Mater. 2019; 311902765Crossref Scopus (160) Google Scholar, 6.Schleder G.R. et al.From DFT to machine learning: recent approaches to materials science–a review.J. Phys. Mater. 2019; 2032001Crossref Scopus (225) Google Scholar]). Instead, we offer a personal perspective on how these new techniques, heavily relying on sophisticated algorithms and large data sets, compete, complement, challenge, and/or benefit from more traditional heuristic approaches. Since the early days of chemistry, scientists have looked for patterns from often limited sets of data. This led to many of today’s widely used chemical heuristics such as the periodic system of elements, electronegativities, and atomic radii. These heuristic models were built by combining the knowledge and creativity of the scientist with simplified physical pictures. For example, Pettifor introduced a completely new chemical scale that enabled separation of structure types of AB compounds within a 2D map. This scale was based on the size, valence, and electronegativity of the constituting atoms [7.Pettifor D.G. A chemical scale for crystal-structure maps.Solid State Commun. 1984; 51: 31-34Crossref Scopus (171) Google Scholar]. We will call this traditional approach classical or chemical heuristics throughout this article (Figure 1, Key Figure). Recently, the traditional heuristic approach has started to be replaced by machine learning techniques. This is not only due to improvements in machine learning methods that are now widely available through open source software but also to an ever-increasing amount of data available mainly through high-throughput ab initio computations [8.Jain A. et al.Commentary: the materials project: a materials genome approach to accelerating materials innovation.APL Mater. 2013; 1011002Crossref Scopus (3051) Google Scholar, 9.Draxl C. Scheffler M. The NOMAD laboratory: from data sharing to artificial intelligence.J. Phys. Mater. 2019; 2036001Crossref Scopus (83) Google Scholar, 10.Curtarolo S. et al.AFLOWLIB.ORG: a distributed materials properties repository from high-throughput ab initio calculations.Comput. Mater. Sci. 2012; 58: 227-235Crossref Scopus (572) Google Scholar, 11.Álvarez-Moreno M. et al.Managing the computational chemistry big data problem: the IoChem-BD platform.J. Chem. Inf. Model. 2015; 55: 95-103Crossref PubMed Scopus (258) Google Scholar]. These machine learning approaches can be divided into two categories. The first approach uses features often based on chemical heuristics (e.g., atomic radii or electronegativity) as inputs to machine learning techniques that can provide a relationship between these features and material properties. Here, machine learning brings new relationships to properties from traditional heuristic descriptors. For instance, machine learning has been used to find a mathematical relationship between well-known atomic features (e.g., atomic radii and ionization potential) to discriminate between wurtzite and rock salt forming binaries [12.Ghiringhelli L.M. et al.Big data of materials science: critical role of the descriptor.Phys. Rev. Lett. 2015; 114105503Crossref PubMed Scopus (467) Google Scholar]. The second approach relies only on rules derived from relationships within the data bypassing traditional chemical descriptors. For example, the probability of certain ions to replace each other has been derived in this manner [13.Hautier G. et al.Data mined ionic substitutions for the discovery of new compounds.Inorg. Chem. 2011; 50: 656-663Crossref PubMed Scopus (235) Google Scholar]. No physical/chemical feature such as atomic radius was implied in this study, only the idea that some atoms are more likely to replace other atoms than others. The three different approaches that include traditional chemical heuristics, the extraction of rules between established chemical features using machine learning, and purely data-driven approaches are summarized in Figure 1. The chemical heuristics approaches require less data but are inherently biased towards preconception that could turn out to be incorrect. Conversely, the purely data-driven approach could require data set sizes that are sometimes not available. Many of the heuristics that are still taught in general chemistry courses date back from at least a century ago. The concept of electronegativity was proposed by Avogadro and Berzelius in ~1809 [14.Jensen W.B. Electronegativity from Avogadro to Pauling: part 1: origins of the electronegativity concept.J. Chem. Educ. 1996; 73: 11Crossref Google Scholar], oxidation states by Wöhler (~1835) [15.Karen P. Oxidation state, a long-standing issue!.Angew. Chem. Int. Ed. 2015; 54: 4716-4726Crossref PubMed Scopus (62) Google Scholar], atomic radii by Loschmidt (~1866) [16.Rahm M. et al.Atomic and ionic radii of elements 1-96.Chem. Eur. J. 2016; 22: 14625-14632Crossref PubMed Scopus (134) Google Scholar], and the periodic table of elements dates 150 years back to Mendelejew [17.Mendelejew D. Über die Beziehungen der Eigenschaften zu den Atomgewichten der Elemente.Z. Chem. 1869; 12: 405-406Google Scholar]. Goldschmidt and Pauling derived rules on the stability of crystal structures more than 90 years ago [18.Pauling L. The principles determining the structure of complex ionic crystals.J. Am. Chem. Soc. 1929; 51: 1010Crossref Scopus (1394) Google Scholar,19.Goldschmidt V.M. Die Gesetze der Krystallochemie.Naturwissenschaften. 1926; 14: 477-485Crossref Scopus (1895) Google Scholar]. Without any doubt, chemical heuristics have been instrumental to advances in chemistry and materials sciences over the last 100 years. However, one must be careful to not use these heuristics blindly. Their historical importance should not exclude them from critical assessment. Rahm and colleagues have recently shown that two of our most fundamental chemical heuristics, the periodic table and electronegativities, can change drastically at high pressures [20.Rahm M. et al.Squeezing all elements in the periodic table: electron configuration and electronegativity of the atoms under compression.J. Am. Chem. Soc. 2019; 141: 10253-10271Crossref PubMed Scopus (73) Google Scholar]. Here, we face the common issue of extrapolation. For example, Li changes its valence and becomes a p group element at 300 gigapascal (GPa). K and heavier alkali metals become transitional metals. Furthermore, Na becomes the most electropositive s1 element. Thus, the structure of the periodic table and chemical reactivities will change completely for some elements with rising pressure. Similar changes should be expected for typical oxidation states. Chemical heuristics might have only limited transferability, needing to be adapted to allow for the exploration of more extreme conditions. Even in more common conditions, very well-established rules can turn out to be less powerful than expected when evaluated with modern techniques and larger data sets. Hautier and colleagues [21.George J. et al.The limited predictive power of the Pauling rules.Angew. Chem. Int. Ed. 2020; 59: 7569-7575Crossref PubMed Scopus (21) Google Scholar] have recently evaluated the predictive power of the Pauling rules, which connects the coordination environments of a crystal with its stability. For instance, the first Pauling rule links the preferred coordination environment of a cation to the cation–anion ratio (Figure 2A ). This analysis on oxides shows that Pauling’s first rule is only fulfilled for 66% of all tested local environments, with important deviations in alkali and alkali–earth chemistries, for instance (Figure 2A). Strikingly, and despite being a corner stone of solid-state chemistry, only 13% of a data set of 5000 oxides fulfil the four other Pauling rules [21.George J. et al.The limited predictive power of the Pauling rules.Angew. Chem. Int. Ed. 2020; 59: 7569-7575Crossref PubMed Scopus (21) Google Scholar]. This new assessment could only be performed using modern tools of information technology. First, a diverse set of oxide crystal structures had not only to be determined by crystallographers but also made readily available in databases such as the Inorganic Crystal Structure Database (ICSD), the Open Crystallographic Database, or the Cambridge Structural Database [22.Groom C.R. et al.The Cambridge structural database.Acta Cryst B. 2016; 72: 171-179Crossref Scopus (5369) Google Scholar, 23.Zagorac D. et al.Recent developments in the inorganic crystal structure database: theoretical crystal structure data and related features.J. Appl. Crystallogr. 2019; 52: 918-925Crossref PubMed Scopus (76) Google Scholar, 24.Gražulis S. et al.Crystallography Open Database – an open-access collection of crystal structures.J. Appl. Crystallogr. 2009; 42: 726-729Crossref PubMed Scopus (811) Google Scholar]. Without these efforts of the crystallographic community, many of today’s data-driven searches would not be possible. Furthermore, automatic tools for the determination of coordination environments had to be developed [25.Waroquiers D. et al.Statistical analysis of coordination environments in oxides.Chem. Mater. 2017; 29: 8346Crossref Scopus (71) Google Scholar,26.Waroquiers D. et al.ChemEnv: a fast and robust coordination environment identification tool.Acta Cryst B. 2020; 76: 683-695Crossref Scopus (4) Google Scholar] and a quantitative assessment of the Pauling rules had to be set up [21.George J. et al.The limited predictive power of the Pauling rules.Angew. Chem. Int. Ed. 2020; 59: 7569-7575Crossref PubMed Scopus (21) Google Scholar]. In today’s world of unprecedented access to information and automation, it is staggering to realize that only several hundred crystal structures had been determined when Pauling proposed his rules and that no computer was available to process the data [27.Behrens H. Data import and validation in the inorganic crystal structure database.J. Res. Natl. Inst. Stand. Technol. 1996; 101: 365Crossref PubMed Scopus (7) Google Scholar]. In a similar spirit of challenging established rules, Filip and Giustino have shown that a modified and improved Goldschmidt tolerance factor that relates the ionic radii of the ions building a perovskite structure to the stability of perovskites has predictive power (Figure 2B) [28.Filip M.R. Giustino F. The geometric blueprint of perovskites.Proc. Natl. Acad. Sci. U. S. A. 2018; 115: 5397-5402Crossref PubMed Scopus (67) Google Scholar]. The Goldschmidt tolerance factor is based on the idea of a tightly packed structure of ionic spheres. To improve the basic idea of the tolerance factor, Filip and Giustino included further geometric constraints in their prediction that were initially missing in the definition of Goldschmidt, based on the idea of a high stability of tightly packed structures. This improved tolerance factor can be used to predict stable perovskites with a high fidelity (80%). Almost needless to say, the dataset of stable perovskite structure available to Filip and Giustino was much larger than the one available to Goldschmidt in 1926. Based on this improved tolerance factor, Filip and Giustino predicted 90 000 hitherto unknown perovskites. Since the initial development of most chemical rules, both the amount of data available and computing processing power have grown substantially. Now is the time for the community to test chemical heuristics more widely and rigorously. This includes assessing their validity in less common conditions (e.g., high pressure and/or temperature) and developing new rules following the spirit of previous heuristics but updated using modern machine learning techniques and larger data sets. Advancing further along the data arrow in Figure 1, another popular approach has been to use concepts from the classical heuristics as features or descriptors for machine learning studies of materials properties (Figure 3) [3.Schmidt J. et al.Recent advances and applications of machine learning in solid-state materials science.NPJ Comput. Mater. 2019; 5: 1-36Crossref Scopus (594) Google Scholar]. The material is here represented by traditional physical and chemical features such as electronegativities or atomic radii, and machine learning models that link these descriptors to materials properties are trained on large data sets. To get back to the previous examples of the Pauling rules, such approaches could, for instance, start from similar descriptors (e.g., local environments or atomic radii) but search through machine learning for relationships that deviate from these empirical rules. Various materials properties such as the band gap, bulk and shear modulus, and vibrational properties have successfully been machine learned in this way [3.Schmidt J. et al.Recent advances and applications of machine learning in solid-state materials science.NPJ Comput. Mater. 2019; 5: 1-36Crossref Scopus (594) Google Scholar]. These studies can link composition to properties, including, if necessary, crystal structure features. For instance, researchers have shown that vibrational entropy could be inferred from composition and crystal structure features, including traditional heuristic descriptors such as atomic numbers, radii, position in the periodic table, Pettifor scales, or electronegativities (Figure 4A ) [29.Legrain F. et al.How chemical composition alone can predict vibrational free energies and entropies of solids.Chem. Mater. 2017; 29: 6220-6227Crossref Scopus (64) Google Scholar,30.Tawfik S.A. et al.Predicting thermal properties of crystals using machine learning.Adv. Theor. Simul. 2020; 31900208Crossref Scopus (12) Google Scholar].Figure 4Importance of the Descriptor for Machine Learning.Show full caption(A) The figure shows the mean absolute error for the prediction of different vibrational properties. Five different types of descriptors are tested: they are based on the chemical composition, elemental properties of the atoms, pair correlation functions (PCF), solid angles as defined by O’Keeffe, and bispectrum components based on the local atomic density around each atom. Reprinted, with permission, from [29.Legrain F. et al.How chemical composition alone can predict vibrational free energies and entropies of solids.Chem. Mater. 2017; 29: 6220-6227Crossref Scopus (64) Google Scholar]. Copyright 2017 American Chemical Society. (B) Energy differences between rock salt and zinc blend structures of binary AB materials as a function of the optimal 2D descriptor that was identified in [12.Ghiringhelli L.M. et al.Big data of materials science: critical role of the descriptor.Phys. Rev. Lett. 2015; 114105503Crossref PubMed Scopus (467) Google Scholar]. Here, the descriptor clearly separates the compounds preferring rock salt and zinc blend structure type within a 2D plot. Reprinted, with permission, from [12.Ghiringhelli L.M. et al.Big data of materials science: critical role of the descriptor.Phys. Rev. Lett. 2015; 114105503Crossref PubMed Scopus (467) Google Scholar]. The figure is licensed under a CC BY 3.0 license.View Large Image Figure ViewerDownload (PPT) (A) The figure shows the mean absolute error for the prediction of different vibrational properties. Five different types of descriptors are tested: they are based on the chemical composition, elemental properties of the atoms, pair correlation functions (PCF), solid angles as defined by O’Keeffe, and bispectrum components based on the local atomic density around each atom. Reprinted, with permission, from [29.Legrain F. et al.How chemical composition alone can predict vibrational free energies and entropies of solids.Chem. Mater. 2017; 29: 6220-6227Crossref Scopus (64) Google Scholar]. Copyright 2017 American Chemical Society. (B) Energy differences between rock salt and zinc blend structures of binary AB materials as a function of the optimal 2D descriptor that was identified in [12.Ghiringhelli L.M. et al.Big data of materials science: critical role of the descriptor.Phys. Rev. Lett. 2015; 114105503Crossref PubMed Scopus (467) Google Scholar]. Here, the descriptor clearly separates the compounds preferring rock salt and zinc blend structure type within a 2D plot. Reprinted, with permission, from [12.Ghiringhelli L.M. et al.Big data of materials science: critical role of the descriptor.Phys. Rev. Lett. 2015; 114105503Crossref PubMed Scopus (467) Google Scholar]. The figure is licensed under a CC BY 3.0 license. While materials descriptions based solely on composition were the norm in the earlier days, there is a growing set of methods that very elegantly incorporate structural information. For example, graph-based structure representations have shown very promising results [31.Xie T. Grossman J.C. Crystal graph convolutional neural networks for an accurate and interpretable prediction of material properties.Phys. Rev. Lett. 2018; 120145301Crossref PubMed Scopus (487) Google Scholar,32.Chen C. et al.Graph networks as a universal machine learning framework for molecules and crystals.Chem. Mater. 2019; 31: 3564-3572Crossref Scopus (210) Google Scholar]. For instance, crystal graph convolutional neural networks allowed learning of different properties (e.g., formation energy or band gaps) for a variety of structure types and compositions [31.Xie T. Grossman J.C. Crystal graph convolutional neural networks for an accurate and interpretable prediction of material properties.Phys. Rev. Lett. 2018; 120145301Crossref PubMed Scopus (487) Google Scholar]. These crystal graph convolutional neural networks are based on a representation of atoms as nodes and bonds as edges within a graph. Additional information on the atom and bonds that are partially based on chemical heuristics (e.g., position in periodic table or electronegativities) are included in node and edge feature vectors. On top of this graph, a convolutional neural network is built that allows automatic extraction of the optimal representations of target properties. In a similar way, Chen and colleagues introduced MatErials Graph Networks (MEGNet) to represent crystals and molecules within machine learning models of material properties [32.Chen C. et al.Graph networks as a universal machine learning framework for molecules and crystals.Chem. Mater. 2019; 31: 3564-3572Crossref Scopus (210) Google Scholar]. There is an abundance of chemical features to use and efforts have started to group them in easily accessible codes and databases (matminer [33.Ward L. et al.Matminer: an open source toolkit for materials data mining.Comput. Mater. Sci. 2018; 152: 60-69Crossref Scopus (209) Google Scholar]). However, not all descriptors are adequate to predict a property. While previous chemical knowledge could help choosing descriptors (e.g., the Pauling rules could motivate to focus on atomic radii and local environments to predict material stability), it is also possible to find the adequate descriptors from data only. The approach is here to start with many descriptors or features and select the most relevant ones. This is a common task implemented in many machine learning methods [12.Ghiringhelli L.M. et al.Big data of materials science: critical role of the descriptor.Phys. Rev. Lett. 2015; 114105503Crossref PubMed Scopus (467) Google Scholar,34.Ghiringhelli L.M. et al.Learning physical descriptors for materials science by compressed sensing.New J. Phys. 2017; 19023017Crossref Scopus (72) Google Scholar,35.Ouyang R. et al.SISSO: a compressed-sensing method for identifying the best low-dimensional descriptor in an immensity of offered candidates.Phys. Rev. Mater. 2018; 2083802Crossref Scopus (184) Google Scholar]. Work from Ghiringhelli and colleagues [12.Ghiringhelli L.M. et al.Big data of materials science: critical role of the descriptor.Phys. Rev. Lett. 2015; 114105503Crossref PubMed Scopus (467) Google Scholar] illustrates the importance of selecting the right descriptors by performing a classification of AB binaries between zinc blende and rock salt formers. Using the least absolute shrinkage and selection operator (LASSO), which helps select descriptors in a large pool of typical atomic features, the authors discovered which descriptors would be the most predictive and established a simple relationship to be used for classification (Figure 4B). Here, they found that atomic ionization potentials, electron affinities, and radii describing where the radial probability density of the valence s or p orbitals show its maximum were the most effective descriptors on a large pool of features. Figure 4B also illustrates that the relationship between the selected descriptor and the target property is relatively simple. Feature selection can also be efficiently combined with less-interpretable feed-forward neural networks on crystal structure graphs [36.De Breuck P.-P. et al.Machine learning materials properties for small datasets.arXiv. 2020; (Published online April 30, 2020. arXiv:2004.14766 [cond-mat])Google Scholar]. The importance of feature selection for these neural networks has been shown to be critical when data sets are small (around hundreds of data points). The third approach of machine learning is to directly extract relationships between the raw data without using the intermediate step of chemical or atomistic features. For instance, one would like to directly link the crystal structure description (position and chemical elements on the sites) to properties. This strategy is appealing as it does not require knowledge a priori of what features or descriptors would be predictive, but it usually requires much more data (Figure 1). An interesting example of such a data-driven approach comes from the field of crystal structure prediction. Based on atomic radii and groups in the periodic table, chemists have foreseen which ions can substitute each other for more than 100 years. In recent years, a model based on machine learning that can quantify how likely certain cations can substitute each other has been developed [13.Hautier G. et al.Data mined ionic substitutions for the discovery of new compounds.Inorg. Chem. 2011; 50: 656-663Crossref PubMed Scopus (235) Google Scholar]. Essentially, the model builds a probability function providing the likelihood for two ions or elements to substitute each other. No assumption is made on what chemical features (ionic radius, electronegativity, position in the periodic table) drives these substitutions. The likelihood of substitution between two ions is given as a matrix in Figure 5. This algorithm was successfully used and combined with density functional theory (DFT) to predict new Li-ion battery, luminescent, or ternary nitride materials [37.Wang Z. et al.Mining unexplored chemistries for phosphors for high-color-quality white-light-emitting diodes.Joule. 2018; 2: 914-926Abstract Full Text Full Text PDF Scopus (72) Google Scholar, 38.Chen H. et al.Carbonophosphates: a new family of cathode materials for Li-ion batteries identified computationally.Chem. Mater. 2012; 24: 2009-2016Crossref Scopus (112) Google Scholar, 39.Sun W. et al.A map of the inorganic ternary metal nitrides.Nat. Mater. 2019; 18: 732-739Crossref PubMed Scopus (140) Google Scholar] that were confirmed experimentally. Some of those substitutions were surprising and not easily rationalizable on standard chemical grounds. For instance, the substitution model suggested a Ba2+ with Sr2+, Re7+ with Al3+, and N3– with O2– substitution to form a new phosphor host material, Sr2LiAlO4, which was successfully synthesized [37.Wang Z. et al.Mining unexplored chemistries for phosphors for high-color-quality white-light-emitting diodes.Joule. 2018; 2: 914-926Abstract Full Text Full Text PDF Scopus (72) Google Scholar]. Another field where data-driven models are replacing traditional physical models is interatomic potentials [5.Deringer V.L. et al.Machine learning interatomic potentials as emerging tools for materials science.Adv. Mater. 2019; 311902765Crossref Scopus (160) Google Scholar,40.Bartók A.P. et al.Gaussian approximation potentials: the accuracy of quantum mechanics, without the electrons.Phys. Rev. Lett. 2010; 104136403Crossref PubMed Scopus (1074) Google Scholar,41.Behler J. Parrinello M. Generalized neural-network representation of high-dimensional potential-energy surfaces.Phys. Rev. Lett. 2007; 98146401Crossref PubMed Scopus (1550) Google Scholar]. Traditional force-fields such as Lennard–Jones potentials [42.Jones J.E. Chapman S. On the determination of molecular fields. —II. From the equation of state of a gas.Proc. Math. Phys. Eng. Sci. 1924; 106: 463-477Crossref Google Scholar] or embedded atom models [43.Daw M.S. et al.The embedded-atom method: a review of theory and applications.Mater. Sci. Rep. 1993; 9: 251-310Crossref Scopus (1227) Google Scholar] have strong physical assumptions embedded. Recently developed machine learning potentials bypass these assumptions by directly connecting energetics and forces to crystal structure without much functional assumption about their relationships. In a sense, they are purely data-driven with minimal physical assumptions. To fit these potentials, a large set of reference data (e.g., from DFT) is needed. There are several types of these machine learning interatomic potentials: they differ by the descriptor used to encode the structure and the type of regression. Popular ones are the Gaussian approximation potentials by Bartók and Csányi, where a ‘smooth overlap of atomic positions’ (SOAP) descriptor is used, and the neuronal networks by Behler, where atom centered symmetry functions are used [40.Bartók A.P. et al.Gaussian approximation potentials: the accuracy of quantum mechanics, without the electrons.Phys. Rev. Lett. 2010; 104136403Crossref PubMed Scopus (1074) Google Scholar,41.Behler J. Parrinello M. Generalized neural-network representation of high-dimensional potential-energy surfaces.Phys. Rev. Lett. 2007; 98146401Crossref PubMed Scopus (1550) Google Scholar,44.Bartók A.P. et al.On representing chemical environments.Phys. Rev. B. 2013; 87184115Crossref Scopus (825) Google Scholar,45.Behler J. Atom-centered symmetry functions for constructing high-dimensional neural network potentials.J. Chem. Phys. 2011; 134074106Crossref PubMed Scopus (636) Google Scholar]. The SOAP descriptor encodes atomic geometries with the help of a local expansion of Gaussian smeared atomic densities. The possibilities offered by these potentials are immense, allowing simulations of phenomena requiring many atoms (e.g., grain boundaries [46.Hu C. et al.Genetic algorithm-guided deep learning of grain boundary diagrams: addressing the challenge of five degrees of freedom.Mater. Today. 2020; 38: 49-57Crossref Scopus (21) Google Scholar,47.Yokoi T. et al.Neural-network interatomic potential for grain boundary structures and their energetics in silicon.Phys. Rev. Mater. 2020; 4014605Crossref Scopus (15) Google Scholar] or amorphous phases [48.Sosso G.C. et al.Neural network interatomic potential for the phase change material GeTe.Phys. Rev. B. 2012; 85174103Crossref Scopus (153) Google Scholar,49.Deringer V.L. Csányi G. Machine learning based interatomic potential for amorphous carbon.Phys. Rev. B. 2017; 95094203Crossref Scopus (286) Google Scholar]) or accelerating the search for new crystal structures [50.Deringer V.L. et al.Data-driven learning and prediction of inorganic crystal structures.Faraday Discuss. 2018; 211: 45-59Crossref PubMed Google Scholar]. It is expected that they will impact many fields and they have already been used in the investigation of battery and potential thermoelectric materials [51.Deringer V.L. Modelling and understanding battery materials with machine-learning-driven atomistic simulations.J. Phys. Energy. 2020; 2041003Crossref Scopus (28) Google Scholar,52.George J. et al.Combining phonon accuracy with high transferability in Gaussian approximation potential models.J. Chem. Phys. 2020; 153044104Crossref PubMed Scopus (11) Google Scholar]. While these data-driven approaches are attractive because they do not rely on descriptors that could turn out to be inadequate, we should keep in mind that they could simply relearn well-established chemical rules. An illustrative example comes from a search for water-splitting materials among the ABX3 perovskites [53.Jain A. et al.Performance of genetic algorithms in search for water splitting perovskites.J. Mater. Sci. 2013; 48: 6519Crossref Scopus (32) Google Scholar]. Using a genetic algorithm, a search among all perovskites was performed, focusing on stability but also band gap and band edges. This search was purely data-driven using no heuristics (Figure 6, blue line, best genetic algorithm). The authors performed another search on the same data set, including constraints from typical chemical rules: the oxidation states in one material should sum up to 0, only materials with an even number of electrons in the primitive unit cell are considered to avoid metals and the classical Goldschmidt tolerance factor is used for ranking of the materials. The search based on traditional chemical heuristics was similarly successful as the genetic algorithms (Figure 6, red line, chemical). Obviously, here, the machine had been ‘reinventing the wheel’ or more precisely relearning known rules such as charge balance. Interestingly, it is the combination of data-driven optimization and heuristics that lead to the best performances. Joining the two approaches together prevents reinvention of the wheel while capturing relationships that might be unknown from traditional knowledge (Figure 6, green line). One of the earlier mentioned graph-based approaches (MEGNet) allows combination of the purely data-driven approach with heuristic features as well and it enables the researcher to decide on the amount of traditional heuristic knowledge (from only atomic numbers and simple bonding information to the inclusion of more complex chemical heuristics) used in the developed models depending on the data availability [32.Chen C. et al.Graph networks as a universal machine learning framework for molecules and crystals.Chem. Mater. 2019; 31: 3564-3572Crossref Scopus (210) Google Scholar]. Chemical heuristics and machine learning are rooted in the same desire to build models explaining patterns in data. Instead of seeing traditional chemical heuristics and machine learning studies only as competitors, we think they instead feed on each other. Machine learning approaches can benefit from old chemical heuristics as they provide natural features to represent materials (see Outstanding Questions). The traditional chemical rules also provide standard benchmarks for machine learning studies. Researchers should be encouraged to not only validate their machine learning models by cross-validation but also compare the results and performance of artificial intelligence with traditional rules. In our opinion, machine learning provides a true added value only when it convincingly beats traditional approaches. If the outcome of a new neural network is to simply relearn the rule of charge balance, one can question the advance in knowledge achieved by the machine learning tool. Chemical heuristics should also be challenged by machine learning and large-scale statistical studies. The historical significance of a heuristic rule should not preclude its critical assessment. In fact, researchers should remember that many traditional concepts and rules in chemistry have been developed on relatively small datasets and might, therefore, be highly biased. The boom of information technology presents an opportunity for chemists to get back to the drawing board and design new rules based on large datasets with the help of machine learning approaches.Outstanding QuestionsWill the intuitive understanding of materials science profit as much from machine learning as from traditional chemical heuristics? Traditional chemical heuristics are typically taught in chemistry courses in university and are core to our chemical understanding. Machine learning should allow for a similar contribution to chemical understanding as these traditional chemical heuristics.Are current datasets of materials biased by traditional chemical heuristics? Most inorganic compounds have been discovered based on serendipity and traditional chemical heuristics. Therefore, it might be possible that current materials datasets and, therefore, also assessments based on these datasets are biased based on the use of traditional chemical heuristics during their discovery.Should we build frameworks that allow for easy testing against classical chemical heuristics? True scientific progress based on machine learning would only be made if those approaches beat classical chemical heuristics. Currently, there are no straightforward ways to compare those approaches with each other. Will the intuitive understanding of materials science profit as much from machine learning as from traditional chemical heuristics? Traditional chemical heuristics are typically taught in chemistry courses in university and are core to our chemical understanding. Machine learning should allow for a similar contribution to chemical understanding as these traditional chemical heuristics. Are current datasets of materials biased by traditional chemical heuristics? Most inorganic compounds have been discovered based on serendipity and traditional chemical heuristics. Therefore, it might be possible that current materials datasets and, therefore, also assessments based on these datasets are biased based on the use of traditional chemical heuristics during their discovery. Should we build frameworks that allow for easy testing against classical chemical heuristics? True scientific progress based on machine learning would only be made if those approaches beat classical chemical heuristics. Currently, there are no straightforward ways to compare those approaches with each other. J.G. acknowledges funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 837910. We thank Pierre-Paul De Breuck for helpful comments on the manuscript. Figure 4 was created with the help of VESTA [54.Momma K. Izumi F. VESTA 3 for three-dimensional visualization of crystal, volumetric and morphology data.J. Appl. Crystallogr. 2011; 44: 1272-1276Crossref Scopus (10969) Google Scholar]. ‘intelligent’ behavior of machines. Machine learning can be considered a subfield of artificial intelligence. intuitive strategies that guide chemists in solving problems and making decisions. a crystal graph, where nodes represent the atoms (and their properties) and the edges represent the connections between the atoms, and a convolutional neural network built on top of the crystal graph. The latter consists of convolutional and pooling layers, which produce the overall feature vector of the crystal by convoluting the feature vectors of neighboring atoms. focuses on algorithms and principles to extract patterns from large amounts of data. It is closely related to the fields of machine learning and data mining and these terms are sometimes used interchangeably. indicates the stability of crystals with perovskite structure types based on the ionic radii of ions constituting the crystal structure. a regression with a penalty in the form of an L1-norm, which ensures both selection and shrinkage of the variables in the regression. provides algorithms that can analyze large data sets and it is therefore an important subfield of data science. a structure–stability relationship; they connect the stability/existence of crystals to the local environments of cations and the way those local environments are connected.