Atom-centered Symmetry Functions Research Articles

DFTB Simulation of Charged Clusters Using Machine Learning Charge Inference.

We present a modification to self-consistent charge density functional-based tight binding (SCC-DFTB), which allows computation based on approximate atomic charges. We obtain these charges by means of a machine learning (ML) process that combines a Coulomb model with a neural network. This allows us to avoid the SCC cycles in the SCC-DFTB calculation while maintaining its accuracy. The main input of the model is the atomic positions characterized by a set of atom-centered symmetry functions. The charge inference from our ML algorithm is as close as 10-2 units of charge from the exact SCC solution. Our ML-DFTB approach provides a good approximation of the density matrix and of the energy and forces with only a single diagonalization. This is a significant computational saving with respect to the complete SCC algorithm, which allows us to investigate a bigger ensemble of atoms. We show the quality of our approach in the case of charged silicon carbide (SiC) clusters. The ML-DFTB potential energy surface (PES) mimics the SCC-DFTB PES rather well, despite its simplicity. This allows us to obtain the same geometric structure ordering with respect to energy for small clusters. The dissociation barriers for ion emission are well-reproduced, which opens the way to investigating ion field emission and charged cluster stability. The ML-DFTB approach is obviously not limited to charged clusters or SiC materials. It opens a new route to investigate larger clusters than those investigated by standard SCC-DFTB, as well as surface and solid-state chemistry at the atomic level.

An Unsupervised Machine Learning Approach for the Automatic Construction of Local Chemical Descriptors.

Condensing the many physical variables defining a chemical system into a fixed-size array poses a significant challenge in the development of chemical Machine Learning (ML). Atom Centered Symmetry Functions (ACSFs) offer an intuitive featurization approach by means of a tedious and labor-intensive selection of tunable parameters. In this work, we implement an unsupervised ML strategy relying on a Gaussian Mixture Model (GMM) to automatically optimize the ACSF parameters. GMMs effortlessly decompose the vastness of the chemical and conformational spaces into well-defined radial and angular clusters, which are then used to build tailor-made ACSFs. The unsupervised exploration of the space has demonstrated general applicability across a diverse range of systems, spanning from various unimolecular landscapes to heterogeneous databases. The impact of the sampling technique and temperature on space exploration is also addressed, highlighting the particularly advantageous role of high-temperature Molecular Dynamics (MD) simulations. The reliability of the resulting features is assessed through the estimation of the atomic charges of a prototypical capped amino acid and a heterogeneous collection of CHON molecules. The automatically constructed ACSFs serve as high-quality descriptors, consistently yielding typical prediction errors below 0.010 electrons bound for the reported atomic charges. Altering the spatial distribution of the functions with respect to the cluster highlights the critical role of symmetry rupture in achieving significantly improved features. More specifically, using two separate functions to describe the lower and upper tails of the cluster results in the best performing models with errors as low as 0.006 electrons. Finally, the effectiveness of finely tuned features was checked across different architectures, unveiling the superior performance of Gaussian Process (GP) models over Feed Forward Neural Networks (FFNNs), particularly in low-data regimes, with nearly a 2-fold increase in prediction quality. Altogether, this approach paves the way toward an easier construction of local chemical descriptors, while providing valuable insights into how radial and angular spaces should be mapped. Finally, this work opens the possibility of encoding many-body information beyond angular terms into upcoming ML features.

Lifelong Machine Learning Potentials.

Machine learning potentials (MLPs) trained on accurate quantum chemical data can retain the high accuracy, while inflicting little computational demands. On the downside, they need to be trained for each individual system. In recent years, a vast number of MLPs have been trained from scratch because learning additional data typically requires retraining on all data to not forget previously acquired knowledge. Additionally, most common structural descriptors of MLPs cannot represent efficiently a large number of different chemical elements. In this work, we tackle these problems by introducing element-embracing atom-centered symmetry functions (eeACSFs), which combine structural properties and element information from the periodic table. These eeACSFs are key for our development of a lifelong machine learning potential (lMLP). Uncertainty quantification can be exploited to transgress a fixed, pretrained MLP to arrive at a continuously adapting lMLP, because a predefined level of accuracy can be ensured. To extend the applicability of an lMLP to new systems, we apply continual learning strategies to enable autonomous and on-the-fly training on a continuous stream of new data. For the training of deep neural networks, we propose the continual resilient (CoRe) optimizer and incremental learning strategies relying on rehearsal of data, regularization of parameters, and the architecture of the model.

Atomic structure generation from reconstructing structural fingerprints

Data-driven machine learning methods have the potential to dramatically accelerate the rate of materials design over conventional human-guided approaches. These methods would help identify or, in the case of generative models, even create novel crystal structures of materials with a set of specified functional properties to then be synthesized or isolated in the laboratory. For crystal structure generation, a key bottleneck lies in developing suitable atomic structure fingerprints or representations for the machine learning model, analogous to the graph-based or SMILES representations used in molecular generation. However, finding data-efficient representations that are invariant to translations, rotations, and permutations, while remaining invertible to the Cartesian atomic coordinates remains an ongoing challenge. Here, we propose an alternative approach to this problem by taking existing non-invertible representations with the desired invariances and developing an algorithm to reconstruct the atomic coordinates through gradient-based optimization using automatic differentiation. This can then be coupled to a generative machine learning model which generates new materials within the representation space, rather than in the data-inefficient Cartesian space. In this work, we implement this end-to-end structure generation approach using atom-centered symmetry functions as the representation and conditional variational autoencoders as the generative model. We are able to successfully generate novel and valid atomic structures of sub-nanometer Pt nanoparticles as a proof of concept. Furthermore, this method can be readily extended to any suitable structural representation, thereby providing a powerful, generalizable framework towards structure-based generation.

Systematic Identification of Atom-Centered Symmetry Functions for the Development of Neural Network Potentials.

Neural network potentials are emerging as promising classical force fields that can enable long-time and large-length scale simulations at close to ab initio accuracies. They learn the underlying potential energy surface by mapping the Cartesian coordinates of atoms to system energies using elemental neural networks. To ensure invariance with respect to system translation, rotation, and atom index permutations, in the Behler-Parrinnello type of neural network potential (BP-NNP), the Cartesian coordinates of atoms are transformed into "structural fingerprints" using atom-centered symmetry functions (ACSFs). Development of an accurate BP-NNP for any chemical system critically relies on the choice of these ACSFs. In this work, we have proposed a systematic framework for the identification of an optimal set of ACSFs for any target system, which not only considers the diverse atomic environments present in the training dataset but also inter-ACSF correlations. Our method is applicable to different kinds of ACSFs and across diverse chemical systems. We demonstrate this by building accurate BP-NNPs for water and Cu2S systems.

Accurate, affordable, and generalizable machine learning simulations of transition metal x-ray absorption spectra using the XANESNET deep neural network.

The affordable, accurate, and generalizable prediction of spectroscopic observables plays a key role in the analysis of increasingly complex experiments. In this article, we develop and deploy a deep neural network-XANESNET-for predicting the lineshape of first-row transition metal K-edge x-ray absorption near-edge structure (XANES) spectra. XANESNET predicts the spectral intensities using only information about the local coordination geometry of the transition metal complexes encoded in a feature vector of weighted atom-centered symmetry functions. We address in detail the calibration of the feature vector for the particularities of the problem at hand, and we explore the individual feature importance to reveal the physical insight that XANESNET obtains at the Fe K-edge. XANESNET relies on only a few judiciously selected features-radial information on the first and second coordination shells suffices along with angular information sufficient to separate satisfactorily key coordination geometries. The feature importance is found to reflect the XANES spectral window under consideration and is consistent with the expected underlying physics. We subsequently apply XANESNET at nine first-row transition metal (Ti-Zn) K-edges. It can be optimized in as little as a minute, predicts instantaneously, and provides K-edge XANES spectra with an average accuracy of ∼±2%-4% in which the positions of prominent peaks are matched with a >90% hit rate to sub-eV (∼0.8eV) error.

Manifolds of quasi-constant SOAP and ACSF fingerprints and the resulting failure to machine learn four-body interactions.

Atomic fingerprints are commonly used for the characterization of local environments of atoms in machine learning and other contexts. In this work, we study the behavior of two widely used fingerprints, namely, the smooth overlap of atomic positions (SOAP) and the atom-centered symmetry functions (ACSFs), under finite changes of atomic positions and demonstrate the existence of manifolds of quasi-constant fingerprints. These manifolds are found numerically by following eigenvectors of the sensitivity matrix with quasi-zero eigenvalues. The existence of such manifolds in ACSF and SOAP causes a failure to machine learn four-body interactions, such as torsional energies that are part of standard force fields. No such manifolds can be found for the overlap matrix (OM) fingerprint due to its intrinsic many-body character.

Molecular partition coefficient from machine learning with polarization and entropy embedded atom-centered symmetry functions.

Efficient prediction of the partition coefficient (log P) between polar and non-polar phases could shorten the cycle of drug and materials design. In this work, a descriptor, named 〈q - ACSFs〉conf, is proposed to take the explicit polarization effects in the polar phase and the conformation ensemble of energetic and entropic significance in the non-polar phase into consideration. The polarization effects are involved by embedding the partial charge directly derived from force fields or quantum chemistry calculations into the atom-centered symmetry functions (ACSFs), together with the entropy effects, which are averaged according to the Boltzmann distribution of different conformations taken from the similarity matrix. The model was trained with high-dimensional neural networks (HDNNs) on a public dataset PhysProp (with 41 039 samples). Satisfactory log P prediction performance was achieved on three other datasets, namely, Martel (707 molecules), Star & Non-Star (266) and Huuskonen (1870). The present 〈q - ACSFs〉conf model was also applicable to n-carboxylic acids with the number of carbons ranging from 2 to 14 and 54 kinds of organic solvent. It is easy to apply the present method to arbitrary sized systems and give a transferable atom-based partition coefficient.

Theoretical studies on triplet-state driven dissociation of formaldehyde by quasi-classical molecular dynamics simulation on machine-learning potential energy surface.

The H-atom dissociation of formaldehyde on the lowest triplet state (T1) is studied by quasi-classical molecular dynamic simulations on the high-dimensional machine-learning potential energy surface (PES) model. An atomic-energy based deep-learning neural network (NN) is used to represent the PES function, and the weighted atom-centered symmetry functions are employed as inputs of the NN model to satisfy the translational, rotational, and permutational symmetries, and to capture the geometry features of each atom and its individual chemical environment. Several standard technical tricks are used in the construction of NN-PES, which includes the application of clustering algorithm in the formation of the training dataset, the examination of the reliability of the NN-PES model by different fitted NN models, and the detection of the out-of-confidence region by the confidence interval of the training dataset. The accuracy of the full-dimensional NN-PES model is examined by two benchmark calculations with respect to ab initio data. Both the NN and electronic-structure calculations give a similar H-atom dissociation reaction pathway on the T1 state in the intrinsic reaction coordinate analysis. The small-scaled trial dynamics simulations based on NN-PES and ab initio PES give highly consistent results. After confirming the accuracy of the NN-PES, a large number of trajectories are calculated in the quasi-classical dynamics, which allows us to get a better understanding of the T1-driven H-atom dissociation dynamics efficiently. Particularly, the dynamics simulations from different initial conditions can be easily simulated with a rather low computational cost. The influence of the mode-specific vibrational excitations on the H-atom dissociation dynamics driven by the T1 state is explored. The results show that the vibrational excitations on symmetric C-H stretching, asymmetric C-H stretching, and C=O stretching motions always enhance the H-atom dissociation probability obviously.

High-dimensional neural network potentials for magnetic systems using spin-dependent atom-centered symmetry functions

Machine learning potentials have emerged as a powerful tool to extend the time and length scales of first-principles quality simulations. Still, most machine learning potentials cannot distinguish different electronic spin arrangements and thus are not applicable to materials in different magnetic states. Here we propose spin-dependent atom-centered symmetry functions as a type of descriptor taking the atomic spin degrees of freedom into account. When used as an input for a high-dimensional neural network potential (HDNNP), accurate potential energy surfaces of multicomponent systems can be constructed, describing multiple collinear magnetic states. We demonstrate the performance of these magnetic HDNNPs for the case of manganese oxide, MnO. The method predicts the magnetically distorted rhombohedral structure in excellent agreement with density functional theory and experiment. Its efficiency allows to determine the Néel temperature considering structural fluctuations, entropic effects, and defects. The method is general and is expected to be useful also for other types of systems such as oligonuclear transition metal complexes.

Machine Learning Approach for Describing Water OH Stretch Vibrations.

A machine learning approach employing neural networks is developed to calculate the vibrational frequency shifts and transition dipole moments of the symmetric and antisymmetric OH stretch vibrations of a water molecule surrounded by water molecules. We employed the atom-centered symmetry functions (ACSFs), polynomial functions, and Gaussian-type orbital-based density vectors as descriptor functions and compared their performances in predicting vibrational frequency shifts using the trained neural networks. The ACSFs perform best in modeling the frequency shifts of the OH stretch vibration of water among the types of descriptor functions considered in this paper. However, the differences in performance among these three descriptors are not significant. We also tried a feature selection method called CUR matrix decomposition to assess the importance and leverage of the individual functions in the set of selected descriptor functions. We found that a significant number of those functions included in the set of descriptor functions give redundant information in describing the configuration of the water system. We here show that the predicted vibrational frequency shifts by trained neural networks successfully describe the solvent-solute interaction-induced fluctuations of OH stretch frequencies.

A bin and hash method for analyzing reference data and descriptors in machine learning potentials

In recent years the development of machine learning potentials (MLPs) has become a very active field of research. Numerous approaches have been proposed, which allow one to perform extended simulations of large systems at a small fraction of the computational costs of electronic structure calculations. The key to the success of modern MLPs is the close-to first principles quality description of the atomic interactions. This accuracy is reached by using very flexible functional forms in combination with high-level reference data from electronic structure calculations. These data sets can include up to hundreds of thousands of structures covering millions of atomic environments to ensure that all relevant features of the potential energy surface are well represented. The handling of such large data sets is nowadays becoming one of the main challenges in the construction of MLPs. In this paper we present a method, the bin-and-hash (BAH) algorithm, to overcome this problem by enabling the efficient identification and comparison of large numbers of multidimensional vectors. Such vectors emerge in multiple contexts in the construction of MLPs. Examples are the comparison of local atomic environments to identify and avoid unnecessary redundant information in the reference data sets that is costly in terms of both the electronic structure calculations as well as the training process, the assessment of the quality of the descriptors used as structural fingerprints in many types of MLPs, and the detection of possibly unreliable data points. The BAH algorithm is illustrated for the example of high-dimensional neural network potentials using atom-centered symmetry functions for the geometrical description of the atomic environments, but the method is general and can be combined with any current type of MLP.

Four Generations of High-Dimensional Neural Network Potentials.

Since their introduction about 25 years ago, machine learning (ML) potentials have become an important tool in the field of atomistic simulations. After the initial decade, in which neural networks were successfully used to construct potentials for rather small molecular systems, the development of high-dimensional neural network potentials (HDNNPs) in 2007 opened the way for the application of ML potentials in simulations of large systems containing thousands of atoms. To date, many other types of ML potentials have been proposed continuously increasing the range of problems that can be studied. In this review, the methodology of the family of HDNNPs including new recent developments will be discussed using a classification scheme into four generations of potentials, which is also applicable to many other types of ML potentials. The first generation is formed by early neural network potentials designed for low-dimensional systems. High-dimensional neural network potentials established the second generation and are based on three key steps: first, the expression of the total energy as a sum of environment-dependent atomic energy contributions; second, the description of the atomic environments by atom-centered symmetry functions as descriptors fulfilling the requirements of rotational, translational, and permutation invariance; and third, the iterative construction of the reference electronic structure data sets by active learning. In third-generation HDNNPs, in addition, long-range interactions are included employing environment-dependent partial charges expressed by atomic neural networks. In fourth-generation HDNNPs, which are just emerging, in addition, nonlocal phenomena such as long-range charge transfer can be included. The applicability and remaining limitations of HDNNPs are discussed along with an outlook at possible future developments.

An assessment of the structural resolution of various fingerprints commonly used in machine learning

Atomic environment fingerprints are widely used in computational materials science, from machine learning potentials to the quantification of similarities between atomic configurations. Many approaches to the construction of such fingerprints, also called structural descriptors, have been proposed. In this work, we compare the performance of fingerprints based on the overlap matrix, the smooth overlap of atomic positions, Behler–Parrinello atom-centered symmetry functions, modified Behler–Parrinello symmetry functions used in the ANI-1ccx potential and the Faber–Christensen–Huang–Lilienfeld fingerprint under various aspects. We study their ability to resolve differences in local environments and in particular examine whether there are certain atomic movements that leave the fingerprints exactly or nearly invariant. For this purpose, we introduce a sensitivity matrix whose eigenvalues quantify the effect of atomic displacement modes on the fingerprint. Further, we check whether these displacements correlate with the variation of localized physical quantities such as forces. Finally, we extend our examination to the correlation between molecular fingerprints obtained from the atomic fingerprints and global quantities of entire molecules.

Transferable Multilevel Attention Neural Network for Accurate Prediction of Quantum Chemistry Properties via Multitask Learning.

The development of efficient models for predicting specific properties through machine learning is of great importance for the innovation of chemistry and material science. However, predicting global electronic structure properties like Frontier molecular orbital highest occupied molecular orbital (HOMO) and lowest unoccupied molecular orbital (LUMO) energy levels and their HOMO-LUMO gaps from the small-sized molecule data to larger molecules remains a challenge. Here, we develop a multilevel attention neural network, named DeepMoleNet, to enable chemical interpretable insights being fused into multitask learning through (1) weighting contributions from various atoms and (2) taking the atom-centered symmetry functions (ACSFs) as the teacher descriptor. The efficient prediction of 12 properties including dipole moment, HOMO, and Gibbs free energy within chemical accuracy is achieved by using multiple benchmarks, both at the equilibrium and nonequilibrium geometries, including up to 110,000 records of data in QM9, 400,000 records in MD17, and 280,000 records in ANI-1ccx for random split evaluation. The good transferability for predicting larger molecules outside the training set is demonstrated in both equilibrium QM9 and Alchemy data sets at the density functional theory (DFT) level. Additional tests on nonequilibrium molecular conformations from DFT-based MD17 data set and ANI-1ccx data set with coupled cluster accuracy as well as the public test sets of singlet fission molecules, biomolecules, long oligomers, and protein with up to 140 atoms show reasonable predictions for thermodynamics and electronic structure properties. The proposed multilevel attention neural network is applicable to high-throughput screening of numerous chemical species in both equilibrium and nonequilibrium molecular spaces to accelerate rational designs of drug-like molecules, material candidates, and chemical reactions.

PyXtal_FF: a python library for automated force field generation

We present PyXtal_FF—a package based on Python programming language—for developing machine learning potentials (MLPs). The aim of PyXtal_FF is to promote the application of atomistic simulations through providing several choices of atom-centered descriptors and machine learning regressions in one platform. Based on the given choice of descriptors (including the atom-centered symmetry functions, embedded atom density, SO4 bispectrum, and smooth SO3 power spectrum), PyXtal_FF can train MLPs with either generalized linear regression or neural network models, by simultaneously minimizing the errors of energy/forces/stress tensors in comparison with the data from ab-initio simulations. The trained MLP model from PyXtal_FF is interfaced with the Atomic Simulation Environment (ASE) package, which allows different types of light-weight simulations such as geometry optimization, molecular dynamics simulation, and physical properties prediction. Finally, we will illustrate the performance of PyXtal_FF by applying it to investigate several material systems, including the bulk SiO2, high entropy alloy NbMoTaW, and elemental Pt for general purposes. Full documentation of PyXtal_FF is available at https://pyxtal-ff.readthedocs.io.

Physically inspired atom-centered symmetry functions for the construction of high dimensional neural network potential energy surfaces

Among different atomistic neural network (AtNN) potential energy surfaces (PESs), the Behler-Parrinello neural network (BPNN) based on atom-centered symmetry functions (ACSFs) has been proved to be capable of constructing accurate PESs for various crystals. A judicious setting of the parameters of the ACSFs largely determines the accuracy of a BPNN PES. However, this is typically an ad hoc and tedious task requiring highly acute chemical intuition. To address this issue, we derived a set of physically inspired ACSFs from the effective densities of atoms, in which the radii of atoms are naturally incorporated. Therefore, the parameters of the physically inspired ACSFs can be directly chosen based on the types of chemical bonds within a target system. Compared with the original ones, the physically inspired ACSFs are more suitable for complex systems based on its better performance on predicting the formation enthalpies of molecules in QM9 database. Moreover, the physically inspired ACSFs can also effectively accelerate the convergence of the atomic forces during the training of an AtNN PES. With the physically inspired ACSFs, we constructed a highly accurate AtNN PES for a solid electrolyte Li10GeP2S12. Based on the AtNN PES, we studied the bulk Li ion diffusion within Li10GeP2S12 by molecular dynamics (MD) simulations. The MD results well reproduced the experimental results, indicating the high accuracy of the AtNN PES constructed with the physically inspired ACSFs.

An accurate machine-learning calculator for optimization of Li-ion battery cathodes

There is significant interest in improving the performance of batteries to increase electrification of transportation and aviation. Recently, performance improvements have been in large part due to changes in the composition of the cathode material family, LiNixMnyCo(1-x-y)O2 (e.g., 111-622-811). Despite the importance of these materials and tremendous progress with density functional theory (DFT) calculations in understanding basic design principles, it is computationally prohibitively expensive to make this problem tractable. Specifically, predicting the open circuit voltage for any cathode material in this family requires evaluation of stability in a quaternary phase space. In this work, we develop machine-learning potentials using fingerprinting based on atom-centered symmetry functions, used with a neural network model, trained on DFT calculations with a prediction accuracy of 3.7 meV/atom and 0.13 eV/Å for energy and force, respectively. We perform hyperparameter optimization of the fingerprinting parameters using Bayesian optimization through the Dragonfly package. Using this ML calculator, we first test its performance in predicting thermodynamic properties within the Debye-Grüneisen model and find good agreement for most thermodynamic properties, including the Gibbs free energy and entropy. Then, we use this to calculate the Li-vacancy ordering as a function of Li composition to simulate the process of discharging/charging of the cathode using grand canonical Monte Carlo simulations. The predicted voltage profiles are in good agreement with the experimental ones and provide an approach to rapidly perform design optimization in this phase space. This study serves as a proof-point of machine-learned DFT surrogates to enable battery materials optimization.

Machine learning approach for describing vibrational solvatochromism.

Machine learning is becoming a more and more versatile tool describing condensed matter systems. Here, we employ the feed-forward and the convolutional neural networks to describe the frequency shifts of the amide I mode vibration of N-methylacetamide (NMA) in water. For a given dataset of configurations of an NMA molecule solvated by water, we obtained comparable or improved results for describing vibrational solvatochromic frequency shift with the neural network approach, compared to the previously developed differential evolution algorithm approach. We compared the performance of the atom centered symmetry functions (ACSFs) and simple polynomial functions as descriptors for the solvated system and found that the polynomial function performs better than the ACSFs employed in the description of the amide I vibrational solvatochromism.

DScribe: Library of descriptors for machine learning in materials science

DScribe is a software package for machine learning that provides popular feature transformations (“descriptors”) for atomistic materials simulations. DScribe accelerates the application of machine learning for atomistic property prediction by providing user-friendly, off-the-shelf descriptor implementations. The package currently contains implementations for Coulomb matrix, Ewald sum matrix, sine matrix, Many-body Tensor Representation (MBTR), Atom-centered Symmetry Function (ACSF) and Smooth Overlap of Atomic Positions (SOAP). Usage of the package is illustrated for two different applications: formation energy prediction for solids and ionic charge prediction for atoms in organic molecules. The package is freely available under the open-source Apache License 2.0. Program summaryProgram Title: DScribeProgram Files doi:http://dx.doi.org/10.17632/vzrs8n8pk6.1Licensing provisions: Apache-2.0Programming language: Python/C/C++Supplementary material: Supplementary Information as PDFNature of problem: The application of machine learning for materials science is hindered by the lack of consistent software implementations for feature transformations. These feature transformations, also called descriptors, are a key step in building machine learning models for property prediction in materials science.Solution method: We have developed a library for creating common descriptors used in machine learning applied to materials science. We provide an implementation the following descriptors: Coulomb matrix, Ewald sum matrix, sine matrix, Many-body Tensor Representation (MBTR), Atom-centered Symmetry Functions (ACSF) and Smooth Overlap of Atomic Positions (SOAP). The library has a python interface with computationally intensive routines written in C or C++. The source code, tutorials and documentation are provided online. A continuous integration mechanism is set up to automatically run a series of regression tests and check code coverage when the codebase is updated.