Neural Network Force Fields Research Articles

First-principles based deep neural network force field for molecular dynamics simulation of N-Ga-Al semiconductors.

Accurate interatomic force fields are of paramount importance for molecular dynamics simulations to explore the thermal transport at the GaN/AlN heterogenous interface, which is a key factor hindering heat dissipation and limiting the performance of GaN power electronic devices. In this work, an interatomic potential (force field) based on a deep neural network technique and first-principles calculations is developed for N-Ga-Al semiconductors to predict the elastic and thermodynamic properties. Using our deep neural network potential (NNP), the precise structural features, elastic constants, and thermal conductivities of GaN, AlN, and their alloy are obtained, which are well consistent with those from experiments and first-principles calculations. The interfacial thermal conductance of GaN/AlN heterostructures with different interfacial morphologies are further studied using molecular dynamics simulations with the NNP. It is found that atomic interdiffusion and disorder at the interfaces dramatically reduces the interfacial thermal conductance. The developed NNP exhibits a larger effective dimension with respect to classical empirical potentials and reaches competitive performances, thus pointing towards attractive advantages in the study of GaN heterostructures and devices with the NNP.

Robust and scalable uncertainty estimation with conformal prediction for machine-learned interatomic potentials

Uncertainty quantification (UQ) is important to machine learning (ML) force fields to assess the level of confidence during prediction, as ML models are not inherently physical and can therefore yield catastrophically incorrect predictions. Established a-posteriori UQ methods, including ensemble methods, the dropout method, the delta method, and various heuristic distance metrics, have limitations such as being computationally challenging for large models due to model re-training. In addition, the uncertainty estimates are often not rigorously calibrated. In this work, we propose combining the distribution-free UQ method, known as conformal prediction (CP), with the distances in the neural network’s latent space to estimate the uncertainty of energies predicted by neural network force fields. We evaluate this method (CP+latent) along with other UQ methods on two essential aspects, calibration, and sharpness, and find this method to be both calibrated and sharp under the assumption of independent and identically-distributed (i.i.d.) data. We show that the method is relatively insensitive to hyperparameters selected, and test the limitations of the method when the i.i.d. assumption is violated. Finally, we demonstrate that this method can be readily applied to trained neural network force fields with traditional and graph neural network architectures to obtain estimates of uncertainty with low computational costs on a training dataset of 1 million images to showcase its scalability and portability. Incorporating the CP method with latent distances offers a calibrated, sharp and efficient strategy to estimate the uncertainty of neural network force fields. In addition, the CP approach can also function as a promising strategy for calibrating uncertainty estimated by other approaches.

Erratum: "Physics-based, neural network force fields for reactive molecular dynamics: Investigation of carbene formation from [EMIM+][OAc-]" [J. Chem. Phys. 155, 104112 (2021)

Erratum: "Physics-based, neural network force fields for reactive molecular dynamics: Investigation of carbene formation from [EMIM+][OAc-]" [J. Chem. Phys. 155, 104112 (2021)

Neural-network-backed evolutionary search for SrTiO3(110) surface reconstructions.

The determination of atomic structures in surface reconstructions has typically relied on structural models derived from intuition and domain knowledge. Evolutionary algorithms have emerged as powerful tools for such structure searches. However, when density functional theory is used to evaluate the energy the computational cost of a thorough exploration of the potential energy landscape is prohibitive. Here, we drive the exploration of the rich phase diagram of TiO x overlayer structures on SrTiO3(110) by combining the covariance matrix adaptation evolution strategy (CMA-ES) and a neural-network force field (NNFF) as a surrogate energy model. By training solely on SrTiO3(110) 4×1 overlayer structures and performing CMA-ES runs on 3×1, 4×1 and 5×1 overlayers, we verify the transferability of the NNFF. The speedup due to the surrogate model allows taking advantage of the stochastic nature of the CMA-ES to perform exhaustive sets of explorations and identify both known and new low-energy reconstructions.

Deep learning neural network potential for simulating gaseous adsorption in metal–organic frameworks

This study proposes ab initio neural network force fields with physically motivated features to offer superior accuracy in describing adsorbate–adsorbent interactions of nonpolar (CO2) and polar (H2O and CO) molecules in metal–organic frameworks with open-metal sites.

A Differentiable Neural-Network Force Field for Ionic Liquids.

We present NeuralIL, a model for the potential energy of an ionic liquid that accurately reproduces first-principles results with orders-of-magnitude savings in computational cost. Built on the basis of a multilayer perceptron and spherical Bessel descriptors of the atomic environments, NeuralIL is implemented in such a way as to be fully automatically differentiable. It can thus be trained on ab initio forces instead of just energies, to make the most out of the available data, and can efficiently predict arbitrary derivatives of the potential energy. Using ethylammonium nitrate as the test system, we obtain out-of-sample accuracies better than 2 meV atom–1 (<0.05 kcal mol–1) in the energies and 70 meV Å–1 in the forces. We show that encoding the element-specific density in the spherical Bessel descriptors is key to achieving this. Harnessing the information provided by the forces drastically reduces the amount of atomic configurations required to train a neural network force field based on atom-centered descriptors. We choose the Swish-1 activation function and discuss the role of this choice in keeping the neural network differentiable. Furthermore, the possibility of training on small data sets allows for an ensemble-learning approach to the detection of extrapolation. Finally, we find that a separate treatment of long-range interactions is not required to achieve a high-quality representation of the potential energy surface of these dense ionic systems.

Physics-based, neural network force fields for reactive molecular dynamics: Investigation of carbene formation from [EMIM+][OAc-

Reactive molecular dynamics simulations enable a detailed understanding of solvent effects on chemical reaction mechanisms and reaction rates. While classical molecular dynamics using reactive force fields allows significantly longer simulation time scales and larger system sizes compared with ab initio molecular dynamics, constructing reactive force fields is a difficult and complex task. In this work, we describe a general approach following the empirical valence bond framework for constructing ab initio reactive force fields for condensed phase simulations by combining physics-based methods with neural networks (PB/NNs). The physics-based terms ensure the correct asymptotic behavior of electrostatic, polarization, and dispersion interactions and are compatible with existing solvent force fields. NNs are utilized for a versatile description of short-range orbital interactions within the transition state region and accurate rendering of vibrational motion of the reacting complex. We demonstrate our methodology for a simple deprotonation reaction of the 1-ethyl-3-methylimidazolium cation with acetate to form 1-ethyl-3-methylimidazol-2-ylidene and acetic acid. Our PB/NN force field exhibits ∼1 kJ mol-1 mean absolute error accuracy within the transition state region for the gas-phase complex. To characterize the solvent modulation of the reaction profile, we compute potentials of mean force for the gas-phase reaction as well as the reaction within a four-ion cluster and benchmark against ab initio molecular dynamics simulations. We find that the surrounding ionic environment significantly destabilizes the formation of the carbene product, and we show that this effect is accurately captured by the reactive force field. By construction, the PB/NN potential may be directly employed for simulations of other solvents/chemical environments without additional parameterization.

Comparative Studies of the Structural and Transport Properties of Molten Salt FLiNaK Using the Machine-Learned Neural Network and Reparametrized Classical Forcefields.

Despite surging interest in molten salt reactors and thermal storage systems, knowledge of the physicochemical properties of molten salts are still inadequate due to demanding experiments that require high temperature, impurity control, and corrosion mitigation. Therefore, the ability to predict these properties for molten salts from first-principles computations is urgently needed. Herein, we developed and compared a machine-learned neural network force field (NNFF) and a reparametrized rigid ion model (RIM) for a prototypical molten salt LiF-NaF-KF (FLiNaK). We found that NNFF was able to reproduce both the structural and transport properties of the molten salt with first-principles accuracy and classical-MD computational efficiency. Furthermore, the correlation between the local atomic structures and the dynamics was identified by comparing with RIMs, suggesting the significance of polarization of anions implicitly embedded in the NNFF. This work demonstrated a computational framework that can facilitate the screening of molten salts with different chemical compositions, impurities, and additives, and at different thermodynamic conditions suitable for the next-generation nuclear reactors and thermal energy storage facilities.

Accurate and scalable graph neural network force field and molecular dynamics with direct force architecture

Recently, machine learning (ML) has been used to address the computational cost that has been limiting ab initio molecular dynamics (AIMD). Here, we present GNNFF, a graph neural network framework to directly predict atomic forces from automatically extracted features of the local atomic environment that are translationally-invariant, but rotationally-covariant to the coordinate of the atoms. We demonstrate that GNNFF not only achieves high performance in terms of force prediction accuracy and computational speed on various materials systems, but also accurately predicts the forces of a large MD system after being trained on forces obtained from a smaller system. Finally, we use our framework to perform an MD simulation of Li7P3S11, a superionic conductor, and show that resulting Li diffusion coefficient is within 14% of that obtained directly from AIMD. The high performance exhibited by GNNFF can be easily generalized to study atomistic level dynamics of other material systems.

Temperature dependence of nuclear quantum effects on liquid water via artificial neural network model based on SCAN meta-GGA functional.

We investigate the temperature dependence of nuclear quantum effects (NQEs) on structural and dynamic properties of liquid water by training a neural network force field using first-principles molecular dynamics (FPMD) based on the strongly constrained and appropriately normed meta-generalized gradient approximation exchange-correlation approximation. The FPMD simulation based on density functional theory has become a powerful computational approach for studying a wide range of condensed phase systems. However, its large computational cost makes it difficult to incorporate NQEs in the simulation and investigate temperature dependence of various properties. To circumvent this difficulty, we use an artificial neural network model and employ the thermostatted ring polymer MD approach for studying the temperature dependence of NQEs on various properties. The NQEs generally bring the radial distribution functions closer to the experimental measurements. Translational diffusivity and rotational dynamics of water molecules are both slowed down by the NQEs. The competing inter-molecular and intra-molecular quantum effects on hydrogen bonds, as discussed by Habershon, Markland, and Manolopoulos [J. Chem. Phys. 131(2), 024501 (2019)], can explain the observed temperature dependence of the NQEs on the dynamical properties in our simulation.

Spatial density neural network force fields with first-principles level accuracy and application to thermal transport

Constrained by the fixed mathematical form of most empirical potentials used in classical molecular dynamics (MD) simulations, many properties of materials cannot be captured within experimental accuracy. On the other hand, accurate electronic structure calculations based on quantum theory, most notably density functional theory (DFT), are limited to several hundred atoms within a picosecond, which makes the method inadequate for modeling systems beyond the nanoscale. A combination of speed from classical MD and fidelity from DFT can be achieved through machine learning methods. Herewith, we developed an approach named spatial density neural network force fields (SDNNFFs) by training neural networks to ``learn'' and predict DFT-level forces. Our model focuses on the usage of a three-dimensional mesh of density functions, which together act as a mapping of the atomic environment and provides a physical representation of the forces acting on the central atom. Several notable advantages arise from the SDNNFF, including (1) the avoidance of the chain rule on the total energy and other variables by direct calculation of the forces from the neural network, (2) the ever large $N\ifmmode\times\else\texttimes\fi{}t$ scaling of the training data, where $N$ is the number of atoms in a supercell and $t$ is the number of evaluated structures by first-principles, and (3) the significant reduction in parameters and human effort needed to successfully train a force- and/or property-converged neural network force field. Overall, we focus on modeling DFT-level forces with minimal computational cost and parametrization for rapid prediction of phonon-based properties and future molecular dynamics of large-scale systems. To demonstrate the SDNNFF, we trained several models on diamond structures, including bulk silicon (Si), diamond, silicon carbide (SiC), and boron arsenide (BAs), and predicted their phonon dispersions and lattice thermal conductivities using the direct solution to the phonon Boltzmann transport equation. For phonon properties, we utilized a fitting method for obtaining the second- and third-order force constants, which outperforms the highly force-sensitive finite displacement method when employing neural network force fields. In comparison to DFT lattice thermal conductivity, we obtained high precision results from our SDNNFF within 0.7% for Si, 6.2% for diamond, 2.76% for SiC, and 7.46% for BAs, with further agreement with experiments. The phonon dispersions from the SDNNFF also matched those from direct DFT and experiments. The developed approach for accurately predicting phonon transport properties of crystalline materials would largely benefit the design of advanced materials with improved performance, such as complex thermoelectric devices and low thermal resistance interfaces for nanoelectronics. Future applications of our SDNNFF model could be extended toward including atomic energy into the algorithm and simulating large-scale heterogeneous systems for quasielectronic representations for various properties.

Neural Network Force Fields for Metal Growth Based on Energy Decompositions.

A method using machine learning (ML) is proposed to describe metal growth for simulations, which retains the accuracy of ab initio density functional theory (DFT) and results in a thousands-fold reduction in the computational time. This method is based on atomic energy decomposition from DFT calculations. Compared with other ML methods, our energy decomposition approach can yield much more information with the same DFT calculations. This approach is employed for the amorphous sodium system, where only 1000 DFT molecular dynamics images are enough for training an accurate model. The DFT and neural network potential (NNP) are compared for the dynamics to show that similar structural properties are generated. Finally, metal growth experiments from liquid to solid in a small and larger system are carried out to demonstrate the ability of using NNP to simulate the real growth process.

A fast neural network approach for direct covariant forces prediction in complex multi-element extended systems

Neural network force field (NNFF) is a method for performing regression on atomic structure-force relationships, bypassing expensive quantum mechanics calculation which prevents the execution of long ab-initio quality molecular dynamics simulations. However, most NNFF methods for complex multi-element atomic systems indirectly predict atomic force vectors by exploiting just atomic structure rotation-invariant features and the network-feature spatial derivatives which are computationally expensive. We develop a staggered NNFF architecture exploiting both rotation-invariant and covariant features separately to directly predict atomic force vectors without using spatial derivatives, thereby reducing expensive structural feature calculation by ~180-480x. This acceleration enables us to develop NNFF which directly predicts atomic forces in complex ternary and quaternary-element extended systems comprised of long polymer chains, amorphous oxide, and surface chemical reactions. The staggered rotation-invariant-covariant architecture described here can also directly predict complex covariant vector outputs from local physical structures in domains beyond computational material science.

Reactive molecular dynamics for the [Cl–CH3–Br]− reaction in the gas phase and in solution: a comparative study using empirical and neural network force fields

The nucleophilic substitution reaction [Cl–CH3–Br]− is used for a comparative study of several reactive molecular dynamics schemes in the gas phase and in solution. Multi surface adiabatic reactive MD (MS-ARMD) and multi-state valence bond (MS-VALBOND) use empirical force fields to allow bond-breaking and bond-formation in dynamics studies. As a third alternative, machine learning is used to obtain a reactive force field from training a neural network. The focus of the present work is on highlighting differences of the parametrisation strategy and the associated computational cost and parametrization effort, as well as discussing the transferability and the ease with which the methods can be applied to a given chemical reaction. All methods are able to fit the reference data with R2 = 0.99 or better. Free energy barrier heights in the gas phase from all three methods for the SN2 reaction compare to within 3.5 kcal mol−1 for the forward and to within 0.6 kcal mol−1 for the reverse reaction. For the reaction in solution only the MS-ARMD and MS-VALBOND approaches can be used as training a NN for this would be computationally extremely prohibitive. Overall, MS-VALBOND yields the best results compared to experiment with differences in the barrier heights of ∼1 kcal mol−1 for the reaction in solution. Potential improvements for all three methods are discussed and aim to guide computational investigation of chemical reactions applying these three methods.

Density functional theory based neural network force fields from energy decompositions

In order to develop force fields (FF) for molecular dynamics simulations that retain the accuracy of ab initio density functional theory (DFT), we developed a machine learning protocol based on an energy decomposition scheme that extracts atomic energies from DFT calculations. Our DFT to FF (DFT2FF) approach provides almost hundreds of times more data for the DFT energies, which dramatically improves accuracy with less DFT calculations. In addition, we use piecewise cosine basis functions to systematically construct symmetry invariant features into the neural network model. We illustrate this DFT2FF approach for amorphous silicon where only 800 DFT configurations are sufficient to achieve an accuracy of 1 meV/atom for energy and 0.1 eV/A for forces. We then use the resulting FF model to calculate the thermal conductivity of amorphous Si based on long molecular dynamics simulations. The dramatic speedup in training in our DFT2FF protocol allows the adoption of a simulation paradigm where an accurate and problem specific FF for a given physics phenomenon is trained on-the-spot through a quick DFT precalculation and FF training.

Neural network force fields for simple metals and semiconductors: construction and application to the calculation of phonons and melting temperatures.

We present a practical procedure to obtain reliable and unbiased neural network based force fields for solids. Training and test sets are efficiently generated from global structural prediction runs, at the same time assuring the structural variety and importance of sampling the relevant regions of phase space. The neural networks are trained to yield not only good formation energies, but also accurate forces and stresses, which are the quantities of interest for molecular dynamics simulations. Finally, we construct, as an example, several force fields for both semiconducting and metallic elements, and prove their accuracy for a variety of structural and dynamical properties. These are then used to study the melting of bulk copper and gold.

Maximization of learning speed in motor cortex due to neuron redundancy

AbstractMany kinds of redundancies play important roles in the motor system. Many combinations of muscle activity can generate an identical movement. One of the functional and computational roles of this muscle redundancy is to control joint impedance for stabilizing movements against applied external perturbation. Many combinations of joint angles can yield an identical fingertip position. A functional and computational role of this kinematic redundancy is to ensure motor accuracy, if a joint angle is perturbed, by adjusting redundant ones. Neurons are overwhelmingly more numerous than muscles in our motor system. However, an open question remains as to what the functional and computational roles of this neuron redundancy are. To generate desired motor commands, our motor system adapts to changes in both the internal and external environment, caused by muscle fatigue, force fields, etc. Faster learning is more desirable than slower learning because accumulated error can be minimized. Our analysis on a neural network model, assuming visuomotor or force field adaptation, revealed that learning speed reaches its maximum value if the model includes sufficient neuron redundancy. We subsequently ran numerical simulations of more biologically plausible neural network models. The results suggest that neuron redundancy accelerates learning speed even if the models include recurrent connections, a two-layer structure, or a skewed bimodal distribution of preferred directions. Neuron redundancy also yields the equivalence between a learning rule of synaptic weights and a model for sensorimotor learning, which is based on the results of behavioral experiments. These results suggest that neuron redundancy contributes to maximizing learning speed and also that the learning curves observed in behavioral experiments are tuned to be as fast as possible.