Neural Network Force Fields Research Articles

Training machine learning interatomic potentials for accurate phonon properties

Abstract One of the major challenges in the development of universal machine learning interatomic potentials is accurately reproducing phonon properties. This issue appears to arise from the limitations of available datasets rather than the models themselves. To address this, we develop an extensive dataset of phonon calculations using density-functional perturbation theory (DFPT). We then show how this dataset can be used to train neural-network force fields, by implementing the training and the prediction of force constants in periodic crystals. This approach improves the quality of phonon properties prediction while reducing the number of structures needed for neural network training. We demonstrate the efficiency of this method using two examples of ternary phase diagrams: Ti–Nb–Ta and Li–B–C. In both cases, neural network predictions for the energy and forces show a considerable improvement, while phonon properties are predicted with high precision for all structures across the entire phase diagrams.

From Graphene Oxide to Graphene: Changes in Interfacial Water Structure and Reactivity Using Deep Neural Network Force Fields

From Graphene Oxide to Graphene: Changes in Interfacial Water Structure and Reactivity Using Deep Neural Network Force Fields

Enhanced deep potential model for fast and accurate molecular dynamics: application to the hydrated electron.

In molecular simulations, neural network force fields aim at achieving ab initio accuracy with reduced computational cost. This work introduces enhancements to the Deep Potential network architecture, integrating a message-passing framework and a new lightweight implementation with various improvements. Our model achieves accuracy on par with leading machine learning force fields and offers significant speed advantages, making it well-suited for large-scale, accuracy-sensitive systems. We also introduce a new iterative model for Wannier center prediction, allowing us to keep track of electron positions in simulations of general insulating systems. We apply our model to study the solvated electron in bulk water, an ostensibly simple system that is actually quite challenging to represent with neural networks. Our trained model is not only accurate, but can also transfer to larger systems. Our simulation confirms the cavity model, where the electron's localized state is observed to be stable. Through an extensive run, we accurately determine various structural and dynamical properties of the solvated electron.

Evidence of ferroelectric features in low-density supercooled water from ab initio deep neural-network simulations

Over the last decade, an increasing body of evidence has emerged, supporting the existence of a metastable liquid-liquid critical point in supercooled water whereby two distinct liquid phases of different densities coexist. Analyzing long molecular dynamics simulations performed using deep neural-network force fields trained to accurate quantum mechanical data, we demonstrate that the low-density liquid phase displays a strong propensity toward spontaneous polarization, as witnessed by large and long-lived collective dipole fluctuations. Our findings suggest that the dynamical stability of the low-density phase, and hence the transition from high-density to low-density liquid, is triggered by a collective process involving an accumulation of rotational angular jumps, which could ignite large dipole fluctuations. This dynamical transition involves subtle changes in the electronic polarizability of water molecules which affects their rotational mobility within the two phases. These findings hold the potential for catalyzing activity in the search for dielectric-based probes of the putative second critical point.

Unusual dynamics of tetrahedral liquids caused by the competition between dynamic heterogeneity and structural heterogeneity.

Tetrahedral liquids exhibit intriguing thermodynamic and transport properties because of the various ways tetrahedra can be packed and connected. Recently, an unusual temperature dependence of the stretching exponent β in a model tetrahedral liquid ZnCl2 from Tm + 85K to Tm + 35K has been reported using neutron-spin echo spectroscopy. This discovery stands in sharp contrast to other glass-forming liquids. In this study, we conducted neural network force field driven molecular dynamic simulations of ZnCl2. We found a non-monotonic temperature dependence of β from liquid to supercooled liquid temperatures. Further structural decomposition and dynamic analysis suggest that this unusual dynamic behavior is a result of the competition between the decrease in the diversity of tetrahedra motifs (structural heterogeneity) and the increase in glassy dynamic heterogeneity. This result may contribute to new understandings of the structural relaxation of other network liquids.

Exploring fracture of H-BN and graphene by neural network force fields

Extreme mechanical processes such as strong lattice distortion and bond breakage during fracture often lead to catastrophic failure of materials and structures. Understanding the nucleation and growth of cracks is challenged by their multiscale characteristics spanning from atomic-level structures at the crack tip to the structural features where the load is applied. Atomistic simulations offer ‘first-principles’ tools to resolve the progressive microstructural changes at crack fronts and are widely used to explore the underlying processes of mechanical energy dissipation, crack path selection, and dynamic instabilities (e.g. kinking, branching). Empirical force fields developed based on atomic-level structural descriptors based on atomic positions and the bond orders do not yield satisfying predictions of fracture, especially for the nonlinear, anisotropic stress–strain relations and the energy densities of edges. High-fidelity force fields thus should include the tensorial nature of strain and the energetics of bond-breaking and (re)formation events during fracture, which, unfortunately, have not been taken into account in either the state-of-the-art empirical or machine-learning force fields. Based on data generated by density functional theory calculations, we report a neural network-based force field for fracture (NN-F3) constructed by using the end-to-end symmetry preserving framework of deep potential—smooth edition (DeepPot-SE). The workflow combines pre-sampling of the space of strain states and active-learning techniques to explore the transition states at critical bonding distances. The capability of NN-F3 is demonstrated by studying the rupture of hexagonal boron nitride (h-BN) and twisted bilayer graphene as model problems. The simulation results elucidate the roughening physics of fracture defined by the lattice asymmetry in h-BN, explaining recent experimental findings, and predict the interaction between cross-layer cracks in twisted graphene bilayers, which leads to a toughening effect.

Phonon Transport in Defect-Laden Bilayer Janus PtSTe Studied Using Neural-Network Force Fields.

We explore the phonon transport properties of defect-laden bilayer PtSTe using equilibrium molecular dynamics simulations based on a neural-network force field. Defects prove very efficient at depressing the thermal conductivity of the structure, and flower defects have a particularly powerful effect, comparable to that of double vacancies. Furthermore, the conductivity of the structure with flower defects exhibits an unusual temperature dependence due to structural instability at high temperatures. We look into the distortion to normal modes around the defect by means of the projected phonon density of states and find diverse phenomena including localized modes and blue shifts.

Seebeck Coefficient of Ionic Conductors from Bayesian Regression Analysis.

We propose a novel approach to evaluating the ionic Seebeck coefficient in electrolytes from relatively short equilibrium molecular dynamics simulations, based on the Green-Kubo theory of linear response and Bayesian regression analysis. By exploiting the probability distribution of the off-diagonal elements of a Wishart matrix, we develop a consistent and unbiased estimator for the Seebeck coefficient, whose statistical uncertainty can be arbitrarily reduced in the long-time limit. We assess the efficacy of our method by benchmarking it against extensive equilibrium molecular dynamics simulations conducted on molten CsF using empirical force fields. We then employ this procedure to calculate the Seebeck coefficient of molten NaCl, KCl, and LiCl using neural network force fields trained on ab initio data over a range of pressure-temperature conditions.

Equivariant neural network force fields for magnetic materials

Neural network force fields have significantly advanced ab initio atomistic simulations across diverse fields. However, their application in the realm of magnetic materials is still in its early stage due to challenges posed by the subtle magnetic energy landscape and the difficulty of obtaining training data. Here we introduce a data-efficient neural network architecture to represent density functional theory total energy, atomic forces, and magnetic forces as functions of atomic and magnetic structures. Our approach incorporates the principle of equivariance under the three-dimensional Euclidean group into the neural network model. Through systematic experiments on various systems, including monolayer magnets, curved nanotube magnets, and moiré-twisted bilayer magnets of CrI3, we showcase the method’s high efficiency and accuracy, as well as exceptional generalization ability. The work creates opportunities for exploring magnetic phenomena in large-scale materials systems.

EGraFFBench: evaluation of equivariant graph neural network force fields for atomistic simulations

EGraFFBench: a framework for evaluating equivariant graph neural network force fields on dynamic atomistic simulations.

Exploring inhomogeneous surfaces: Ti-rich SrTiO3(110) reconstructions via active learning.

The investigation of inhomogeneous surfaces, where various local structures coexist, is crucial for understanding interfaces of technological interest, yet it presents significant challenges. Here, we study the atomic configurations of the (2 × m) Ti-rich surfaces at (110)-oriented SrTiO3 by bringing together scanning tunneling microscopy and transferable neural-network force fields combined with evolutionary exploration. We leverage an active learning methodology to iteratively extend the training data as needed for different configurations. Training on only small well-known reconstructions, we are able to extrapolate to the complicated and diverse overlayers encountered in different regions of the inhomogeneous SrTiO3(110)-(2 × m) surface. Our machine-learning-backed approach generates several new candidate structures, in good agreement with experiment and verified using density functional theory. The approach could be extended to other complex metal oxides featuring large coexisting surface reconstructions.

Coarse-Graining with Equivariant Neural Networks: A Path Toward Accurate and Data-Efficient Models.

Machine learning has recently entered into the mainstream of coarse-grained (CG) molecular modeling and simulation. While a variety of methods for incorporating deep learning into these models exist, many of them involve training neural networks to act directly as the CG force field. This has several benefits of which the most significant is accuracy. Neural networks can inherently incorporate multibody effects during the calculation of CG forces, and a well-trained neural network force field outperforms pairwise basis sets generated from essentially any methodology. However, this comes at a significant cost. First, these models are typically slower than pairwise force fields, even when accounting for specialized hardware, which accelerates the training and integration of such networks. The second and the focus of this paper is the need for a considerable amount of data to train such force fields. It is common to use 10s of microseconds of molecular dynamics data to train a single CG model, which approaches the point of eliminating the CG model's usefulness in the first place. As we investigate in this work, this "data-hunger" trap from neural networks for predicting molecular energies and forces can be remediated in part by incorporating equivariant convolutional operations. We demonstrate that, for CG water, networks that incorporate equivariant convolutional operations can produce functional models using data sets as small as a single frame of reference data, while networks without these operations cannot.

Unraveling pyrolysis mechanisms of lignin dimer model compounds: Neural network-based molecular dynamics simulation investigations

Unraveling the pyrolysis mechanisms of lignin is of great importance for effective lignin utilization. However, both conventional quantum mechanical method and molecular dynamics (MD) simulations based on empirical potentials, e.g., the ReaxFF, cannot guarantee simultaneous accuracy and efficiency in exploring lignin pyrolysis mechanisms. Here, we developed a neural network (NN) force field for four types of representative lignin dimers based on 56,164 structures at the level of density functional theory (DFT). Such NN model was demonstrated to possess more accurate predictive power in both energetic and atomistic force properties than ReaxFF and was far more efficient than QM method. The MD simulations with the NN model showed that entire pyrolysis process can be divided into three stages dependent on the reaction temperature: homolytic cleavage of C–C bond and C-O bond, the changes from methoxy group to hydroxy, methyl, and formyl group, ring-opening reactions and the ring structure evolutions, and some novel reactions. The NN-based MD has provided us an atomic-level understanding of lignin pyrolysis and enabled us to build a comprehensive lignin pyrolysis network by means of the reactions disclosed based on the NN-based MD, which provided a useful guide for taking advantage of lignin more effectively.

Unlocking phonon properties of a large and diverse set of cubic crystals by indirect bottom-up machine learning approach

Although first principles based anharmonic lattice dynamics is one of the most common methods to obtain phonon properties, such method is impractical for high-throughput search of target thermal materials. We develop an elemental spatial density neural network force field as a bottom-up approach to accurately predict atomic forces of ~80,000 cubic crystals spanning 63 elements. The primary advantage of our indirect machine learning model is the accessibility of phonon transport physics at the same level as first principles, allowing simultaneous prediction of comprehensive phonon properties from a single model. Training on 3182 first principles data and screening 77,091 unexplored structures, we identify 13,461 dynamically stable cubic structures with ultralow lattice thermal conductivity below 1 Wm−1K−1, among which 36 structures are validated by first principles calculations. We propose mean square displacement and bonding-antibonding as two low-cost descriptors to ease the demand of expensive first principles calculations for fast screening ultralow thermal conductivity. Our model also quantitatively reveals the correlation between off-diagonal coherence and diagonal populations and identifies the distinct crossover from particle-like to wave-like heat conduction. Our algorithm is promising for accelerating discovery of novel phononic crystals for emerging applications, such as thermoelectrics, superconductivity, and topological phonons for quantum information technology.

Toward a general neural network force field for protein simulations: Refining the intramolecular interaction in protein.

Molecular dynamics (MD) is an extremely powerful, highly effective, and widely used approach to understanding the nature of chemical processes in atomic details for proteins. The accuracy of results from MD simulations is highly dependent on force fields. Currently, molecular mechanical (MM) force fields are mainly utilized in MD simulations because of their low computational cost. Quantum mechanical (QM) calculation has high accuracy, but it is exceedingly time consuming for protein simulations. Machine learning (ML) provides the capability for generating accurate potential at the QM level without increasing much computational effort for specific systems that can be studied at the QM level. However, the construction of general machine learned force fields, needed for broad applications and large and complex systems, is still challenging. Here, general and transferable neural network (NN) force fields based on CHARMM force fields, named CHARMM-NN, are constructed for proteins by training NN models on 27 fragments partitioned from the residue-based systematic molecular fragmentation (rSMF) method. The NN for each fragment is based on atom types and uses new input features that are similar to MM inputs, including bonds, angles, dihedrals, and non-bonded terms, which enhance the compatibility of CHARMM-NN to MM MD and enable the implementation of CHARMM-NN force fields in different MD programs. While the main part of the energy of the protein is based on rSMF and NN, the nonbonded interactions between the fragments and with water are taken from the CHARMM force field through mechanical embedding. The validations of the method for dipeptides on geometric data, relative potential energies, and structural reorganization energies demonstrate that the CHARMM-NN local minima on the potential energy surface are very accurate approximations to QM, showing the success of CHARMM-NN for bonded interactions. However, the MD simulations on peptides and proteins indicate that more accurate methods to represent protein-water interactions in fragments and non-bonded interactions between fragments should be considered in the future improvement of CHARMM-NN, which can increase the accuracy of approximation beyond the current mechanical embedding QM/MM level.

RLEKF: An Optimizer for Deep Potential with Ab Initio Accuracy

It is imperative to accelerate the training of neural network force field such as Deep Potential, which usually requires thousands of images based on first-principles calculation and a couple of days to generate an accurate potential energy surface. To this end, we propose a novel optimizer named reorganized layer extended Kalman filtering (RLEKF), an optimized version of global extended Kalman filtering (GEKF) with a strategy of splitting big and gathering small layers to overcome the O(N^2) computational cost of GEKF. This strategy provides an approximation of the dense weights error covariance matrix with a sparse diagonal block matrix for GEKF. We implement both RLEKF and the baseline Adam in our alphaDynamics package and numerical experiments are performed on 13 unbiased datasets. Overall, RLEKF converges faster with slightly better accuracy. For example, a test on a typical system, bulk copper, shows that RLEKF converges faster by both the number of training epochs (x11.67) and wall-clock time (x1.19). Besides, we theoretically prove that the updates of weights converge and thus are against the gradient exploding problem. Experimental results verify that RLEKF is not sensitive to the initialization of weights. The RLEKF sheds light on other AI-for-science applications where training a large neural network (with tons of thousands parameters) is a bottleneck.

Deep ensembles vs committees for uncertainty estimation in neural-network force fields: Comparison and application to active learning.

A reliable uncertainty estimator is a key ingredient in the successful use of machine-learning force fields for predictive calculations. Important considerations are correlation with error, overhead during training and inference, and efficient workflows to systematically improve the force field. However, in the case of neural-network force fields, simple committees are often the only option considered due to their easy implementation. Here, we present a generalization of the deep-ensemble design based on multiheaded neural networks and a heteroscedastic loss. It can efficiently deal with uncertainties in both energy and forces and take sources of aleatoric uncertainty affecting the training data into account. We compare uncertainty metrics based on deep ensembles, committees, and bootstrap-aggregation ensembles using data for an ionic liquid and a perovskite surface. We demonstrate an adversarial approach to active learning to efficiently and progressively refine the force fields. That active learning workflow is realistically possible thanks to exceptionally fast training enabled by residual learning and a nonlinear learned optimizer.

Neural-network-backed effective harmonic potential study of the ambient pressure phases of hafnia

Machine learning methods, and in particular neural-network force fields (NNFF), are bridging the gap between macroscopically relevant parameters and quantum mechanical simulations. This work describes an NNFF training strategy and uses it as the back end for an effective harmonic potential study of phase transitions in hafnia. While good agreement with experiment is found regarding the monoclinic-tetragonal transition, the commonly assumed transition to a Fm$\overline{3}$m cubic phase is found to be unlikely for the stoichiometric material

Million-scale data integrated deep neural network for phonon properties of heuslers spanning the periodic table

Existing machine learning potentials for predicting phonon properties of crystals are typically limited on a material-to-material basis, primarily due to the exponential scaling of model complexity with the number of atomic species. We address this bottleneck with the developed Elemental Spatial Density Neural Network Force Field, namely Elemental-SDNNFF. The effectiveness and precision of our Elemental-SDNNFF approach are demonstrated on 11,866 full, half, and quaternary Heusler structures spanning 55 elements in the periodic table by prediction of complete phonon properties. Self-improvement schemes including active learning and data augmentation techniques provide an abundant 9.4 million atomic data for training. Deep insight into predicted ultralow lattice thermal conductivity (<1 Wm−1 K−1) of 774 Heusler structures is gained by p–d orbital hybridization analysis. Additionally, a class of two-band charge-2 Weyl points, referred to as “double Weyl points”, are found in 68% and 87% of 1662 half and 1550 quaternary Heuslers, respectively.

Size and Quality of Quantum Mechanical Data Set for Training Neural Network Force Fields for Liquid Water.

Molecular dynamics simulations have been used in different scientific fields to investigate a broad range of physical systems. However, the accuracy of calculation is based on the model considered to describe the atomic interactions. In particular, ab initio molecular dynamics (AIMD) has the accuracy of density functional theory (DFT) and thus is limited to small systems and a relatively short simulation time. In this scenario, Neural Network Force Fields (NNFFs) have an important role, since they provide a way to circumvent these caveats. In this work, we investigate NNFFs designed at the level of DFT to describe liquid water, focusing on the size and quality of the training data set considered. We show that structural properties are less dependent on the size of the training data set compared to dynamical ones (such as the diffusion coefficient), and a good sampling (selecting data reference for the training process) can lead to a small sample with good precision.