Potential Of Neural Networks Research Articles

Synthetic pre-training for neural-network interatomic potentials

Machine learning (ML) based interatomic potentials have transformed the field of atomistic materials modelling. However, ML potentials depend critically on the quality and quantity of quantum-mechanical reference data with which they are trained, and therefore developing datasets and training pipelines is becoming an increasingly central challenge. Leveraging the idea of ‘synthetic’ (artificial) data that is common in other areas of ML research, we here show that synthetic atomistic data, themselves obtained at scale with an existing ML potential, constitute a useful pre-training task for neural-network (NN) interatomic potential models. Once pre-trained with a large synthetic dataset, these models can be fine-tuned on a much smaller, quantum-mechanical one, improving numerical accuracy and stability in computational practice. We demonstrate feasibility for a series of equivariant graph-NN potentials for carbon, and we carry out initial experiments to test the limits of the approach.

Nuclear quantum effects in the acetylene:ammonia plastic co-crystal.

Organic molecular solids can exhibit rich phase diagrams. In addition to structurally unique phases, translational and rotational degrees of freedom can melt at different state points, giving rise to partially disordered solid phases. The structural and dynamic disorder in these materials can have a significant impact on the physical properties of the organic solid, necessitating a thorough understanding of disorder at the atomic scale. When these disordered phases form at low temperatures, especially in crystals with light nuclei, the prediction of material properties can be complicated by the importance of nuclear quantum effects. As an example, we investigate nuclear quantum effects on the structure and dynamics of the orientationally disordered, translationally ordered plastic phase of the acetylene:ammonia (1:1) co-crystal that is expected to exist on the surface of Saturn's moon Titan. Titan's low surface temperature (∼90K) suggests that the quantum mechanical behavior of nuclei may be important in this and other molecular solids in these environments. By using neural network potentials combined with ring polymer molecular dynamics simulations, we show that nuclear quantum effects increase orientational disorder and rotational dynamics within the acetylene:ammonia (1:1) co-crystal by weakening hydrogen bonds. Our results suggest that nuclear quantum effects are important to accurately model molecular solids and their physical properties in low-temperature environments.

Theoretical Prediction of Thermal Expansion Anisotropy for Y2Si2O7 Environmental Barrier Coatings Using a Deep Neural Network Potential and Comparison to Experiment.

Environmental barrier coatings (EBCs) are an enabling technology for silicon carbide (SiC)-based ceramic matrix composites (CMCs) in extreme environments such as gas turbine engines. However, the development of new coating systems is hindered by the large design space and difficulty in predicting the properties for these materials. Density Functional Theory (DFT) has successfully been used to model and predict some thermodynamic and thermo-mechanical properties of high-temperature ceramics for EBCs, although these calculations are challenging due to their high computational costs. In this work, we use machine learning to train a deep neural network potential (DNP) for Y2Si2O7, which is then applied to calculate the thermodynamic and thermo-mechanical properties at near-DFT accuracy much faster and using less computational resources than DFT. We use this DNP to predict the phonon-based thermodynamic properties of Y2Si2O7 with good agreement to DFT and experiments. We also utilize the DNP to calculate the anisotropic, lattice direction-dependent coefficients of thermal expansion (CTEs) for Y2Si2O7. Molecular dynamics trajectories using the DNP correctly demonstrate the accurate prediction of the anisotropy of the CTE in good agreement with the diffraction experiments. In the future, this DNP could be applied to accelerate additional property calculations for Y2Si2O7 compared to DFT or experiments.

A Multiple-Fidelity Method for Accurate Simulation of MoS2 Properties Using JAX-ReaxFF and Neural Network Potentials.

Reactive force field (ReaxFF) is a commonly used force field for modeling chemical reactions at the atomic level. Recently, JAX-ReaxFF, combined with automatic differentiation, has been used to efficiently parametrize ReaxFF. However, its analytical formula may lead to inaccurate predictions. While neural network-based potentials (NNPs) trained on density functional theory-labeled data offer a more accurate method, it requires a large amount of training data to be trained from scratch. To overcome these issues, we present a multiple-fidelity method that combines JAX-ReaxFF and NNP and apply the method on MoS2, a promising two-dimensional semiconductor for flexible electronics. By incorporating implicit prior physical information, ReaxFF can serve as a cost-effective way to generate pretraining data, facilitating more accurate simulations of MoS2. Moreover, in the Mo-S-H system, the pretraining strategy can reduce root-mean-square errors of energy by 20%. This approach can be extended to a wide variety of material systems, accelerating their computational research.

Rapid and accurate predictions of perfect and defective material properties in atomistic simulation using the power of 3D CNN-based trained artificial neural networks

This article introduces an innovative approach that utilizes machine learning (ML) to address the computational challenges of accurate atomistic simulations in materials science. Focusing on the field of molecular dynamics (MD), which offers insight into material behavior at the atomic level, the study demonstrates the potential of trained artificial neural networks (tANNs) as surrogate models. These tANNs capture complex patterns from built datasets, enabling fast and accurate predictions of material properties. The article highlights the application of 3D convolutional neural networks (CNNs) to incorporate atomistic details and defects in predictions, a significant advancement compared to current 2D image-based, or descriptor-based methods. Through a dataset of atomistic structures and MD simulations, the trained 3D CNN achieves impressive accuracy, predicting material properties with a root-mean-square error below 0.65 GPa for the prediction of elastic constants and a speed-up of approximately 185 to 2100 times compared to traditional MD simulations. This breakthrough promises to expedite materials design processes and facilitate scale-bridging in materials science, offering a new perspective on addressing computational demands in atomistic simulations.

Knowledge distillation of neural network potential for molecular crystals.

Organic molecular crystals exhibit various functions due to their diverse molecular structures and arrangements. Computational approaches are necessary to explore novel molecular crystals from the material space, but quantum chemical calculations are costly and time-consuming. Neural network potentials (NNPs), trained on vast amounts of data, have recently gained attention for their ability to perform energy calculations with accuracy comparable to quantum chemical methods at high speed. However, NNPs trained on datasets primarily consisting of inorganic crystals, such as the Materials Project, may introduce bias when applied to organic molecular crystals. This study investigates the strategies to improve the accuracy of a pre-trained NNP for organic molecular crystals by distilling knowledge from a teacher model. The most effective knowledge transfer was achieved when fine-tuning using only soft targets, i.e., the teacher model's inference values. As the ratio of hard target loss increased, the efficiency of knowledge transfer decreased, leading to overfitting. As a proof of concept, the NNP created through knowledge distillation was used to predict elastic properties, resulting in improved accuracy compared to the pre-trained model.

Uncovering the decomposition mechanism of nitrate ester plasticized polyether (NEPE): a neural network potential simulation.

Nitrate ester plasticized polyether (NEPE) propellants have attracted widespread attention due to their high energy density and excellent low-temperature mechanical properties. However, little is known about the thermal decomposition process of the NEPE propellant, particularly lacking microscale models and interaction mechanisms. This work aims to establish a high-precision and efficient neural network potential (NNP) model covering the NEPE matrix, describing its mechanical behavior and detailed thermal decomposition mechanisms. The model accuracy, including atomic energies and forces, was validated through density functional theory (DFT) results, and the NEPE propellant decomposition model was verified via molecular dynamics (MD) simulations with DFT precision. The results demonstrate that the NNP model accurately predicts the energies and forces of the NEPE matrix for single and mixed systems at the DFT-level precision, and reproduces the mechanical properties consistent with DFT calculations. Meanwhile, the thermal decomposition order of the NEPE matrix predicted by NNP is consistent with the experimental results, accurately capturing complex physical phenomena and detailed decomposition processes among components. It is also revealed that the addition of a binder can improve the stability of the propellant and extend its energy release time. This study applies innovative machine learning algorithms to develop an NNP computational model for the NEPE matrix with DFT precision, which is crucial for practical propellant formulation design.

A first-principles exploration of the conformational space of sodiated di-saccharides assisted by semi-empirical methods and neural network potentials.

Previous exploration of the conformational space of sodiated mono-saccharides using a random search algorithm leads to ∼103 structurally distinct conformers covering an energy range of ∼150 kJ mol-1. Thus, it is reasonable to expect that the number of distinct conformers for a given disaccharide would be on the order of 106. Efficient identification of distinct conformers at the first-principles level has been demonstrated with the assistance of neural network potential (NNP) with an accuracy of ∼1 kJ mol-1 compared to DFT. Leveraging a local minima database of neutral and sodiated glucose (Glc), we develop algorithms to systematically explore the conformation landscape of 19 Glc-based sodiated disaccharides. To accelerate the exploration, the NNP method is implemented. The NNP achieves an accuracy of ∼2.3 kJ mol-1 compared to DFT, offering a comparable quality to that of DFT. Through a multi-model approach integrating DFTB3, NNP and DFT, we can rapidly locate low-energy disaccharide conformers at the first-principles level. The methodology we show here can be used to efficiently explore the potential energy landscape of any di-saccharides when first-principles accuracy is required.

Long-range proton and hydroxide ion transfer dynamics at the water/CeO2 interface in the nanosecond regime: reactive molecular dynamics simulations and kinetic analysis.

The structural properties, dynamical behaviors, and ion transport phenomena at the interface between water and cerium oxide are investigated by reactive molecular dynamics (MD) simulations employing neural network potentials (NNPs). The NNPs are trained to reproduce density functional theory (DFT) results, and DFT-based MD (DFT-MD) simulations with enhanced sampling techniques and refinement schemes are employed to efficiently and systematically acquire training data that include diverse hydrogen-bonding configurations caused by proton hopping events. The water interfaces with two low-index surfaces of (111) and (110) are explored with these NNPs, and the structure and long-range proton and hydroxide ion transfer dynamics are examined with unprecedented system sizes and long simulation times. Various types of proton hopping events at the interface are categorized and analyzed in detail. Furthermore, in order to decipher the proton and hydroxide ion transport phenomena along the surface, a counting analysis based on the semi-Markov process is formulated and applied to the MD trajectories to obtain reaction rates by considering the transport as stochastic jump processes. Through this model, the coupling between hopping events, vibrational motions, and hydrogen bond networks at the interface are quantitatively examined, and the high activity and ion transport phenomena at the water/CeO2 interface are unequivocally revealed in the nanosecond regime.

The thermal decomposition mechanism of RDX/AP composites: ab initio neural network MD simulations.

A neural network potential (NNP) is developed to investigate the decomposition mechanism of RDX, AP, and their composites. Utilizing an ab initio dataset, the NNP is evaluated in terms of atomic energy and forces, demonstrating strong agreement with ab initio calculations. Numerical stability tests across a range of timesteps reveal excellent stability compared to the state-of-the-art ReaxFF models. Then the thermal decomposition of pure RDX, AP, and RDX/AP composites is performed using NNP to explore the coupling effect between RDX and AP. The results highlight a dual interaction between RDX and AP, i.e., AP accelerates RDX decomposition, particularly at low temperatures, and RDX promotes AP decomposition. Analyzing RDX trajectories at the RDX/AP interface unveils a three-part decomposition mechanism involving N-N bond cleavage, H transfer with AP to form Cl-containing acid, and chain-breaking reactions generating small molecules such as N2, CO, and CO2. The presence of AP enhances H transfer reactions, contributing to its role in promoting RDX decomposition. This work studies the reaction kinetics of RDX/AP composites from the atomic point of view, and can be widely used in the establishment of reaction kinetics models of composite systems with energetic materials.

A machine learning approach for dynamical modelling of Al distributions in zeolites via23Na/27Al solid-state NMR.

One of the main limitations in supporting experimental characterization of Al siting/pairing via modelling is the high computational cost of ab initio calculations. For this reason, most works rely on static or very short dynamical simulations, considering limited Al pairing/siting combinations. As a result, comparison with experiment suffers from a large degree of uncertainty. To alleviate this limitation we have developed neural network potentials (NNPs) which can dynamically sample across broad configurational and chemical spaces of sodium-form aluminosilicate zeolites, preserving the level of accuracy of the ab initio (dispersion-corrected metaGGA) training set. By exploring a wide range of Al/Na arrangements and a combination of experimentally relevant Si/Al ratios, we found that the 23Na NMR spectra of dehydrated high-silica CHA zeolite offer an opportunity to assess the distribution and pairing of Al atoms. We observed that the 23Na chemical shift is sensitive not only to the location of sodium in 6- and 8MRs, but also to the Al-Sin-Al sequence length. Furthermore, neglect of thermal and dynamical contributions was found to lead to errors of several ppm, and has a profound influence on the shape of the spectra and the dipolar coupling constants, thus necessitating the long-term dynamical simulations made feasible by NNPs. Finally, we obtained a predictive regression model for the 23Na chemical shift in CHA (Si/Al = 35, 17, 11) that circumvents the need for expensive NMR density functional calculations and can be easily extended to other zeolite frameworks. By combining NNPs and regression methods, we can expedite the simulations of NMR properties and capture the effect of dynamics on the spectra, which is often overlooked in computational studies despite its clear manifestation in experimental setups.

Impact of vacancy defects on the thermal conductivity of BaAgBi: a comprehensive study using molecular dynamics simulations with neural network potentials.

The presence of vacancy defects significantly impacts thermal properties of materials. In this research, we delve into the effects of vacancy defects on the thermal conductivity of ternary alloy BaAgBi, employing molecular dynamics simulations coupled with a deep neural network potential (NNP). Initially, we validate the precision of our NNP by comparing their predictions for energy, atomic forces, phonon dispersion curves, phonon density of states, and vacancy formation energy with density functional theory calculations, ensuring a high degree of accuracy. Our findings reveal that the reduction in thermal conductivity due to vacancies aligns with the Debye-Callaway model, with variations depending on the type of vacancy. Specifically, Ba vacancies result in the most notable decrement in thermal conductivity, attributable to their low phonon participation ratio and high lattice distortion, both factors that enhance phonon scattering. Besides, we find that the high energy barrier (∼1.66 eV) indicates that Ba vacancies hardly migrate at 300 K. This study helps us understand how vacancies affect thermal conductivity in BaAgBi and how different vacancy types affect it.

Structure and properties of the FLiNaK – LaF3 melt obtained with neural network potential

Accurate and efficient prediction of the thermochemical properties of the melts applicable in molten salt reactors could be made if the proper machine learning model is used. In this paper, the neural network potential for simulation of 85 % (LiF – NaF – KF)eut. – 15 % LaF3 molten mixture was developed based on ab initio data. In spite of multiple atomic types, the model of a moderate size showed small root mean squared errors in energy and forces of 0.5 meV/atom and 39 meV/Å, respectively. Then the neural network potential was employed to calculate local structure, density, self-diffusion coefficients, heat capacity, thermal conductivity, and thermal diffusivity for a range of temperatures. We found that the addition of LaF3 to the eutectic mixture of alkali fluorides results in a reduction the melt ability to store and transfer heat. The strong effect observed here is the reduction in heat capacity by 20–30 %. The analysis of the local structure details reveals the existence of [LaF6], [LaF7] and [LaF8] groupings, with the most probable being [LaF7] and the La – F separation averaged over the ensemble of 2.35 Å.

Determining the mechanical and decomposition properties of high energetic materials (α-RDX, β-HMX, and ε-CL-20) using a neural network potential.

Molecular simulations of high energetic materials (HEMs) are limited by efficiency and accuracy. Recently, neural network potential (NNP) models have achieved molecular simulations of millions of atoms while maintaining the accuracy of density functional theory (DFT) levels. Herein, an NNP model covering typical HEMs containing C, H, N, and O elements is developed. The mechanical and decomposition properties of 1,3,5-trinitroperhydro-1,3,5-triazine (RDX), hexahydro-1,3,5-trinitro-1,3,5-triazine (HMX), and 2,4,6,8,10,12-hexanitrohexaazaisowurtzitane (CL-20) are determined by employing the molecular dynamics (MD) simulations based on the NNP model. The calculated results show that the mechanical properties of α-RDX, β-HMX, and ε-CL-20 agree with previous experiments and theoretical results, including cell parameters, equations of state, and elastic constants. In the thermal decomposition simulations, it is also found that the initial decomposition reactions of the three crystals are N-NO2 homolysis, corresponding radical intermediates formation, and NO2-induced reactions. This decomposition trajectory is mainly divided into two stages separating from the peak of NO2: pyrolysis and oxidation. Overall, the NNP model for C/H/N/O elements in this work is an alternative reactive force field for RDX, HMX, and CL-20 HEMs, and it opens up new potential for future kinetic study of nitramine explosives.

Germanium distributions in zeolites derived from neural network potentials

This work uses newly developed machine learning potentials to predict how germanium distributes within the zeolite catalysts, depending on both germanium content and the framework topology, aiding the rational zeolite design.

Searching low-energy conformers of neutral and protonated di-, tri-, and tetra-glycine using first-principles accuracy assisted by the use of neural network potentials.

In the last ten years, combinations of state-of-the-art gas-phase spectroscopies and quantum chemistry calculations have suggested several intuitive trends in the structure of small polypeptides that may not hold true. For example, the preference for the cis form of the peptide bond and multiple protonated sites was proposed by comparing experimental spectra with low-energy minima obtained from limited structural sampling using various density functional theory methods. For understanding the structures of polypeptides, extensive sampling of their configurational space with high-accuracy computational methods is required. In this work, we demonstrated the use of deep-learning neural network potential (DL-NNP) to assist in exploring the structure and energy landscape of di-, tri-, and tetra-glycine with the accuracy of high-level quantum chemistry methods, and low-energy conformers of small polypeptides can be efficiently located. We hope that the structures of these polypeptides we found and our preliminary analysis will stimulate further experimental investigations.

Active learning of neural network potentials for rare events

Developing an automated active learning framework for Neural Network Potentials, focusing on accurately simulating bond-breaking in hexane chains through steered molecular dynamics sampling and assessing model transferability.

Block-Wise Partner Learning for Model Compression.

Despite the great potential of convolutional neural networks (CNNs) in various tasks, the resource-hungry nature greatly hinders their wide deployment in cost-sensitive and low-powered scenarios, especially applications in remote sensing. Existing model pruning approaches, implemented by a "subtraction" operation, impose a performance ceiling on the slimmed model. Self-knowledge distillation (Self-KD) resorts to auxiliary networks that are only active in the training phase for performance improvement. However, the knowledge is holistic and crude, and the learning-based knowledge transfer is mediate and lossy. Here, we propose a novel model-compression method, termed block-wise partner learning (BPL), which comprises "extension" and "fusion" operations and liberates the compressed model from the bondage of baseline. Different from the Self-KD, the proposed BPL creates a partner for each block for performance enhancement in training. For the model to absorb more diverse information, a diversity loss (DL) is designed to evaluate the difference between the original block and the partner. Besides, the partner is fused equivalently instead of being discarded directly. After training, we can simply adopt the fused compressed model that contains the enhancement information of partners but with fewer parameters and less inference cost. As validated using the UC Merced land-use, NWPU-RESISC45, and RSD46-WHU datasets, the BPL demonstrates superiority over other compared model-compression approaches. For example, it attains a substantial floating-point operations (FLOPs) reduction of 73.97% with only 0.24 accuracy (ACC.) loss for ResNet-50 on the UC Merced land-use dataset. The code is available at https://github.com/zhangxin-xd/BPL.

Comparative Analysis of Vanilla CNN and Transfer Learning Models for Glaucoma Detection

Glaucoma is a leading cause of blindness worldwide and results from high eye pressure‐induced damage to the optic nerves, thereby preventing visual information from reaching the brain. While glaucoma is incurable in its advanced stages, its early detection improves the treatment outcome. Recently, computational models for image classification have enabled early detection of glaucoma from OCT scans of patients and will potentially augment the medical diagnosis of this disease. Models that are based on the convolutional neural networks (CNNs) have shown promise in the early detection of glaucoma. These models vary in their architectures and their accuracies depend largely on the intrinsic nature of the training datasets used. Hence, in this work, a comparative analysis is performed on vanilla CNN, AlexNet, GoogLeNet, and ResNet50 using two popular glaucoma datasets (ACRIMA and ORIGA). With careful attention to exhaustive image processing, an impressive training accuracy of 89% and validation accuracy of 50% was obtained from the vanilla CNN, showing a high sensitivity of 88% in detecting glaucomatous patients from OCT scans. However, the other models (AlexNet, GoogLeNet, and ResNet5) overfitted with a large difference between the obtained training accuracy and validation accuracy. The results also reveal that ResNet50 has the highest computational cost compared to the rest of the models. The obtained results demonstrate the peculiarity of the dataset, its selectiveness of the most appropriate model, and the potential of deep neural networks (DNNs) as an effective screening tool for glaucoma, enabling prompt interventions, reducing healthcare costs, and helping optometrists make swift decisions.

Actualization of linguistic markers in political Internet discourse, and its peculiarities in the context of artificial neural network operation, and hate speech detection

The article argues that the actualization of linguistic markers (lexical, grammatical, pragmatic, semantic) in the texts of political Internet discourse is crucial for understanding the context (social, emotional, modal, etc.) in them. It is emphasized that the use of innovative tools (in particular, artificial neural networks) is in line with the trends towards the integration of the mathematical paradigm into the humanities, which is manifested in the latter’s tendency to the humanities digital sciences, affective computing, etc. It is noted that the use of the outlined innovative tools creates the need to take into account an integrated approach to such research, which is manifested in the integration of the methodology of other (primarily related) sciences, which, in turn, will allow to take into account contextual determination, cultural features (gaps, realities, etc.), as well as to use the full potential of artificial neural networks in modern linguistic research. The author emphasizes the need to further develop the originality of the use of the latter in such studies: developing datasets, building a training algorithm, testing and retraining (if necessary), as well as integrating different layers into their structures and optimizing such a structure specifically for linguistic research (which combinations are productive and why, for what purpose, etc.) The author emphasizes the originality of the use of innovative tools (deep learning, machine learning, artificial neural networks) to deal with hate speech, misinformation, disinformation, and propaganda, which can take the form of sarcasm, irony, and absurdity in political Internet discourse, which requires further research and development of a new methodology to accurately identify such data and produce an inclusive and safe Internet environment.