Potential Of Deep Neural Networks Research Articles

Ab initio informed machine learning potential for tribochemistry and mechanochemistry: Application for eco–friendly gallate lubricant additive

Mechanochemistry and tribochemistry processes involve multiple physical/chemical interactions induced by extreme conditions including molecular confinement, high temperatures and mechanical stress applied. Simulating these processes by molecular dynamics is very challenging. While force fields fall short reproducing the enhanced reactivity arising by quantum effects, ab initio molecular dynamics is severely limited by the complexity of the systems of interest, their sizes, and the long-time scale on which relevant events take place. In this work using an active learning approach, a landmark deep neural network potential has been developed which reproduces the accuracy of ab initio interactions at the classical molecular dynamics computational cost and permits to successfully simulate the tribochemical processes occurring at the interface between butyl gallate molecules and iron substrates under tribological conditions. The simulations of the dynamics of the Fe-gallate system when sliding under an imposed external load reveal the key atomistic mechanisms underlying the formation of the friction reducing lubricant tribofilm and permit to characterize the tribological properties of the explored systems, clearly exposing the shortcoming of reactive force field based approaches. The successful development of neural network potentials, making it possible to push the limits of molecular dynamics marrying accuracy with system sizes and long time scales, paves the route toward a new area in computational tribochemistry.

Audio analysis speeding detection techniques based on metaheuristic-optimized machine learning models

Law enforcement plays an important role in road safety across the world. Speed enforcement merits special attention as the correlation between exceeding speed limits and the risk of fatalities and serious injuries is well-known. Methods based on radar and light detection and ranging techniques have established themselves as effective methods for measuring vehicle speeds in real time on roads. However, the price of instruments used for these measurements limits their widespread applicability. This work explored the potential of coupling artificial intelligence and audio analysis for vehicle speed assessment. The potential of deep neural networks and extreme gradient boosting is explored on a recently proposed real-world vehicle speed dataset. Frequency analysis techniques are also applied to help reduce the computational demands of experimentation. Additionally, a modified optimization algorithm is introduced to help select optimal control parameters and improve the performance of the suggested method. Five experiments were executed to demonstrate the potential for detecting the exact vehicle speed using regression techniques as well as utilizing classifications to detect vehicles breaking the speed limit. The proposed methods demonstrated promising potential when applied to this pressing challenge.

Autonomous Car Behavioral Training Using Deep Neural Network

Autonomous driving is becoming increasingly prevalent nowadays. With the help of a number of images of car movement from the Kaggle self-driving dataset, we explore the feasibility of utilizing the images obtained to train a deep neural network to detect and predict the steering angle, which is the critical part of the car behavior. Since deep neural networks have emerged as powerful tools for training autonomous cars and learning about and improving their driving behaviors, we incorporate convolutional layers and additional layers in the deep neural network architecture so that it can capture the behaviors appropriately and provide effective results. We demonstrate that the implementation of this approach is successful and that the corresponding implementation highlights the potential of deep neural network in advancing autonomous car technology. Our comprehensive evaluation suggests that further research should concentrate on refining the network architecture and enhancing perception capabilities in order to deliver promising advances to the field.

Theoretical Prediction of Thermal Expansion Anisotropy for Y2Si2O7 Environmental Barrier Coatings Using a Deep Neural Network Potential and Comparison to Experiment.

Environmental barrier coatings (EBCs) are an enabling technology for silicon carbide (SiC)-based ceramic matrix composites (CMCs) in extreme environments such as gas turbine engines. However, the development of new coating systems is hindered by the large design space and difficulty in predicting the properties for these materials. Density Functional Theory (DFT) has successfully been used to model and predict some thermodynamic and thermo-mechanical properties of high-temperature ceramics for EBCs, although these calculations are challenging due to their high computational costs. In this work, we use machine learning to train a deep neural network potential (DNP) for Y2Si2O7, which is then applied to calculate the thermodynamic and thermo-mechanical properties at near-DFT accuracy much faster and using less computational resources than DFT. We use this DNP to predict the phonon-based thermodynamic properties of Y2Si2O7 with good agreement to DFT and experiments. We also utilize the DNP to calculate the anisotropic, lattice direction-dependent coefficients of thermal expansion (CTEs) for Y2Si2O7. Molecular dynamics trajectories using the DNP correctly demonstrate the accurate prediction of the anisotropy of the CTE in good agreement with the diffraction experiments. In the future, this DNP could be applied to accelerate additional property calculations for Y2Si2O7 compared to DFT or experiments.

Graph attention network and radial basis function neural network-based hybrid framework for epileptic seizure detection from EEG signal

<p>Epileptic seizure is a neurological disorder characterized by recurrent, abrupt behavioral changes attributed to transient shifts in excessive electrical discharges within specific brain cell groups. Electroencephalogram (EEG) signals are the primary modality for capturing seizure activity, offering real-time, computer-assisted detection through long-term monitoring. Over the last decade, extensive experiments through deep learning techniques on EEG signal analysis, and automatic seizure detection. Nevertheless, realizing the full potential of deep neural networks in seizure detection remains a challenge, primarily due to limitations in model architecture design and their capacity to handle time series brain data. The fundamental drawback of current deep learning methods is their struggle to effectively represent physiological EEG recordings; as it is irregular and unstructured in nature, which is difficult to fit into matrix format in traditional methods. Because of this constraint, a significant research gap remains in this research field. In this context, we propose a novel approach to bridge this gap, leveraging the inherent relationships within EEG data. Graph neural networks (GNNs) offer a potential solution, capitalizing on their ability to naturally encapsulate relational data between variables. By representing interacting nodes as entities connected by edges with weights determined by either temporal associations or anatomical connections, GNNs have garnered substantial attention for their potential in configuring brain anatomical systems. In this paper, we introduce a hybrid framework for epileptic seizure detection, combining the Graph Attention Network (GAT) with the Radial Basis Function Neural Network (RBFN) to address the limitations of existing approaches. Unlike traditional graph-based networks, GAT automatically assigns weights to neighbouring nodes, capturing the significance of connections between nodes within the graph. The RBFN supports this by employing linear optimization techniques to provide a globally optimal solution for adjustable weights, optimizing the model in terms of the minimum mean square error (MSE). Power spectral density is used in the proposed method to analyze and extract features from electroencephalogram (EEG) signals because it is naturally simple to analyze, synthesize, and fit into the graph attention network (GAT), which aids in RBFN optimization. The proposed hybrid framework outperforms the state-of-the-art in seizure detection tasks, obtaining an accuracy of 98.74%, F1-score of 96.2%, and Area Under Curve (AUC) of 97.3% in a comprehensive experiment on the publicly available CHB-MIT EEG dataset.</p>

Mitigating the influence of domain shift in skin lesion classification: A benchmark study of unsupervised domain adaptation methods

The potential of deep neural networks in skin lesion classification has already been demonstrated to be on-par if not superior to the dermatologists’ diagnosis in experimental settings. However, the performance of these models usually deteriorates in real-world scenarios, where the test data differs significantly from the training data (i.e. domain shift). This concerning limitation for models intended to be used in real-world skin lesion classification tasks poses a risk to patients. For example, different image acquisition systems or previously unseen anatomical sites on the patient can suffice to cause such domain shifts. Mitigating the negative effect of such shifts is therefore crucial, but developing effective methods to address domain shift has proven to be challenging. In this study, we carry out a comparative analysis of eight different unsupervised domain adaptation methods to analyze their effectiveness in improving generalization for dermoscopic datasets. To ensure robustness of our findings, we test each method on a total of ten derived datasets, thereby covering a variety of possible domain shifts. In addition, we investigated which factors in the domain shifted datasets have an impact on the effectiveness of domain adaptation methods. Our findings show that all of the eight domain adaptation methods result in improved AUPRC for the majority of analyzed datasets. Altogether, these results indicate that unsupervised domain adaptations generally lead to performance improvements for the binary melanoma-nevus classification task regardless of the nature of the domain shift. However, small or heavily imbalanced datasets lead to a reduced conformity of the results due to the influence of these factors on the methods’ performance.

Probing the self-ionization of liquid water with ab initio deep potential molecular dynamics.

The chemical equilibrium between self-ionized and molecular water dictates the acid-base chemistry in aqueous solutions, yet understanding the microscopic mechanisms of water self-ionization remains experimentally and computationally challenging. Herein, Density Functional Theory (DFT)-based deep neural network (DNN) potentials are combined with enhanced sampling techniques and a global acid-base collective variable to perform extensive atomistic simulations of water self-ionization for model systems of increasing size. The explicit inclusion of long-range electrostatic interactions in the DNN potential is found to be crucial to accurately reproduce the DFT free energy profile of solvated water ion pairs in small (64 and 128 H2O) cells. The reversible work to separate the hydroxide and hydronium to a distance [Formula: see text] is found to converge for simulation cells containing more than 500 H2O, and a distance of [Formula: see text] 8 Å is the threshold beyond which the work to further separate the two ions becomes approximately zero. The slow convergence of the potential of mean force with system size is related to a restructuring of water and an increase of the local order around the water ions. Calculation of the dissociation equilibrium constant illustrates the key role of long-range electrostatics and entropic effects in the water autoionization process.

Reversible and irreversible photon-absorption in amorphous SiO2 revealed by deep potential

Understanding the kinetic balance between reversible and irreversible photon-absorption (IPA/RPA) in amorphous-SiO2 (a-SiO2) is crucial for mitigating optical damage and assessing the lifetime of irradiated optics. Experimental work has placed some thermodynamic bounds on the formation of defects from damage precursor, but the formation kinetics and underlying microscopic mechanisms are yet to be elucidated. Here, by employing deep neural network potential, we find that the formation of defects are temperature-dependent and density-dependent, where competitions between IPA and RPA take place. The IPA prevails in the early formation of E’-center, and the ratio of IPA/RPA is related to the strained bonds, SiSi bonds and 5-coordinated silicon (Si5). Furthermore, the Si5 is critical to the propagation of E’-center and densification of a-SiO2. Additionally, we develop quantitative models of defects in regard with irradiation temperature of specified temperature and irradiation photon-energy, respectively.

Realizing the potential of deep neural network for analyzing neutron star observables and dense matter equation of state

Realizing the potential of deep neural network for analyzing neutron star observables and dense matter equation of state

Deep neural network potential for simulating hydrogen blistering in tungsten

Deep neural network potential for simulating hydrogen blistering in tungsten

The value of deep neural networks in the pathological classification of thyroid tumors

BackgroundTo explore the distinguishing diagnostic value and clinical application potential of deep neural networks (DNN) for pathological images of thyroid tumors.MethodsA total of 799 pathological thyroid images of 559 patients with thyroid tumors were retrospectively analyzed. The pathological types included papillary thyroid carcinoma (PTC), medullary thyroid carcinoma (MTC), follicular thyroid carcinoma (FTC), adenomatous goiter, adenoma, and normal thyroid gland. The dataset was divided into a training set and a test set. Resnet50, Resnext50, EfficientNet, and Densenet121 were trained using the training set data and tested with the test set data to determine the diagnostic efficiency of different pathology types and to further analyze the causes of misdiagnosis.ResultsThe recall, precision, negative predictive value (NPV), accuracy, specificity, and F1 scores of the four models ranged from 33.33% to 100.00%. The area under curve (AUC) ranged from 0.822 to 0.994, and the Kappa coefficient ranged from 0.7508 to 0.7713. However, the performance of diagnosing FTC, adenoma, and adenomatous goiter was slightly inferior to other types of pathological tissues.ConclusionThe DNN model achieved satisfactory results in the task of classifying thyroid tumors by learning thyroid pathology images. These results indicate the potential of the DNN model for the efficient diagnosis of thyroid tumor histopathology.

Comparison of Matlantis and VASP Bulk Formation and Surface Energies in Metal Hydrides, Carbides, Nitrides, Oxides, and Sulfides

Abstract Generic neural network potentials without forcing users to train potentials could result in significant acceleration of total energy calculations. Takamoto et al. [Nat. Commun. (2022), 13, 2991] developed such a deep neural network potential (NNP) and made it available in their Matlantis package. Matlantis bulk formation energies of metal hydrides, carbides, nitrides, oxides, and sulfides were consistently ∼0.1 eV/atom larger and the surface energies were typically ∼10 meV/Å2 smaller than our previously calculated PBEsol(+U) VASP energies.

Deep learning networks in the segmentation of the left atrial appendage in 2D ultrasound: A comparative analysis.

Left atrial appendage (LAA) is the major source of thromboembolism in patients with non-valvular atrial fibrillation. Currently, LAA occlusion can be offered as a treatment for these patients, obstructing the LAA through a percutaneously delivered device. Nevertheless, correct device sizing is a complex task, requiring manual analysis of medical images. This approach is sub-optimal, time-demanding, and highly variable between experts. Different solutions were proposed to improve intervention planning, but, no efficient solution is available to 2D ultrasound, which is the most used imaging modality for intervention planning and guidance. In this work, we studied the performance of recently proposed deep learning methods when applied for the LAA segmentation in 2D ultrasound. For that, it was created a 2D ultrasound database. Then, the performance of different deep learning methods, namely Unet, UnetR, AttUnet, TransAttUnet was assessed. All networks were compared using seven metrics: i) Dice coefficient; ii) Accuracy iii) Recall; iv) Specificity; v) Precision; vi) Hausdorff distance and vii) Average distance error. Overall, the results demonstrate the efficiency of AttUnet and TransAttUnet with dice scores of 88.62% and 89.28%, and accuracy of 88.25% and 86.30%, respectively. The current results demonstrate the feasibility of deep learning methods for LAA segmentation in 2D ultrasound.Clinical relevance- Our results proved the clinical potential of deep neural networks for the LAA anatomical analysis.

Automatic Knee Osteoarthritis Severity Grading using Deep Neural Networks: Comparative Analysis of Network Architectures and Optimization Functions

Knee osteoarthritis (OA) is a prevalent degenerative joint disease that requires accurate assessment of its severity for effective treatment planning. In this study, we propose an automatic knee OA severity-grading system based on deep neural networks. Specifically, we explore various network architectures, including VGG-16, VGG-19, ResNet-101, EfficientNet-B7, and EfficientNet-B6, along with different optimization functions such as SGD, ADAM, Nadam, AdamW, and AdaDelta. Furthermore, we investigate two loss functions, namely, the novel ordinal loss and the cross-entropy loss. The proposed system is evaluated on a carefully curated dataset, and comprehensive experimental settings are employed to ensure reliable results. Our findings indicate that the combination of the EfficientNet-B7 network with the Nadam optimizer yields the best performance, achieving an accuracy of 70.1% in knee OA severity grading. These results demonstrate the potential of deep neural networks in automating the grading process, offering a valuable tool for clinicians and researchers in the field of knee osteoarthritis management.

Deep learning for sleep analysis on children with sleep-disordered breathing: Automatic detection of mouth breathing events

IntroductionSleep-disordered breathing (SDB) can range from habitual snoring to severe obstructive sleep apnea (OSA). A common characteristic of SDB in children is mouth breathing, yet it is commonly overlooked and inconsistently diagnosed. The primary aim of this study is to construct a deep learning algorithm in order to automatically detect mouth breathing events in children from polysomnography (PSG) recordings.MethodsThe PSG of 20 subjects aged 10–13 years were used, 15 of which had reported snoring or presented high snoring and/or high OSA values by scoring conducted by a sleep technologist, including mouth breathing events. The separately measured mouth and nasal pressure signals from the PSG were fed through convolutional neural networks to identify mouth breathing events.ResultsThe finalized model presented 93.5% accuracy, 97.8% precision, 89% true positive rate, and 2% false positive rate when applied to the validation data that was set aside from the training data. The model's performance decreased when applied to a second validation data set, indicating a need for a larger training set.ConclusionThe results show the potential of deep neural networks in the analysis and classification of biological signals, and illustrates the usefulness of machine learning in sleep analysis.

Multistage linguistic conditioning of convolutional layers for speech emotion recognition

IntroductionThe effective fusion of text and audio information for categorical and dimensional speech emotion recognition (SER) remains an open issue, especially given the vast potential of deep neural networks (DNNs) to provide a tighter integration of the two.MethodsIn this contribution, we investigate the effectiveness of deep fusion of text and audio features for categorical and dimensional SER. We propose a novel, multistage fusion method where the two information streams are integrated in several layers of a DNN, and contrast it with a single-stage one where the streams are merged in a single point. Both methods depend on extracting summary linguistic embeddings from a pre-trained BERT model, and conditioning one or more intermediate representations of a convolutional model operating on log-Mel spectrograms.ResultsExperiments on the MSP-Podcast and IEMOCAP datasets demonstrate that the two fusion methods clearly outperform a shallow (late) fusion baseline and their unimodal constituents, both in terms of quantitative performance and qualitative behavior.DiscussionOverall, our multistage fusion shows better quantitative performance, surpassing alternatives on most of our evaluations. This illustrates the potential of multistage fusion in better assimilating text and audio information.

Unravelling the dissolution dynamics of silicate minerals by deep learning molecular dynamics simulation: A case of dicalcium silicate

Quantitative analyses of the thermodynamics and kinetics of silicate minerals dissolution at atomic level are difficult currently, both experimentally and computationally. Here, we apply the deep learning, enhancing sampling molecular dynamics and density functional theory to build up a deep neural network potential with quantum mechanics accuracy, which can determine the free energy surfaces, minimum free-energy reaction pathways and kinetic rates for Ca dissolution from the water/dicalcium silicate interface at different temperatures. We find that the Ca dissolution is a spontaneous reaction and follows different minimum free-energy reaction pathways at different temperatures. The dissolution time of the five-coordinated Ca ion is on the order of hundreds of seconds at ambient temperature and increases to the order of nanoseconds after heating. The relatively slow dissolution kinetics comparing to the low free energy barriers of each elementary reaction is attributed to the multi-directional and multi-step nature of the dissolution reaction. This new atomistic insight promotes a better understanding of the dicalcium silicate and cement hydration.

Development and Validation of Versatile Deep Atomistic Potentials for Metal Oxides.

Machine learning interatomic potentials powered by neural networks have been shown to readily model a gradient of compositions in metallic systems. However, their application to date on ionic systems tends to focus on specific compositions and oxidation states owing to their more heterogeneous chemical nature. Herein we show that a deep neural network potential (DNP) can model various properties of metal oxides with different oxidation states without additional charge information. We created and validated DNPs for AgxOy, CuxOy MgxOy, PtxOy, and ZnxOy, whereby each system was trained without any limitations on oxidation states. We illustrate how the database can be augmented to enhance the DNP transferability for a new polymorph, surface energies, and thermal expansion. In addition, we show that these potentials can correctly interpolate significant pressure and temperature ranges, exhibit stability over long molecular dynamics simulation time scales, and replicate nonharmonic thermal expansion, consistent with experimental results.

Accessing the thermal conductivities of Sb2Te3 and Bi2Te3/Sb2Te3 superlattices by molecular dynamics simulations with a deep neural network potential.

Phonon thermal transport is a key feature for the operation of thermoelectric materials, but it is challenging to accurately calculate the thermal conductivity of materials with strong anharmonicity or large cells. In this work, a deep neural network potential (NNP) is developed using a dataset based on density functional theory (DFT) and applied to describe the lattice dynamics of Sb2Te3 and Bi2Te3/Sb2Te3 superlattices. The lattice thermal conductivities of Sb2Te3 are first predicted using equilibrium molecular dynamics (EMD) simulations combined with an NNP and the results match well with experimental values. Then, through further exploration of weighted phase spaces and the Grüneisen parameter, we find that there is a stronger anharmonicity in the out-of-plane direction in Sb2Te3, which is the reason why the thermal conductivities are overestimated more in the out-of-plane direction than in the in-plane direction by solving the phonon Boltzmann transport equation (BTE) with only three-phonon scattering processes being considered. More importantly, the lattice thermal conductivities of Bi2Te3/Sb2Te3 superlattices with different periods are accurately predicted using non-equilibrium molecular dynamics (NEMD) simulations together with an NNP, which serves as a good example to explore the thermal transport physics of superlattices using a deep neural network potential.

Tuning the lattice thermal conductivity of Sb2Te3 by Cr doping: a deep potential molecular dynamics study.

Element doping is a prominent method for reducing the lattice thermal conductivity and optimizing the thermoelectric performance of materials in the thermoelectric field. However, determination of the thermal conductivity of element-doped systems is a challenging task, especially when the elements are randomly doped. In this work, a first-principles based deep neural network potential (NNP) is developed to investigate the lattice thermal transport properties of Cr-doped Sb2Te3 using molecular dynamics simulations. Compared with pure Sb2Te3, the thermal conductivity of orderly Cr-doped Sb2Te3 with Cr atoms locating at specific atomic layer positions decreases slightly in the in-plane direction, but sharply in the out-of-plane direction. The decrease of the low frequency phonon density of states and the enhancement of phonon scattering near 2.5 THz are the primary reasons for the decrease in the thermal conductivity of Cr-doped Sb2Te3, while the decrease of phonon velocity due to band flattening is the reason for the sharp decrease of thermal conductivity in the out-of-plane direction. Moreover, the thermal conductivities of randomly Cr-doped Sb2Te3 with different Cr concentrations are also investigated using the NNP. It is found that the thermal conductivities in both the in-plane and out-of-plane directions are reduced by 76% and 80%, respectively, for Sb36Cr36Te108. Furthermore, the influence of different Cr dopant arrays on the thermal conductivity of Sb2Te3 is also predicted using the NNP. Our work provides a good example for predicting the thermal conductivity of element-doped systems using the NNP combined with molecular dynamics simulations.