Deep Learning Neural Network Model Research Articles

Novel Ensemble Approach of Deep Learning Neural Network (DLNN) Model and Particle Swarm Optimization (PSO) Algorithm for Prediction of Gully Erosion Susceptibility.

This study aims to evaluate a new approach in modeling gully erosion susceptibility (GES) based on a deep learning neural network (DLNN) model and an ensemble particle swarm optimization (PSO) algorithm with DLNN (PSO-DLNN), comparing these approaches with common artificial neural network (ANN) and support vector machine (SVM) models in Shirahan watershed, Iran. For this purpose, 13 independent variables affecting GES in the study area, namely, altitude, slope, aspect, plan curvature, profile curvature, drainage density, distance from a river, land use, soil, lithology, rainfall, stream power index (SPI), and topographic wetness index (TWI), were prepared. A total of 132 gully erosion locations were identified during field visits. To implement the proposed model, the dataset was divided into the two categories of training (70%) and testing (30%). The results indicate that the area under the curve (AUC) value from receiver operating characteristic (ROC) considering the testing datasets of PSO-DLNN is 0.89, which indicates superb accuracy. The rest of the models are associated with optimal accuracy and have similar results to the PSO-DLNN model; the AUC values from ROC of DLNN, SVM, and ANN for the testing datasets are 0.87, 0.85, and 0.84, respectively. The efficiency of the proposed model in terms of prediction of GES was increased. Therefore, it can be concluded that the DLNN model and its ensemble with the PSO algorithm can be used as a novel and practical method to predict gully erosion susceptibility, which can help planners and managers to manage and reduce the risk of this phenomenon.

Electroencephalography Might Improve Diagnosis of Acute Stroke and Large Vessel Occlusion.

Clinical methods have incomplete diagnostic value for early diagnosis of acute stroke and large vessel occlusion (LVO). Electroencephalography is rapidly sensitive to brain ischemia. This study examined the diagnostic utility of electroencephalography for acute stroke/transient ischemic attack (TIA) and for LVO. Patients (n=100) with suspected acute stroke in an emergency department underwent clinical exam then electroencephalography using a dry-electrode system. Four models classified patients, first as acute stroke/TIA or not, then as acute stroke with LVO or not: (1) clinical data, (2) electroencephalography data, (3) clinical+electroencephalography data using logistic regression, and (4) clinical+electroencephalography data using a deep learning neural network. Each model used a training set of 60 randomly selected patients, then was validated in an independent cohort of 40 new patients. Of 100 patients, 63 had a stroke (43 ischemic/7 hemorrhagic) or TIA (13). For classifying patients as stroke/TIA or not, the clinical data model had area under the curve=62.3, whereas clinical+electroencephalography using deep learning neural network model had area under the curve=87.8. Results were comparable for classifying patients as stroke with LVO or not. Adding electroencephalography data to clinical measures improves diagnosis of acute stroke/TIA and of acute stroke with LVO. Rapid acquisition of dry-lead electroencephalography is feasible in the emergency department and merits prehospital evaluation.

Crime Scene Prediction by Identify Thunderous Objects Using Convolutional Neural Network Deep Learning Model

Crime Scene Prediction by Identify Thunderous Objects Using Convolutional Neural Network Deep Learning Model

Evaluation of immune infiltrating of thyroid cancer based on the intrinsic correlation between pair-wise immune genes

IntroductionUnlike most mutation-driven cancers, thyroid cancer is thought to be highly dependent on changes in human hormone levels. It has become research hotspot using the change of gene expression level as a detection and diagnostic marker. The internal relationship between two genes and disease development is used to avoid the instability caused by single gene fluctuation.AimIt is possible to achieve early diagnosis in thyroid cancer during tumorigenesis and recurrence using IGPS (immune gene pairs). MethodsWe extracted thyroid cancer data from The Cancer Genome Atlas (TCGA), using CIBERSORT algorithm to infiltrate out 22 immune cells types. We screened out IGPS that differ significantly between different groups, then used LinearSVC model to learn and screen features, combined with deep learning neural network model to predict benign and malignant cancer as well as patients at different groups. Key findingsThere are significant differences of immune cell ratio in tumor stages and relapse samples. We screen out 42 and 64 IGPS for in normal-tumor and non-relapsed groups respectively, for example ASCC3-MAP3K7 and ATF2-SOCS5, have significant correlation in IGPS expression. Then we use the IGPS to train the tumor diagnostic classifier, obtain average AUC are both 0.99 after ten times cross-validation. SignificanceThe IGPS gives us new insight to explore immune cell infiltration of thyroid cancer, deep learning model can be further used in early diagnosis of thyroid cancer and estimation of the risk of recurrence.

Derin Öğrenme Mimarilerinde Akustik ve Fonotaktik Öznitelikleri Kullanan Türkçe Ağız Tanıma

Dialects are local forms of speech separated by a certain rate from a standard language. Dialect recognition is one of the popular topics studied in speech recognition. In particular, the spoken dialect is asked to be identified first in order to improve the performance of large scale speech recognition systems. The phonetic differences of speech can be determined by examining the acoustic properties at the physical level. Features such as Log mel-spectrograms are used for this purpose. In addition, the phonotactic term corresponds to the arrangement rules of phonemes in a language/dialect. Phoneme sequences and the frequency of this sequence vary from dialect to dialect. Phoneme sequences are obtained by phoneme recognizers. Another topic that has become popular in recent years is deep learning neural networks. Convolutional Neural Networks (CNN), which is a special kind of deep learning neural networks, are often used in image and speech recognition. Long Short-Term Memory Neural Networks (LSTM) is a deep learning neural network model that produces more successful results than n-gram models in language modeling. In this study, the classification of Turkish dialects with CNN and LSTM type neural networks in terms of acoustic and phonotactic features were discussed. Also, LSTM neural networks are used for language modeling in phonotactic approach. In the experimental study, the proposed approaches were tested and interpreted on the Turkish Dialects Dataset that we collected. As a result of the study, it has been observed that the approaches used reaches 85.1% accuracy rate for Turkish dialect recognition.

Bidirectional Gated Recurrent Deep Learning Neural Networks for Smart Acoustic Emission Sensing of Natural Fiber–Reinforced Polymer Composite Machining Process

Abstract Natural fiber–reinforced polymer (NFRP) composites are increasingly considered in the industry for creating environmentally benign product alternatives. The complex structure of the fibers and their random distribution within the matrix basis impede the machinability of NFRP composites as well as the resulting product quality. This article investigates a smart process monitoring approach that employs acoustic emission (AE)—elastic waves sourced from various plastic deformation and fracture mechanisms—to characterize the variations in the NFRP machining process. The state-of-the-art analytic tools are incapable of handling the transient dynamic patterns with long-term correlations and bursts in AE and how process conditions and the underlying material removal mechanisms affect these patterns. To address this gap, we investigated two types of the bidirectional gated recurrent deep learning neural network (BD-GRNN) models, viz., bidirectional long short-term memory and bidirectional gated recurrent unit to predict the process conditions based on dynamic AE patterns. The models are tested on the AE signals gathered from orthogonal cutting experiments on NFRP samples performed at six different cutting speeds and three fiber orientations. The results from the experimental study suggest that BD-GRNNs can correctly predict (around 87 % accuracy) the cutting conditions based on the extracted temporal-spectral features of AE signals.

Taking Matters into Your Own Hands.

Taking Matters into Your Own Hands.

A method of sample enhancement based on partial discharge PRPS spectrum

In this paper, we summarize the problems existing in the traditional partial discharge PRPS sample enhancement method, and propose a method based on the partial discharge PRPS spectrum sample enhancement, including threshold denoising, noise superposition, data density, data sparsity, phase shift, data enhancement method matrix and so on. It solves the problem that the difficulty of collecting the spectrum of partial discharge cases restricts the application of intelligent algorithms such as deep neural network. The practical application shows that the local partial discharge PRPS spectrum sample enhancement method can effectively enhance the sample size, make the sample distribution uniform and representative, and the deep learning neural network model trained by the sample set has high accuracy and generalization ability for PRPS spectrum diagnosis.

Analysis on block chain financial transaction under artificial neural network of deep learning

In order to conduct an in-depth study on financial transactions of block chain, the classical back propagation (BP) neural network based on the artificial neural network (ANN) model is selected, and its propagation mode, weight change, and learning process are analyzed. For the problem of slow convergence speed and local minimum value of BP neural network, based on the idea of deep learning, the initial value and training step are changed by auto-encoder and restricted Boltzmann machine, and the theory is analyzed. Taking the stock index futures trading in the block chain financial trading as an example, the stock price trading of stock index futures is studied using the two deep learning neural network models to predict the price changes. The results show that the auto-encoder, as an unsupervised learning system, performs better than the restricted Boltzmann machine in setting the initial weights and thresholds, with fewer iterations, faster convergence rate, and smaller convergence error. The results obtained by the auto-encoder can be used as initialization settings and data analysis. The prediction accuracy of the whole model is around 59%. When the transaction cost is not considered, the transaction can be conducted based on the prediction signal of the deep learning model. Therefore, deep learning neural network model can be applied to block chain financial transactions as a reference for financial transactions, which has a good practical significance for the development of this field.

Improving the Spatial Prediction of Soil Organic Carbon Content in Two Contrasting Climatic Regions by Stacking Machine Learning Models and Rescanning Covariate Space

Understanding the spatial distribution of soil organic carbon (SOC) content over different climatic regions will enhance our knowledge of carbon gains and losses due to climatic change. However, little is known about the SOC content in the contrasting arid and sub-humid regions of Iran, whose complex SOC–landscape relationships pose a challenge to spatial analysis. Machine learning (ML) models with a digital soil mapping framework can solve such complex relationships. Current research focusses on ensemble ML models to increase the accuracy of prediction. The usual ensemble method is boosting or weighted averaging. This study proposes a novel ensemble technique: the stacking of multiple ML models through a meta-learning model. In addition, we tested the ensemble through rescanning the covariate space to maximize the prediction accuracy. We first applied six state-of-the-art ML models (i.e., Cubist, random forests (RF), extreme gradient boosting (XGBoost), classical artificial neural network models (ANN), neural network ensemble based on model averaging (AvNNet), and deep learning neural networks (DNN)) to predict and map the spatial distribution of SOC content at six soil depth intervals for both regions. In addition, the stacking of multiple ML models through a meta-learning model with/without rescanning the covariate space were tested and applied to maximize the prediction accuracy. Out of six ML models, the DNN resulted in the best modeling accuracies, followed by RF, XGBoost, AvNNet, ANN, and Cubist. Importantly, the stacking of models indicated a significant improvement in the prediction of SOC content, especially when combined with rescanning the covariate space. For instance, the RMSE values for SOC content prediction of the upper 0–5 cm of the soil profiles of the arid site and the sub-humid site by the proposed stacking approaches were 17% and 9% respectively, less than that obtained by the DNN models—the best individual model. This indicates that rescanning the original covariate space by a meta-learning model can extract more information and improve the SOC content prediction accuracy. Overall, our results suggest that the stacking of diverse sets of models could be used to more accurately estimate the spatial distribution of SOC content in different climatic regions.

Beluga whale acoustic signal classification using deep learning neural network models.

Over a decade after the Cook Inlet beluga (Delphinapterus leucas) was listed as endangered in 2008, the population has shown no sign of recovery. Lack of ecological knowledge limits the understanding of, and ability to manage, potential threats impeding recovery of this declining population. National Oceanic and Atmospheric Administration Fisheries, in partnership with the Alaska Department of Fish and Game, initiated a passive acoustics monitoring program in 2017 to investigate beluga seasonal occurrence by deploying a series of passive acoustic moorings. Data have been processed with semi-automated tonal detectors followed by time intensive manual validation. To reduce this labor intensive and time-consuming process, in addition to increasing the accuracy of classification results, the authors constructed an ensembled deep learning convolutional neural network model to classify beluga detections as true or false. Using a 0.5 threshold, the final model achieves 96.57% precision and 92.26% recall on testing dataset. This methodology proves to be successful at classifying beluga signals, and the framework can be easily generalized to other acoustic classification problems.

Controlling the Properties of Additively Manufactured Cellular Structures Using Machine Learning Approaches

Cellular structures are lightweight‐engineered materials that have gained much attention with the development of additive manufacturing technologies. This article introduces a precise approach to predict the mechanical properties of additively manufactured lattice structures using deep‐learning approaches. Diamond‐shaped nodal lattice structures are designed by varying strut length, strut diameter, and strut orientation angle. The samples are manufactured using laser powder bed fusion (LPBF) of Ti−64 alloy and subjected to compression testing to measure the ultimate strength, elastic modulus, and specific strength. Machine learning approaches such as shallow neural network (SNN), deep neural network (DNN), and deep learning neural network (DLNN) are developed and compared to the statistical design of experiment (DoE) approach. The trained DLNN model show the highest performance when compared with DNN, DoE, and SNN with a mean percentage error of 5.26%, 14.60%, and 9.39% for the ultimate strength, elastic modulus, and specific strength, respectively. The DLNN model is used to create process maps, and is further validated. The results show that although deep learning is preferred for big data, the optimized DLNN model outperform the statistical DoE approach and can be a favorable tool for lattice structure prediction with limited data.

Different sampling strategies for predicting landslide susceptibilities are deemed less consequential with deep learning

Predictive capability of landslide susceptibilities is assumed to be varied with different sampling techniques, such as (a) the landslide scarp centroid, (b) centroid of landslide body, (c) samples of the scrap region representing the scarp polygon, and (d) samples of the landslide body representing the entire landslide body. However, new advancements in statistical and machine learning algorithms continuously being updated the landslide susceptibility paradigm. This paper explores the predictive performance power of different sampling techniques in landslide susceptibility mapping in the wake of increased usage of artificial intelligence. We used logistic regression (LR), neural network (NNET), and deep learning neural network (DNN) model for testing and validation of the models. The tests were applied to the 2018 Hokkaido Earthquake affected areas using a set of 11 predictor variables (seismic, topographic, and hydrological). We found that the prediction rates are inconsequential with the DNN model irrespective of the sampling technique (AUC: 0.904 – 0.919). Whereas, testing with LR (AUC: 0.825 – 0.785) and NNET (AUC: 0.882 – 0.858) produces larger differences in the accuracies between the four datasets. Nonetheless, the highest success rates were obtained for samples within the landslide scarp area. The analogy was then validated with a published landslide inventory from the 2015 Gorkha earthquake. We, therefore, suggest that DNN models as an appropriate technique to increase the predictive performance of landslide susceptibilities if the landslide scarp and body are not characterized properly in an inventory.

Detection and Classification of Cardiac Arrhythmias by a Challenge-Best Deep Learning Neural Network Model.

SummaryElectrocardiograms (ECGs) are widely used to clinically detect cardiac arrhythmias (CAs). They are also being used to develop computer-assisted methods for heart disease diagnosis. We have developed a convolution neural network model to detect and classify CAs, using a large 12-lead ECG dataset (6,877 recordings) provided by the China Physiological Signal Challenge (CPSC) 2018. Our model, which was ranked first in the challenge competition, achieved a median overall F1-score of 0.84 for the nine-type CA classification of CPSC2018's hidden test set of 2,954 ECG recordings. Further analysis showed that concurrent CAs were adequately predictive for 476 patients with multiple types of CA diagnoses in the dataset. Using only single-lead data yielded a performance that was only slightly worse than using the full 12-lead data, with leads aVR and V1 being the most prominent. We extensively consider these results in the context of their agreement with and relevance to clinical observations.

An effective deep learning neural network model for short‐term load forecasting

SummaryEnergy load forecasting plays an important role in the smart grid, which can affect the promoting energy production and consumption decision‐making processes. In this paper, the state‐of‐the‐art deep learning (DL) neural models are used in the short‐term load forecasting, including the multilayer perceptron (MLP), the convolutional neural network (CNN), and the long short‐term memory (LSTM). A novel loss function is proposed for the load forecasting, and two commonly used benchmarks are used to verify the validity of the proposed function. The simulation results show that the mean absolute percentage error (MAPE) of the proposed loss function is 19.63% lower than cross‐entropy and 2.34% lower than mean absolute error (MAE). We compared the mentioned neural networks in different aspects, and the results show that in energy load forecasting, CNN has superior performance than MLP and LSTM in terms of high accuracy and robustness to weather changes.

Comparing the prediction performance of a Deep Learning Neural Network model with conventional machine learning models in landslide susceptibility assessment

The main objective of the current study was to introduce a Deep Learning Neural Network (DLNN) model in landslide susceptibility assessments and compare its predictive performance with state-of-the-art machine learning models. The efficiency of the DLNN model was estimated for the Kon Tum Province, Viet Nam, an area characterized by the presence of landslide phenomena. Nine landslide related variables, elevation, slope angle, aspect, land use, normalized difference vegetation index, soil type, distance to faults, distance to geology boundaries, lithology cover, and 1,657 landslide locations, were used so as to produce the training and validation datasets during the landslide susceptibility assessment. The Frequency Ratio method was used so as to estimate the existing relation between the landslide-related variables and the presence of landslides, assigning to each variable class a weight value. Based on the results concerning the predictive ability of the landslide related variables which was evaluated using the Information ration method, all variables were further processed since they appear as highly predictive. The learning ability of the DLNN model has been evaluated and compared with a Multi Layer Preceptron Neural Network, a Support Vector Machine, a C4.5-Decision Tree model and a Random Forest model using the training dataset, whereas the predictive performance of each model has been evaluated and compared using the validation datasets. In order to evaluate their learning and predictive capacity of each model the classification accuracy, the sensitivity, the specificity and the area under the success and predictive rate curves (AUC) were calculated. Results showed that the proposed DLNN model had a higher performance than the four benchmark models. Although DLNN has been used seldom in landslide susceptibility assessments, the study highlights that the usage of deep learning approach could be considered as a satisfactory alternative approach for landslide susceptibility mapping.

A spatially explicit deep learning neural network model for the prediction of landslide susceptibility

With the increasing threat of recurring landslides, susceptibility maps are expected to play a bigger role in promoting our understanding of future landslides and their magnitude. This study describes the development and validation of a spatially explicit deep learning (DL) neural network model for the prediction of landslide susceptibility. A geospatial database was generated based on 217 landslide events from the Muong Lay district (Vietnam), for which a suite of nine landslide conditioning factors was derived. The Relief-F feature selection method was employed to quantify the utility of the conditioning factors for developing the landslide predictive model. Several performance metrics demonstrated that the DL model performed well both in terms of the goodness-of-fit with the training dataset (AUC = 0.90; accuracy = 82%; RMSE = 0.36) and the ability to predict future landslides (AUC = 0.89; accuracy = 82%; RMSE = 0.38). The efficiency of the model was compared to the quadratic discriminant analysis, Fisher's linear discriminant analysis, and multi-layer perceptron neural network. A comparative analysis using the Wilcoxon signed-rank tests revealed a significant improvement of landslide prediction using the spatially explicit DL model over these other models. The insights provided from this study will be valuable for further development of landslide predictive models and spatially explicit assessment of landslide-prone regions around the world.

Deciphering tip-enhanced Raman imaging of carbon nanotubes with deep learning neural networks.

Recent release of open-source machine learning libraries presents opportunities to unify machine learning with nanoscale research, thus improving effectiveness of research methods and characterization protocols. This paper outlines and demonstrates the effectiveness of such a synergy with artificial neural networks to provide for an accelerated and enhanced characterization of individual carbon nanotubes deposited over a surface. Our algorithms provide a rapid diagnosis and analysis of tip-enhanced Raman spectroscopy mappings and the results show an improved spectral assignment of spectral features and spatial contrast of the collected images. Using several examples, we demonstrate the robustness and versatility of our deep learning neural network models. We highlight the use of machine learning and data science in tandem with tip-enhanced Raman spectroscopy technique enables a fast and accurate understanding of experimental data, thus leading to a powerful and comprehensive imaging analysis applied to spectroscopic measurements.

Semi‐supervised joint learning for longitudinal clinical events classification using neural network models

The success of deep learning neural network models often relies on the accessibility of a large number of labelled training data. In many health care settings, however, only a small number of accurately labelled data are available while unlabelled data are abundant. Further, input variables such as clinical events in the medical settings are usually of longitudinal nature, which poses additional challenges. In this paper, we propose a semi‐supervised joint learning method for classifying longitudinal clinical events. Specifically, our model consists of a sequence generative model and a label prediction model, and the two parts are learned end to end using both labelled and unlabelled data in a joint manner to obtain better prediction performance. Using five mortality‐related classification tasks on the Medical Information Mart for Intensive Care (MIMIC) III database, we demonstrate that the proposed method outperforms the purely supervised method that uses labelled data only and existing two‐step semi‐supervised methods.

PPE Compliance Detection using Artificial Intelligence in Learning Factories

This project demonstrates the application of Artificial Intelligence (AI) and machine vision for the identification of Personal Protective Equipment (PPE), particularly safety glasses in zones of the Learning Factory, where safety risks exist. The objective is to design and implement an automated system for ensuring the safety of personnel when they are in the vicinity of machinery that presents potential risks to the eyes. Microsoft Azure Custom Vision AI and Intelligent AI Services, in conjunction with low-cost vision devices with lightweight onboard AI capability, provide a platform for a deep learning neural network model using publicly available images under the Creative Commons License. A combination of cloud-based and on-premises AI is used in this proof of concept system to provide a real-time vision-based safety system capable of detecting and recording potential safety breaches, promoting compliance, and ultimately preventing accidents before they happen. This system can be used to initiate different control actions in the event of safety violations and can detect multiple forms of protective wear. The flexibility of the system offers multiple benefits to learning factories and manufacturing organizations such as improved user safety, reduced insurance costs, and better detection and recording of safety violations. The hybrid AI architecture approach allows for flexibility in training and deployment based on the capability of local computing resources.