Deep Learning Artificial Neural Network Research Articles

Temperature Regimes and Chemistry for Stabilizing Precipitation Hardening Phases in Al–Sc Alloys: Combined CALPHAD–Deep Machine Learning

Abstract In this work, CALPHAD-based calculations provided with data for various stable and metastable phases in 2XXX, 6XXX, and 7XXX classes of aluminum-based alloys. These data were scaled and then used to develop Deep Learning Artificial Neural Network (DLANN) models for all these phases as a function of composition and temperature. Code was written in the python programming language using TensorFlow/Keras libraries. DLANN models were used for determining the amount of various phases for new compositions and temperatures. The resulting data were further analyzed through the concept of Self-organizing Maps (SOM) and a few candidates were chosen for studying the precipitation kinetics of Al3Sc phase under the framework of CALPHAD approach. This work reports on heat-treatment simulation for one case of 6XXX alloy where the nucleation site was on dislocation, while a detailed study for other alloys is reported in a previously published work. Grain-growth simulations presented in this work are valid for single crystals only.

Deep Learning Algorithms for Complex Traits Genomic Prediction

The underlying perception of genomic selection (GS) is to use genome-wide from DNA sequence (“SNP markers”) along with phenotypes from an observed population to make prediction for the phenotypic outcomes of untested individuals in crop and livestock breeding programs. GS was firstly described by Meuwissen et al.(2001) in dairy cattle to identify genetically superior animals at an early age. The aim was to capture specific genes across the whole genome that are associated with desired traits. The major challenge in using GS programs is to predict the effect of many SNP markers using phenotypic information from a few individuals (aka small n big p problem, or p >> n). Many approaches including naïve and scaled elastic net, ridge regression BLUP Bayesian approaches (BayesA, BayesB, BayesCπ, BayesDπ) LASSO, Support Vector Regression have been conducted to address the small n big p (aka, p >> n) problem. These methods all perform well for (p>>n) by using linear approximation to set a functional relationship between genotypes and phenotypes. However, these methods may not fully capture non-linear effects which are possible to be crucial for complex traits. To deal with this limitation, many methods including neural networks (NN) were recommended to cover non-linearity for GS. Artificial NNs (ANNs) for GS was first presented by Okut et al. (2011) who establish a fully connected regularized multi-layer ANN (MLANN) comprising one hidden layer to predict the body mass index (BMI) in mice using dense molecular markers. Since then, rather complex ANNs approaches have been applied including deep learning (DL) networks. The different DL algorithms have their own advantages to deal with specific problems in complex trait GS. Four different major classes of DL approaches such as fully connected deep learning artificial neural networks (DL-MLANN), recurrent neural networks (RNN), convolutional neural networks (CNN) and long-short term memory (LSTM) and some variation of these network architectures will be summarized here.

Development of deep learning artificial neural networks models to predict temperature and power demand variation for demand response application in cold storage

Food warehouses and cold rooms have a significant potential for Demand Response (DR) application (stopping or reducing the power of fans and compressors of the refrigeration system) due to thermal inertia of food products. However, as air and food temperature might increase beyond acceptable limits during DR periods, DR needs to be carefully applied in order to respect the food temperature regulation and to maintain quality and safety of the products. It is thus important to predict the system behaviour in case of DR application in order to evaluate its potential impacts and to decide if DR can be performed or not. Four deep learning artificial neural networks (ANN) models, traditional Long Short-Term Memory LSTM, stacked LSTM, bidirectional LSTM and convolutional LSTM, were developed to predict future temperature and power demand perturbations due to the application of DR in cold storage. The aims of this work are: first, to assess the performance of those models in predicting the system behaviours, in particular the sudden variations during and after DR applications, and second, to identify the impact of data availability (number of sensors, their positions) and data characteristic (quality, quantity and DR patterns) on the prediction performance. The results have shown the high potential of deep learning ANN models in supporting DR application in cold storage.

Analysis of early warning of corporate financial risk via deep learning artificial neural network

To forecast the financial crisis of manufacturing corporations more accurately, a risk warning model of corporate finance is constructed based on back propagation (BP) neural network to forecast the financial crisis. Firstly, based on the principle of index selection, the forecast indexes are selected and the index system of financial risk early warning is constructed. Then the index system is optimized by factor analysis. Finally, the BP neural network algorithm model is adopted to forecast the financial crisis of 200 manufacturing corporations in 2018 and 2019, and the forecasting results are compared with the traditional method. The results show that the prediction accuracy of the enterprise financial risk early warning model based on the BP neural network for 2018 is above 85%, and the prediction accuracy for 2019 is above 95%, or even 100%. Through comparison with other traditional methods, the prediction accuracy of the BP neural network in 2018 (above 88%) is higher than that of other algorithms (below 87%). In 2019, the prediction accuracy of BP neural network (above 90%) is higher than other algorithms (less than 88%). The accuracy of the proposed financial risk warning model is 95%, and the accuracy is at least 2% higher than traditional method, which prove that the risk early warning model constructed in this study can accurately forecast the financial crisis of the corporation. This study is of important reference value for the establishment of efficient financial crisis forecasting model under deep learning.

Developing a robust and flexible smart tool to predict a full range Critical Heat Flux (CHF) in different LWRs by using deep learning Artificial Neural Networks (ANN) via parallel multi-processing

In this paper, a practical smart tool has been developed to predict CHF in LWRs. The tool is based on the strengthen cascade feed forward ANNs in addition of some of the deep learning features. The most complete experimental CHF data available for LWRs are chosen as the training and test data sets. Input layer can get different thermodynamic qualities, mass flow rates, and pressures to predict the CHF. Then it gains a full range CHF simulator for LWRs fortunately covering both anticipated and unknown transients. Although training such a huge ANN structure and the huge training data set is time consuming, but after the training it acts fast especially via a multi-processing platform. Results are quite satisfactory due to the very good network performance and regressions. Briefly, this is a fast, flexible, and robust tool to predict the CHF deep inside the physics of the most broadening CHF data set ever published for the LWRs.

Multi-stage optimal scheduling of multi-microgrids using deep-learning artificial neural network and cooperative game approach

This article proposes a two-stage system for the daily energy management of micro-grids (MGs) in the presence of wind turbines, photovoltaic (PV) panels, and electrical energy storage systems (ESSs). Each MG uses historical data to predict its consumers' load demand, wind speed, and solar irradiance in the first stage. In the second stage, the cooperative game method is used to determine the MG's daily dispatch and energy transaction. The paper develops a prediction model using artificial neural network (ANN) and rough neuron water cycle (RNWC) algorithms, called deep learning artificial neural network (DLANN), which is a combination of technology from the artificial neural network and WC algorithm in order to predict uncertain parameters. The above model is implemented in the 33bus power distribution system; the simulation results show that the DLANN method provides more accurate predictions than the ANN method. The results also show that a MG can achieve energy cost savings through an alliance of MGs using the cooperative game approach. Furthermore, analysis of the impact of the ESS on the operation of the MG shows that the absence of the ESS will reduce the power output of the wind turbine.

Parametric analysis of lignocellulosic ultrafiltration in lab scale cross flow module using pore blocking and artificial neural network model

In this study, fouling mechanism and modelling analysis of synthetic lignocellulose biomass and agricultural palm oil effluent was studied using polyethersulfone (PES) ultrafiltration (UF) 10 kDa membrane. The impact of process variables (transmembrane pressure (TMP), pH and concentration of feed solution) on lignocellulosic flux was analysed using pore blocking model. The feasible approaches on utilising deep learning artificial neural network (ANN) to predict smaller flux datasets are studied. Among the input variables, pH of lignin feed solution has significant control towards flux and lignin rejection coefficient for both lignin and lignocellulosic solution. Alteration in the structure of lignin at different pH conditions contributed in the improvement of lignin rejection coefficient to 0.98 at the feed pH of 9. A maximum steady state flux of 52.03 L/m2h was observed at the lower lignin concentration (0.25 g/L), TMP of 200 kPa and feed pH of 3. At high TMP and concentration, lignin rejection decreased due to enhancement of feed concentration on membrane surface. The mechanistic model exhibited that cake layer phenomena was dominant in both lignin and lignocellulosic solution. The proposed ANN model showed good correlation (R2-1.00) with experimental non-linear flux dynamic data of both lignin and synthetic lignocellulosic solution. In ANN analysis, activation function, algorithm and neuron effect have significant effect in design of accurate model for prediction of small flux datasets. Aerobically-treated palm oil mill filtration analysis also showed that cake layer phenomenon was dominant. A water recovery of 82 % was achieved even at low TMP under short durations.

Human EEG and Recurrent Neural Networks Exhibit Common Temporal Dynamics During Speech Recognition.

Recent deep-learning artificial neural networks have shown remarkable success in recognizing natural human speech, however the reasons for their success are not entirely understood. Success of these methods might be because state-of-the-art networks use recurrent layers or dilated convolutional layers that enable the network to use a time-dependent feature space. The importance of time-dependent features in human cortical mechanisms of speech perception, measured by electroencephalography (EEG) and magnetoencephalography (MEG), have also been of particular recent interest. It is possible that recurrent neural networks (RNNs) achieve their success by emulating aspects of cortical dynamics, albeit through very different computational mechanisms. In that case, we should observe commonalities in the temporal dynamics of deep-learning models, particularly in recurrent layers, and brain electrical activity (EEG) during speech perception. We explored this prediction by presenting the same sentences to both human listeners and the Deep Speech RNN and considered the temporal dynamics of the EEG and RNN units for identical sentences. We tested whether the recently discovered phenomenon of envelope phase tracking in the human EEG is also evident in RNN hidden layers. We furthermore predicted that the clustering of dissimilarity between model representations of pairs of stimuli would be similar in both RNN and EEG dynamics. We found that the dynamics of both the recurrent layer of the network and human EEG signals exhibit envelope phase tracking with similar time lags. We also computed the representational distance matrices (RDMs) of brain and network responses to speech stimuli. The model RDMs became more similar to the brain RDM when going from early network layers to later ones, and eventually peaked at the recurrent layer. These results suggest that the Deep Speech RNN captures a representation of temporal features of speech in a manner similar to human brain.

A Deep Learning Artificial Neural Network Algorithm for Instance-based Arabic Language Authorship Attribution

One of the common examples of cybercrime are identity theft and violating of intellectual property that commonly occur in social media. Authorship attribution (AA) techniques are used to extract and use several features of the text in order to identify the original author. These features are used to differentiate the writing style of one author from others. Several machine learning methods have been used to identify the AA using different languages. Few studies were conducted for Arabic AA. This paper aims to investigate the performance of deep learning-based artificial neural network (ANN) for identifying the attribution of authors using Arabic text. The applied model helps protect users in social media from identity theft and violating of their intellectual property. The experiments of this study used a dataset that includes 4,686 Arabic texts for 15 different authors. The performance of the deep learning method was compared with several machine learning methods. The experimental results showed the superior performance of deep learning for AA in Arabic language using different evaluation criteria such as F-score, accuracy, precision, and recall measures.

Advancing Stress Detection Methodology with Deep Learning Techniques Targeting UX Evaluation in AAL Scenarios: Applying Embeddings for Categorical Variables

Physiological measurements have been widely used by researchers and practitioners in order to address the stress detection challenge. So far, various datasets for stress detection have been recorded and are available to the research community for testing and benchmarking. The majority of the stress-related available datasets have been recorded while users were exposed to intense stressors, such as songs, movie clips, major hardware/software failures, image datasets, and gaming scenarios. However, it remains an open research question if such datasets can be used for creating models that will effectively detect stress in different contexts. This paper investigates the performance of the publicly available physiological dataset named WESAD (wearable stress and affect detection) in the context of user experience (UX) evaluation. More specifically, electrodermal activity (EDA) and skin temperature (ST) signals from WESAD were used in order to train three traditional machine learning classifiers and a simple feed forward deep learning artificial neural network combining continues variables and entity embeddings. Regarding the binary classification problem (stress vs. no stress), high accuracy (up to 97.4%), for both training approaches (deep-learning, machine learning), was achieved. Regarding the stress detection effectiveness of the created models in another context, such as user experience (UX) evaluation, the results were quite impressive. More specifically, the deep-learning model achieved a rather high agreement when a user-annotated dataset was used for validation.

Deep learning artificial neural networks for non-destructive archaeological site dating

This article introduces artificial neural networks as a computational tool to utilize legacy archaeological data for precisely and accurately estimating dates of residential site occupation. The implementation of this deep learning algorithm can provide high-resolution demographic reconstructions of a study area from non-collection, non-invasive, and non-destructive data collection methods that only record frequencies of artifact types on the contemporary ground surface. The utility of this deep learning algorithm is presented through an example from the central Mesa Verde region in the northern US Southwest. Results show a properly trained artificial neural network predicts annual residential occupation with an average 92.8% accuracy from AD 450–1300. An annual demographic reconstruction of the central Mesa Verde region using occupation predictions from the artificial neural network is also presented.

Modeling the influence of the composition of refractory elements on the heat resistance of nickel alloys by a deep learning artificial neural network

Nickel alloys are widely used in the manufacture of gas turbine parts. The alloys show resistance to mechanical and chemical degradation under prolonged loads and high temperatures. The properties of an alloy are determined by its composition and are subject to careful modeling. A set of basic refractory elements makes a special contribution to the parameters of the alloy. One of the main mechanical properties of the alloys is the high‐temperature tensile strength. Determining the influence of certain elements on certain properties of an alloy is a complex scientific and engineering problem that affects the time and cost of developing new materials. Simulation is a great chance to cut costs. In this article, we predict high temperature strength based on the composition of refractory elements in alloys using a specially designed deep learning artificial neural network. We build the model based on prior knowledge of alloy composition, information on the role of alloying elements, type of crystallization, test temperature and time, and tensile strength. Successful simulation results show the applicability of this method in practice.

Eyelid basal cell carcinoma classification using shallow and deep learning artificial neural networks

Eyelid basal cell carcinoma classification using shallow and deep learning artificial neural networks

Development of mounted system for controlling row crop cultivator in automatic mode

Weed control automation appears to be a very promising technology based on the tremendous advances in computer processing, machine vision and robotics. Machine vision systems are based on size differences between crops and weeds and or on the regular structure of crop rows, allowing the system to recognize crop plants and control surrounding weeds. The paper provides description of the mounted system for orienting the row cultivator in rows and its displacement relative to the tractor using vision systems and automatic control. The developed technical vision system is capable of clearly identifying sugar beet leaves based on the deep learning artificial neural networks technology, and a specially developed algorithm for identifying the center of the row spacing to control the moving part of the cultivator in the required direction to level the inaccuracy of the tractor. The use of controlled mounted device with a tractor and an implement based on vision systems, the accuracy of technological operations for the row crops care, will increase the technical level of the machine and tractor unit, and reduce labor costs. Technical vision systems and automatic control of the cultivator will improve the quality of inter-row cultivation of row crops and reduce the pesticide load on the environment. The results of these studies can be used to create machines for cultivation of agricultural crops with an automated control system.

Technology Readiness and Cryptocurrency Adoption: PLS-SEM and Deep Learning Neural Network Analysis

Today’s world is increasingly dependent on technology directly or indirectly. The rapid technological advancement has impacted people to adopt the technology. As cryptocurrency recently commenced, few studies have attempted to investigate this use of technology. In this study, the technology readiness aspects- Optimism, Innovativeness, Discomfort, and Insecurity are used to understand the people’s adoption of cryptocurrency. A multi-approach of Partial Least Squares- Structural Equation Modeling (PLS-SEM) and Deep learning Artificial Neural Network (ANN) analysis was performed. Deep learning Artificial Neural Network (ANN) analysis was performed to complement PLS-SEM findings and predict higher accuracy. This study shows that technology readiness dimensions - Optimism, Innovativeness, Discomfort, and Insecurity have meaningful relationships with cryptocurrency adoption.

English Grammar Detection Based on LSTM-CRF Machine Learning Model.

Deep learning and neural network have been widely used in the field of speech, vocabulary, text, pictures, and other information processing fields, which has achieved excellent research results. Neural network algorithm and prediction model were used in this paper for the study and exploration of English grammar. Aiming at the application requirements of English grammar accuracy and standardization, we proposed a machine learning model based on LSTM-CRF to detect and analyze English grammar. This paper briefly summarized the development trend of deep learning and neural network algorithm and designed the structure pattern of radial basis function neural network in grammar semantic detection and analysis on the basis of deep learning artificial neural network theory. Based on the morphological features of English grammar, a grammar database was established according to the rules of English word segmentation. In this paper, we proposed an improved conditional random field CRF (Conditional Random Field) network model based on LSTM (Long Short-Term Memory) neural network. It can improve the problem that the traditional machine learning model relies on feature point selection in English grammar detection. The machine learning model based on LSTM-CRF was used to recognize English grammar text entities. The results show that the English grammar detection system based on the LSTM-CRF model can simplify the process structure in the recognition process, reduce the unnecessary operation cycle, and improve the overall detection accuracy.

Radiomics AI prediction for head and neck squamous cell carcinoma (HNSCC) prognosis and recurrence with target volume approach.

Objectives:To evaluate the performance of radiomics features extracted from planning target volume (PTV) and gross tumor volume (GTV) in the prediction of the death prognosis and cancer recurrence rate for head and neck squamous cell carcinoma (HNSCC).Methods:188 HNSCC patients’ planning CT images with radiotherapy structures sets were acquired from Cancer Imaging Archive (TCIA). The 3D slicer (v. 4.10.2) with the PyRadiomics extension (Computational Imaging and Bioinformatics Lab, Harvard medical School) was used to extract radiomics features from the radiotherapy planning images. An in-house developed deep learning artificial neural networks (DL-ANN) model was used to predict death prognosis and cancer recurrence rate based on the features extracted from GTV and PTV of the CT images.Results:The PTV radiomics features with DL-ANN model could achieve 77.7% accuracy with overall AUC equal to 0.934 and 0.932 when predicting HNSCC-related death prognosis and cancer recurrence respectively. Furthermore, the DL-ANN model can achieve an accuracy of 74.3% with AUC equal to 0.947 and 0.956 for the HNSCC-related death prognosis and cancer recurrence respectively using GTV features.Conclusion:Using both GTV and PTV radiomics features in the DL-ANN model, can aid in predicting HNSCC-related death prognosis and cancer recurrence. Clinicians may find it helpful in formulating different treatment regimens and facilitate personized medicine based on the predicted outcome when performing GTV and PTV delineation.Advances in knowledge:Radiomics features of GTV and PTV are reliable prognosis and recurrence predicting tools, which may help clinicians in GTV and PTV delineation to facilitate delivery of personalized treatment.

Discovery of New Ti-Based Alloys Aimed at Avoiding/Minimizing Formation of α” and ω-Phase Using CALPHAD and Artificial Intelligence

In this work, we studied a Ti-Nb-Zr-Sn system for exploring novel composition and temperatures that will be helpful in maximizing the stability of β phase while minimizing the formation of α” and ω-phase. The Ti-Nb-Zr-Sn system is free of toxic elements. This system was studied under the framework of CALculation of PHAse Diagram (CALPHAD) approach for determining the stability of various phases. These data were analyzed through artificial intelligence (AI) algorithms. Deep learning artificial neural network (DLANN) models were developed for various phases as a function of alloy composition and temperature. Software was written in Python programming language and DLANN models were developed utilizing TensorFlow/Keras libraries. DLANN models were used to predict various phases for new compositions and temperatures and provided a more complete dataset. This dataset was further analyzed through the concept of self-organizing maps (SOM) for determining correlations between phase stability of various phases, chemical composition, and temperature. Through this study, we determined candidate alloy compositions and temperatures that will be helpful in avoiding/minimizing formation of α” and ω-phase in a Ti-Zr-Nb-Sn system. This approach can be utilized in other systems such as ω-free shape memory alloys. DLANN models can even be used on a common Android mobile phone.

Comparison of River Basin Water Level Forecasting Methods: Sequential Neural Networks and Multiple-Input Functional Neural Networks

To precisely forecast downstream water levels in catchment areas during typhoons, the deep learning artificial neural networks were employed to establish two water level forecasting models using sequential neural networks (SNNs) and multiple-input functional neural networks (MIFNNs). SNNs, which have a typical neural network structure, are network models constructed using sequential methods. To develop a network model capable of flexibly consolidating data, MIFNNs are employed for processing data from multiple sources or with multiple dimensions. Specifically, when images (e.g., radar reflectivity images) are used as input attributes, feature extraction is required to provide effective feature maps for model training. Therefore, convolutional layers and pooling layers were adopted to extract features. Long short-term memory (LSTM) layers adopted during model training enabled memory cell units to automatically determine the memory length, providing more useful information. The Hsintien River basin in northern Taiwan was selected as the research area and collected relevant data from 2011 to 2019. The input attributes comprised one-dimensional data (e.g., water levels at river stations, rain rates at rain gauges, and reservoir release) and two-dimensional data (i.e., radar reflectivity mosaics). Typhoons Saola, Soudelor, Dujuan, and Megi were selected, and the water levels 1 to 6 h after the typhoons struck were forecasted. The results indicated that compared with linear regressions (REG), SNN using dense layers (SNN-Dense), and SNN using LSTM layers (SNN-LSTM) models, superior forecasting results were achieved for the MIFNN model. Thus, the MIFNN model, as the optimal model for water level forecasting, was identified.

Use of Deep Learning Networks and Statistical Modeling to Predict Changes in Mechanical Parameters of Contaminated Bone Cements.

The purpose of the study was to test the usefulness of deep learning artificial neural networks and statistical modeling in predicting the strength of bone cements with defects. The defects are related to the introduction of admixtures, such as blood or saline, as contaminants into the cement at the preparation stage. Due to the wide range of applications of deep learning, among others in speech recognition, bioinformation processing, and medication design, the extent was checked to which it is possible to obtain information related to the prediction of the compressive strength of bone cements. Development and improvement of deep learning network (DLN) algorithms and statistical modeling in the analysis of changes in the mechanical parameters of the tested materials will enable determining an acceptable margin of error during surgery or cement preparation in relation to the expected strength of the material used to fill bone cavities. The use of the abovementioned computer methods may, therefore, play a significant role in the initial qualitative assessment of the effects of procedures and, thus, mitigation of errors resulting in failure to maintain the required mechanical parameters and patient dissatisfaction.