Abstract Artificial Neural Network Model Research Articles

123 Benchmarking of Artificial Neural Network Models for Genomic Prediction of Quantitative Traits in Pigs

Abstract Artificial neural networks (ANN) are a type of machine learning model that has been applied to various genomic problems, with the ability to learn non-linear relationships and model high-dimensional data. ANNs also have the potential in genomic prediction by capturing the intricate relationship between genetic variants and phenotypes. However, there is currently a limited effort to investigate the performance and feasibility of ANNs for pig genomic predictions. In this study, we evaluated the predictive performance of TensorFlow’s ANN models with one-layer, two-layer, and three-layer structures (with zero, one, and two hidden layers, respectively), in comparison with five linear methods, including GBLUP, LDAK, BayesR, SLEMM and scikit-learn’s ridge regression using data of six quantitative traits including off-test body weight (WT), off-test back fat thickness (BF), off-test loin muscle depth (MS), number of piglets born alive (NBA), number of piglets born dead (NBD), and number of piglets weaned (NW). Furthermore, we assessed the computational efficiency of ANNs on both CPU and GPU. The benchmarking was based on cross-validations of 26,190 genotyped pigs. We employed hyperband tuning to optimize the hyper-parameters and select the best model among one-layer, two-layer, and three-layer structures. Results showed that the one-layer structure, which is equivalent to ridge regression, yielded the best performance comparable to that of GBLUP. Using the optimal hyper-parameters for two-layer and three-layer structures, ANNs underperformed GBLUP in terms of accuracy. Of the five linear methods, BayesR and SLEMM performed similarly and the best, followed by LDAK, scikit-learn’s ridge regression, and GBLUP. Moreover, SLEMM was the fastest, which completed training with 21k individuals and 30k SNPs in 2.6 minutes. Compared with CPUs, GPUs exhibited a comparable computational speed for one-layer ANN but offered significant gains in computational efficiency for multi-layer ANNs. Based on our analysis of optimal hyper-parameters for two-layer ANN with BF, we found that using a GPU can lead to a five-fold increase in processing speed compared with using a conventional CPU, but it is still slower than GBLUP. In addition, hyper-parameter tuning (particularly for L2 regularization and the number of dense units in hidden layers) is critical for improving the genomic prediction accuracy in pigs. In conclusion, we found ANN with up to three layers could not improve genomic predictions compared with routine linear methods for pig quantitative traits.

Optimization of ANN Architecture and Training Parameters Using Taguchi Method

Artificial neural networks (ANNs) have gained significant popularity for modeling and optimization. Choosing appropriate training and design parameters for an ANN, on the other hand, is still a difficult task. These parameters are typically chosen by a trial-and-error strategy in which a large number of ANN models are developed and compared. This evidence stated how the Taguchi approach can be used to enhance an ANN model. To show the technique, a case study of a minimization of Green Sand-casting defects in a green sand-casting process is provided. Total 175 samples were taken for the study. L18 orthogonal array was used to construct the ANN training and architectural parameters. A Taguchi-optimized ANN model was created and showed excellent prediction accuracy. Analyses and experiments have shown that the best ANN training and architectural parameters may be found in a systematic manner, minimizing the time-consuming trial-and-error method.

River flow modelling for flood prediction using artificial neural network in ungauged Perkerra catchment, Baringo County, Kenya

Abstract Artificial neural network (ANN) modelling has been applied successfully in hydrology to predict future flows based on previous rainfall-runoff values. For a long time, flooding has been experienced in the surrounding areas of the Rift Valley lakes including Lake Baringo, fed by the River Perkerra, due to the rising water levels because of the above-normal rainfall season, resulting in massive socioeconomic losses. The study aims at predicting the occurrence of floods in River Perkerra using an ANN model with the input data being 417 consistent pairs of daily rainfall and discharge, and simulated runoff as the output. The model was trained, tested and validated producing a best fit regression with R2 of 0.951 for training, 0.938 for validation, 0.953 for testing giving an average of 0.949 indicating a close relationship between the input and output values. The overall best validation performance, RMSE, was 0.9204 m3/s indicating high efficiency of the FFNN model developed to predict floods. Flows greater than 14 m3/s, Q1, were the extreme flood events closely associated with socioeconomic losses. This prediction of Q1 value is crucial in the formulation and implementation of measures and policies by the County Government that will mitigate adverse impacts of predicted floods in the catchment.

Chaos-generalized regression neural network prediction model of mine water inflow

Artificial neural network (ANN) provides a new way for mine water inflow prediction. However, the effectiveness of prediction using ANN model would not be guaranteed if the influencing factors of water inflow are difficult to quantify or there are only a few observation data. Chaos theory can recover the rich dynamic information hidden in time series. By reconstructing water inflow time series in phase space, the multi-dimensional matrix could be obtained, with each column representing an influencing factor of water inflow and its value representing the change of the influencing factor with time. Therefore, a new prediction model of mine water inflow can be established by combining ANN with chaos theory when lacking data on the influencing factors of water inflow. In the present study, the No. 12 coal mine of Pingdingshan China was selected as the study site. The Chaos-GRNN model and Chaos- BPNN model of mine, water inflow were established by using the water inflow data from February 1976 to December 2013. The model was verified by using the water inflow values in the 24 months from 2014 to 2015. The number embedded dimension (M) of influencing factors of water inflow determined by phase space reconstruction was 7, meaning that there were 7 influencing factors of water inflow and 7 neurons in GRNN input layer, and the time delay was 13 months. The value of GRNN input layer neurons was determined accordingly. The maximum Lyapunov index was 0.0530, and the prediction time of GRNN was 19 months. The two models were evaluated by using four evaluation indices (R, RMSE, MAPE, NSE) and violin plot. It was found that both models can realize the long-term prediction of water inflow, and the prediction effectiveness of Chaos-GRNN model is better than that of Chaos-BPNN model.

Combining genetic risk score with artificial neural network to predict the efficacy of folic acid therapy to hyperhomocysteinemia

Artificial neural network (ANN) is the main tool to dig data and was inspired by the human brain and nervous system. Several studies clarified its application in medicine. However, none has applied ANN to predict the efficacy of folic acid treatment to Hyperhomocysteinemia (HHcy). The efficacy has been proved to associate with both genetic and environmental factors while previous studies just focused on the latter one. The explained variance genetic risk score (EV-GRS) had better power and could represent the effect of genetic architectures. Our aim was to add EV-GRS into environmental factors to establish ANN to predict the efficacy of folic acid therapy to HHcy. We performed the prospective cohort research enrolling 638 HHcy patients. The multilayer perception algorithm was applied to construct ANN. To evaluate the effect of ANN, we also established logistic regression (LR) model to compare with ANN. According to our results, EV-GRS was statistically associated with the efficacy no matter analyzed as a continuous variable (OR = 3.301, 95%CI 1.954–5.576, P < 0.001) or category variable (OR = 3.870, 95%CI 2.092–7.159, P < 0.001). In our ANN model, the accuracy was 84.78%, the Youden’s index was 0.7073 and the AUC was 0.938. These indexes above indicated higher power. When compared with LR, the AUC, accuracy, and Youden’s index of the ANN model (84.78%, 0.938, 0.7073) were all slightly higher than the LR model (83.33% 0.910, 0.6687). Therefore, clinical application of the ANN model may be able to better predict the folic acid efficacy to HHcy than the traditional LR model. When testing two models in the validation set, we got the same conclusion. This study appears to be the first one to establish the ANN model which added EV-GRS into environmental factors to predict the efficacy of folic acid to HHcy. This model would be able to offer clinicians a new method to make decisions and individual therapeutic plans.

Learning Nonlinear Constitutive Laws Using Neural Network Models Based on Indirectly Measurable Data

Abstract Artificial neural network (ANN) models are used to learn the nonlinear constitutive laws based on indirectly measurable data. The real input and output of the ANN model are derived from indirect data using a mechanical system, which is composed of several subsystems including the ANN model. As the ANN model is coupled with other subsystems, the input of the ANN model needs to be determined during the training. This approach integrates measurable data, mechanics, and ANN models so that the ANN models can be trained without direct data which is usually not available from experiments. Two examples are provided as an illustration of the proposed approach. The first example uses two-dimensional (2D) finite element (FE) analysis to train an ANN model to learn the nonlinear in-plane shear constitutive law. The second example applies a continuum damage model to train an ANN model to learn the damage accumulation law. The results show that the trained ANN models achieve great accuracy based on the proposed approach.

Development and application of a GIS-based artificial neural network system for water quality prediction: a case study at the Lake Champlain area

Artificial Neural Network (ANN) models have been extensively applied in the prediction of water resource variables, and Geographical Information System (GIS) includes powerful functions to visualize spatial data. In order to provide an efficient tool for environmental assessment and management that combines the advantages of these two modules, a GIS-based ANN water quality prediction system was developed in the present study. The ANN module and ArcGIS Engine module, along with a dynamic database, were imbedded in the system, which integrates water quality prediction via the ANN model and spatial presentation of the model results. The structure of the ANN model could be modified through the graphical user interface to optimize the model performance. The developed system was applied to a real case study for the prediction of the total phosphorus concentration in the Lake Champlain area. The prediction results were verified with the monitoring data, and the performance of the developed model was further evaluated through graphical techniques and quantitative statistical methods. Overall, the developed system provided satisfactory prediction results, and spatial distribution maps of the predicted results were obtained, which coincided with the monitored values. The developed GIS-based ANN water quality prediction system could serve as an efficient tool for engineers and decision makers.

Long-term prediction of biosignals from greenhouse-grown tomato

Artificial neural networks (ANN) have been successfully used for modelling climate and plant signals in greenhouses. This work shows the combination of two ANN models, built and trained separately, and then coupled together. Both models were trained using a data set consisting of 3 complete cultivation periods (2011 until 2013) of 2 venlo-type greenhouses located at the Humboldt-Universitat zu Berlin. The greenhouses were used for soilless cultivation of tomato. The first model considered (climate prediction: CP) estimated the expected values of temperature and relative humidity in the near future (one-step prediction, OSP). The output was then recursively fed to the model in order to make further predictions. Each time step represented 5 min, and the maximum number of prediction steps was set to 6. On the other hand, the output of the second model (plant signals: PM) was not a prediction in time, but an estimation of plant signals (leaf temperature, transpiration rate, photosynthesis rate) as expected to be measured by a phytomonitoring system (BERMONIS). The combined models took the 6 predicted steps of climate (long-term prediction: LTP) produced by CM and used them to feed PM, thus generating a Biosignals LTP. As expected, the fit of the LTP decreased with the number of forward steps. For example, the R2 for leaf temperature range from 0.946 for the first simulation until 0.704 for the 6th simulated step. For transpiration and photosynthesis rate, these ranges were (0.874, 0.809) and (0.772, 0.692), respectively. These results show that the estimated plant responses could support a predictive control system, in order to avoid plant damage due to extreme climate conditions.

Dynamic Modeling of Streptomyces hygroscopicus Fermentation Broth Microfiltration by Artificial Neural Networks

Artificial neural networks (ANNs) have been used to dynamically model cross-flow microfiltration of Streptomyces hygroscopicus fermentation broths. The aim is to predict permeate flux as a function of temperature, feed flow, transmembrane pressure and processing time. Dynamic modeling of microfiltration performance of complex systems (such as broths) is very important for design of new processes and better understanding of the present. The results of ANN model analysis suggest that the coefficients of the determination have high values. The application of the Bayesian regularization gave better results to the performance of the neural network compared to the Levenberg-Marquet algorithm. The optimal number of neurons in the hidden layer is eight. Analysis of the absolute relative error showed excellent permeate flux estimates for 100 % of the data points, with an error less than 5 % for the data obtained during microfiltration in the presence of a turbulence promoter. Whilst in the case of microfiltration without turbulence promoter 90 % of predictions have an error less than 10 %. The results of applying the concept of neural networks in the dynamic modeling of microfiltration of Streptomyces hygroscopicus fermentative broths with and without a turbulence promoter clearly show the validity of proposed method for simulation and prediction of microfiltration experimental results.

Packaging Domain-Based Named Entity Recognition with Multi-Layer Neural Networks

Artificial neural networks (ANNs) are the greatest success story that inspired by biological neural networks and neuroscience; ANNs model realistic problems by a network of neurons which are designed by simulating biological neurons. This paper attempts to design a multi-layer neural network to recognize the named entities in packaging domain. For this purpose, a neural network language model was designed to automatically learn the distributed word features and partial speech features. Based on these distributed features, a multi-layer deep neural network model was constructed for the NER of packaging products. The experiments prove that our model can automatically extract more and better advanced features than traditional methods, thus minimizing the workloads of manual feature selection. The results show that the model outperformed the traditional sequence labelled CRF model by 10% in precision and 6% in recall, and that the four-layer neural network with two hidden layers boasted the best NER of packaging products.

Performance Analysis of Artificial Neural Networks Training Algorithms and Transfer Functions for Medium-Term Water Consumption Forecasting

Artificial Neural Network (ANN) is a widely used machine learning pattern recognition technique in predicting water resources based on historical data. ANN has the ability to forecast close to accurate prediction given the appropriate training algorithm and transfer function along with the model’s learning rate and momentum. In this study, using the Neuroph Studio platform, six models having different combination of training algorithms, namely, Backpropagation, Backpropagation with Momentum and Resilient Propagation and transfer functions, namely, Sigmoid and Gaussian were compared. After determining the ANN model’s input, hidden and output neurons from its respective layers, this study compared data normalization techniques and showed that Min-Max normalization yielded better results in terms of Mean Square Error (MSE) compared to Max normalization. Out of the six models tested, Model 1 which was composed of Backpropagation training algorithm and Sigmoid transfer function yielded the lowest MSE. Moreover, learning rate and momentum value for the models of 0.2 and 0.9 respectively resulted to very minimal error in terms of MSE. The results obtained in this research clearly suggest that ANN can be a viable forecasting technique for medium-term water consumption forecasting.

A FPGA-Based, Granularity-Variable Neuromorphic Processor and Its Application in a MIMO Real-Time Control System.

Artificial Neural Networks (ANNs), including Deep Neural Networks (DNNs), have become the state-of-the-art methods in machine learning and achieved amazing success in speech recognition, visual object recognition, and many other domains. There are several hardware platforms for developing accelerated implementation of ANN models. Since Field Programmable Gate Array (FPGA) architectures are flexible and can provide high performance per watt of power consumption, they have drawn a number of applications from scientists. In this paper, we propose a FPGA-based, granularity-variable neuromorphic processor (FBGVNP). The traits of FBGVNP can be summarized as granularity variability, scalability, integrated computing, and addressing ability: first, the number of neurons is variable rather than constant in one core; second, the multi-core network scale can be extended in various forms; third, the neuron addressing and computing processes are executed simultaneously. These make the processor more flexible and better suited for different applications. Moreover, a neural network-based controller is mapped to FBGVNP and applied in a multi-input, multi-output, (MIMO) real-time, temperature-sensing and control system. Experiments validate the effectiveness of the neuromorphic processor. The FBGVNP provides a new scheme for building ANNs, which is flexible, highly energy-efficient, and can be applied in many areas.

Prediction of Thermal Properties of Sweet Sorghum Bagasse as a Function of Moisture Content Using Artificial Neural Networks and Regression Models

Abstract Artificial neural networks (ANN) and traditional regression models were developed for prediction of thermal properties of sweet sorghum bagasse as a function of moisture content and room temperature. Predictions were made for three thermal properties: 1) thermal conductivity, 2) volumetric specific heat, and 3) thermal diffusivity. Each thermal property had five levels of moisture content (8.52%, 12.93%, 18.94%, 24.63%, and 28.62%, w. b.) and room temperature as inputs. Data were sub-partitioned for training, testing, and validation of models. Backpropagation (BP) and Kalman Filter (KF) learning algorithms were employed to develop nonparametric models between input and output data sets. Statistical indices including correlation coefficient (R) between actual and predicted outputs were produced for selecting the suitable models. Prediction plots for thermal properties indicated that the ANN models had better accuracy from unseen patterns as compared to regression models. In general, ANN models were able to strongly generalize and interpolate unseen patterns within the domain of training.

Correlation and Prediction of the Solubility of Air Gases in Saline Solutions for Mining Processes, Using Artificial Neural Networks

Artificial neural networks (ANN) are used to correlate and predict the solubility of air gases (oxygen, nitrogen, and argon) in saline solutions. Several network architectures were considered in the study and correlated and predicted values of solubility were compared with values obtained from the literature. The model is general in the sense that just one model can correlate and predict the solubility of any of the three gases. As a measure of the accuracy of the method, the average relative deviation, the average relative absolute deviation, and the maximum relative absolute deviation are evaluated. The ANN model obtained give lower deviations than analytical models presented in the literature. The chosen ANN model give absolute deviations lower than 1% with some few exceptions and maximum relative absolute deviations are always below 6%.

Clustering-based classification of road traffic accidents using hierarchical clustering and artificial neural networks

Artificial neural networks (ANNs) have been widely used in predicting the severity of road traffic crashes. All available information about previously occurred accidents is typically used for building a single prediction model (i.e., classifier). Too little attention has been paid to the differences between these accidents, leading, in most cases, to build less accurate predictors. Hierarchical clustering is a well-known clustering method that seeks to group data by creating a hierarchy of clusters. Using hierarchical clustering and ANNs, a clustering-based classification approach for predicting the injury severity of road traffic accidents was proposed. About 6000 road accidents occurred over a six-year period from 2008 to 2013 in Abu Dhabi were used throughout this study. In order to reduce the amount of variation in data, hierarchical clustering was applied on the data set to organize it into six different forms, each with different number of clusters (i.e., clusters from 1 to 6). Two ANN models were subsequently built for each cluster of accidents in each generated form. The first model was built and validated using all accidents (training set), whereas only 66% of the accidents were used to build the second model, and the remaining 34% were used to test it (percentage split). Finally, the weighted average accuracy was computed for each type of models in each from of data. The results show that when testing the models using the training set, clustering prior to classification achieves (11%–16%) more accuracy than without using clustering, while the percentage split achieves (2%–5%) more accuracy. The results also suggest that partitioning the accidents into six clusters achieves the best accuracy if both types of models are taken into account.

A Hybrid NIDS Model Using Artificial Neural Network and D-S Evidence

Artificial Neural Networks (ANNs), especially back-propagation (BP) neural network, can improve the performance of intrusion detection systems. However, for the current network intrusion detection methods, the detection precision, especially for low-frequent attacks, detection stability and training time are still needed to be enhanced. In this paper, a new model which based on optimized BP neural network and Dempster-Shafer theory to solve the above problems and help NIDS to achieve higher detection rate, less false positive rate and stronger stability. The general process of the authors' model is as follows: firstly dividing the main extracted feature into several different feature subsets. Then, based on different feature subsets, different ANN models are trained to build the detection engine. Finally, the D-S evidence theory is employed to integration these results, and obtain the final result. The effectiveness of this method is verified by experimental simulation utilizing KDD Cup1999 dataset.

Prediction of flow regime using ANN for air-water flow through small diameter tubes in horizontal plane

Artificial neural network (ANN) modelling for the classifications of flow regimes in air-water flow through 1 mm to 5 mm tubes is presented. Two hundred eighteen data points based on the experimental investigation in 3 and 4 mm tubes and 2,114 data points from various experimental results collected from the literature for air-water two-phase flow through tubes having small diameter have been used. Five different artificial neural network training algorithms have been used to predict the flow regime. The ANN model based on radial basis function gives slightly better predictability over the other networks.

Prediction of the Characteristics of a New Sonochemical Reactor Using an Expert Model

Artificial neural network (ANN) models were developed to estimate the characteristics of a novel high-frequency sonoreactor. Mean residence time and ultrasound dissipated power were considered to evaluate the performance of this sonoreactor. These parameters were calculated from predicted tracer concentration and temperature variation by ANN models. The best network configurations were determined as two layers network including 10 and 12 neurons in hidden layer for the concentration and temperature prediction, respectively. For both networks Levenberg–Marquardt was the best training algorithm with average absolute relative error (AARE) below than 2%. The estimated characteristics were in good agreements with experimental data. Also, sensitivity analyses were done to find the relative importance of each input variable in networks training.

Inflation Forecasting in Ghana-Artificial Neural Network Model Approach

Artificial Neural Network (ANN) is a modelling technique which is based on the way the human brain process information. ANNs have proved to be good forecasting models in several fields including economics and finance. The ANN methodology is used by some central banks to predict various macroeconomic indicators such as the inflation, money supply, GDP growth etc. The use of the ANN for prediction is common in the forecasting literature but rare in Ghana. This paper forecasts inflation with the ANN method using the Ghanaian data. The monthly y-o-y data between 1991:01 and 2010:12 are used to estimate and forecast for the period 2011:01 to 2011:12. The result of the ANNs are also compared with traditional time series models such as the AR (12) and VAR (14) which use the same set of variables. The basis of comparison is the out-of-sample forecast error (RMSFE). The results show that the RMSFE of the ANNs are lower than their econometric counterparts. That is, by this comparative criterion forecast based on ANN models are more accurate.

Prediction of the Vertical Swelling Percentage of Expansive Clays Using a Two-Stage Artificial Neural Networks Methodology

Abstract Artificial Neural Networks (ANN), which are used in many different areas, have been applied to predict the vertical swelling percentage of expansive clays. In contrast to previous models that estimated ANN Models in a single phase, this paper proposes an alternative analysis in based on the following two-stage operation: (a) conducting an ANN analysis on the swelling-pressure test results (i.e., the ASTM 4546 Method C test results) to obtain the swelling-pressure model for any given clay characteristics, and (b) performing an additional ANN analysis on the swelling-percentage test results (i.e., the ASTM 4546 Method B test results), including the former ones, with the given independent variables of the clay characteristics. This second stage includes a defined expression containing the given surcharge pressure and the predicted value of the swelling pressure as obtained from the model of the previous stage. Two final ANN Models, each with a different arrangement of the given independent variables, were derived from this two-stage procedure. Their statistical fit was clearly found to be superior in comparison to previous models estimated with the same data set. Furthermore, one of these two models exhibited the expected geophysical behavior. As this new ANN Model yields higher predicted swelling-percentage values, it can definitely be regarded as a preferable one in the sense of enlarging the safety margin in heave calculations.