Three-layer Feedforward Neural Network Research Articles

A New Type of Hybrid Learning Algorithm for Three-Layered Feed-Forward Neural Networks

The problem of local minimum cannot be avoided when it comes to nonlinear optimization in the learning algorithm of neural network parameters, and the larger the optimization space is, the more obvious the problem becomes. This paper proposes a new type of hybrid learning algorithm for three-layered feed-forward neural networks. This algorithm is based on three-layered feed-forward neural networks with output layer function, namely linear function, combining a quasi Newton algorithm with adaptive decoupled step and momentum (QNADSM) and iterative least square method to export. Simulation proves that this hybrid algorithm has strong self-adaptive capability, small calculation amount and fast convergence speed. It is an effective engineer practical algorithm.

A simple modelling approach for prediction of standard state real gas entropy of pure materials

The performance of an energy conversion system depends on exergy analysis and entropy generation minimisation. A new simple four-parameter equation is presented in this paper to predict the standard state absolute entropy of real gases (SSTD). The model development and validation were accomplished using the Linear Genetic Programming (LGP) method and a comprehensive dataset of 1727 widely used materials. The proposed model was compared with the results obtained using a three-layer feed forward neural network model (FFNN model). The root-mean-square error (RMSE) and the coefficient of determination (r2) of all data obtained for the LGP model were 52.24 J/(mol K) and 0.885, respectively. Several statistical assessments were used to evaluate the predictive power of the model. In addition, this study provides an appropriate understanding of the most important molecular variables for exergy analysis. Compared with the LGP based model, the application of FFNN improved the r2 to 0.914. The developed model is useful in the design of materials to achieve a desired entropy value.

Estimating hydrogen sulfide solubility in ionic liquids using a machine learning approach

For the design and development of new processes of gas sweetening using ionic liquids (ILs), as promising candidates for amine solutions, an amazing model to predict the solubility of acid gases is of great importance. In this direction, in the current study, the capability of artificial neural networks (ANNs) trained with back propagation (BP) and particle swarm optimization (PSO), to correlate the solubility of H2S in 11different ILs have been investigated. Different structures of three-layer feed forward neural network using acentric factor (ω), critical temperature (Tc), critical pressure (Pc) of ILs accompanied by pressure (P) and temperature (T), as input parameters, were examined and an optimized architecture has been proposed as 5–9–1.Implementation of these models for 465 experimental data points collected from the literature shows coefficient of determination (R2) of 0.99218 and mean squared error (MSE) of 0.00025 from experimental values for PSO-ANN predicted solubilities while the values of R2=0.95151 and MSE=0.00335 were obtained for BP-ANN model. Therefore, through PSO training algorithm we are able to attain significantly better results than with BP training procedure based on the statistical criteria.

Neural Network Model for the Prediction of Safe Work Behavior in Construction Projects

AbstractA model has been developed employing an artificial neural network (ANN) to predict the safe work behavior of employees using 10 safety climate constructs determined through literature review. The model utilizes safety climate constructs (determinants) as inputs and safe work behavior as an output. Two hundred twenty-two responses from several construction projects across India were collected through a questionnaire survey. A three-layer feed-forward back-propagation neural network (10-11-1) was appropriate in building this model which has been trained, validated, and tested with sufficient data sets. The model predicts the safe work behavior of employees reasonably well. In addition, a sensitivity analysis was carried out to study the impact of each construct on the safe work behavior of employees. As a result, safety climate constructs like supervisory environment, work pressure, employees’ involvement, personal appreciation of risk, and supportive environment were significantly associated with t...

A Study and Design of Compressed Encoder Based on Neural Network for Remote Sensing Image

The characteristics of remote sensing image are big data volume and high resolution. It is difficult to meet actual demands by using traditional techniques for transmission and storage. This paper studies and designs remote sensing image compression encoder with FPGA technology and three-layer feedforward BP neural network. The neural network can parallelly process large amounts of data, and has a structural characteristic of self-learning and self-organizing. The encoder has a simple structure, safety, fast speed, good reconfiguration. It overcomes shortcomings of conventional compression techniques which compress high-resolution images ineffectively. The study has some theoretical values in the field of image compression.

Estimation of Binary Infinite Dilute Diffusion Coefficient Using Artificial Neural Network

In this study, the use of the three-layer feed forward neural network has been investigated for estimating of infinite dilute diffusion coefficient ( D12 ) of supercritical fluid (SCF), liquid and gas binary systems. Infinite dilute diffusion coefficient was spotted as a function of critical temperature, critical pressure, critical volume, normal boiling point, molecular volume in normal boiling point, molecule diameter, Lennard-Jones’s (LJ) energy parameter, temperature and pressure. For each set of SCF, liquid and gas systems a three-layer network has been applied with training algorithm of Levenberg-Marquard (LM). The obtained results of models have shown good accuracy of artificial neural network (ANN) for estimating infinite dilute diffusion coefficient of SCF, liquid and gas binary systems with mean relative error (MRE) of 2.88 % for 231 systems containing 4078 data points (mean relative error for ANN model in SCF, liquid and gas binary systems are 3.00, 2.99 and 1.21 %, respectively)

Classical and Neural Network–Based Approach of Model Predictive Control for Binary Continuous Distillation Column

Abstract The objective of present research work is to develop a neural network–based model predictive control scheme (NN-MPC) for distillation column. To fulfill this objective, an existing laboratory setup of continuous binary-type distillation column (BDC) is used. An equation-based model that uses the fundamental physical and chemical laws along with valid normal assumptions is validated for this experimental setup. Model predictive control (MPC) is one of the main process control techniques explored in the recent past for various chemical engineering applications; therefore, the conventional MPC scheme and the proposed NN-MPC scheme are applied on the equation-based model to control the methanol composition. In NN-MPC scheme, a three-layer feedforward neural network model has been developed and is used to predict the methanol composition over a prediction horizon using the MPC algorithm for searching the optimal control moves. The training data is acquired by the simulation of the equation-based model under the variation of manipulated variables in the defined range. Two cases have been considered, one is for set point tracking and another is for feed flow disturbance rejection. The performance of the control schemes is compared on the basis of performance parameters namely overshoot and settling time. NN-MPC and MPC schemes are also compared with conventional PID controller. The results show the improvement in settling time with NN-MPC scheme as compared to MPC and conventional PID controller for both the cases.

Competitive adsorption of methylene blue and brilliant green onto graphite oxide nano particle following: Derivative spectrophotometric and principal component-artificial neural network model methods for their simultaneous determination

In this work, the competitive adsorption of methylene blue (MB) and brilliant green (BG) onto graphite oxide (GO) nanoparticles followed by their accurate and reproducible determination by second order derivative spectrophotometry (SODS) and principal component–artificial neural network model (PCA–ANN) model has been studied. The evaluation of kinetic and isotherm studies was investigated at optimum experimental conditions set as pH=7.0, 8mg of GO and 14min contact time in binary systems. The equilibrium amounts of MB and BG dyes in binary mixture adsorbed onto GO-NP has opposite correlation with their initial concentration. Principal component analysis (PCA) used to minimize the dimensionality of large data sets via reducing the number of spectral data by a three-layered feed-forward artificial neural network (ANN) trained by Levenberg–Marquardt back-propagation algorithm. The ANN model was able to predict the concentrations of both dyes in mixtures with a tangent sigmoid transfer function (tansig) at hidden layer with 20 neurons and a linear transfer function (purelin) at output layer. Several isotherm models were applied to experimental data and the isotherm constants were calculated for BG and MB dyes. Among the applied models, the extended Freundlich isotherm model adequately predicts the multi-component adsorption equilibrium data at moderate ranges of concentration.

A novel artificial neural network method for biomedical prediction based on matrix pseudo-inversion

Biomedical prediction based on clinical and genome-wide data has become increasingly important in disease diagnosis and classification. To solve the prediction problem in an effective manner for the improvement of clinical care, we develop a novel Artificial Neural Network (ANN) method based on Matrix Pseudo-Inversion (MPI) for use in biomedical applications. The MPI-ANN is constructed as a three-layer (i.e., input, hidden, and output layers) feed-forward neural network, and the weights connecting the hidden and output layers are directly determined based on MPI without a lengthy learning iteration. The LASSO (Least Absolute Shrinkage and Selection Operator) method is also presented for comparative purposes. Single Nucleotide Polymorphism (SNP) simulated data and real breast cancer data are employed to validate the performance of the MPI-ANN method via 5-fold cross validation. Experimental results demonstrate the efficacy of the developed MPI-ANN for disease classification and prediction, in view of the significantly superior accuracy (i.e., the rate of correct predictions), as compared with LASSO. The results based on the real breast cancer data also show that the MPI-ANN has better performance than other machine learning methods (including support vector machine (SVM), logistic regression (LR), and an iterative ANN). In addition, experiments demonstrate that our MPI-ANN could be used for bio-marker selection as well.

Estimation of the phenolic waste attenuation capacity of some fine-grained soils with the help of ANN modeling

In the present investigation, batch experiments were undertaken in the laboratory for different initial phenol concentration ranging from 10 to 40 mg/L using various types of fine-grained soils namely types A, B, C, D, and E based on physical compositions. The batch kinetic data were statistically analyzed with a three-layered feed-forward artificial neural network (ANN) model for predicting the phenol removal efficiency from the water environment. The input parameters considered were the adsorbent dose, initial phenol concentration, contact time, and percentage of clay and silt content in soils. The response output of the ANN model was considered as the phenol removal efficiency. The predicted results of phenol removal efficiency were compared with the experimental values as obtained from batch tests and also tests for goodness of fitting in ANN model with experimental results. The estimated values of coefficient of correlation (R = 0.99) and mean squared error (MSE = 0.006) reveals a reasonable closeness of experimental and predicted values. Out of five different types of soil, type E exhibited the highest removal efficiency (31.6 %) corresponding to 20 mg/L of initial phenol concentration. A sensitivity analysis was also carried out on the ANN model to ascertain the degree of effectiveness of various input variables.

Density prediction of liquid alkali metals and their mixtures using an artificial neural network method over the whole liquid range

In this study, the application of artificial neural network (ANN) method in predicting the density of alkali metals and their mixtures is investigated. A total number of 595 different data points of these compounds were used to train, validate and test the model. A typical three-layer feedforward backpropagation neural network has been trained by the Levenberg Marquardt algorithm. The tansig-tansig transfer functions with 15 neurons in the hidden layer makes the least error, so a network with (8-15-1) structure was used to design the ANN model. The average relative deviations for train, validation, and test sets are 0.1029, 0.1396, and 0.1002, respectively. A comparison between our results and those obtained from some previous works shows that this work, as an excellent alternative, can provide a simple procedure to predict the density of these compounds in a better accord with experimental data up to high temperature, high pressure (HTHP) conditions.

Determination of the normal boiling point of chemical compounds using a quantitative structure–property relationship strategy: Application to a very large dataset

In this work, the quantitative structure–property relationship (QSPR) strategy is applied to predict the normal boiling point (NBP) of pure chemical compounds. In order to propose a comprehensive, reliable, and predictive model, a large dataset of 17,768 pure chemical compounds was exploited. The sequential search mathematical method has been observed to be the only viable search method capable for selection of appropriate model parameters (molecular descriptors) with regard to a data set as large as is used in this study. To develop the model, a three-layer feed forward artificial neural network has been optimized using the Levenberg–Marquardt (LM) optimization strategy. Using this dedicated strategy, satisfactory results were obtained and are quantified by the following statistical parameters: average absolute relative deviations of the predicted properties from existing literature values: 3.2%, and squared correlation coefficient: 0.94.

Improved annual rainfall-runoff forecasting using PSO–SVM model based on EEMD

Rainfall-runoff simulation and prediction in watersheds is one of the most important tasks in water resources management. In this research, an adaptive data analysis methodology, ensemble empirical mode decomposition (EEMD), is presented for decomposing annual rainfall series in a rainfall-runoff model based on a support vector machine (SVM). In addition, the particle swarm optimization (PSO) is used to determine free parameters of SVM. The study data from a large size catchment of the Yellow River in China are used to illustrate the performance of the proposed model. In order to measure the forecasting capability of the model, an ordinary least-squares (OLS) regression and a typical three-layer feed-forward artificial neural network (ANN) are employed as the benchmark model. The performance of the models was tested using the root mean squared error (RMSE), the average absolute relative error (AARE), the coefficient of correlation (R) and Nash–Sutcliffe efficiency (NSE). The PSO–SVM–EEMD model improved ANN model forecasting (65.99%) and OLS regression (64.40%), and reduced RMSE (67.7%) and AARE (65.38%) values. Improvements of the forecasting results regarding the R and NSE are 8.43%, 18.89% and 182.7%, 164.2%, respectively. Consequently, the presented methodology in this research can enhance significantly rainfall-runoff forecasting at the studied station.

Neural-network-based optimal tracking control scheme for a class of unknown discrete-time nonlinear systems using iterative ADP algorithm

In this paper, an optimal tracking control scheme is proposed for a class of unknown discrete-time nonlinear systems using iterative adaptive dynamic programming (ADP) algorithm. First, in order to obtain the dynamics of the system, an identifier is constructed by a three-layer feedforward neural network (NN). Second, a feedforward neuro-controller is designed to get the desired control input of the system. Third, via system transformation, the original tracking problem is transformed into a regulation problem with respect to the state tracking error. Then, the iterative ADP algorithm based on heuristic dynamic programming is introduced to deal with the regulation problem with convergence analysis. In this scheme, feedforward NNs are used as parametric structures for facilitating the implementation of the iterative algorithm. Finally, simulation results are also presented to demonstrate the effectiveness of the proposed scheme.

Evaluation of seismogenesis behavior in Himalayan belt using data mining tools for forecasting

AbstractIn the proposed study, non-linear behavioral patterns in the seismic regime for earthquakes in the Himalayan basin have been studied using a complete, verified EQ catalogue comprised of all major events and their aftershock sequences in the Himalayan basin for the past 110 years [1900–2010]. The dataset has been analyzed to give better decision making criteria for impending earthquakes. A series of statistical tests based on multi-dimensional rigorous statistical studies, inter-event distance analyses, and statistical time analyses have been used to obtain correlation dimensions. The time intervals of earthquakes within a seismic regime have been used to train the neural network to analyze the nature of earthquake patterns in the different clusters. The results obtained from descriptive statistics show high correlation with previously conducted gravity studies and radon anomaly variation. A study of the time of recurrence of the numerical properties of the regime for 60 years from 1950 to 2010 for the Himalayan belt for analysis of significant EQ failure events has been done to find the best fit for an empirical data probability distribution. The distribution of waiting time of swarm events occurring in the Himalayan basin follows a power-law model, while independent events do not fit the power-law distribution. This suggests that probability of the occurrence of swarm events [M ⩽ 6.0] with frequent shaking may be more frequent than that of the occurrence of independent events of magnitude [M >6.0] in the Himalayan belt. We propose a three-layer feed forward neural network model to identify factors, with the actual occurrence of the maximum earthquake level M as input and target vectors in Himalayan basin area. We infer through a series of statistical results and evaluations that probabilistic forecasting of earthquakes can be achieved by finding the meta-stable cluster zones of the Himalayan clusters for the spatio-temporal distribution of earthquakes in the area.

An Accurate Heave Signal Prediction Using Artificial Neural Network

An accurate heave modeling is required for several applications, including hydrographic surveying. This paper proposes an adaptive heave signal modeling, which uses a neural network-based modelling. A recurrent neural network and three-layer feed forward neural network trained using the Levenberg-Marquardt learning algorithm is used for this purpose. Computational results with five different datasets of real time heave are provided to validate the effectiveness of the artificial neural network-based model. It is shown that the new neural network-based model give a reliable heave model with excellent performance. Also, a comparison is made between the developed artificial neural network models and the autoregressive models. It is shown that the new artificial neural network-based model give the best performance results (i.e., the mean square error MSE).

Testing selected optimal descriptors with artificial neural networks

Eleven properties have been modeled with the objective of checking the importance for model purposes of mixed descriptors made of empirical parameters, molecular connectivity indices and random numbers. The mixed descriptors with random indices have a descriptive character which is satisfactorily confirmed by the leave-one-out method of statistical analysis. The introduction of a partition of the set of compounds into training and evaluation sets decreases drastically the probability to find a mixed descriptor with random indices with good model quality. Two properties, the magnetic susceptibility and the elutropic values, insist on having optimal descriptors with random indices. The overall model study underlines the importance of semiempirical descriptors made of experimental parameters and molecular connectivity indices, as well as the importance of a perturbation parameter that has been introduced into the valence delta number to encode the contribution of the depleted hydrogen atoms. The use of complete graphs to encode the core electrons of higher-row atoms is also underlined. The model quality of the mixed descriptors obtained with combinatorial regressive methods has also been tested with three-layered feed-forward artificial neural network (ANN) methods. This methodology not only confirms the validity of the descriptors but also improves their model quality widening, thus, their predictive ability.

Data Imputation for Regional Traffic Index Based on GSO-NN

The incomplete traffic data in urban road networks greatly affect the quality and performance of traffic information platform. The importance of effectively imputing the missing values emerges. The paper introduces a three-layer feed-forward neural network to missing traffic index prediction, where the group search optimization (GSO) is proposed to optimize the connection weights and thresholds to solve the problem of falling into the local optima. It is the first time to apply the technique to missing data imputation. Simulations are presented to demonstrate the accuracies and effectiveness of the proposed methodologies. This reveals that it is a promising approach in missing data prediction.

Reconstruction of hydrometeorological time series and its uncertainties for the Kaidu River Basin using multiple data sources

This paper tried to reconstruct the time series (TS) of monthly average temperature (MAT), monthly accumulated precipitation (MAP), and monthly accumulated runoff (MAR) during 1901–1960 in the Kaidu River Basin using the Delta method and the three-layered feed forward neural network with backpropagation algorithm (TLBP-FFNN) model. Uncertainties in the reconstruction of hydrometeorological parameters were also discussed. Available monthly observed hydrometeorological data covering the period 1961–2000 from the Kaidu River Basin, the monthly observed meteorological data from three stations in Central Asia, monthly grid climatic data from the Climatic Research Unit (CRU), and Coupled Model Intercomparison Project Phase 3 (CMIP3) dataset covering the period 1901–2000 were used for the reconstruction. It was found that the Delta method performed very well for calibrated and verified MAT in the Kaidu River Basin based on the monthly observed meteorological data from Central Asia, the monthly grid climatic data from CRU, and the CMIP3 dataset from 1961 to 2000. Although calibration and verification of MAP did not perform as well as MAT, MAP at Bayinbuluke station, an alpine meteorological station, showed a satisfactory result based on the data from CRU and CMIP3, indicating that the Delta method can be applied to reconstruct MAT in the Kaidu River Basin on the basis of the selected three data sources and MAP in the mountain area based on CRU and CMIP3. MAR at Dashankou station, a hydrological gauge station on the verge of the Tianshan Mountains, from 1961 to 2000 was well calibrated and verified using the TLBP-FFNN model with structure (8,1,1) by taking MAT and MAP of four meteorological stations from observation; CRU and CMIP3 data, respectively, as inputs; and the model was expanded to reconstruct TS during 1901–1960. While the characteristics of annual periodicity were depicted well by the TS of MAT, MAP, and MAR reconstructed over the target stations during the period 1901–1960, different high frequency signals were captured also. The annual average temperature (AAT) show a significant increasing trend during the 20th century, but annual accumulated precipitation (AAP) and annual accumulated runoff (AAR) do not. Although some uncertainties exist in the hydrometeorological reconstruction, this work should provide a viable reference for studying long-term change of climate and water resources as well as risk assessment of flood and drought in the Kaidu River Basin, a region of fast economic development.

Research on Welding Line Defect Recognition of the In-Service Pipeline Using X-Ray Detecting

This paper, based on the practical demands of in-service pipeline detection, a set of X-ray digital image welding line defect intelligent recognition system is established. Taking the welding line image detected by X-ray as objects of study, self-adaptive median filter method filters noise, high frequency enhancement filter method conducts the image edge sharpening enhancement; a edge detection method for X-ray digital image based on morphological gradient is proposed; a group of characteristics parameters that accurately reflects the essence characteristic of defects is selected, using a self-organizing, self-adaptive three-layer feed-forward neural network, applying BP algorithm, the BP neural network recognition system is established, thus, to achieve detection and recognition of weld defects.