Three-layer Back-propagation Neural Network Research Articles

Special issue on pattern recognition and information processing using neural networks

Neural network techniques have proven to be flexible in pattern recognition and information processing in complex environments. They typically include BP networks, RBF networks, support vector machine (SVM) and other similar biologically motivated models. The neural network techniques are able to enhance recognition accuracy, and have found applications in real-world environments. This special issue addresses neural network techniques in pattern recognition and information processing problems. The first paper ‘‘Kernel based improved discriminant analysis and its application to face recognition,’’ coauthored by Dake Zhou and Zhenmin Tang, presents a variant of KDA called kernel-based improved discriminant analysis (KIDA). In the proposed framework, original samples are projected firstly into a feature space by an implicit nonlinear mapping. After reconstructing betweenclass scatter matrix in the feature space by weighted schemes, the kernel method is used to obtain a modified Fisher criterion. Finally, simultaneous diagonalization technique is employed to find lower dimensional nonlinear features with significant discriminant power. The second paper ‘‘A Novel Application of a Self-Organizing Network for Recognition of Facial Expressions from Contours,’’ coauthored by W.F. Gu, Y.V. Venkatesh, and C. Xiang, proposes a self-organizing network for recognizing facial expressions using biologically plausible features: contours of face and its components. Experimental results show that the recognition accuracy of the presented algorithm is superior to that of other algorithms in the literature on the Japanese Female Facial Expression (JAFFE) database. The third paper ‘‘An Automatic Fuzzy C-Means Algorithm for Image Segmentation,’’ co-authored by Yanling Li and Yi Shen, proposes a novel fuzzy clustering algorithm for automatically grouping the pixels of an image into different homogeneous regions when the number of clusters is not known a priori. The new algorithm initiates the first two centroids of clusters by a method based on hard c-means algorithm and automatically determines the appropriate cluster number for image segmentation. The fourth paper ‘‘Content Based Image Classification with Wavelet Relevance Vector Machines,’’ co-authored by Arvind Tolambiya, S. Venkataraman, and Prem K. Kalra, introduces the use of relevance vector machines (RVM) for content-based image classification and compares it with the conventional SVM approach. The authors also propose a new wavelet-based feature extraction method that extracts less number of features as compared to other wavelet-based feature extraction methods. The fifth paper ‘‘A Three-layer Back-propagation Neural Network for Spam Detection Using Artificial Immune Concentration,’’ co-authored by Guangchen Ruan and Ying Tan, uses a three-layer back-propagation neural network F.-C. Sun (&) State Key Lab of Intelligent Technology and Systems, Department of Computer Science and Technology, Tsinghua University, 100084 Beijing, China e-mail: fcsun@tsinghua.edu.cn

A three-layer back-propagation neural network for spam detection using artificial immune concentration

In this paper, a three-layer back-propagation neural network (BPNN) is employed for spam detection by using a concentration based feature construction (CFC) approach. In the CFC approach, `self' and `non-self' concentrations are constructed through `self' and `non-self' gene libraries, respectively, to form a two-element concentration vector for expressing the e-mail efficiently. A three-layer BPNN with two-element input is then employed to classify e-mails automatically. Comprehensive experiments are conducted on two public benchmark corpora PU1 and Ling to demonstrate that the proposed CFC approach based BPNN classifier not only has a very much fast speed but also achieves 97 and 99% of classification accuracy on corpora PU1 and Ling by just using a two-element concentration feature vector.

Complex object 3D measurement based on phase-shifting and a neural network

An accurate phase-height mapping algorithm based on phase-shifting and a neural network is proposed to improve the performance of the structured light system with digital fringe projection. As phase-height mapping is nonlinear, it is difficult to find the best camera model for the system. In order to achieve high accuracy, a trained three-layer back propagation neural network is employed to obtain the complicated transformation. The phase error caused by the non-sinusoidal attribute of the fringe image is analyzed. During the phase calculation process, a pre-calibrated phase error look-up-table is used to reduce the phase error. The detailed procedures of the sample data collection are described. By training the network, the relationship between the image coordinates and the 3D coordinates of the object can be obtained. Experimental results demonstrate that the proposed method is not sensitive to the non-sinusoidal attribute of the fringe image and it can recover complex free-form objects with high accuracy.

Estimation of formation strength index of aquifer from neural networks

The purpose of this study is to construct a model that predicts an aquifer's formation strength index (the ratio of shear modulus and bulk compressibility, G/ C b) from geophysical well logs by using a back-propagation neural network (BPNN). The BPNN model of an aquifer's formation strength index is developed using a set of well logging data. The model is a [4-5-1] three-layer BPNN with a four-neuron input layer (depth, gamma-ray log data, formation density log data, and sonic log data, respectively), a five-neuron hidden layer, and a one-neuron output layer (formation strength index). The optimal learning rate and momentum constant used in the BPNN model are obtained from serial combinative experiments. The inside test and outside test are implemented to check the performance of network learning and the prediction ability of the network, respectively. The results of the inside test, based on 84 training data sets from a total of 105 data sets, show that the network has been well-trained because the mean square error between the network output value and the target value from the inside test is very small (1.1×10 −4). The results of the outside test, based on 21 testing data sets from 105 data sets, show the excellent prediction ability of the BPNN model, because the network prediction values closely track with the target values (the mean square error is 2.1×10 −4).

Artificial neural network vs. nonlinear regression for gold content estimation in pyrometallurgy

Pyrometallurgy is often used in the industrial process for treating gold-bearing slime. Slag compositions have remarkable influences on gold recovery and gold content in slag. In this paper, the relationships between the slag compositions in the soda–borax–silica glass-salt system and the gold content in the slag are investigated by using nonlinear regression and artificial neural network. A neural network model for estimating the gold contents of different slag compositions is presented, including the neural network type, structure and its learning algorithms. The study indicates that the three-layer back propagation neural network model can be applied to estimate gold content in the slag. Compared with the traditional regression methods, the neural network has many advantages.

Applying Neural Networks to the Solution of the Inverse Heat Conduction Problem in a Gun Barrel

The primary cause of gun barrel erosion is the heat generated by the shell as its travels along the barrel. Therefore, calculating the heat flux input to the gun bore is very important when investigating wear problems in the gun barrel and examining its thermomechanical properties. This paper employs the continuous-time analog Hopfield neural network (CHNN) to compute the temperature distribution in various forward heat conduction problems. An efficient technique is then proposed for the solution of inverse heat conduction problems using a three-layered backpropagation neural network (BPN). The weak generalization capacity of BPN networks when applied to the solution of nonlinear function approximations is improved by employing the Bayesian regularization algorithm. The CHNN scheme is used to calculate the temperature in a 155mm gun barrel and the trained BPN is then used to estimate the heat flux of the inner surface of the barrel. The results show that the proposed neural network analysis method successfully solves forward heat conduction problems and is capable of predicting the unknown parameters in inverse problems with an acceptable error.

Modeling and Analysis of Compressive Properties of Porous NiTi Shape Memory Alloy Using Artificial Neural Network

Artificial neural network (ANN) is an intriguing data processing technique. Over the last decade, it was applied widely in the chemistry field, but there were few applications in the porous NiTi shape memory alloy (SMA). In this paper, 32 sets of samples from thermal explosion experiments were used to build a three-layer BP (back propagation) neural network model. According to the registered BP model, the effect of process parameters including heating rate ( ), green density ( ) and particle size of Ti ( d ) on compressive properties of reacted products including ultimate compressive strength ( v D σ ) and ultimate compressive strain (ε ) was analyzed. The predicted results agree with the actual data within reasonable experimental error, which shows that the BP model is a practically very useful tool in the properties analysis and process parameters design of the porous NiTi SMA prepared by thermal explosion method.

ESTIMATION OF PARTICLE CONCENTRATION EMITTED FROM THE STACKS OF KERMAN CEMENT PLANT USING ARTIFICIAL NEURAL NETWORKS

In this study a new approach based on artificial neural networks (ANNs) has been designed to estimate the concentration of PM10 from the Kerman Cement Plant. Some measured data have been used to create an artificial neural network for predicting suspended particle concentration. The data include particle concentration, distance from source, mixing height, lateral and vertical dispersion parameters, and 10 meter wind speed. The present work applies a three-layer back-propagation neural network with 10 neurons in the hidden layer. The results from the network are in good agreement with the measured data, with an average absolute percent deviation of 5.91%. The results of ANNs have also been compared with the results of the Gaussian plume model and multivariable regression model.

Emulating and Modeling for Position Errors of Ultra-Precision Aspherical Grinding

In the process of the ultra-precision grinding, the machining path of the aspherical is the result of motor coordination by several axes for the numerical control system. Since the motion of each axis have errors, there are big errors between the real positions and the theoretical positions, and the position error of the wheel infects the accuracy of the workpiece greatly. This paper analyses the position error property of the wheel and finds the machining approach path has nothing to do with the position error, just do with to the present machining point. In order to solve the problem, the method using the Neural Network optimized by the Genetic Algorithm to establish the position error model is introduced. A three-layer error back propagation (simplified as BP) Neural Network is used to establish the position error model, the position coordinates (x, z) of the program instruction is input layer, and the corroding measured error value ( Δx , Δz ) is output layer. Before training data sample, using the Genetic Algorithm to optimize the Neural Network to improve the predicting accuracy of the Neural Network, and reduce the training time. The emulation results indicate that using the Neural Network model optimized by the Genetic Algorithm can predict the position error in a high degree of accuracy, and at the same time, according to the predicting results, compensating the position error of the wheel is possible.

A Generalized Neural Network Model of Refrigerant Mass Flow Through Adiabatic Capillary Tubes and Short Tube Orifices

Adiabatic capillary tubes and short tube orifices are widely used as expansive devices in refrigeration, residential air conditioners, and heat pumps. In this paper, a generalized neural network has been developed to predict the mass flow rate through adiabatic capillary tubes and short tube orifices. The input/output parameters of the neural network are dimensionless and derived from the homogeneous equilibrium flow model. Three-layer backpropagation (BP) neural network is selected as a universal function approximator. Log sigmoid and pure linear transfer functions are used in the hidden layer and the output layer, respectively. The experimental data of R12, R22, R134a, R404A, R407C, R410A, and R600a from the open literature covering capillary and short tube geometries, subcooled and two-phase inlet conditions, are collected for the BP network training and testing. Compared with experimental data, the overall average and standard deviations of the proposed neural network are 0.75% and 8.27% of the measured mass flow rates, respectively.

Application of Improved BP Neural Network in the Preparation Processing of the SrTiO<sub>3</sub> Nanocrystalline

A three-layer back-propagation neural network model based on the non-linear relationship between the size of the SrTiO3 nanocrystalline and the technology factors, such as reaction time, reaction temperature, raw material adding amount of NaOH and SrCl2, and the rate of TiCl4/Hl, was established. Moreover, in order to accelerate the converging rate and avoid the non-converging situation, the momentum terms are introduced. Besides, the variable learning speed is adopted. At the same time, the input variables were pretreated by using the main component analysis firstly. And the results show that the improved back-propagation neural network model is very efficient for predication of the SrTiO3 nanocrystalline size.

A novel intrusion detection mode based on understandable neural network trees

Several data mining techniques such as Hidden Markov Model (HMM), artificial neural network, statistical techniques and expert systems are used to model network packets in the field of intrusion detection. In this paper a novel intrusion detection mode based on understandable Neural Network Tree (NNTree) is presented. NNTree is a modular neural network with the overall structure being a Decision Tree (DT), and each non-terminal node being an Expert Neural Network (ENN). One crucial advantage of using NNTrees is that they keep the non-symbolic model ENN’s capability of learning in changing environments. Another potential advantage of using NNTrees is that they are actually “gray boxes” as they can be interpreted easily if the number of inputs for each ENN is limited. We showed through experiments that the trained NNTree achieved a simple ENN at each non-terminal node as well as a satisfying recognition rate of the network packets dataset. We also compared the performance with that of a three-layer backpropagation neural network. Experimental results indicated that the NNTree based intrusion detection model achieved better performance than the neural network based intrusion detection model.

Neural-network based analysis and prediction of a compressor’s characteristic performance map

The difficulties, due to a lack of information about stage-by-stage axial-compressor performance, are analyzed. To overcome these issues, a three-layer back-propagation neural-network applied Levenberg–Marquardt algorithm is presented and discussed. The experimental data provided by manufacturers are used for the neural-network training. Through twice training, the compressor’s performance map can be predicted. The results can be used for the development of an off-design model or overall dynamic simulation of the behaviour of a gas-turbine power-plant.

Multi-step predictive control with TDBP method for pneumatic position servo system

This paper presents a new multi-step predictive controller based on neural networks and researches the adaptability of the predictive controller for a pneumatic position servo system which has some typical characteristics of non-linearity and time-varying. A diagonal recurrent neural network (DRNN) is used to predict the system output of the multi-step ahead directly. According to the intrinsic defects of a back-propagation (BP) algorithm that cannot update network weights incrementally, a new hybrid learning algorithm combining the temporal differences (TD) method with the BP algorithm to train the DRNN is put forward. A three-layer feedforward BP neural network is used as a non-linear rolling optimal controller to realize the optimization of control input of the next step according to a single-value predictive control algorithm to simplify computation. Simulation and experimental results indicate that the proposed predictive controller is suitable for real-time control of a pneumatic position servo system because of its characteristics of a simple algorithm, fast calculation of the control input and good tracking effects.

QSAR study of 4-phenylpiperidine derivatives as μ opioid agonists by neural network method

A nonlinear QSAR study was conducted on a series of 4-phenylpiperidine derivatives (4PPs) acting as μ opioid agonists by three-layer back-propagation neural network (NN) method. At first a variety of molecular descriptors were calculated and then selected with two-stage least squares combining partial least squares (PLS) method. The selected four molecular descriptors, out of 292 ones, were correlated with the known analgesic activities of 38 4PPs by NN method. The established QSAR model was further validated by five additional 4PPs, as an external testing set. Moreover, a pharmacophore model was hypothesized based on the results, which would be helpful for structural optimization of 4PPs.

Tyre dynamics modelling of vehicle based on support vector machines

Various methods of tyre modelling are implemented from pure theoretical to empirical or semi-empirical models based on experimental results. A new way of representing tyre data obtained from measurements is presented via support vector machines (SVMs). The feasibility of applying SVMs to steady-state tyre modelling is investigated by comparison with three-layer backpropagation (BP) neural network at pure slip and combined slip. The results indicate SVMs outperform the BP neural network in modelling the tyre characteristics with better generalization performance. The SVMs-tyre is implemented in 8-DOF vehicle model for vehicle dynamics simulation by means of the PAC 2002 Magic Formula as reference. The SVMs-tyre can be a competitive and accurate method to model a tyre for vehicle dynamics simulation.

Determination and Analysis of Interrill Erosion of a Soil with Coarse Fragments in Taiwan

To evaluate the effects of coarse fragments in soil on interrill soil erosion, a programmable rainfall simulator and soil erosion boxes were designed and fabricated. Simulated rainfall erosion tests were conducted on soils with coarse fragment contents of 0%, 7.5%, 15%, 30%, and 45%; at slope steepnesses of 9%, 20%, and 30%; and under simulated rainfall intensities of 40, 60, 80, and 100 mm h-1. A total of 132 data sets were obtained. It was found from the test results that coarse fragments had a mitigating effect on soil erosion at high rainfall intensities, and the steeper the slope, the better the effect. However, there was no apparent positive correlation between such mitigating effect and the percentage of coarse fragments in test samples. When part of the coarse fragments were exposed at the soil surface, both positive and negative effects might result, and a quantitative evaluation thereof requires further study. Furthermore, regression analysis and an artificial neural network were used in this study to establish a general equation and a three-layer back-propagation neural network model for estimating interrill soil erosion. While the regression equation is in harmony with the conceptual mechanisms of soil erosion, with an R2 value of 0.900 and an RMSE of 0.429, the artificial neural network model has a higher R2 value of 0.962 and a lower RMSE of 0.342, indicating that the artificial neural network model may provide better estimation of interrill soil erosion.

A neural network approach to predict survival/death and growth/no-growth interfaces for Escherichia coli O157:H7

An artificial neural network (ANN) model was developed to predict survival/death and growth/no-growth interfaces for Escherichia coli O157:H7 in a mayonnaise-type system. Temperature, pH, acetic acid, sucrose and salt were the inputs to a three-layer back-propagation neural network. The ANN model was trained using the data-set of McKellar et al. [2002. A probability model describing the interface between survival and death of E. coli O157:H7 in a mayonnaise model system. Food Microbiol. 19, 235–247] that consisted of 1820 treatment combinations from controlled experiments with a cocktail of five strains of E. coli O157:H7. After training, the model correctly predicted the growth/no-growth in 1810 combinations (99.5%) with 8 false positives and 2 false negatives, and survival/death in 1804 combinations (99.1%) with 13 false positives and 3 false negatives. Classification accuracy was validated using additional literature data-sets for growth of E. coli O157:H7 under various environmental conditions. The ANN model accurately predicted the survival/death in 27 of 30 cases (90%) in experimental mayonnaise inoculated with E. coli O157:H7, with 3 fail-positive predictions and all observed growth (100%). Simulations were used to estimate the influence of incubation temperature on survival and growth for specific combinations of acetic acid, salt, pH and sucrose. The ANN model is recommended as an alternative tool for classification of survival and growth conditions in predictive microbiology.

Modeling water and carbon fluxes above summer maize field in North China Plain with back-propagation neural networks

In this work, datasets of water and carbon fluxes measured with eddy covariance technique above a summer maize field in the North China Plain were simulated with artificial neural networks (ANNs) to explore the fluxes responses to local environmental variables. The results showed that photosynthetically active radiation (PAR), vapor pressure deficit (VPD), air temperature (T) and leaf area index (LAI) were primary factors regulating both water vapor and carbon dioxide fluxes. Three-layer back-propagation neural networks (BP) could be applied to model fluxes exchange between cropland surface and atmosphere without using detailed physiological information or specific parameters of the plant.

Artificial Neural Network Models for Identifying Flow Regimes and Predicting Liquid Holdup in Horizontal Multiphase Flow

SummaryThis paper presents two artificial neural network (ANN) models to identify the flow regime and calculate the liquid holdup in horizontal multiphase flow. These models are developed with 199 experimental data sets and with three-layer back-propagation neural networks (BPNs). Superficial gas and liquid velocities, pressure, temperature, and fluid properties are used as inputs to the model. Data were divided into three portions: training, cross validation, and testing. The results show that the developed models provide better predictions and higher accuracy than the empirical correlations developed specifically for these data groups. The developed flow-regime model predicts correctly for more than 97% of the data points. The liquid-holdup model outperforms the published models; it provides holdup predictions with an average absolute percent error of 9.407, a standard deviation of 8.544, and a correlation coefficient of 0.9896.