Back-propagation Network Training Research Articles

Implementation and efficient evaluation of backpropagation network training algorithms in parametric simulations of soil hydraulic conductivity curve

The estimation of the soil hydraulic conductivity curve (SHCC) through machine learning tools for determining pedotransfer functions (PTFs) is essential in various agriculture, hydrological, and environmental management-related studies. The accuracy and reliability of machine learning tools such as backpropagation neural network (BNN) for estimating SHCC mainly depends on the optimization of its architecture, specifically the selection of the network training algorithms. By comparing the performance of network training algorithms, various researchers related to agriculture science, hydrology and environmental can choose the appropriate algorithm that provides the best balance of accuracy, speed, computational efficiency, and robustness for hydrological modeling purposes. Therefore, the present study assessed and compared the performance of 4 training algorithms including, gradient descent with momentum (GDM), Quasi-Newton (QN), Bayesian regularization (BR) and Levenberg-Marquardt (LM) for the estimation of the parametric-based PTFs of the van Genuchten-Mualem hydraulic conductivity equation as novelty of this research, which has not been investigated, so far. In this study four GDM, BR, LM and QN-based PTFs were derived to estimate the SHCC from different combinations of readily available properties of 86 soil samples from UNSODA database. The integral root mean square error (IRMSE) and coefficient of determination (R2) and were used to assess the estimations. The performance of BR algorithm-based PTFs on testing data varied from 4.634 to 6.690 with average 5.476 cm day−1 in terms of IRMSE, and 0.958 to 0.966 with average 0.963 in terms of R2, which provided better simulation than other training algorithms. The outcome of the present study on testing data showed that the BR algorithm was stable and robust to heterogeneous and noisy data compared to other training algorithms. Finally, development of accurate and reliable PTFs using efficient algorithms can play a crucial role in soil resource assessments, water resource management, irrigation planning, and designing effective drainage systems, because accurate knowledge of SHCC is vital for optimizing irrigation and fertilization practices and minimizing wastage of water and nutrients.

Using Taguchi and neural network approaches in the optimum design of product development process

ABSTRACTIn product design, accuracy of the product information greatly affects design quality. The Taguchi method simplifies the analysis method and provides an effective product design approach by confirming the variable characteristics and determining the optimum combination of characteristics. However, determination of the levels of the variable characteristics in the Taguchi method relies on human experience and might not achieve the optimum situation. The research objective is to use evolutionary neural networks into a robust product design to help designers search for an optimum combination of variable characteristic values for a given product design problem. In the design procedure, the data resulting from the experimental design in the Taguchi method are forwarded to the back-propagation network training process and genetic algorithm simulation to predict the most suitable combination of variable characteristic values. The recommended combination of variable characteristic values is represented in 3D form. A design case of a lat bar for pull-down fitness stations is used to demonstrate the applicability of the design procedures. Note that the signal-to-noise ratios are derived from experiments that measure the muscle responses using an electromyography (EMG) apparatus. The results indicate that the proposed procedures could enhance the efficiency of product design efforts.

A study on estimation of human personality from location visiting preference

It is widely believed that human personality and human-preferred locations are closely related. In this paper, a statistical study to determine the relationship between personality and preferred location, quantitatively, was conducted. Human personality can be represented through the big five factor (BFF) model. Human-preferred locations have been determined from a positioning dataset collected by the smartphones and other mobile devices of volunteers. The positioning dataset can be analyzed and clustered into preferred location datasets. Such analysis is composed of two phases. The first phase estimates a human personality from the dataset of human-preferred locations. The second phase shows the effect of human personality on the selection of preferred locations. In each phase, the location and personality data are preprocessed, trained using a back propagation network (BPN), and then validated and studied through a regression analysis. Location clustering, BPN training, and a regression analysis are the major methods used. The results of this paper demonstrate the relationship between personality and preferred location, which can be presented quantitatively. In particular, the estimation of human personality based on favored locations is the key to our research. For example, preference for a home location is highly related with extraversion, and rarely related with conscientiousness.

Artificial neural networks predict the incidence of portosplenomesenteric venous thrombosis in patients with acute pancreatitis

Objective To construct and validate artificial neural networks (ANNs) for predicting the occurrence of portosplenomesenteric venous thrombosis (PSMVT) and compare the predictive ability of the ANNs with that of logistic regression. Methods The ANNs and logistic regression modeling were constructed using simple clinical and laboratory data of 72 acute pancreatitis (AP) patients. The ANNs and logistic modeling were first trained on 48 randomly chosen patients and validated on the remaining 24 patients. The accuracy and the performance characteristics were compared between these two approaches by SPSS17.0 software. Results The training set and validation set did not differ on any of the 11 variables. After training, the back propagation network training error converged to 1 × 10-20 , and it retained excellent pattern recognition ability. When the ANNs model was applied to the validation set, it revealed a sensitivity of 80%, specificity of 85.7%, a positive predictive value of 77.6% and negative predictive value of 90.7%. The accuracy was 83.3%. Differences could be found between ANNs modeling and logistic regression modeling in these parameters (10.0% [95% CI, -14.3 to 34.3%], 14.3% [95% CI, -8.6 to 37.2%], 15.7% [95% CI, -9.9 to 41.3%], 11.8% [95% CI, -8.2 to 31.8%], 22.6% [95% CI, -1.9 to 47.1%], respectively). When ANNs modeling was used to identify PSMVT, the area under receiver operating characteristic curve was 0.849 (95% CI, 0.807-0.901), which demonstrated better overall properties than logistic regression modeling (AUC = 0.716) (95% CI, 0.679-0.761). Conclusions ANNs modeling was a more accurate tool than logistic regression in predicting the occurrence of PSMVT following AP. More clinical factors or biomarkers may be incorporated into ANNs modeling to improve its predictive ability.

A Fast C++ Implementation of Neural Network Backpropagation Training Algorithm: Application to Bayesian Optimal Image Demosaicing

Recent years have seen a surge of interest in multilayer neural networks fueled by their successful applications in numerous image processing and computer vision tasks. In this article, we describe a C++ implementation of the stochastic gradient descent to train a multilayer neural network, where a fast and accurate acceleration of tanh(.) is achieved with linear interpolation. As an example of application, we present a neural network able to deliver state-of-the-art performance in image demosaicing.

A novel and generalized approach in the inversion of geoelectrical resistivity data using Artificial Neural Networks (ANN)

The non-linear apparent resistivity problem in the subsurface study of the earth takes into account the model parameters in terms of resistivity and thickness of individual subsurface layers using the trained synthetic data by means of Artificial Neural Networks (ANN). Here we used a single layer feed-forward neural network with fast back propagation learning algorithm. So on proper training of back propagation networks it tends to give the resistivity and thickness of the subsurface layer model of the field resistivity data with reference to the synthetic data trained in the appropriate network. During training, the weights and biases of the network are iteratively adjusted to make network performance function level more efficient. On adequate training, errors are minimized and the best result is obtained using the artificial neural networks. The network is trained with more number of VES data and this trained network is demonstrated by the field data. The accuracy of inversion depends upon the number of data trained. In this novel and specially designed algorithm, the interpretation of the vertical electrical sounding has been done successfully with the more accurate layer model.

Research on Human Resource Strategy under Customer Demand Using System Dynamics for the Thin Film Sputtering Target Material Industry

This study integrates the forecast capacity and superiority of dynamic simulation of back-propagation network (BPN) and system dynamics, thereby enabling the target industries to formulate coping strategies and to make adjustment in both production capabilities and manpower. The main purpose of this study is to assess and confirm what the most effective policy is for production and human resources. The forecasting result of this study indicates that Correlation and R-squared values reach 0.99852 and 0.994961 respectively after a training of BPN. Certainly, BPN serves as an effective method of forecasting target sales. Finally, the forecast value of the next three years is worked out through situational simulation of complex production models in system dynamics. The final result of this study shows that the optimum manpower should be 10 workers for the years 2011-2013, as well as the projected result will be a rise in production yield of up to 2.54%. It is the most profitable scenario for the company.

Research on Thermal Error Compensation of Gear Hobbing Machine

The purpose of this research is to improve the machining accuracy of YK3610 Hobbing machine through thermal error compensation. This study presents the whole process of thermal error modeling and compensation by using Back Propagation Network (BPN) and ant colony optimization is introduced into the training of BPN. The results show that the BPN model based on ant colony algorithm improves the prediction accuracy of thermal errors on the gear hobbing machine and the thermal drift has been reduced from 14.2 μm to 4.5 μm after compensation.

Analisis Fungsi Aktivasi Jaringan Syaraf Tiruan untuk Mendeteksi Karakteristik Bentuk Gelombang Spektra Babi dan Sapi

Artificial Neural Network (ANN) is beginning little by little to replace the task of an expert, even with the ANN can be a tool to replace a doctor. One of kind of ANN is Backpropagation networks, this network can be used to training programs in order to be able to recognize whether it is pig or cow wave spectra. To determine the output in Backpropagation training required suitable activation functions. Therefore, in this research will be compared to some of the activation function that can be used in training. Activation functions will be tested with the ratio test to determine the interval convergence. After tested with the ratio test it was found that the activation function was the best activation function to use the Backpropagation network training, because it has a weight range that can meet the methods used in the determination of weights. When tested with the data, the activation function is able to recognize correctly all trial datas. Expected in future research to examine the weight that makes the interval training to achieve fast convergence and the error bit.

Implementation of Neural Network Back Propagation Training Algorithm on FPGA

ABSTRACT This work presents the implementation of trainable Artificial Neural Network (ANN) chip, which can be trained to implement certain functions. Usually training of neural networks is done off-line using software tools in the computer system. The neural networks trained off-line are fixed and lack the flexibility of getting trained during usage. In order to overcome this disadvantage, training algorithm can implemented on-chip with the neural network. In this work back propagation algorithm is implemented in its gradient descent form, to train the neural network to function as basic digital gates and also for image compression. The working of back propagation algorithm to train ANN for basic gates and image compression is verified with intensive MATLAB simulations. In order to implement the hardware, verilog coding is done for ANN and training algorithm. The functionality of the verilog RTL is verified by simulations using ModelSim XE III 6.2c simulator tool. The verilog code is synthesized using Xilinx ISE 10.1 tool to get the netlist of ANN and training algorithm. Finally the netlist was mapped to FPGA and the hardware functionality was verified using Xilinx Chipscope Pro Analyzer 10.1 tool. Thus the concept of neural network chip that is trainable on-line is successfully implemented.

Investigating the performance of neural network backpropagation algorithms for TEC estimations using South African GPS data

Abstract. In this work, results obtained by investigating the application of different neural network backpropagation training algorithms are presented. This was done to assess the performance accuracy of each training algorithm in total electron content (TEC) estimations using identical datasets in models development and verification processes. Investigated training algorithms are standard backpropagation (SBP), backpropagation with weight delay (BPWD), backpropagation with momentum (BPM) term, backpropagation with chunkwise weight update (BPC) and backpropagation for batch (BPB) training. These five algorithms are inbuilt functions within the Stuttgart Neural Network Simulator (SNNS) and the main objective was to find out the training algorithm that generates the minimum error between the TEC derived from Global Positioning System (GPS) observations and the modelled TEC data. Another investigated algorithm is the MatLab based Levenberg-Marquardt backpropagation (L-MBP), which achieves convergence after the least number of iterations during training. In this paper, neural network (NN) models were developed using hourly TEC data (for 8 years: 2000–2007) derived from GPS observations over a receiver station located at Sutherland (SUTH) (32.38° S, 20.81° E), South Africa. Verification of the NN models for all algorithms considered was performed on both "seen" and "unseen" data. Hourly TEC values over SUTH for 2003 formed the "seen" dataset. The "unseen" dataset consisted of hourly TEC data for 2002 and 2008 over Cape Town (CPTN) (33.95° S, 18.47° E) and SUTH, respectively. The models' verification showed that all algorithms investigated provide comparable results statistically, but differ significantly in terms of time required to achieve convergence during input-output data training/learning. This paper therefore provides a guide to neural network users for choosing appropriate algorithms based on the availability of computation capabilities used for research.

A robust behavior of Feed Forward Back propagation algorithm of Artificial Neural Networks in the application of vertical electrical sounding data inversion

The applications of intelligent techniques have increased exponentially in recent days to study most of the non-linear parameters. In particular, the behavior of earth resembles the non-linearity applications. An efficient tool is needed for the interpretation of geophysical parameters to study the subsurface of the earth. Artificial Neural Networks (ANN) perform certain tasks if the structure of the network is modified accordingly for the purpose it has been used. The three most robust networks were taken and comparatively analyzed for their performance to choose the appropriate network. The single-layer feed-forward neural network with the back propagation algorithm is chosen as one of the well-suited networks after comparing the results. Initially, certain synthetic data sets of all three-layer curves have been taken for training the network, and the network is validated by the field datasets collected from Tuticorin Coastal Region (78°7′30"E and 8°48′45"N), Tamil Nadu, India. The interpretation has been done successfully using the corresponding learning algorithm in the present study. With proper training of back propagation networks, it tends to give the resistivity and thickness of the subsurface layer model of the field resistivity data concerning the synthetic data trained earlier in the appropriate network. The network is trained with more Vertical Electrical Sounding (VES) data, and this trained network is demonstrated by the field data. Groundwater table depth also has been modeled.

Multi-sensors data fusion system for wireless sensors networks of factory monitoring via BPN technology

This study attempts to apply a back-propagation network (BPN) for multi-sensors data fusion in a wireless sensor networks (WSNs) system with a node-sink mobile network structure. This investigate is to finish the factory monitoring at environment monitoring services (EMS). These practice wireless sensor network circuits include temperature, humidity, ultraviolet, and illumination four variable measurement components. These data fields of each sensor nodes contain the properties and specifications of that signal process rules, the remote engineers can manage the multi-sensors data fusion using the browser, and the WSNs system then classification the data fusion database via the Internet and mobile network. Moreover, The BPN training approach is significant that improves data fusion system in accuracy and classification with parallel computing for data fusion efficiency. The final phase of the classification fusion system applies parallel BPN technology to process data fusion, and can solve the problem of various signals states. This study is considered implemented on the Yang-Fen Automation Electrical Engineering Company as a case study. The experiment is continued for six months, and engineers are also used to operating the web-based classification fusion system. Therefore, the cooperative plan described above is analyzed and discussed here. Finally, these papers propose the tradition methods compare with the innovative BPN methods.

A Generalized Neural Network Model of Refrigerant Mass Flow Through Adiabatic Capillary Tubes and Short Tube Orifices

Adiabatic capillary tubes and short tube orifices are widely used as expansive devices in refrigeration, residential air conditioners, and heat pumps. In this paper, a generalized neural network has been developed to predict the mass flow rate through adiabatic capillary tubes and short tube orifices. The input/output parameters of the neural network are dimensionless and derived from the homogeneous equilibrium flow model. Three-layer backpropagation (BP) neural network is selected as a universal function approximator. Log sigmoid and pure linear transfer functions are used in the hidden layer and the output layer, respectively. The experimental data of R12, R22, R134a, R404A, R407C, R410A, and R600a from the open literature covering capillary and short tube geometries, subcooled and two-phase inlet conditions, are collected for the BP network training and testing. Compared with experimental data, the overall average and standard deviations of the proposed neural network are 0.75% and 8.27% of the measured mass flow rates, respectively.

Process control of a laboratory combustor using artificial neural networks

Active process control of nitric oxide (NO) emissions from a two-stage combustor burning ethylene (doped with ammonia) in air is demonstrated using two clusters of feed-forward multi-layer-perceptron neural networks. Steady-state experimental data are used for static back-propagation network training. The first cluster consists of two neural networks. The first network identifies the amount of ammonia in the feed. Based on that value and the NO set point, the second network adjusts the first-stage fuel equivalence ratio φ 1. The second cluster also consists of two neural networks. It is the process emulator and serves as a Smith time-delay compensator. A visual basic interface control program accepts incoming concentration and flow rate data signals, accesses the neural networks, and outputs feedback control signals to selected electronic valves. Closed-loop results are compared to the open-loop results. The neural network-based controller successfully brought NO emissions into control after a step disturbance in the feed composition stream (ammonia dopant). The neural network-based controller shows a superior performance over the conventional proportional–integral-derivative controller.

Convergence index for BPN training

The Back Propagation Network (BPN) is one of the most widely used neural networks. It has a distinct training phase and then it is put to use. It is observed that certain training sets would get trained and certain others would not get trained in a BPN. This is an inherent property of the training set and the BPN architecture. This paper proposes a new training index that may be evaluated after a certain number of training epochs and would indicate the ability of the BPN to train for the existing topology and training set. The index is evaluated for several logic gates and the results are presented.

Concentration determination of gas by organic thin film sensor and back propagation network

In this work, vacuum deposited thin films of PbPc, NiPc, VOPc, TiOPc and CoPc were employed as gas sensor to detect NO 2 and NO. Data collected from sensor responses were used to train a back-propagation network (BPN) for identifying the gas species and quantifying its concentration. The results show that among the metallophthalocyanines tested, PbPc and NiPc have better sensing characteristics towards NO 2 and NO. In BPN training, maximum error occurs for data collected by the TiOPc sensor, and minimum error occurs for array of PbPc and NiPc sensors. In the concentration prediction of NO or NO 2, the maximum predicted error is 6.94%. When Two-Stage BPN or Single-Stage BPN was use to identify and quantify a single gas (NO 2 or NO), the accuracy of recognition approaches 100% and the maximum error for concentration prediction is 7.45%.

A neural network approach to elevated temperature creep–fatigue life prediction

A new approach using a back-propagation neural network for life prediction was developed and demonstrated for predicting the elevated temperature (0.7–0.8 T m) creep–fatigue behavior of Ni-base alloy INCONEL 690. The neural network was trained with five extrinsic parameters, characterized via a 2 5–1 fractional factorial design methodology, and an intrinsic parameter (final grain size). The back-propagation network training error, prediction error and training time were minimized using a second fractional factorial design. Life prediction accuracy using only 11 training sets, few training iterations (<20,000) and a simple network architecture (6-2-2-1) showed significant improvement, for sets not previously used for training, when compared to Coffin–Manson, linear life fraction and hysteresis energy prediction techniques.

Training MLPs layer by layer using an objective function for internal representations

A new constructive algorithm for designing and training multilayer perceptrons is proposed. This algorithm involves the optimization of an objective function for internal representations, which does not require any computation of the network's outputs. Coupled with a strategy for recruiting units during the learning process, this concept provides a scheme for training a multilayer network layer by layer, until self-encoding of the pattern categories is achieved in the final, highest-level representations. Two objective functions are proposed. For discrimination problems, recent experimental and theoretical results concerning back-propagation training of networks with one hidden layer and linear outputs suggest the introduction of a particular measure of class separability. For problems involving the approximation of a continuous function, we show that the minimization of the mean squared output error can be achieved by maximizing a statistical measure (the sample coefficient of multiple determination) in the last hidden layer. Simulations are used to illustrate the process of network construction, and to demonstrate the improvements brought by this approach over back-propagation in terms of performance and robustness.

Accelerated training of backpropagation networks by using adaptive momentum step

Considerable research into the training of neural networks by the backpropagation technique has been undertaken in recent years. Introduction of a momentum term into the training equation can accelerate the training process. In the Letter a new momentum step and a scheme for dynamically selecting the momentum rate are described. It is shown that these give improved acceleration of training and strong global convergence characteristics. Results are presented for four benchmark training tasks.