Resilient Backpropagation Algorithm Research Articles

Neuro-Prony and Taguchi's Methodology-Based Adaptive Autoreclosure Scheme for Electric Transmission Systems

This paper presents a novel intelligent autoreclosure technique to discriminate temporary faults from permanent faults, and to accurately determine fault extinction time. A variety of fault simulations is carried out on a specified transmission line on the standard IEEE 9-bus electric power system by using MATLAB/SimPowerSytems. Prony analysis is employed to extract data features from each simulated fault. The fault identification prior to reclosing is accomplished by an artificial neural network trained by Levenberg Marquardt and resilient backpropagation algorithms, which are developed by using MATLAB. Some important parameters which strongly affect the entire training process are fine-tuned to their corresponding best values with the help of Taguchi's method. Test results show the robustness and efficacy of the proposed autoreclosure scheme.

Artificial Neural Network based Defect Detection of Welds in TOFD Technique

Time of Flight Diffraction Technique is one of the NDE methods, used in weld inspection to identify the weld defects. The classification of defects using the TOFD technique depends on the knowledge and experience of the operator. The classification reliability of defects detected by this technique can be improved by applying the Artificial Neural Network. In this work, four austenitic stainless steel weldments with defects viz, Lack of Fusion, Lack of Penetration, Slag, Porosity and one with out any Defect were fabricated. TOFD experiment is conducted on these weldments. Discrete wavelet transform based denoising methods were applied to denoise the resultant A scan signals. Time scale features are extracted from the denoised signals. A multi layer feed forward network with Resilient Back Propagation algorithm has been applied for classification of the signals. The number of hidden layers in the network are increased from 0 to 6. Various performance functions are also employed to achieve a better classification efficiency. The results are promising to proceed the automatic defect classification by TOFD technique.

Analysis of natural convection from a column of cold horizontal cylinders using Artificial Neural Network

Artificial Neural Networks (ANNs) offer an alternative way to tackle complex problems. They can learn from the examples and once trained can perform predictions and generalizations at high speed. They are particularly useful in behavior or system identification. According to the above advantages of ANN in the present paper ANN is used to predict natural convection heat transfer and fluid flow from a column of cold horizontal circular cylinders having uniform surface temperature. Governing equations are solved in a few specified cases by finite volume method to generate the database for training the ANN in the range of Rayleigh numbers of 105–108 and a range of cylinder spacing of 0.5, 1.0, and 1.5 diameters, thereafter a Multi-Layer Perceptron (MLP) network is used to capture the behavior of flow and temperature fields and then generalized this behavior to predict the flow and temperature fields for any other Rayleigh numbers. Different training algorithms are used and it is found that the resilient back-propagation algorithm is the best algorithm regarding the faster training procedure. To validate the accuracy of the trained network, comparison is performed among the ANN and available CFD results. It is observed that ANN can be used more efficiently to determine cold plume and thermal field in lesser computational time. Based on the generalized results from the ANN new correlations are developed to estimate natural convection from a column of cold horizontal cylinders with respect to a single horizontal cylinder.

Control chart pattern recognition using neural networks and efficient features: a comparative study

Control chart patterns (CCPs) are important statistical process control tools for determining whether a process is run in its intended mode or in the presence of unnatural patterns. This paper investigates the design of an efficient system for recognition of the control chart patterns. This system includes two main modules: a feature extraction module and a classification module. The feature extraction module extracts a combination set of the shape features and statistical features. In the classifier module, several multilayer perceptron neural networks with different number of layers and training algorithms are investigated. The performances of the networks for speed of convergence and accuracy classification are evaluated for six classes of the CCPs. Among the different training algorithms, the resilient back-propagation (RP) algorithm represented the best convergence rate and the Levenberg---Marquardt (LM) algorithm achieved the best overall detection accuracy.

Investigation of Linear and Nonlinear Chemometrics Methods in Modeling of Retention Time of Phenol Derivatives Based on Molecular Descriptors

Quantitative Structure-Retention Relationship (QSRR) studies were performed for predicting the gas chromatographic retention times of phenol derivatives on Rtx-200 stationary phase with medium polarity. First, a number of descriptors were calculated using Hyperchem and Mopac softwares. Partial least squares (PLS) and multiple linear regressions (MLR) were used as linear modeling methods. Then, selected descriptors using the MLR model were used as inputs for artificial neural networks with different weight update functions including the Levenberg-Marquardt back propagation algorithm (LM-ANN), the resilient back propagation algorithm (RP-ANN), and the variable learning rate algorithm (GDX-ANN). The stability and the validity of the models were tested by cross-validation, Y-randomization, and external validation set. Moreover, the mean effect of the descriptors indicates that molecular weight (MW) is the most important factor affecting the retention behavior of molecules.

A New Intelligent Autoreclosing Scheme Using Artificial Neural Network and Taguchi's Methodology

This paper presents a novel intelligent autoreclosure technique to discriminate temporary faults from permanent faults and accurately determine fault extinction time. A variety of fault simulations are carried out on a specified transmission line on the standard IEEE 9-bus electric power system using MATLAB/SimPowerSytems. FFT and Prony analysis methods are employed to extract data features from each simulated fault. The fault identification prior to reclosing is accomplished by an artificial neural network trained by standard Error Back-Propagation, Levenberg Marquardt, and Resilient Back-Propagation algorithms which are developed using MATLAB. Some important parameters which strongly affect the entire training process are fine tuned with Taguchi's method to their corresponding best values. The robustness of the developed ANN identifier is verified by testing it with the data patterns which consists of high impedance faults obtained from IEEE 14-bus benchmark system. Test results show the efficacy of the proposed AR scheme.

Artificial neural networks workflow and its application in the petroleum industry

We develop a neural network workflow, which provides a systematic approach for tackling various problems in petroleum engineering. The workflow covers several design issues for constructing neural network models, especially in terms of developing the network structure. We apply the model to predict water saturation in an oilfield in Oman. Water saturation can be accurately obtained from data measured from cores removed from the oil field, but this information is limited to a few wells. Wireline log data are more abundantly available in most wells, and they provide valuable, but indirect, information about rock properties. A three-layered neural network model with five hidden neurons and a resilient back-propagation algorithm is found to be the best design for the saturation prediction. The input variables to the model are density, neutron, resistivity, and photo-electric wireline logs, and the model is trained using core water saturation. The model is able to predict the saturation directly from wireline logs with a correlation coefficient (r) of 0.91 and an error of 2.5 saturation units on the testing data.

Fault diagnosis and classification based on wavelet transform and neural network

In this paper we present a novel method in fault recognition and classification in Nuclear Power Plant (NPP) using wavelet transform based Artificial Neural Network (ANN). We first simulate 10 design basis accidents (DBA) of a VVER-1000 using 15 input parameters with employing a Multilayer Perceptron (MLP) Neural Network with Resilient Backpropagation (RBP) algorithm. Afterwards we present the application of wavelet transform for its temporal shift property and multiresolution analysis characteristics to reduce disturbing perturbations in input training set data. Simulation of Artificial Neural Network and wavelet transform was performed using MATLAB software. The results show an enhanced accuracy and speed in fault recognition and high degree of robustness.

Blending PSO and ANN for Optimal Design of FSS Filters With Koch Island Patch Elements

This work presents a new fast and accurate electromagnetic (EM) optimization technique blending the Particle Swarm Optimization (PSO) algorithm and a Multilayer Perceptrons (MLP) Artificial Neural Network (ANN). The proposed technique was applied for optimal design of Koch fractal Frequency Selective Surface (FSS) with desired stop-band filter specification. Initially, a full-wave parametric analysis was carried out for accurate EM-characterization of FSS filters. From obtained EM-dataset, a MLP network was trained with the first-order Resilient Backpropagation (RPROP) algorithm. The developed MLP model for FSS synthesis was used for efficient evaluation of cost function in PSO iterations. The advantages in the optimal design of FSS through the PSO-ANN technique were discussed in terms of convergence and computational cost. Two optimized FSS prototypes were built and measured. The accuracy of the proposed optimization technique was verified through the excellent agreement obtained by means of comparisons between theoretical and experimental results.

Affectively intelligent and adaptive car interfaces

In this article, we describe a new approach to enhance driving safety via multi-media technologies by recognizing and adapting to drivers’ emotions with multi-modal intelligent car interfaces. The primary objective of this research was to build an affectively intelligent and adaptive car interface that could facilitate a natural communication with its user (i.e., the driver). This objective was achieved by recognizing drivers’ affective states (i.e., emotions experienced by the drivers) and by responding to those emotions by adapting to the current situation via an affective user model created for each individual driver. A controlled experiment was designed and conducted in a virtual reality environment to collect physiological data signals (galvanic skin response, heart rate, and temperature) from participants who experienced driving-related emotions and states (neutrality, panic/fear, frustration/anger, and boredom/sleepiness). k-Nearest Neighbor (KNN), Marquardt-Backpropagation (MBP), and Resilient Backpropagation (RBP) Algorithms were implemented to analyze the collected data signals and to find unique physiological patterns of emotions. RBP was the best classifier of these three emotions with 82.6% accuracy, followed by MBP with 73.26% and by KNN with 65.33%. Adaptation of the interface was designed to provide multi-modal feedback to the users about their current affective state and to respond to users’ negative emotional states in order to decrease the possible negative impacts of those emotions. Bayesian Belief Networks formalization was employed to develop the user model to enable the intelligent system to appropriately adapt to the current context and situation by considering user-dependent factors, such as personality traits and preferences.

A new approach for deriving temperature and salinity fields in the Indian Ocean using artificial neural networks

This work reports a new methodology for deriving monthly averages of temperature (T) and salinity (S) fields for the Indian Ocean based on the use of an artificial neural network (ANN). Investigation and analysis were performed for this region with two distinct datasets: (1) monthly climatological data for T and S fields (in 1° × 1° grid boxes) at standard depth levels of the World Ocean Atlas 1994 (WOA94), and; (2) heterogeneous randomly distributed in situ ARGO, ocean station data (OSD) and profiling (PFL) floats. A further numerical experiment was conducted with these two distinct datasets to train the neural network model. Nonlinear regression mapping utilizing a multilayer perceptron (MLP) is employed to tackle nonlinearity in the data. This study reveals that a feed-forward type of network with a resilient backpropagation algorithm is best suited for deriving T and S fields; this is demonstrated by independently using WOA94 and in situ data, which thus tests the robustness of the ANN model. The suppleness of the T and S fields derived from the ANN model provides the freedom to generate a new grid at any desired level with a high degree of accuracy. Comprehensive training, testing and validation exercises were performed to demonstrate the robustness of the model and the consistency of the derived fields. The study points out that the parameters derived from the ANN model using scattered, inhomogeneous in situ data show very good agreement with state-of-the-art WOA climatological data. Using this approach, improvements in ocean climatology can be expected to occur in a synergistic manner with in situ observations. Our investigation of the Indian Ocean reveals that this approach can be extended to model global oceans.

Application of artificial intelligence in Fault Detection and Isolation of uncertain parameter systems

In this article, two original intelligent methods for Fault Detection and Isolation (FDI) affecting sensors and actuators in case of bond graph modelled uncertain parameter systems are proposed. First, a fuzzy approach based on residual processing is proposed for FDI offline, binary approach and fuzzy approach are compared through an illustrative example. Secondly, a multilayer perceptron trained with resilient backpropagation algorithm is proposed for FDI online.

Identification and Prediction of Internet Traffic Using Artificial Neural Networks

This paper presents the development of an artificial neural network (ANN) model based on the multi-layer perceptron (MLP) for analyzing internet traffic data over IP networks. We applied the ANN to analyze a time series of measured data for network response evaluation. For this reason, we used the input and output data of an internet traffic over IP networks to identify the ANN model, and we studied the performance of some training algorithms used to estimate the weights of the neuron. The comparison between some training algorithms demonstrates the efficiency and the accu-racy of the Levenberg-Marquardt (LM) and the Resilient back propagation (Rp) algorithms in term of statistical crite-ria. Consequently, the obtained results show that the developed models, using the LM and the Rp algorithms, can successfully be used for analyzing internet traffic over IP networks, and can be applied as an excellent and fundamental tool for the management of the internet traffic at different times.

PID-Like Neural Network Nonlinear Adaptive Control for Uncertain Multivariable Motion Control Systems

A mix locally recurrent neural network was used to create a proportional-integral-derivative (PID)-like neural network nonlinear adaptive controller for uncertain multivariable single-input/multi-output system. It is composed of a neural network with no more than three neural nodes in hidden layer, and there are included an activation feedback and an output feedback, respectively, in a hidden layer. Such a special structure makes the exterior feature of the neural network controller able to become a P, PI, PD, or PID controller as needed. The closed-loop error between directly measured output and expected value of the system is chosen to be the input of the controller. Only a group of initial weights values, which can run the controlled closed-loop system stably, are required to be determined. The proposed controller can update weights of the neural network online according to errors caused by uncertain factors of system such as modeling error and external disturbance, based on stable learning rate. The resilient back-propagation algorithm with sign instead of the gradient is used to update the network weights. The basic ideas, techniques, and system stability proof were presented in detail. Finally, actual experiments both of single and double inverted pendulums were implemented, and the comparison of effectiveness between the proposed controller and the linear optimal regulator were given.

Detection of premature ventricular contractions using MLP neural networks: A comparative study

This paper proposes a three stage technique for detection of premature ventricular contraction (PVC) from normal beats and other heart diseases. This method includes a denoising module, a feature extraction module and a classification module. In the first module we investigate the application of stationary wavelet transform (SWT) for noise reduction of the electrocardiogram (ECG) signals. The feature extraction module extracts 10 ECG morphological features and one timing interval feature. Then a number of multilayer perceptron (MLP) neural networks with different number of layers and nine training algorithms are designed. The performances of the networks for speed of convergence and accuracy classifications are evaluated for seven files from the MIT–BIH arrhythmia database. Among the different training algorithms, the resilient back-propagation (RP) algorithm illustrated the best convergence rate and the Levenberg–Marquardt (LM) algorithm achieved the best overall detection accuracy.

Estimation of Medicine Amount Used Anesthesia by an Artificial Neural Network

In this study, Elman's recurrent neural networks using Resilient Back Propagation (RP) algorithm and feed-forward neural networks using adaptive learning rate algorithm (gdx) have been compared in order to determine the depth of anesthesia in the continuation stage of anesthesia and to estimate the amount of medicine to be applied at that moment. EEG data have been recorded by being sampled once in every 2ms. From 30 patients, 57 distinct EEG recordings have been collected prior to during anaesthesia of different levels. The applied artificial neural network is composed of three layers, namely the input layer, the middle layer and the output layer. The nonlinear activation function sigmoid (sigmoid function) has been used in the hidden layer and the output layer. Prediction has been made by means of ANN. Training and testing the ANN have been used previous anaesthesia amount, total power/normal power and total power/previous. When Elman Resilient BP and feed-forward network are compared, it is observed that resilient back propagation algorithm has generated values which are quite close to the applied anesthesia amount compared to gdx which is an adaptive learning algorithm. The system has been able to correctly purposeful responses in average accuracy of 95% of the cases. This method is also computationally fast and acceptable real-time clinical performance has been obtained.

ANN-based energy reconstruction procedure for TACTIC [formula omitted]-ray telescope and its comparison with other conventional methods

The energy estimation procedures employed by different groups, for determining the energy of the primary γ-ray using a single atmospheric Cherenkov imaging telescope, include methods like polynomial fitting in SIZE and DISTANCE, general least square fitting and look-up table based interpolation. A novel energy reconstruction procedure, based on the utilization of Artificial Neural Network (ANN), has been developed for the TACTIC telescope. The procedure uses a 3:30:1 ANN configuration with resilient backpropagation algorithm to estimate the energy of a γ-ray like event on the basis of its image SIZE, DISTANCE and zenith angle. The new ANN-based energy reconstruction method, apart from yielding an energy resolution of ∼26%, which is comparable to that of other single imaging telescopes, has the added advantage that it considers zenith angle dependence as well. Details of the ANN-based energy estimation procedure along with its comparative performance with other conventional energy reconstruction methods are presented in the paper and the results indicate that amongst all the methods considered in this work, ANN method yields the best results. The performance of the ANN-based energy reconstruction has also been validated by determining the energy spectrum of the Crab Nebula in the energy range 1–16TeV, as measured by the TACTIC telescope.

Typing SNP based on the near-infrared spectroscopy and artificial neural network

Based on the near-infrared spectra (NIRS) of the measured samples as the discriminant variables of their genotypes, the genotype discriminant model of SNP has been established by using back-propagation artificial neural network (BP-ANN). Taking a SNP (857G > A) of N-acetyltransferase 2 (NAT2) as an example, DNA fragments containing the SNP site were amplified by the PCR method based on a pair of primers to obtain the three-genotype (GG, AA, and GA) modeling samples. The NIRS-s of the amplified samples were directly measured in transmission by using quartz cell. Based on the sample spectra measured, the two BP-ANN-s were combined to obtain the stronger ability of the three-genotype classification. One of them was established to compress the measured NIRS variables by using the resilient back-propagation algorithm, and another network established by Levenberg–Marquardt algorithm according to the compressed NIRS-s was used as the discriminant model of the three-genotype classification. For the established model, the root mean square error for the training and the prediction sample sets were 0.0135 and 0.0132, respectively. Certainly, this model could rightly predict the three genotypes (i.e. the accuracy of prediction samples was up to100%) and had a good robust for the prediction of unknown samples. Since the three genotypes of SNP could be directly determined by using the NIRS-s without any preprocessing for the analyzed samples after PCR, this method is simple, rapid and low-cost.

A committee neural network for prediction of normalized oil content from well log data: An example from South Pars Gas Field, Persian Gulf

Normalized oil content (NOC) is an important geochemical factor for identifying potential pay zones in hydrocarbon source rocks. The present study proposes an optimal and improved model to make a quantitative and qualitative correlation between NOC and well log responses by integration of neural network training algorithms and the committee machine concept. This committee machine with training algorithms (CMTA) combines Levenberg–Marquardt (LM), Bayesian regularization (BR), gradient descent (GD), one step secant (OSS), and resilient back-propagation (RP) algorithms. Each of these algorithms has a weight factor showing its contribution in overall prediction. The optimal combination of the weights is derived by a genetic algorithm. The method is illustrated using a case study. For this purpose, 231 data composed of well log data and measured NOC from three wells of South Pars Gas Field were clustered into 194 modeling dataset and 37 testing samples for evaluating reliability of the models. The results of this study show that the CMTA provides more reliable and acceptable results than each of the individual neural networks differing in training algorithms. Also CMTA can accurately identify production pay zones (PPZs) from well logs.

Enhanced Neural Network Based Fault Detection of a VVER Nuclear Power Plant With the Aid of Principal Component Analysis

This paper presents a neural network based fault diagnosing approach which allows dynamic fault identification. The method utilizes the principal component analysis (PCA) technique to dramatically reduce the problem dimension. Such a dimension reduction approach leads to faster diagnosing and allows a better graphical presentation of the results. To show the effectiveness of the proposed approach, two methodologies are used to train the neural network (NN). At first, a training matrix composed of 15 variables is used to train a multilayer perceptron neural network (MLP) with resilient backpropagation (RP) algorithm. Employing the proposed method, a more accurate and simpler network is designed where the input size is reduced from 15 to 6 variables for training the NN. In short, the application of PCA highly reduces the network topology and allows employing more efficient training algorithms. The developed networks use, as input, a short set (in a moving temporal window (MTW)) of recent measurements of each variable avoiding the necessity of using starting events. The accuracy, generalization ability and reliability of the designed networks are verified using 10 simulated events data from a VVER-1000 simulator. Noise is added to the data to evaluate robustness of the method, and the method again shows to be effective and powerful.