Reliable classification using neural networks: a genetic algorithm and backpropagation comparison

Comparing performances of backpropagation and genetic algorithms in the data classification

The main objective of this work is to develop an analytical method for designing translation invariant operators via neural network training. A new neural network architecture, called modular morphological neural network (MMNN), is defined using a fundamental result of minimal representations for translation invariant set mappings via mathematical morphology, proposed by Banon and Barrera (1991). The MMNN general architecture is capable of learning both binary and gray-scale translation invariant operators. For its training, ideas of the backpropagation (BP) algorithm and the methodology proposed by Pessoa and Maragos (see Ph.D. thesis, Georgia Institute of Technology, 1997) for overcoming the problem of non-differentiability of the rank functions are used. An alternative MMNN training method via genetic algorithms (GA) is also developed, and a comparative analysis of BP vs. GA training in problems of image restoration and pattern recognition is provided. The MMNN structure can be viewed as a special case of the morphological/rank/linear neural network (MRL-NN), proposed by Pessoa and Maragos (1997), but with specific architecture and training rules. The effectiveness of the proposed BP and GA training algorithms for MMNNs is encouraging, offering alternative design tools for the important class of translation invariant operators.

Artificial neural networks (ANN) have been widely used in the field of data classification. Normally, training of neural network is applied with the traditional back propagation technique. As, this approach has various drawbacks, training of neural network is done with Particle Swarm Optimization (PSO). PSO has been widely used to solve the diverse kind of optimization problems. Population initialization performs a significant role in meta-heuristic algorithms. This paper describes a new initialization population approach Log Logistic termed as PSOLL-NN to create the initialization of the swarm. The proposed algorithm has been tested for weight optimization of feed forward neural network; and compared with back propagation Algorithm (BPA), standard PSO (PSO-NN), PSO initialized with Halton Sequence (PSOH-NN), Torus sequence (PSOT-NN) and Sobol sequence (PSOS-NN). The experimental results show that the proposed technique performed exceptionally better than the other traditional techniques. Moreover, the outcome of our work presents a foresight that how the proposed initialization technique can be used as an efficient alternative to standard training approaches for the data classification problems.

The backpropagation (BP) algorithm is a one of the most common algorithms used in the training of neural networks. The single offspring technique (SOFT algorithm) is a new technique (see Likartsis, A. et al., Proc. 9th Int. Conf. on Tools with Artificial Intelligence, p.32-6, 1997; Yao, X., Proc. IEEE, vol.87, p.1425-47, 1999) of applying the genetic algorithm in the training of neural networks which reduces the training time as compared with the backpropagation algorithm. We introduce a new technique. This technique is a hybrid SOFT-BP algorithm where the SOFT-algorithm is applied first to obtain an initially good weight vector. This vector is introduced to the backpropagation algorithm, which improves the precession of the weight vector to reach an acceptable error limit. The results show an acceptable improvement in the training speed for the hybrid technique as compared with the individual backpropagation or SOFT algorithm. We also study the success ratio (how many times the algorithm succeeds in finding a solution to the total number of trials) for the new hybrid algorithm. A recommended range of the switching error limit at which to switch from the SOFT algorithm to the BP algorithm is suggested.

Performance of the Levenberg–Marquardt neural network training method in electronic nose applications

PurposeThe paper aims to investigate the application of particle swarm optimisation and back propagation in weights optimisation and training of artificial neural networks within the mass appraisal industry and to compare the performance with standalone back propagation, genetic algorithm with back propagation and regression models.Design/methodology/approachThe study utilised linear regression modelling before the semi-log and log-log models with a sample of 3,242 single-family dwellings. This was followed by the hybrid systems in the selection of optimal attribute weights and training of the artificial neural networks. Also, the standalone back propagation algorithm was used for the network training, and finally, the performance of each model was evaluated using accuracy test statistics.FindingsThe study found that combining particle swarm optimisation with back propagation in global and local search for attribute weights enhances the predictive accuracy of artificial neural networks. This also enhances transparency of the process, because it shows relative importance of attributes.Research limitations/implicationsA robust assessment of the models’ predictive accuracy was inhibited by fewer accuracy test statistics found in the software. The research demonstrates the efficacy of combining two models in the assessment of property values.Originality/valueThis work demonstrated the practicability of combining particle swarm optimisation with back propagation algorithms in finding optimal weights and training of the artificial neural networks within the mass appraisal environment.

The Backpropagation Algorithm (BA) is the standard method for training multilayer Artificial Neural Networks (ANN), although it converges very slowly and can stop in a local minimum. We present a new method for neural network training using the BA inspired on constuctivism, an alphabetization method proposed by Emilia Ferreiro (1985) based on Piaget philosophy. Simulation results show that the proposed configuration usually obtained a lower final mean square error, when compared with the standard BA and with the BA with momentum factor.

There are a number of problems associated with training neural networks with backpropagation algorithm. The algorithm scales exponentially with increased complexity of the problem. It is very often trapped in local minima, and is not robust to changes of network parameters such as number of hidden layer neurons and learning rate. The use of genetic algorithms is a recent trend, which is good at exploring a large and complex search space, to overcome such problems. In this paper a genetic algorithm is proposed for training feedforward neural networks and its performances is investigated. The results are analyzed and compared with those obtained by the backpropagation algorithm.

This chapter aimed to evaluate heuristic approach performances for artificial neural networks (ANN) training. For this purpose, software that can perform ANN training application was developed using four different algorithms. First of all, training system was developed via back propagation (BP) algorithm, which is the most commonly used method for ANN training in the literature. Then, in order to compare the performance of this method with the heuristic methods, software that performs ANN training with genetic algorithm (GA), particle swarm optimization (PSO), and artificial immunity (AI) methods were designed. These designed software programs were tested on the breast cancer dataset taken from UCI (University of California, Irvine) database. When the test results were evaluated, it was seen that the most important difference between heuristic algorithms and BP algorithm occurred during the training period. When the training-test durations and performance rates were examined, the optimal algorithm for ANN training was determined as GA.

Chaos appears in many real and artificial systems. Inspired from the presence of chaos in human brain, we attempt to formulate neural network (NN) training method. The method uses a composite chaotic learning rate (CCLR) to train a neural network. CCLR generates a composite chaotic time series consisting of three different chaotic sources such as Mackey Glass, Logistic Map and Lorenz Attractor and a rescaled version of the series is used as learning rate (LR) during NN training. It gives two advantages - similarity with biological phenomena and possibility of jumping from local minima. In addition, the weight update may be accelerated in the local minimum zone due to chaotic variation of LR. CCLR is extensively tested on five real world benchmark classification problems such as diabetes, time series, horse, glass and soybean. The proposed CCLR outperforms the existing BP and BPCL in terms of generalization ability and also convergence rate.

On‐chip training of neural networks (NNs) is regarded as a promising training method for neuromorphic systems with analog synaptic devices. Herein, a novel on‐chip training method called direct gradient calculation (DGC) is proposed to substitute conventional backpropagation (BP). In this method, the gradients of a cost function with respect to the weights are calculated directly by sequentially applying a small temporal change to each weight and then measuring the change in cost value. DGC achieves a similar accuracy to that of BP while performing a handwritten digit classification task, validating its training feasibility. In particular, DGC can be applied to analog hardware‐based convolutional NNs (CNNs), which is considered to be a challenging task, enabling appropriate on‐chip training. A hybrid method is also proposed that efficiently combines DGC and BP for training CNNs, and the method achieves a similar accuracy to that of BP and DGC while enhancing the training speed. Furthermore, networks utilizing DGC maintain a higher level of accuracy than those using BP in the presence of variations in hardware (such as synaptic device conductance and neuron circuit component variations) while requiring fewer circuit components.

In this study, performances comparison to discriminate five mental states of five artificial neural network (ANN) training methods were investigated. Wavelet Packet Transform (WPT) was used for feature extraction of the relevant frequency bands from raw electroencephalogram (EEG) signals. The five ANN training methods used were (a) Gradient Descent Back Propagation (b) Levenberg-Marquardt (c) Resilient Back Propagation (d) Conjugate Learning Gradient Back Propagation and (e) Gradient Descent Back Propagation with movementum.

The traditional BP neural network training method processes the training dataset serially on one machine, so the efficiency is quite low. The massive data that need to be explored brings great challenge for BP neural network. The traditional serial training method of BP neural network will encounter many problems, such as costing too much time and insufficient memory to finish the training process. To solve these problems, this paper proposes a new parallel training method that is based on MapReduce and genetic algorithm, and the new training method is called MR-GAIBP (MapReduce based Genetic Algorithm Improved Back Propagation). MR-GAIBP includes two parts: MapReduce based BP algorithm and MapReduce based genetic algorithm. Genetic algorithm is first iterated for a few times to find appropriate initial weights of BP neural network, then BP algorithm is used to find the appropriate weights that meets the requirement. In the phase of BP algorithm, local iteration is used to speed up the convergence. Experiment results demonstrate that MR-GAIBP has faster convergence rate and higher accuracy compared with the previous MapReduce based algorithm proposed in other papers.

Surface deflection is a phenomenon that causes fine wrinkles on the outer surfaces of sheet metal and deteriorates product external appearance. It is quantitatively defined as the difference between the section curve of the sheet and the ideal curve. In this study, using neural networks, a prediction model for surface deflection according to material properties was constructed and combined with a genetic algorithm; the combination of the material properties was studied to predict the minimum surface deflection. Because of the limited number of simulation data, neural networks were developed using several sampling methods such as central composite design, Latin hypercube sampling, and random sampling. In the training of the neural networks, the optimal hyper-parameter of the neural network was found automatically using Latin hypercube sampling. In conclusion, for prediction of surface deflection in rectangular embossing, neural networks made by central composite design showed the best performance. In addition, it was confirmed that the procedure of combining automatic training of a neural network and the genetic algorithm accurately predicted the set of material properties that generates the minimum surface deflection. Also, the quantity of surface deflection predicted by the neural network was very close to that predicted by finite element analysis.

5 - Forecasting of electricity prices, demand, and renewable resources