Studies on Optimization Algorithms for Some Artificial Neural Networks Based on Genetic Algorithm (GA)

Abstract

Artificial Neural Networks (ANNs) are the nonlinear and adaptive information processing systems which are combined by numerous processing units, with the characteristics of self-adapting, self-organizing and real- time learning, and play an important in pattern recognition, machine learning and data mining. But we've encountered many problems, such as the selection of the structure and the parameters of the networks, the selection of the learning samples, the selection of the initial values, the convergence of the learning algorithms and so on. Genetic algorithms (GA) is a kind of random search algorithm, on one hand, it simulates the nature selection and evolution, on the other, it has the advantages of good global search abilities and learning the approximate optimal solution without the gradient information of the error functions. In this paper, some optimization algorithms for ANNs with GA are studied. Firstly, an optimizing BP neural network is set up. It is using GA to optimize the connection weights of the neural network, and using GA to optimize both the connection weights and the architecture. Secondly, an optimizing RBF neural network is proposed. It used hybrid encoding method, that is, to encode the network by binary encoding and the weights by real encoding, the network architecture is self-adapted adjusted, the weights are learned, and the network is further adjusted by pseudo- inverse method or LMS method. Then they are used in real world classification tasks, and compared with the modified BP algorithm with adaptive learning rate. Experiments prove that the network got by this method has a better architecture and stronger classification ability, and the time of constructing the network artificially is saved. The algorithm is a self-adapted and intelligent learning algorithm.