Extreme learning machine based on improved genetic algorithm

Abstract

This paper puts forward a novel algorithm called extreme learning machine (ELM)which is optimized by improved genetic algorithm(IGA), and points out the weaknesses of ELM. The input weights and thresholds randomly generated by ELM are optimized by IGA. After it, ELM can get the more effective input weights and thresholds and be better applied in function approximation. The results of simulation shows that the optimized algorithm has a high approximation accuracy and faster convergence speed. Introduction The principle of classic function approximation has the rigorous theoretical analysis and mature system, but it results in many algorithms with some common faults such as a large amountof calculation, poorer adaptability, higher demand for the model and data, strong dependence,etc. The superiority of neural network applied in function approximation can be shown in many cases. For example , the mode characteristics of the data is not very clear and the data is fuzzy, nonlinear or with more noise, etc.The research of function approximation by neural network has a very good theoretical value[1]. Huang et al., in 2004, put forward a novel feed-forward neural network[2]called extreme learning machine(ELM). In ELM, the input weights and thresholds in the hidden neurons are randomly assigned, and the output weight matrix is calculated by Moore-Penrose(MP). Compared to the traditional feed-forward neural network, ELM has the features of faster learning speed, higher precision, simple parameter adjustment and so on. However, although the randomly generated input weights and thresholds avoid the problems of local optimal value and over-fitting, in ELM it results in the failure of some nodes in the hidden layer. Therefore, the purpose of this paper is to use IGA to optimize ELM and get the more effective input weights and thresholds which can improve the effectiveness of nodes in the hidden layer in ELM and then improve the function approximation accuracy of ELM as a whole. The principle of function approximation Function approximation[3][4] is an important part of function theory and important in the numerical calculation The problem of how to approximate the function is a basic problem of function approximation. Using simple function ( ) g x to replace function ( ) f x approximatively is one of the most basic concepts and methods in computational mathematics. Approximate replace is also known as approximation. Function ( ) f x is described as approximated function and function ( ) g x as approximate function. The difference between the two is ( ) ( ) ( ) R x f x g x = − (1) 5th International Conference on Information Engineering for Mechanics and Materials (ICIMM 2015) © 2015. The authors Published by Atlantis Press 199 It is called as the error or remainder of approximation. The problem solved by function approximation is how to solve the approximate equation with the small amount of calculation under the given precision. It is described as: for functions A , a class of function ( ) f x is required to find a function ( ) g x in functions B ( B A ⊂ ) ,which is simple and easy to calculate, to make the difference between ( ) g x and ( ) f x reach the minimum in a way. Generally speaking, the most common functions A are the continuous functions in the interval of [ ] a,b . The most commonly used functions B are rational fraction, trigonometric polynomial, algebraic polynomial, etc. Extreme learning machine ELM [5] [6] is a novel algorithm for training single-layer feed-forward neural network [4]. Supposing having N learning samples, its set is described as ( ) { } 1 2 , | 1, 2, , ; ; d d n n n n N x y n N x R y R = = ∈ ∈  . If its structure has K nodes in the hidden layer, where ( ) g x is the activation function, the input and output of the ELM network is described as: ( ) 1 K q k k q k k T g x b b ω = = ∗ + ∑ (2) ( ) 1 1 X g X e = + (3) Where 1 1 2 , , , T k k k kd ω ω ω ω   =    or 2 1 2 , , , T k k k kd b b b b   =    represents the connection weights between the input layer or the output layer and k th node in the hidden layer ; k b is the bias of the k th hidden node. Formula (2) can be written as the following form: H T b = (4) Where H represents ( ) ( )