Improved Adaptive Learning Algorithm forConstructive Neural Networks

Abstract

Artificial Neural Networks (ANN) are biologicallyinspired models of computation. They are networks with elementary processing units called neurons massively interconnected by trainable connections called weights. ANN algorithms involve training the connection weights through a systematic procedure. Learning in ANN refers to searching for an optimal network topology and weights so as to accomplish a given goal-dictated task. Learning can be categorized as Supervised or Unsupervised. The Supervised learning refers to the presence of inputs and desired outputs for training. Unsupervised learning refers to determining the output categories or correlation inherent in inputs for training. ANNs are capable of generalization, adaptation and performing computation in parallel resembling the human brain. A layer of neurons in ANN have common functionality. The choice of number of neurons in input, hidden and output layers and their functionality depend on the learning algorithm and task needed. The numbers of input neurons are usually same as the total number of attributes of the training patterns, and, the numbers of output neurons are same as the number of categories sought. Determination of number of hidden layers and neurons is dependent on the inherent complexity of the task and number of training examples available. A number of ANN architectures and algorithms have been proposed by researchers, of which Constructive Neural Networks (CoNN) offer an attractive framework for pattern classifications problems. CoNN algorithms provide an optimal way to determine the architecture of a Multi Layer Perceptron network trainable with learning algorithms to determine appropriate weights. These algorithms initially start with small network (usually a single neuron) and dynamically allow the network to grow by adding and training neurons as needed until a satisfactory solution is found[1].