A constructive algorithm for neural networks inspired on decision trees and evolutionary algorithms

Abstract

Inspired on decision trees and evolutionary algorithms, this paper proposes a learning algorithm of constructive neural networks that relies on three principles: to layout the neurons in a tree-like structure; to train each neuron individually; and, to optimize all the weights using an evolutionary approach. This way, it is expected to advance in two main questions concerning multilayer perceptrons (MLPs): how to determine the network architecture and how to build models that are more comprehensible. Based on the normalized information gain of each attribute, the algorithm builds the network architecture. In the process, it automatically creates a set of training examples for each individual neuron and executes single-cell learning. Once the network is created and trained, particle swarm optimization is utilized to evolve the connections of the network. Five metrics were utilized to validate the method when compared to decision trees and MLPs: accuracy, sensitivity, specificity, precision and comprehensibility. The experiments were executed in thirteen different databases and the results suggest that the proposed algorithm can generate neural networks with good classification performance and more comprehensible.