Two Variants of the Constructive Neural Network Tiling Algorithm

Abstract

Unlike conventional neural network (NN) algorithms that require the definition of the NN architecture before learning starts, constructive neural network (CoNN) algorithms enable the NN architecture to be constructed along with the learning process. CoNN algorithms are very dependent on the TLU training algorithm they employ. Generally in their original proposal CoNN algorithms use a Perceptron-based algorithm for training each individual node added to the network during the learning process. This paper proposes two hybrid variants of the CoNN algorithm known as Tiling, referred to as Tiling_V1 and Tiling_V2. The two variants differ from the original Tiling in respect to the algorithm they use for training individual TLUs added to the NN. The master neuron in each hidden layer constructed by Tiling_V1 can be trained either by PRM (Pocket with Ratchet Modification) or BCPMin (Barycentric Correction Procedure) while the auxiliary neurons are always trained using BCPMin. In Tiling_V2 the same algorithm used to train the master neuron of each hidden layer is also used to train the auxiliary neurons. Both variants as well as the original Tiling (using PRM or BCPMin) have been used in learning tasks involving 7 knowledge domains. In 6 out of 7 domains results obtained with one of the variants are in the top two best results.