Deep Multi-Layer Neural Network with Variable-Depth Output

Abstract

In this study, a deep multi-layer neural network (DMLNN) with variable-depth output (VDO), called VDO-DMLNN, is proposed for classification. Unlike the traditional DMLNN, for which a user must define the network architecture in advance, VDO-DMLNN is produced from the top–down, layer by layer, until the classification error rate of VDO-DMLNN no longer decreases. The user thus does not need to define the depth of VDO-DMLNN in advance. The combination of the genetic algorithm (GA) and the self-organizing feature map (SOFM), called GA–SOFM, is proposed to automatically generate the weights and proper number of nodes for each layer in VDO-DMLNN. In addition, the output nodes can be at different levels in VDO-DMLNN rather than all being at the last layer, as in the traditional DMLNN. Thus, the average of computing time required for the recognition of an input sample in VDO-DMLNN is less than that in traditional DMLNN when they have the same classification error rate. Finally, VDO-DMLNN is compared with some state-of-the-art neural networks in the experiments.