Efficient Ensemble Broad Learning System Based on Dropout and DropConnect

Abstract

Broad Learning System is an emerged efficient algorithm for training single hidden layer feedforward neural network (SLFN) with fast speed and good generalization ability. However, it is very difficult to determine the appropriate broad learning system structure and broad learning system may perform overfitting due to the dependence between nodes in processing fully connected network. In order to deal with these problems, efficient ensemble broad learning system based on Dropout and DropConnect are proposed in this paper. The proposed Dropout Ensemble Broad Learning System randomly discards hidden nodes to improve diversity between individuals and reduce the synergy between nodes to improve prediction stability. The DropConnect Ensemble Broad Learning System randomly drops connect weights to generate more complementary models by adding input attribute disturbance. The experimental results and statistical analysis on UCI data sets confirm that the proposed method can solve the problem of model overfitting caused by the strong dependence between the nodes of ensemble broad learning system and also show that the proposed approaches outperform the original BLS on prediction stability and classification accuracy.