Sharing training patterns in neural network ensembles

Abstract

The need for the design of complex and incremental training algorithms in multiple neural network systems has motivated us to study combining methods from the cooperation perspective. One way of achieving effective cooperation is through sharing resources such as information and components. The degree and method by which multiple classifier systems share training resources can be a measure of cooperation. Despite the growing number of interests in data modification techniques, such as bagging and k-fold cross-validation, there is no guidance for whether sharing or not sharing training patterns results in higher accuracy and under what conditions. We implemented several partitioning techniques and examined the effect of sharing training patterns by varying the size of overlap between 0-100% of the size of training subsets. Under most conditions studied, multinet systems showed improvement over the presence of larger overlap subsets.