Design of a modular neural network based on an improved soft subspace clustering algorithm

Abstract

Being a commonly used way for task decomposition in modular neural network (MNN), clustering analysis is employed to decompose the complex task into several simple subtasks for learning. Recent studies mainly focus on hard clustering, but the clusters might be not sufficiently represented when the cluster boundary is ambiguous, which may degenerate the learning performance of subnetworks in MNN. To solve this problem, we design a modular neural network based on an improved soft subspace clustering (IESSC-MNN) algorithm in this study. Firstly, we propose an improved soft subspace clustering algorithm for task decomposition in MNN, which divides the original space into several interactive feature subspaces and allocates a weight item to each subspace to describe the contribution of the subtasks at the same time. Secondly, each RBF subnetwork is adaptively constructed using a structure growing strategy, and all subnetworks learning the corresponding subtask in parallel. Finally, all subnetworks’ outputs are integrated by weighted summation using the contribution weight of subnetworks. The simulation results of the proposed model on five benchmark data and a practical dataset indicate that IESSC-MNN improves the modeling accuracy and generalization performance with a simple structure when compared with other MNNs.