Cross-entropy based pruning of the hierarchical mixtures of experts

Abstract

The paper presents a pruning scheme for the hierarchical mixtures of experts (HME), which is a hierarchical and tree-like modular neural network trained using the EM-algorithm. The pruning scheme is in the style of the classification and regression tree (CART), and consists of using cross-entropy to select and cut out sub-trees of the HME to create a series of nested HMEs. The right sized HME can then be selected by using cross-validation. Experiments are carried out to demonstrate the successful operation of the scheme.