Incremental Learning and Optimization of Hierarchical Clusterings with Art-Based Modular Networks

Abstract

This chapter introduces HART-S, a modular neural network that can incrementally learn stable hierarchical clusterings of arbitrary sequences of input patterns by self-organisation. The network is a cascade of Adaptive Resonance Theory (ART) modules, in which each module learns to cluster the differences between the input pattern and the selected category prototype at the previous module. Input patterns are first classified into a few broad categories, and successive ART modules find increasingly specific categories until a threshold is reached, the level of which can be controlled by a global parameter called “resolution”. The network thus essentially implements a divisive (or splitting) hierarchical clustering algorithm: hence the name HART-S (for “Hierarchical ART with Splitting”). HART-S is also compared and contrasted to HART-J (for “Hierarchical ART with Joining”), another variant that was proposed earlier by the first author. The network dynamics are specified and some useful properties of both networks are given and then proven. Experiments were carried out on benchmark datasets to demonstrate the representational and learning capabilities of both networks and to compare the developed clusterings with those of two classical methods and a conceptual clustering algorithm. Two optimisation methods for the HART-S network are also introduced.