An Incremental Adaptive Network for On-line Supervised Learning and Probability Estimation

Abstract

In this paper, a novel hybrid utilization of the Fuzzy ARTMAP (FAM) neural network and the Probabilistic Neural Network (PNN) is proposed for on-line learning and probability estimation tasks. There are two distinct advantages to the hybrid network. First, FAM is used as an underlying clustering algorithm to classify the input patterns into different recognition categories during the learning phase, resulting in a significant reduction in the number of pattern nodes required in the PNN. Second, a non-parametric posterior probability distribution estimation procedure, in accordance with the PNN paradigm (i.e. the Parzen-windows estimator), is employed during the prediction phase, where a probabilistic interpretation corresponding to Bayes decision theory can be provided for the predictions of FAM. In addition, several modifications are proposed to integrate both of the networks effectively into a unified platform for enhancing generalization. This hybrid approach also realizes an incremental learning system in which the necessity to specify a static network configuration a priori is eliminated as the network is able to “grow” to accommodate new input patterns sequentially and can thus operate in non-stationary environments. The performance of the network is evaluated with benchmark classification tasks and the results are compared with other approaches. Simulation results indicate that this hybrid network is capable of achieving a value near to the Bayes optimal classification rate. © 1997 Elsevier Science Ltd.