Mitigating Catastrophic Forgetting In Adaptive Class Incremental Extreme Learning Machine Through Neuron Clustering

Abstract

Catastrophic forgetting is a major problem that affects neural networks during progressive learning. In it, the previously learned representation vanishes as the network learns new information. The extreme learning machine is one of the variants of the neural network. It is used in many domains due to fast training and good generalization ability. However, like other neural networks, it suffers from catastrophic forgetting and negative forward and backward transfer during the progression of neurons in incremental learning. The study hypothesizes that it is due to overlapping in hidden neurons and output weights. The global representation by an activation function further supports this hypothesis. To address this, the study proposes a neuron clustering approach to mitigate it in an adaptive class incremental extreme learning machine. The neuron clustering method activates k nearest neurons during learning and testing. It helps to partition the network to select overlapping subnetwork. Experimental results on four food datasets show that the proposed approach reduces negative forward and backward transfer when neurons are added incrementally during progressive learning.