Ensemble-learning based neural networks for novelty detection in multi-class systems

Abstract

In most real-world systems or processes, determining the complete set of classes during the training phase is generally impossible. There is a high chance that novelties or abnormal data can appear in future phases which might severely affect the performance of the machine learning system. Novelty detection is of great importance in many critical systems and domains, such as business intelligence, process monitoring, information security, clinical decision support etc. Most of the available methods for novelty detection use a one-class classification (OCC) criterion, i.e. treating multiple known classes as a single ”Normal” class, whose aim is to distinguish data samples between “Normal” and “Not Normal” classes. In this paper, the problem of novelty detection in multi-class systems is addressed through ensemble based learning of neural networks (EBNN), capable of both detecting novelties and classifying the known normal samples in future datasets. Moreover, the model is analogous to the semi-supervised learning system as it is trained using only the available normal classes. Evaluation of the proposed model (EBNN) on UCI machine learning datasets showed that the model not only outperforms other models in detecting novelties but also has a better multi-class classification accuracy for known normal classes. The proposed model implements a novel activation function in its framework and differs from the commonly available novelty detection models in three aspects. First, the model is much simpler to implement and does not need any initial assumptions about the model. Second, the model does not require any novel or abnormal data during training phase (semi-supervised learning). Third, it can be used as a two in one system to detect novelties and at the same time to classify data based on known classes.