Detection of Novel Classes in Dynamic Data Streams

Abstract

Every second, a huge volume of multi-dimensional data is generated in fields such as Social Networking, Industrial Internet of Things, Stock market and E-commerce applications. Knowledge and pattern extraction are a challenging task in the evolving nature of data stream. Major issues are (i) ‘concept drift’ occurs as a result of pattern changes in the data distribution and (ii) ‘concept evolution’ occurs when a new class evolves in the data stream. These issues degrade the performance of learning models. In this paper, we focus on detection of concept evolution and enhance the performance of classifiers. For this, we propose a new model to identify novel classes, namely, Detection of Novel Classes (DNC). The proposed method adopts long short term memory to continuously observe the streaming data in order to detect emerging classes. The continuous monitoring allows the model to distinguish between existing classes and the novel classes which save time and memory. Also, the proposed method is demonstrated for identifying more than one novel class. The experiments are performed over seven different datasets. The results confirm the efficiency is increased ranging from 6% to 34% by the proposed method in identifying new concepts in the evolving data stream than the existing methods available in the literature.