Building Adaptive Data Mining Models on Streaming Data in Real-Time, an Outlook on Challenges, Approaches and Ongoing Research

Abstract

Advances in hardware and software, over the past two decades have enabled the capturing, recording and processing of potentially large and infinite streaming data. The field of research in Data Stream Mining (DSM) has emerged to respond to the challenges and opportunities of developing the required analytics to unlock valuable knowledge. Thus DSM is focused on building Data Mining models, workflows and algorithms enabling the efficient and effective analysis of such streaming data at a large scale- the so-called “Big Data”. Examples of application areas of Data Stream Mining techniques include real-time telecommunication data, telemetric data from large industrial plants, credit card transactions, cyber security threat modelling, social media data, etc. For some applications it is acceptable to provide data processing, modelling and analysis in batch mode using the traditional Data Mining approaches. However, for other application, particularly where continuous monitoring and contingent response are required, the model building and analytics have to take place in real-time as soon as new data becomes available i.e. to accommodate infinite streams and fast changing concepts in the data. This article highlights some of the key concepts and emergent techniques in DSM as presented in the authors’ recent publications as also outlined in a talk given at the UK Symposium on Knowledge Discovery from Data in London on May 24th, 2019 discussing the challenges, opportunities and innovative solutions in Data Stream Mining.