Adaptive methods for efficient burst and correlative burst detection

Abstract

In recent year, advances in hardware and information technology have led to large flow of data across over IP networks. Continuous burst detection in data streams has important applications in fraud credit card detection, real-time IP traffic analysis, sensor readings, email/SMS/blog and hot trends detection in social text sources. The arrival and departure of data objects in a streaming manner impose new challenges for burst detection algorithms, especially in time and space efficiency. Moreover, sample time window size and burst defined threshold values are important to detect accurate burst in continuous data stream. Thus in this work, we describe how to adjust time windows sizes and burst defined threshold values for burst trends detection in real time. And then, correlated bursty terms are detected by finding correlations among them. The burst detection is the task of finding unexpected change in some quantity in real time tweet stream. The main objective of this work is timely detection of bursty trends in Twitter tweet stream which have happened recently and discovery of their evolutionary patterns along the timeline. Our experimental results show the scalability, applicability and effectiveness of our approach in real time processing.