Distributed Sequence Pattern Detection Over Multiple Data Streams

Abstract

Sequence pattern detection over streaming data has many real world applications. Most of the present work is aimed to process sequence queries over single data stream. Situations where streaming data arrive from multiple sources have not been explored much. In traditional approaches a single centralized machine handles and processes sequence queries over multiple data streams. While running sequence queries on a single server, even though many of the events in data streams do not lead to successful pattern detection they are still handled and processed by the server. This consumes precious network bandwidth, server’s computing resources and precious time. In this paper we focus on sequence pattern detection, where patterns are defined on chains of events that arrive from multiple distributed data streams. We propose a three layer distributed framework to avoid unnecessary event processing by the server, and to efficiently process sequence queries to detect sequence patterns relying upon chains of events. The bottom layer of data sources sends continuous data streams to the middle layer, which then performs pattern detection locally, and on the basis of the feedback received from the top layer of global server, sends events to the global server to detect complete patterns. Our present work is aimed to detect sequence patterns over multiple data streams, but, our proposed model can be extended to many other areas of distributed stream processing.