PARS: A parallel model for scaled processing of complex events

Abstract

The need for fast processing of high volume of event streams has triggered the deployment of parallel processing models and techniques for complex event processing. It is however hard to parallelize stateful operators, making the implementations of distributed complex event processing systems very challenging. A well-defined parallel processing model is a pre-requisite to the implementation of any high performance distributed complex event processing system. Most recent works use partition key or shared memory for parallelizing stateful operators but suffer from imbalanced distribution of keys and slow access time to shared memory. This paper proposes a new parallel model called PARS using three partitioning techniques to allow scalable processing of complex events without using partition key or shared memory. We define PARS formally and use this formalism to prove its soundness and completeness. To present a proof-of-concept and evaluate PARS, we use an event generator to simulate event sources and show significant improvement in system scalability. Compared to other works, we experience 10%–20% higher throughput in our experimental results. Our experiments demonstrate processing speeds of up to 5,200,000 complex event detection per second on a multi-machine cluster.