EEM: An elastic event matching framework for content-based publish/subscribe systems

Abstract

The inevitable workload fluctuations in large-scale IoT applications make it difficult to ensure the QoS of data distribution services. Usually, this problem can be solved by dynamically adjusting resource supply. However, deploying more resources not only takes time, it is also difficult to achieve cost-effective results. In this paper, we propose an elastic event matching (EEM) framework and we explore the idea of endowing matching algorithms with adaptability in performance. The strategy is to enable seamless switching between exact matching and approximate matching, achieving a trade-off between matching precision and matching speed. First, we establish a predicate skipping adjustment (PSA) mechanism which quantifies the relationship between false positives and the number of skipped predicates. In addition, we design a performance adjustment decision (PAD) algorithm according to fluctuating workloads. We implemented an EEM prototype based an Kafka, which uses an existing matching algorithm enhanced by PSA as the engine. The prototype is evaluated through a series of experiments based on both synthetic data and real-world stock traces. Experiment results show that adjusting the performance of matching algorithm at the price of a small false positive rate of less than 0.1% can shorten event latency by up to 14.34 times, which clearly demonstrates the effectiveness of the EEM framework.