A Low-Load Distributed Stream Processing System for Continuous Conjunctive Normal Form Queries

Abstract

One of the research issues for stream data is fast notification of continuous query results. Recent researches assume that query processing servers receive data from remote data sources continuously. Therefore, when a large number of processing servers are distributed in networks, large amounts of communication traffic for those are produced. This is the main cause of delaying query result notifications. To tackle this fundamental problem, we focus on the conjunction of operations and conditions in continuous queries for distributed processing systems. When queries consist of some series of operations and conditions, processing servers can stop query executions if the remaining operations or condition checking are dispensable. Thus, communications less arise and communication traffic is further reduced. Our proposed system represents continuous queries by conjunctive normal forms (CNF queries). The proposed system constructs trigger trees for evaluating the sum terms of CNF queries and determines the timing for their evaluations, in order to reduce communication traffic. We confirmed that one of the proposed methods can reduce the average amount of communication traffic by 23 percent compared with a sensor data serial aggregation method in an evaluation situation.