Just-In-Time Processing of Continuous Queries

Abstract

In a data stream management system, a continuous query is processed by an execution plan consisting of multiple operators connected via the consumer-producer relationship, i.e., the output of an operator (the producer) feeds to another downstream operator (the consumer) as input. Existing techniques execute each operator separately and push all results to its consumers, without considering whether the consumers need them. Consequently, considerable CPU and memory resources are wasted on producing and storing useless intermediate results. Motivated by this, we propose just-in-time (JIT) processing, a novel methodology that enables a consumer to return feedback expressing its current demand to the producer. The latter selectively generates results based on this information. We show, through extensive experiments, that JIT achieves significant savings in terms of both CPU time and memory consumption.