Efficient and Adaptive Processing of Multiple Continuous Queries

Abstract

Continuous queries are queries executed on data streams within a potentially open-ended time interval specified by the user and are usually long running. The data streams are likely to exhibit fluctuating characteristics such as varying inter-arrival times, as well as varying data characteristics during the query execution. In the presence of such unpredictable factors, continuous query systems must still be able to efficiently handle large number of queries, as well as to offer acceptable individual query performance.In this paper, we propose and discuss a novel framework, called AdaptiveCQ, for the efficient processing of multiple continuous queries. In our framework, multiple queries share intermediate results at a fine level of granularity. Unlike previous approaches to sharing or reusing that relied on materialization to disk, AdaptiveCQ allows on-the-fly sharing of results. We show that this feature improves both the initial query response time, and the overall response time. Finally, AdaptiveCQ, which extrapolates the idea proposed by the eddy query-processing model, adapts well to fluctuations of the data streams characteristics by this combination of fine grain and on-the-fly sharing. We implemented AdaptiveCQ from scratch in Java and made use of it to conduct the experiments. We present experimental results that substantiate our claim that AdaptiveCQ can provide substantial performance improvements over existing methods of reusing intermediate results that relied on materialization to disk. In addition, we also show that AdaptiveCQ can adapt well to fluctuations in the query environment.