Cloud‐based data streams optimization

Abstract

Many modern applications of sensor networks and transaction analysis require real‐time processing of their stream data sets. These data streams vary continuously over time. Current stream processing approaches focus on only one of the two optimization perspectives, proposing optimization techniques for data streams processing regardless of the processing environment or improving the processing environment only. In this paper, a brief survey of recent approaches to data streams processing coming from the two optimizations perspectives is proposed; their shortcomings are presented as well. Then, a proposal to an innovative and integrative framework is developed; it is referred to as the continuous query optimization based on multiple plans (CQOMP) for data streams over the cloud environment. CQOMP combines the two optimization perspectives and provides an optimized stream clusters processing using multiple split query plans. Each plan is constructed for a cluster of data that has nearest characteristics and it processes streams tuples over the cloud. We also propose a novel algorithm called the optimized multiple plans (OMP) for processing data streams clusters on Cloud Computing. The OMP algorithm efficiently divides data streams and generates optimized multiple split plans. Each plan is for processing a group of data streams on the cloud. We present the experimental results of the OMP solution compared to the alternative state‐of‐the‐art data stream approaches. The experiments show the efficiency and the scalability of the combined OMP algorithm on different cloud environments, the real Amazon cloud environment, and the simulated windows azure cloud environment.This article is categorized under: Fundamental Concepts of Data and Knowledge > Big Data Mining Technologies > Classification Technologies > Data Preprocessing Technologies > Structure Discovery and Clustering