Load shedding techniques based on windows in data stream systems

Abstract

Many applications need to process streams, for analyzing and monitoring their data for inferring useful information. Database analysts are structuring Data Stream Management systems (DSMS) so that applications can subject queries to get timely information from streams. Unlike the traditional database system, managing and processing of stream database raise several challenges. In this paper, we portrayed a new load shedding system for queries consisting of one or more aggregate operators with sliding windows. We utilized different types of window aggregate function to drop the tuple in DataStream. This method is conscious of the window properties of its window aggregate operators in the query plan. Accordingly, it plausibly divides the input stream into windows and probabilistically decides which tuple to drop based on the window function. This decision is further encoded into tuple by marking the ones that are disallowed from starting new windows. Unlike previous methods, our method conserve consistency of windows all over a query plan, and always distributes subsets of original query responds with negligible deprivation in the quality of the result.