Intelligent query placement strategy for progressive-real time analytics in big data

Abstract

Over the past few years there has been humongous increase in the data volume by the explosion of unstructured machine-generated logs, sensors, networks and devices. This Changing business conditions raise a necessity to process real-time data asset in a faster way. Stream processing frameworks provide solutions to address high volume of data in real time with a scalable, highly available and fault tolerant manner which enables analysis of data in motion. Stream processing takes place on the inbound data before its getting stored whereas traditional database models first store and index the data before being processed by queries. Addressing continuous queries over continuous stream data hence makes sense in almost every industry-through human activities, machine data or sensor data. We propose a mechanism to exploit the common characteristics existing among continuous queries by implementing query selection placement strategies which will enhance the query execution performance.