Proactive and intelligent evaluation of big data queries in edge clouds with materialized views

Abstract

The rise of big data brings extraordinary benefits and opportunities to businesses and governments. Enterprise users can analyze their consumers’ data and infer the business value, such as purchasing goods correlations and customer preferences. To unveil such hidden values from big data, various big data processing frameworks, such as Hadoop and Tensor-flow, are developed. One fundamental approach of accelerating such big data processing frameworks is to efficiently evaluate queries for big data analytics based on materialization of intermediate results. In this paper, we consider problems of dynamic and proactive QoS-aware query evaluation of big data analytics with intermediate result materialization in a mobile edge cloud. We propose a one-shot online algorithm for the dynamic QoS-aware big data query evaluation within a finite time horizon, which can intelligently determine whether some immediate results during query evaluation need to be materialized for later use of other queries, by making use of the Reinforcement Learning (RL) method with predictions. We also devise an online learning algorithm based on the technique of multi-armed bandits, for the proactive QoS-aware big data query evaluation problem with resource reservations, where the query arrival information and their required datasets are uncertain before the actual evaluation of the queries. We finally investigate the performance of the proposed algorithms by experimental simulations, and results show that the performance of the proposed algorithms is promising, by achieving 12% higher system throughput while reducing 50% and 40% average evaluation cost and time per query compared to the comparison benchmarks.