Profit Optimization for Splitting and Sampling Based Resource Management in Big Data Analytics-as-a-Service Platforms in Cloud Computing Environments

Abstract

Exploring optimal big data analytics solutions to benefit various domains in decision making and problem solving becomes an ever-important research area. Big data Analytics-as-a-Service (AaaS) platforms offer online AaaS to various domains in a pay-as-you-go model. Big data analytics incurs expensive costs and takes lengthy processing times due to large-scale computing requirements. To tackle the cost and time challenges for big data analytics, we focus on proposing efficient and automatic resource management algorithms to maximize profits and minimize query times while guaranteeing Service Level Agreements (SLAs) on Quality of Service (QoS) requirements of queries. For query processing constrained by tight deadlines and limited budgets, our proposed algorithms enable data splitting and sampling based resource scheduling for parallel and approximate processing that significantly reduce data processing times and resource costs. We formulate the multi-objective resource scheduling problem to optimize profits for AaaS platforms while guaranteeing SLAs of queries with minimized response times. We design extensive experiments for algorithm performance evaluation, results show our proposed algorithms outperform state-of-the-art algorithms that maximize profits for AaaS platforms while improving admission rates and minimizing response times for queries. The scheduling algorithms support elastic and automatic large-scale resource configurations to minimize resource costs, and deliver timely, cost-effective, and reliable AaaS with SLA guarantees.