Quality Profile-Based Cloud Service Selection for Fulfilling Big Data Processing Requirements

Abstract

Big data has emerged as promising technology to handle huge and special data. Processing Big data involves selecting the appropriate services and resources thanks to the variety of services offered by different Cloud providers. Such selection is difficult, especially if a set of Big data requirements should be met. In this paper, we propose a dynamic cloud service selection scheme that assess Big data requirements, dynamically map these to the most available cloud services, and then recommend the best match services that fulfill different Big data processing requests. Our selection is conducted in two stages: 1) relies on a Big data task profile that efficiently capture Big data task's requirements and map them to QoS parameters, and then classify cloud providers that best satisfy these requirements, 2) uses the list of selected providers from stage 1 to further select the appropriate Cloud services to fulfill the overall Big Data task requirements. We extend the Analytic Hierarchy Process (AHP) based ranking mechanism to cope with the problem of multi-criteria selection. We conduct a set of experiments using simulated cloud setup to evaluate our selection scheme as well as the extended AHP against other selection techniques. The results show that our selection approach outperforms the others and select efficiently the appropriate cloud services that guarantee Big data task's QoS requirements.