Performance and price analysis for cloud service providers

Abstract

In the cloud computing market, the selection of the best Cloud Service Provider (CSP) is not easy for the customers since no clear comparison between the different CSPs offerings. Each CSP offers its own services and configurations with its own prices. Some CSP can offer fixed size virtual machines (VMs), while others can offer flexible VM configuration. Cloud customers have to choose the CSP that best fits their requirements. In the actual market, and with the limited performance information provided by the CSPs to the cloud users, the choice of the CSP can be a problem for the customers. In our paper, and in the context of the Easi-Clouds [23], a European ITEA 2 research project, we propose a set of performance tests based on real measurements to classify the CSPs based on their performance score as well as their proposed price. We used a set of benchmarks to test the performances of four VMs sizes from each one of the biggest eight CSPs (Amazon, Softlayer, Rackspace, Google, Microsoft Azure, Aruba, Digital Ocean, Joyent). We try to compare the performance based on seven different metrics (CPU performance, Memory performance, Disk I/O performance, Mean Response time (MRT), Provisioning time, Availability, and Variability). In a second step, We include the price to have a performance vs. price value figure. The performance and value figures of merit help the Cloud customers choosing the suitable CSP.