Benchmarking Joyent Smartdatacenter for Hadoop Mapreduce and Mpi Operations

Abstract

Cloud Computing is an ever-growing paradigm shift in computing allowing users commodity access to compute and storage services. As such cloud computing is an emerging promising approach for High Performance Computing (HPC) application development. Automation of resource provision offered by Cloud computing facilitates the eScience programmer usage of computing and storage resources. Currently, there are many commercial services for compute, storage, network and many others from big name companies. However, these services typically do not have performance guarantees associated with them. This results in unexpected performance degradation of user's applications that can be somewhat random to the user. In order to overcome this, a user must be well versed in the tools and technologies that drive Cloud Computing. One of the state of the art cloud systems, Joyent SmartDataCenter, is a cloud system that provides virtual machines(and their processes) the ability to burst CPU capacity automatically and thus is suitable for HPC applications. To help HPC developers, we present a set of Hadoop MapReduce and MPI benchmarks for FlexCloud (a SmartDataCenter installation). Our benchmarks show that this cloud system offers scalable performance for HPC environments.