Realizing integrated prioritized service in the Hadoop cloud system

Abstract

Cloud computing has profoundly influenced how people receive IT services nowadays. As the cloud offers more and more services to users, the Quality of Service (QoS) in cloud has become an important issue that could determine the success or failure of services on the cloud. Among the aspects of cloud QoS, job performance is an area which most cloud systems often have little or no control over it. To effectively manage job performance in cloud QoS, the collaboration between the cloud system and its underlying operating system is a must. Hadoop is one of the most popular cloud systems used today. Unfortunately, it does not support efficient schemes to manage job performance. Previously, we proposed a new approach, namely PYARN, to enable Hadoop to provide prioritized job scheduling service to help manage job performance in cloud QoS. In this paper, we report our efforts to make PYARN collaborate with PMMDO, a Linux module providing prioritized service we developed earlier. Compared with the original Hadoop, experiment results showed that the cooperation between PYARN and PMMDO could further expedite the execution of prioritized jobs by up to around 30% more than what PYARN could achieve alone. Our integrated system demonstrates that the cloud system and its host operating system should work together to help manage cloud QoS with regard to job performance.