CLOUDRB: A framework for scheduling and managing High-Performance Computing (HPC) applications in science cloud

Abstract

In recent years, the Cloud environment has played a major role in running High-Performance Computing (HPC) applications, which are computationally intensive and data intensive in nature. The High-Performance Computing Cloud (HPCC) or Science Cloud (SC) provides the resources to these types of applications in an on demand and scalable manner. Scheduling of jobs or applications in a Cloud environment is NP-Complete and complex in nature due to the dynamicity of resources and on demand user application requirements. The main motivation behind this research study is to design and develop a CLOUD Resource Broker (CLOUDRB) for efficiently managing cloud resources and completing jobs for scientific applications within a user-specified deadline. It is implemented and integrated with a Deadline-based Job Scheduling and Particle Swarm Optimization (PSO)-based Resource Allocation mechanism. Our proposed approach intends to achieve the objectives of minimizing both execution time and cost based on the defined fitness function. It is simulated by modeling the HPC jobs and Cloud resources using the Matlab programming environment. The simulation results prove the effectiveness of the proposed research work by minimizing the completion time, cost and job rejection ratio and maximizing the number of jobs completing their applications within a deadline and meeting the user’s satisfaction. The proposed work has been tested in our Eucalyptus-based cloud environments by submitting real-world HPC applications and observed the improvements in performance.