Job flow management for virtualized resources of heterogeneous distributed computing environment

Abstract

We address to the problem of an integration of heterogeneous computational clusters to the united environment applying the OpenStack platform for a cluster resource virtualization. However, this platform does not interact with the traditional resource management systems such as the PBS Torque or SLURM that are used in cluster resources. To this end, we developed an additional hypervisor shell to run virtual machines through queues of the aforementioned systems. Thus, we expand the OpenStack capabilities for an application job management and provide a computation scalability for virtualized resources of clusters. In order to show the benefits of our approach to the scalable application management in integrated cluster environments, we developed a parameter sweep application for simulation modeling of warehouse logistics and solved three optimization tasks for the real refrigerated warehouse. We compared the selected criteria of task solving processes with the hypervisor shell applying and without it by means of simulation modeling. Experiments show that the hypervisor shell can significantly improve computation speedup and task solving time through reallocating virtual machines to queues of the resource management systems.