Topology-aware Job Allocation in 3D Torus-based HPC Systems with Hard Job Priority Constraints

Abstract

Abstract In this paper, we address the topology-aware job allocation problem on 3D torus-based high performance computing systems, with the objective of reducing system fragmentation. Firstly, we propose a group-based job allocation strategy, which leads to a more global optimization of resource allocation. Secondly, we propose two shape allocation methods to determine the topo-logical shape for each input job, including a zigzag allocation method for communication non-sensitive jobs, and a convex allocation method for communication sensitive jobs. Thirdly, we propose a topology-aware job mapping algorithm to reduce the system fragmentation brought in by the job mapping process, including a target bin selection method and a bi-directional job mapping method. The evaluation results validate the efficiency of our approach in reducing system fragmentation and improving system utilization.