A reinforcement learning based job scheduling algorithm for heterogeneous computing environment

Abstract

Efficient job scheduling for heterogeneous computing environments has attracted widespread attention, jobs are usually modeled as directed acyclic graphs(DAG). Optimizing scheduling can improve system throughput. We propose a two-stage scheduling algorithm, which calculates task selection and processor allocation respectively. In task selection stage, we utilize a bidirectional graph convolution network to learn DAG structural features, and a fully-connected network to generate proper scheduling scheme. In the processor allocation stage, we propose a heuristic based on optimistic cost table(OCT) and task duplication, which trade-off scheduling allocation better. Experiments of various scheduling scenarios have been conducted, and the results show that the proposed algorithm has better scheduling performance than the compared heterogeneous DAG scheduling algorithms.