A meta-scheduling model with multi-agent coordination mechanism in multi-cluster

Abstract

A multi-cluster computing environment is composed of many clusters which are distributed over different sites in an organization or an enterprise. All the clusters can be resource shared, autonomous, load balanced, interconnected and collaborative. However, there is still a problem that the uncertainty of the cluster resources caused difficulty in scheduling jobs effectively. To address this problem, a meta-scheduling model with multi-agent coordination mechanism in multi-cluster is presented. Agents are deployed in each cluster and communicated with coordinator. Coordinator can provide global job scheduling service, can access and monitor the state of all the clusters resources, do prediction and evaluation the performance of the resources. It can schedule the job to the fittest cluster according to the scheduling strategy and the evaluation results of the performance of the clusters. The model is applied to the national seismic network computing application system in china. The national seismic network computing platform is constructed by using the meta-scheduling model with multi-agent coordination. An example of seismic scientific computing application is given. The multi-agent coordination meta-scheduling model has been applied in practice and verified actually.