MSHGN: Multi-scenario adaptive hierarchical spatial graph convolution network for GPU utilization prediction in heterogeneous GPU clusters

Abstract

Accurately predicting GPU utilization is crucial for effectively managing heterogeneous GPU clusters, yet existing prediction methods are tailored to homogeneous clusters or ignore the unique characteristics of heterogeneous ones. To address this problem, we propose the Multi-Scenarios Adaptive Hierarchical Spatial Graph Convolution Network (MSHGN) model. This model leverages the hierarchical relationships among users, tasks, and machines to construct multiple scenarios' undirected graphs and uses Graph Convolution Neural (GCN) to capture the spatial dependency relationships between the computational resources consumed during execution. The MSHGN model accurately predicts the future GPU utilization rates of each machine in a heterogeneous GPU cluster, achieving an RMSE of only 0.0101, an MAE of 0.0072, and an MAPE of 0.0641. We evaluate the MSHGN model on a real-world Alibaba dataset collected from a heterogeneous GPU cluster and find that it achieves superior accuracy and robustness in predicting resource utilization, outperforming other baseline models.