Comparison of analytical and ML-based models for predicting CPU–GPU data transfer time

Abstract

The overhead of data transfer to the GPU poses a bottleneck for the performance of CUDA programs. The accurate prediction of data transfer time is quite effective in improving the performance of GPU analytical modeling, the prediction accuracy of kernel performance, and the composition of the CPU with the GPU for solving computational problems. For estimating the data transfer time between the CPU and the GPU, the current study employs three machine learning-based models and a new analytical model called $$\lambda $$-Model. These models run on four GPUs from different NVIDIA architectures and their performance is compared. The practical results show that the $$\lambda $$-Model is able to anticipate the transmission of large-sized data with a maximum error of 1.643%, which offers better performance than that of machine learning methods. As for the transmission of small-sized data, machine learning-based methods provide better performance and a predicted data transfer time with a maximum error of 4.52%. Consequently, the current study recommends a hybrid model, that is, the $$\lambda $$-Model for large-sized data and machine learning tools for small-sized data.