Empirical Analysis of Hardware-Assisted GPU Virtualization

Abstract

The increasing use of Graphics Processing Unit (GPUs) for accelerating compute intensive tasks and graphics-related computations has led to their inclusion in High Performance Clusters and Cloud setups. Several cloud vendors provide virtual machine instances with GPU capabilities. With the advent of virtualization aware GPU hardware (NVIDIA vGPUs), allocating and sharing physical GPU among virtual machines has become easier and cost-efficient. The sharing mechanism and extent of sharing are determined by the vGPU scheduling algorithm and a configurable vGPU profile. As part of this work, we present a thorough empirical study of the hardware-assisted virtualized GPU setups. In particular, we quantify the virtualization overheads, study the interference effects of concurrently executing homogeneous and heterogeneous workloads, and impact of vGPU scheduling algorithms. We also demonstrate that the best vGPU configuration parameters are sensitive to the mix of workload characteristics that share the GPU. Our study also compares the performance of vGPU based virtualization with PCI passthrough based direct GPU assignment to virtual machines. Based on our evaluation using heterogeneous workloads setups and varying vGPU configurations, we observe that the virtualization overheads are up to 7% (in terms of reduced memory availability) and up to 20% increase in execution times. Further, we demonstrate using our setups, that co-placing of heterogeneous workloads can improve the efficiency of GPU multiplexing and decrease execution times to as low as 20% as compared to homogeneous workload placements.