An Transfer Latency Optimized Solution in GPU-Accelerated De-Duplication

Abstract

Recently, GPU has been introduced as an important tool in general purpose programming due to its powerful computing capacity. In data de-duplication systems, GPU has been used to accelerate the chunking and hashing algorithms. However, the data transfer latency between the memories of CPU to GPU is one of the main challenges in GPU accelerated de-duplication. To alleviate this challenge, our solution strives to reduce the data transfer time between host and GPU memory on parallelized content-defined chunking and hashing algorithm. In our experiment, it has shown 15%~20% performance improvements over already accelerated baseline GPU implementation in data de-duplication.