Ddelta: A deduplication-inspired fast delta compression approach

Abstract

Delta compression is an efficient data reduction approach to removing redundancy among similar data chunks and files in storage systems. One of the main challenges facing delta compression is its low encoding speed, a worsening problem in face of the steadily increasing storage and network bandwidth and speed. In this paper, we present Ddelta, a deduplication-inspired fast delta compression scheme that effectively leverages the simplicity and efficiency of data deduplication techniques to improve delta encoding/decoding performance. The basic idea behind Ddelta is to (1) accelerate the delta encoding and decoding processes by a novel approach of combining Gear-based chunking and Spooky-based fingerprinting for fast identification of duplicate strings for delta calculation, and (2) exploit content locality of redundant data to detect more duplicates by greedily scanning the areas immediately adjacent to already detected duplicate chunks/strings. Our experimental evaluation of a Ddelta prototype based on real-world datasets shows that Ddelta achieves an encoding speedup of 2.5×–8× and a decoding speedup of 2×–20× over the classic delta-compression approaches Xdelta and Zdelta while achieving a comparable level of compression ratio.