Post-deduplication Delta Compression Schemes

Abstract

AbstractDelta compression has been gaining increasing attention in recent years for its ability to remove redundancy among non-duplicate but very similar data files and chunks, for which the data deduplication technology often fails to identify and eliminate. Given a new chunk B and an existing chunk A, delta compression encodes B relative to A and generates their differences. We call the differences a “delta” and the chunk A its “base.” We then only need to transfer or store the delta, rather than the entire chunk B, thus obtaining bandwidth- or space-savings. Due to significant data reduction efficiency, some applications adopt delta compression as a complement for chunk-level deduplication to further reduce the storage space (or bandwidth) requirement. In this chapter, we discuss the problems facing the post-deduplication delta compression and the solutions to address the problems. The rest of this chapter is organized as follows: Section 7.1 presents the state-of-the-art solutions for post-deduplication delta compression techniques. Section 7.2 describes the design and implementations of our proposed deduplication-inspired delta compression approach, called Ddelta. Section 7.3 describes our proposed delta compressed and deduplicated technique with low overhead.