Improvement of Deduplication Efficiency by Two‐Layer Deduplication System

Abstract

SUMMARYThis paper proposes a two‐layer deduplication system for the backup process in an IT environment. The proposed system implements two chunk sizes in a descendent way and activates them simultaneously in a sequence. They eliminate duplicated parts among backup target data more efficiently than any conventional single‐layer deduplication system. The system weakens a substantial trade‐off between the deduplication rate and the performance, which is defined as the number of chunks reduced. Compared to a conventional approach, the proposed system provides an equivalent deduplication rate with much less performance degradation or more performance with less compensation of the deduplication rate. In addition, it achieves better stability of performance when small chunk sizes are used to manipulate heavily duplicated data and densely changed data. The positive effects of the proposed system are evaluated quantitatively and are compared to a conventional approach by using a simulation. The results are certified with a prototype machine.