Storage Deduplication by Virtual Large-Scale Disks

Abstract

Recently, the demand of low cost large scale storages increases. We developed VLSD (Virtual Large Scale Disks) toolkit for constructing virtual disk based distributed storages, which aggregate free spaces of individual disks. VLSD realizes low-cost large-scale storage by collecting free disk spaces from many computers. Generally, the cost of storage is proportional to the number of disks in the storage. Deduplication is effective for reducing the number of disks. Recently, deduplication is one of hot topics in storage. We have developed Dedup Folder Disk, which is a virtual disk that deduplicates blocks on basis of inline method. In this virtual disk, the same contents are stored into a folder. In this paper, we describe the design and implementation of Dedup Folder Disk and evaluate its performance and capacity efficiency. From the result of these evaluations, we conclude that Dedup Folder Disk is suited for middle-scale storage and it is effective in a sense of deduplication. Especially, it is useful when users has same contents. In the other word, its performance is not so good. Therefore, the relationship between performance and block size is trade-off.