BlockDrive: A Deduplication framework for Cloud using edge-level blockchain

Abstract

Data deduplication is an important data compression technique to eliminate duplicate copies of data and has been widely used in cloud storage to reduce the amount of storage space and save bandwidth. However, even though most cloud service providers already employ deduplication techniques, they consume a lot of overhead bandwidth. Hence, we came up with a solution to reduce this overhead bandwidth by creating a middleware deduplication layer that checks the originality of a file without the need to communicate with the cloud. This results in reducing a substantial amount of internet overhead and at the same time making the deduplication process much quicker since it is closer to the edge device. We implement the deduplication layer using a decentralized blockchain structure due to its inherent security and scalability features.To protect the confidentiality of sensitive data while supporting deduplication, the convergent encryption techniques such as double hashing has been proposed to ensure that the user account credentials are secure and not misused. We implement our blockchain layer using Ganache that provides us with an Ethereum powered blockchain to test and deploy our prototype. Hence all the file transactions i.e., uploading, sharing and deletion of files will go through ganache and get added to our local blockchain network. We define Smart contracts to communicate with the ganache layer which mines a block every time a new file is uploaded or shared with another user and file metadata is stored in each of these blocks. Since every unique file will have its own block, the file metadata will help in achieving deduplication at this layer.