Improvised dedupication with keys and chunks in HDFS storage

Abstract

Cloud computing has providing unlimited storage space to store data in cloud. Service providers can increase space in cloud by incorporating with de-duplication systems. In the previous de-duplication systems, they consider single server to store data in cloud. However, as lots of de-duplication system and storage are intend by users and applications for higher reliability and the data are critical. Hence in this system, the concept of de-duplication and security is achieved by means of Proof of Ownership of the File[POF]. We used multiple server to store the data in cloud in this system. We outsource the keys to slave machine securely in order to keep the data very secure. De-key supports both file level and block level de-duplications. The overhead of key management is avoided in this system and provides fault tolerance for key management. In this system, Triple DES technique is used, as the plain text is encrypted triple times with the convergent keys so that the data will be more secured. Scalability increases as de-key achieved efficiently. Cost efficiency is achieved as multiple users of same data is just referred and not newly added. Deleting content of shared file of different user will allow deleting only convergent key reference not co0ntent stored in Hadoop Distributed File system Storage[HDFS].