Dynamic Deduplication Algorithm for Cross-User Duplicate Data in Hybrid Cloud Storage

Abstract

The escalating growth of distributed big data in hybrid cloud storage architecture introduces a new set of challenges. Constantly, content enrichment puts pressure on capacity. Nonetheless, the explosion of user data places a significant strain on broadband and storage capacity. Consequently, many cloud storage providers will implement deduplication to compress data, reduce transfer bandwidth, and reduce cloud storage space. In cloud storage systems, it is a data compression and storage optimization method. By locating and removing redundant data, it can save storage space and bandwidth. An MTHDedup deduplication strategy based on the Merkle hash tree is presented in a hybrid cloud environment to address the issue of convergent encryption algorithms being susceptible to brute-force attacks and ciphertext computation time overhead. Merkle hash trees are constructed using additional encryption algorithms to generate encryption keys during file- and block-level deduplication, ensuring that generated ciphertexts are unpredictable. The method is effective against both internal and external brute-force attacks, thereby increasing data security. Our method reduces the computational burden of ciphertext generation and the key storage space, and the performance advantage increases with the number of privilege sets.