Secure Data Deduplication Research Articles

Secure and Efficient Data Deduplication in JointCloud Storage

Data deduplication can efficiently eliminate data redundancies in cloud storage and reduce the bandwidth requirement of users. However, most previous schemes depending on the help of a trusted key server (KS) are vulnerable and limited because they suffer from revealing information, poor resistance to attacks, great computational overhead, etc. In particular, if the trusted KS fails, the whole system stops working, i.e., single-point-of-failure. In this paper, we propose a Secure and Efficient data Deduplication scheme (named SED) in a JointCloud storage system which provides the global services via collaboration with various clouds. SED also supports dynamic data update and sharing without the help of the trusted KS. Moreover, SED can overcome the single-point-of-failure that commonly occurs in the classic cloud storage system. According to the theoretical analyses, our SED ensures the semantic security in the random oracle model and has strong anti-attack ability such as the brute-force attack resistance and the collusion attack resistance. Besides, SED can effectively eliminate data redundancies with low computational complexity and communication and storage overhead. The efficiency and functionality of SED improves the usability in client-side. Finally, the comparing results show that the performance of our scheme is superior to that of the existing schemes.

SEEDDUP: A Three-Tier SEcurE Data DedUPlication Architecture-Based Storage and Retrieval for Cross-Domains Over Cloud

Data deduplication has become a promising technique that significantly reduces space utilization of cloud storage servers and optimizes the network bandwidth. A variety of deduplication techniques have been proposed to enhance the cloud server’s storage efficiency. Most of the deduplication schemes do not address global indexing and user privacy issues. This paper elucidates a novel three-tier secure data deduplication architecture (referred to as SEEDDUP) for data storage and retrieval in the cloud. SEEDDUP provides data deduplication with proper indexing and user privacy. In this work, the optimal chunk size, for a given file size and format, is calculated using Cuckoo Search Algorithm. The chunk existence in the cloud is verified by the hash value generated via SHA3-512. To improve data deduplication speed, Merkle Hash Tree based MapReduce (MHTMR) is used for fast index lookup, which improves the performance at data storage and retrieval. In addition, data are encrypted based on its sensitivity level (low and high) using Niederreiter Public Key Cryptosystem. With this new SEEDDUP architecture, only legitimate users can request data successfully for limited times (k) and spurious data requests cannot be processed or handled in the cloud. Experimental results show that the proposed SEEDDUP can reduce latency, achieved good throughput and deduplication efficiency (chunk size variance and security measures) in data deduplication.

Secure Deduplication-Based Storage Systems With Resistance to Side-Channel Attacks via Fog Computing

Data deduplication technique could greatly save the storage overhead of the cloud by eliminating duplicated data and retaining one copy. In order to ensure the privacy of the data against an untrusted cloud, many cryptographic schemes have been proposed to make deduplication feasible in ciphertext. A typical scheme is Message-Locked Encryption (MLE), which takes cryptographic hash values of messages as encryption keys. However, MLE is vulnerable to side-channel attacks. Current solutions trying to mitigate these attacks raise either expensive overhead or security drawbacks. In this paper, we propose a secure data deduplication system against an untrusted cloud with resistance to two typical side-channel attacks, namely probe attack and key-cache attack. Our system uses fog computing to devise two new techniques to solve the two side-channel attacks with new security and efficiency trade-offs. The analysis and evaluation show that our system achieves better trade-offs compared with previous works.

A secure data deduplication system for integrated cloud-edge networks

Data redundancy is a significant issue that wastes plenty of storage space in the cloud-fog storage integrated environments. Most of the current techniques, which mainly center around the static scenes, for example, the backup and archive systems, are not appropriate because of the dynamic nature of data in the cloud or integrated cloud environments. This problem can be effectively reduced and successfully managed by data deduplication techniques, eliminating duplicate data in cloud storage systems. Implementation of data deduplication (DD) over encrypted data is always a significant challenge in an integrated cloud-fog storage and computing environment to optimize the storage efficiently in a highly secured manner. This paper develops a new method using Convergent and Modified Elliptic Curve Cryptography (MECC) algorithms over the cloud and fog environment to construct secure deduplication systems. The proposed method focuses on the two most important goals of such systems. On one side, the redundancy of data needs to be reduced to its minimum, and on the other hand, a robust encryption approach must be developed to ensure the security of the data. The proposed technique is well suited for operations such as uploading new files by a user to the fog or cloud storage. The file is first encrypted using the Convergent Encryption (CE) technique and then re-encrypted using the Modified Elliptic Curve Cryptography (MECC) algorithm. The proposed method can recognize data redundancy at the block level, reducing the redundancy of data more effectively. Testing results show that the proposed approach can outperform a few state-of-the-art methods of computational efficiency and security levels.

Secure Data Deduplication Protocol for Edge-Assisted Mobile CrowdSensing Services

Data deduplication for edge-based mobile crowdsensing services removes duplicate data to minimize storage space and enhance communication efficiency. However, secure data deduplication in edge-based mobile crowdsensing applications can be challenging to achieve in concurrent with protecting sensing data and participants (e.g., client and recruited mobile users) from various threats (i.e., inside attacks and outside attacks). In this paper, we propose an efficient and secure data deduplication protocol for edge-assisted mobile crowdsensing services. Specifically, certificateless public key cryptography is first employed to generate the private and public keys of the crowdsensing server center and edge nodes. Then, a novel method is created to detect and eliminate duplicates while preventing the duplicate-linking from being leaked. Next, we demonstrate that the presented scheme achieves both privacy and security attributes typically required of mobile crowdsening applications. Finally, the performance analysis shows that the proposed protocol incurs minimal computation and communication costs compared with the state of the art research.

An efficient secure data deduplication method using radix trie with bloom filter (SDD-RT-BF) in cloud environment

Recent advancements in the domain of cloud computing (CC) and big data technologies leads to an exponential increase in cloud data, huge replica data utilized the available memory space and maximum computation brought a major issue to the restricted cloud storage space. This paper develops an effective radix trie (RT) with Bloom Filter (BF) based secure data deduplication model, abbreviated as SDD-RT-BD. The proposed SDD-RT-BF model involves three major stages namely, authorized deduplication, proof of ownership and role key update. Initially, a convergent encryption approach is applied for preventing the leakage of data and employed role re-encryption process for attaining authorized deduplication resourcefully. Specifically, management centre handles the authorized request, and establish a RT structure to map the relationship among roles and keys. Besides, BF is applied for the implementation of data updating and enhance the retrieval of ownership verifying efficiently. The inclusion of RT along with BF for secure data deduplication shows the novelty of the paper. A detailed simulation experiments takes place for demonstrating the security and effectiveness of the presented model. The experimental outcome pointed out that the SDD-RT-BF model possesses many beneficial features namely Client-side deduplication, Tag consistency preservation, Update of outsourced data and Fault tolerance. The experimental results denoted that under the file size of 8 MB, the SDD-RT-BF model offers maximum deduplication rate of 25.40% whereas the SS, SSIMI and SDM models attains minimum deduplication rate of 24.60%, 23.60% and 22.30% respectively.

Providing Task Allocation and Secure Deduplication for Mobile Crowdsensing via Fog Computing

Mobile crowdsensing enables a crowd of individuals to cooperatively collect data for special interest customers using their mobile devices. The success of mobile crowdsensing largely depends on the participating mobile users. The broader participation, the more sensing data are collected; nevertheless, the more replicate data may be generated, thereby bringing unnecessary heavy communication overhead. Hence it is critical to eliminate duplicate data to improve communication efficiency, a.k.a., data deduplication. Unfortunately, sensing data is usually protected, making its deduplication challenging. In this paper, we propose a fog-assisted mobile crowdsensing framework, enabling fog nodes to allocate tasks based on user mobility for improving the accuracy of task assignment. Further, a fog-assisted secure data deduplication scheme (Fo-SDD) is introduced to improve communication efficiency while guaranteeing data confidentiality. Specifically, a BLS-oblivious pseudo-random function is designed to enable fog nodes to detect and remove replicate data in sensing reports without exposing the content of reports. To protect the privacy of mobile users, we further extend the Fo-SDD to hide users’ identities during data collection. In doing so, Chameleon hash function is leveraged to achieve contribution claim and reward retrieval for anonymous mobile users. Finally, we demonstrate that both schemes achieve secure, efficient data deduplication.

SecDedoop: Secure Deduplication with Access Control of Big Data in the HDFS/Hadoop Environment

With the rapid growth of storage providers, data deduplication is an essential storage optimization technique that greatly minimizes data storage costs by storing a unique copy of duplicate data. Nowadays, deduplication introduces various new challenges such as security and insufficient space issue. Hence, in this article, we propose a secure data deduplication with access control of big data over HDFS (Hadoop Distributed File System)/Hadoop environment, called SecDedoop. First, the system achieves security for data confidentiality by third party vendor using elliptic curve cryptography. There are two types of keys (public key and private key) generated for data retrieval. Second, we consider data deduplication. The user's original file is divided into a number of equal chunks. Then, each chunk (e.g., 1. txt) is tokenized into words and the weight of words is computed by using TF-IDF frequency. The SHA-3 hash computation is performed to the user's original file. If the hash value is not duplicate, then we store data in HDFS. The PSO (particle swarm optimization)-based MapReduce model is the proposed best data node selection. Initially, MapReduce process is finished for the user's original file and it results in the best set of data nodes; then, we apply PSO to compute the fitness value for best data node selection. Further, we consider MongoDB for fast indexing of the user's original files and also apply FCM (fuzzy-C-means clustering) for clustering the user's files. In this article, we consider the modified version of PSO and FCM to eliminate the open issues in conventional PSO and FCM. The performance of our proposed SecDedoop has been evaluated by using various performance metrics and also proved it outperforms better than previous approaches.

Secure data deduplication for Internet-of-things sensor networks based on threshold dynamic adjustment

Large amount of data are being produce by Internet-of-things sensor networks and applications. Secure and efficient deduplication of Internet-of-things data in the cloud is vital to the prevalence of Internet-of-things applications. In order to ensure data security for deduplication, different data should be assigned with different privacy levels. We propose a deduplication scheme based on threshold dynamic adjustment to ensure the security of data uploading and related operations. The concept of the ideal threshold is introduced for the first time, which can be used to eliminate the drawbacks of the fixed threshold in traditional schemes. The item response theory is adopted to determine the sensitivity of different data and their privacy score, which ensures the applicability of data privacy score. It can solve the problem that some users care little about the privacy issue. We propose a privacy score query and response mechanism based on data encryption. On this basis, the dynamic adjustment method of the popularity threshold is designed for data uploading. Experiment results and analysis show that the proposed scheme based on threshold dynamic adjustment has decent scalability and practicability.

Fog assisted task allocation and secure deduplication using 2FBO[formula omitted] and MoWo in cluster-based industrial IoT (IIoT)

Fog-assisted Internet-of-Things have dynamically received interest in the research community. There are more duplicate data transmitted over the Internet due to the rise in IoT devices. In this paper, task allocation and secure deduplication are implemented over four layers of Fog assisted Cluster-based Industrial IoT. IoT device layer used to sense data and mitigate security threats. Devices presented in this layer are registered to the cloud server using Elliptic Curve Cryptography based Hybrid Multiplier. Multi-Objective based Whale Optimization algorithm is accessible for cluster section. SHA-3 is presented in the fog layer for secure data deduplication. ECC HM private key is used for data encryption before transmitted the data. In cloud, layer indexing is constructed using Merkle Hash Tree. This is endeavored to provide query search results for IoT users at the service layer. The simulation results prove enhancement in average latency, user satisfaction, network lifetime, energy consumption, and security strength.

Secure Data De-Duplication Based on Threshold Blind Signature and Bloom Filter in Internet of Things

Within the cloud environment, the availability of storage, as well as bandwidth, can be effectively preserved in virtue of data de-duplication. However, refraining redundancy from additional storage or communication is not trivial due to security concerns. Though intensive researches have been addressed on a convergent cryptosystem for secure data de-duplication, the conflicts amongst functionality, confidentiality, and authority remain unbalanced. More concretely, although data are obfuscated under convergent encryption, a violent dictionary attack is still efficacious since the whole pseudorandom process relies heavily on plaintexts. As for data ownership, the download privilege, which depends on hash value, may also be infringed due to the same reason. To dispose of these problems, we presented a conspiracy-free data de-duplication protocol based on a threshold blind signature in this article. With the help of multiple key servers, the outsourced file and de-duplication label will be computationally indistinguishable from random strings. We used the Boom filter as a tool to implement a proof of ownership, ensuring that the ownership claims made by users are real. It effectively prevents the attacker from using the stolen tag to get the whole file to gain file access without authorization. The most significant innovation of this article is to use homomorphism computation to aggregate and generate partial signature tags, and to introduce a secret sharing mechanism based on The Chinese Remainder Theorem to hide signature keys, thus balancing the security concerns of cloud and client. Compared with existing schemes, both communication and computation performances are preferable in our protocol. As far as we know, our scheme is the only data de-duplication scheme that satisfies the semantic security of ciphertext and label.

Secure Enterprise and Read Performance Enhancement in Data Deduplication for Secondary Storage

With the tremendous growth of available digital data, the use of Cloud Service Providers (CSPs) are gaining more popularity, since these types of services promise to provide convenient and efficient storage services to end-users by taking advantage of a new set of benefits and savings offered by cloud technologies in terms of computational, storage, bandwidth, and transmission costs. we propose solutions for different data types (text, image and video) for secure data de-duplication in cloud environments. Our schemes allow users to upload their data in a secure and efficient manner such that neither a semi-honest CSP nor a malicious user can access or compromise the security of the data. Moreover, we propose proof of storage protocols including Proof of Retrievability (POR) and Proof of Ownership (POW) so that users of cloud storage services are able to ensure that their data has been saved in the cloud without tampering or manipulation. Experimental results are provided to validate the effectiveness of the proposed schemes. proposes a method to improve the read performance by investigating the recently accessed chunks and their locality in the backup set (data stream). Based on this study of the distribution of chunks in the data stream, few chunks are identified that need to be accumulated and stored to serve the future read requests better. This identification and accumulation happen on cached chunks. By this a small degree of duplication of the de-duplicated data is introduced, but by later caching them together during the restore of the same data stream, the read performance is improved. Finally the read performance results obtained through experiments with trace datasets are presented and analyzed to evaluate the design.

A secure deduplication scheme for encrypted data

Cloud storage (CS) is gaining much popularity nowadays because it offers low-cost and convenient network storage services. In this big data era, the explosive growth in digital data moves the users towards CS to store their massive data. This explosive growth of data causes a lot of storage pressure on CS systems because a large volume of this data is redundant. Data deduplication is a most-effective data reduction technique that identifies and eliminates the redundant data. Dynamic nature of data makes security and ownership of data as a very important issue. Proof-of-ownership schemes are a robust way to check the ownership claimed by any owner. However to protect the privacy of data, many users encrypt it before storing in CS. This method affects the deduplication process because encryption methods have varying characteristics. Convergent encryption (CE) scheme is widely used for secure data deduplication, but it destroys the message equality. Although, DupLESS provides strong privacy by enhancing CE, but it is also found insufficient. The problem with the CE-based scheme is that the user can decrypt the cloud data while he has lost his ownership. This paper addresses the problem of ownership revocation by proposing a secure deduplication scheme for encrypted data. The proposed scheme enhances the security against unauthorized encryption and poison attack on the predicted set of data.

Secure and efficient big data deduplication in fog computing

With the rapid development of the Internet of Things, the massive amount of big data generated by the Internet of Things terminals and the real-time processing requirements have brought enormous challenges. A two-tier computing model consisting solely of two entities, cloud and user, will not be sufficient to support processing large numbers of concurrent data requests. Therefore, fog computing was proposed. How to realize the secure and efficient deduplication of ciphertext in fog computing has become a new research topic. In this paper, we firstly present a new decentralized deduplication structure and then show how to apply it to construct a secure and efficient big data deduplication scheme in fog computing. The cloud server, in the proposed paper, can quickly determine which fog server needs to be traversed to search duplicate data, and instead of traversing all fog servers. This significantly improves the efficiency of big data deduplication in fog computing. Furthermore, the proposed scheme allows fog server to verify whether the user possesses the ownership of the data. Performance analysis and experimental results show the proposed scheme has less overheads than existing schemes.

Secure Data Deduplication with Reliable Data Deletion in Cloud

A cloud storage system which incorporates the deletion and deduplication functionalities will have both security and efficiency advantages over exiting solutions which provide only one of them. However, the security models of secure data deletion and data deduplication functionalities are not compatible with each other, which will cause security and efficiency vulnerability under coercive adversaries. To solve these security and efficiency challenges, we define and construct a scheme, whose security relies on the proper erasure of keys in the wrapped key tree and periodical update of the deduplication encryption keys. Moreover, we enhance the efficiency of the proposed scheme by introducing incremental data update, where only the changed part is encrypted/decrypted and uploaded/downloaded in data updating. Further security analysis shows that the proposed scheme is secure against coercive attack. Finally, the practical implementation shows that our scheme is performance efficient in computation, storage and communication for both the cloud storage server and users.

Secure Deduplication Scheme for Cloud Encrypted Data

Cloud storage (CS) is gaining much popularity nowadays because it offers low-cost and convenient network storage services. In this big data era, the explosive growth in digital data moves the users towards CS but this causes a lot of storage pressure on CS systems because a large volume of this data is redundant. Data deduplication is an effective data reduction technique. The dynamic nature of data makes security and ownership of data as a very important issue. Proof-of-ownership schemes are a robust way to check the ownership claimed by any owner. However, this method affects the deduplication process because encryption methods have varying characteristics. A convergent encryption (CE) scheme is widely used for secure data deduplication. The problem with the CE-based scheme is that the user can decrypt the cloud data while he has lost his ownership. This article addresses the problem of ownership revocation by proposing a secure deduplication scheme for encrypted data. The proposed scheme enhances the security against unauthorized encryption and poison attack on the predicted set of data.

CSED: Client-Side encrypted deduplication scheme based on proofs of ownership for cloud storage

As digital data are explosively generated nowadays, data management becomes a critical problem, which makes cloud storage services important and popular. In reality, the storage overhead can be reduced significantly by performing date deduplication. Among the outsourced data, some of them are very personal and sensitive, and should be prevented for any leakage. Generally, if clients conventionally encrypt the data, deduplication is lost. Message-locked encryption (MLE) is a cryptographic primitive supporting encrypted data deduplication. A secure client-side deduplication scheme can be built upon MLE to reduce both communication and computation overhead for cloud storage systems, where a client interacts with the cloud server to check the duplicate data and only the data which has not been outsourced by other clients before is required to be uploaded. However, existing client-side encrypted data deduplication schemes are confronted with brute-force attacks that can recover files falling into a known set. Furthermore, existing schemes are vulnerable to illegal content distribution attacks, where the adversary can distribute data to other users via the cloud server without detecting. In this paper, we propose a secure and efficient client-side encrypted data deduplication scheme (CSED). In CSED, a dedicated key server is introduced in generating MLE keys to resist brute-force attacks. We propose a Bloom filter-based proofs of ownership (PoW) mechanism and integrate it into CSED to resist illegal content distribution attacks. Moreover, a hierarchical storage architecture is employed to improve the I/O efficiency on the cloud server. Security analysis and performance evaluation demonstrate that CSED is secure and efficient.

Attribute Based Storage Supporting with Secure De-Duplication in Cloud

zAttribute based Encryption has been widely used in cloud computing for secure encryption of our files into the cloud. It controls the users who can access our data. It enables the users only with specific credentials to access our data. But this standard Attribute based Encryption does not support Deduplication of data, which helps in saving both the storage and network bandwidth of an organization. In this paper, we propose a system setting in a hybrid cloud, which will provide both features of secure encryption and data Deduplication.

Toward Edge-Assisted Internet of Things: From Security and Efficiency Perspectives

As we are moving toward the Internet of Things (IoT) era, the number of connected physical devices is increasing at a rapid pace. Mobile edge computing is emerging to handle the sheer volume of produced data and reach the latency demand of computation-intensive IoT applications. Although the advance of mobile edge computing on service latency has been well studied, security and efficiency on data usage in mobile edge computing have not been clearly identified. In this article, we examine the architecture of mobile edge computing and explore the potential of utilizing mobile edge computing to enhance data analysis for IoT applications while achieving data security and computational efficiency. Specifically, we first introduce the overall architecture and several promising edge-assisted IoT applications. We then study the security, privacy, and efficiency challenges in data processing for mobile edge computing, and discuss the opportunities to enhance data security and improve computational efficiency with the assistance of edge computing, including secure data aggregation, secure data deduplication, and secure computational offloading. Finally, several interesting directions on edge-empowered data analysis are presented for future research.

Secure Deduplication Based on Rabin Fingerprinting over Wireless Sensing Data in Cloud Computing

The rapid advancements in the Internet of Things (IoT) and cloud computing technologies have significantly promoted the collection and sharing of various data. In order to reduce the communication cost and the storage overhead, it is necessary to exploit data deduplication mechanisms. However, existing data deduplication technologies still suffer security and efficiency drawbacks. In this paper, we propose two secure data deduplication schemes based on Rabin fingerprinting over wireless sensing data in cloud computing. The first scheme is based on deterministic tags and the other one adopts random tags. The proposed schemes realize data deduplication before the data is outsourced to the cloud storage server, and hence both the communication cost and the computation cost are reduced. In particular, variable-size block-level deduplication is enabled based on the technique of Rabin fingerprinting which generates data blocks based on the content of the data. Before outsourcing data to the cloud, users encrypt the data based on convergent encryption technologies, which protects the data from being accessed by unauthorized users. Our security analysis shows that the proposed schemes are secure against offline brute-force dictionary attacks. In addition, the random tag makes the second scheme more reliable. Extensive experimental results indicate that the proposed data deduplication schemes are efficient in terms of the deduplication rate, the system operation time, and the tag generation time.