Level Deduplication Research Articles

Improved storage capacity for health care center using ant lion- ramp secret sharing scheme for data deduplication in cloud computing

The Cloud-based storage is able to store more information in gigabyte size in all formats such as text, image or video and it can access at any time with their login credentials. In such a system, reducing the duplication of data and increasing security is an important factor for efficient storage. In this work, the file level de-duplication process is applied on the Magnetic Resonance Imaging (MRI) brain image by reducing the shares of the image to retrieve an original image from the cloud. To reduce the storage problem in this an optimization-based RSSS is used. The objective of this investigation is to decrease the storage blow-up problem in Cloud storage and reduce the duplicate files in the Cloud storage of the health care centre. The proposed model comprises of two subsets: In the first set, the input image is divided into a number of shares using RSSS scheme. In the second set, the minimum share is determined by using the optimization process and it is encrypted and it is stored in the Cloud. Initially, the image is divided into number of shares for reconstructing using the ramp secret sharing scheme.Without these shares, the original image cannot be recovered. But storing all the shares result in high storage capacity. It is overcome with the help of Ant Lion optimization (ALO) to determine the minimum number of shares required for recovering the image. The ALO works to minimizing the Mean Square Error (MSE) of the image reconstruction to find the minimum shares. Then, the minimum shares are encrypted and converted into hash keys. Those hash keys are stored in the Cloud storage. The proposed ALO-RSSS is achieved its objective by reducing the shares to 2 as compared to the traditional method as well as the PSNR is 27% improved.

Data De-Duplication Process and Authentication Using ERCE with Poisson Filter in Cloud Data Storage

The cloud storage is essential environment for users to access high confidential data. Every single data is most valued by users. If we count, day by day information as well as, memory storage are increasing gradually. Cost of memory will increase when data increases with demand for storage. At present data duplication or redundant data storing in the cloud became hard for storage provider. Also, it makes security issue if repeated data from various users stored in the server. It makes data duplication, which is very efficient for intruders. Also, when same data stored in cloud, there will be a waste of storage. Our research article is focused on security of original data by generating the key from owner and identifying the repeated data, while storing in cloud platform. This process is called as data de-duplication, which is also known as intelligent based computing. Storing the data in single instance is very challenging among cloud computing researchers. In this article we propose a content level de-duplication with re-encryption using enhanced Randomized convergent encryption (ERCE) based on Poisson filter (PF). First the data is encrypted and re-encrypted using the cipher methodology. Generated key only stored and processed by authenticated user. Owner of the data give permission to access key. Then the Poisson filter is used in de-duplication process. if key is authenticated, then the authenticated user can access data from cloud server. Data is stored only once and accessing key decides who can access the data. The result is evaluated with various existing algorithm. our proposed algorithm proves less time in downloading file and less computation cost when comparing with existing system.

Backup Storage Block Level Deduplication with DDUMBFS and BACULA

Backup Storage Block Level Deduplication with DDUMBFS and BACULA

Backup Storage Block Level Deduplication with DDUMBFS and BACULA

Backup Storage Block Level Deduplication with DDUMBFS and BACULA

Multi keyword searchable attribute based encryption for efficient retrieval of health Records in Cloud

Personal Health Record (PHR) is an online electronic application used by patients to store, retrieve and share their health information in a private and secure environment. While outsourcing the PHR into cloud environment, there exist issues in privacy while storing, searching and sharing of health information. To overcome these issues, an efficient retrieval of health records using Multi Keyword Searchable Attribute Based Encryption (MK-SABE) is proposed. To manage the increasing PHR data in the cloud, an Authorized File Level Deduplication technique is adopted. It eliminates redundant files, thereby reducing the communication overhead. Moreover, PHR data is encrypted before outsourcing and to perform searching over encrypted data, the proposed MK-SABE introduces Conjunctive Multi Keyword Searchable Attribute Based Encryption (CM-SABE). This maintains the searchable property after encryption for efficient retrieval of health files using range query. Further to ensure the trustworthiness while sharing the sensitive data, MK-SABE introduces the Location Based Encryption (LBE) and Dynamic Location Based ReEncryption (DLBRE) technique to provide additional security. From the experimental analysis, it is proved that the proposed MK-SABE reduces the storage complexity by 5%, keyword search time by 25% and improves the overall performance of PHR by 40% compared to the existing schemes.

WEIGHT BASED DEDUPLICATION FOR MINIMIZING DATA REPLICATION IN PUBLIC CLOUD STORAGE

The approach to minimize data replication in cloud storage is one of the challenging issues to process text data. The amount of digital data has been increasing exponentially. There is a need to reduce the amount of storage space by storing the data efficiently. In cloud storage environment, the data replication provides high availability with fault tolerance system. An effective approach of deduplication system using a weight based method is proposed at the target level in order to reduce the storage space in cloud. Storage space can be efficiently utilized by removing the unpopular files from the secondary servers. Target level consumes less processing power than Source level deduplication. Input text documents are stored into Dropbox cloud. The Term Frequency (TF) and Named Entity Recognition (NER) of the documents are found. The text features found are stored in database using MySQL. After storing features in database, fresh text documents are collected to find popular and unpopular files. TF and NER are found for the freshly collected text documents and duplicate features are removed to compare with the features stored in the database. On comparison, relevant text documents are listed. After listing text documents, document frequency, document weight and threshold factor are found. Depending on threshold factor, the popular and unpopular files are detected. The popular files are replicated in all the storage nodes to achieve availability. Before deduplication, the storage space occupied in the Dropbox cloud is 8.09MB. After deduplication, the unpopular files are removed from secondary storage nodes and the storage space occupied in the Dropbox cloud is 4.82MB. Finally, data replications are minimized and 45.6% of the cloud storage space is efficiently saved by applying weight based deduplication.

Enhanced Data Security for Hybrid Encryption and Compression using Deduplication

Distributed computing is an arising innovation that gives the information capacity administration and application getting to support in on the web. It has a few preferences and special attributes, for example, request self-administration, expansive network access, asset pooling, quick flexibility and estimated administration. It handles enormous information in powerful climate with simple method of information access and more significant level of security with protection. The proposed strategy is to stay away from information duplication over distributed storage and give security. The information deduplication strategy is utilized to diminish the capacity measure and improve the capacity execution. Here intelligent Message Locked Encryption and Convergent Encryption calculation are utilized. The proposed framework client data is scrambled and re-encoded once more. The capacity space can be decreased with assistance of square level deduplication strategy. The cipher text is scrambled as same as the indistinguishable pieces. The security examination is contrasted and person calculation and half and half calculation. Consequently this proposition demonstrates that the proposed calculation produce the better outcome contrasted with the current calculations.

Secure Block Level De-Duplication on Big Data using Proof of Ownership

De-duplication technology is commonly used to decrease the space and bandwidth requirements of services by eliminating repeated data and store only a single copy of them. Unfortunately, it raises issues relating to security and ownership such as, unauthorized users may claim as owner of that file and security threat from curious server itself. To overcome these issues, Secure Block level de-duplication with Proof of Ownership Scheme is proposed in this work. Proof-of-Ownership Scheme allows any owner of the same data to prove to the server that he owns the data in a robust way. This scheme uses convergent encryption and it protects the data from attackers and honest but curious server. It also reduces storage space efficiently by checking the uniqueness of data at block level

Encryption to Decryption Statistics for Secure Data with Independent Servers for Deduplication

Encoding information on the customer side before transferring it to distributed storage is fundamental for ensuring clients’ protection. Anyway customer side encryption is inconsistent with the standard routine with regards to deduplication in distributed storage administrations. Deduplication of records on the capacity servers has been very much clarified on different arrangements which utilized autonomous servers. An answer has been proposed considering customer side encryption and server side deduplication and guaranteeing security while utilizing a solitary server for the entire procedure. This arrangement was tended to at record level and had the thought to actualize square level deduplication utilizing a similar deduplication convention. We examined the odds of executing square level deduplication considering the single server record level deduplication plan of the base convention. We executed and examined this situation and played out an unpleasant examination with the record level deduplication by recreation. Both the plans appeared to be similar. We exhibit that our plan gives preferable security ensures over past efforts. We present the main secure cross-client deduplication plot that supports customer side encryption without requiring any extra autonomous servers. Curiously, the plan depends on utilizing a PAKE (secret word confirmed key trade) convention.

Minimal re-computation for exploratory data analysis in astronomy

Abstract We present a technique to automatically minimise the re-computation when a data analysis program is iteratively changed, or added to, as is often the case in exploratory data analysis in astronomy. A typical example is flagging and calibration of demanding or unusual observations where visual inspection suggests improvement to the processing strategy. The technique is based on memoization and referentially transparent tasks. We describe the implementation of this technique for the CASA radio astronomy data reduction package. We also propose a technique for optimising efficiency of storage of memoized intermediate data products using copy-on-write and block level de-duplication and measure their practical efficiency. We find that the minimal recomputation technique improves the efficiency of data analysis while reducing the possibility for user error and improving the reproducibility of the final result. It also aids exploratory data analysis on batch-schedule cluster computer systems.

DIFFERENT SECURE DATA DEDUPLICATION APPROACHES FOR CLOUD STORAGE: A REVIEW

With the fast growth of the current era, world are moving to digital storage for accomplishment purposes, there is a rising demand for systems that can deliver safe data storage in effective manner. As more corporate and private users outsource their data to cloud storage providers, new data breach incidents make end-to-end encryption progressively prominent obligation. With the growing awareness of data privacy, more and more cloud users choose to encrypt their sensitive data before outsourcing them to the cloud storage. Data deduplication is single instance data storage widely used in cloud storage system to reduce space and upload bandwidth. Duplication-less storage system which de-duplicates the data using file level deduplication and block level deduplication. In this paper we gives an overview of the state of data Deduplication approachesand describes the various application to produce specification according need of current generation.

Secure and duplication detection in cloud using cryptographic hashing method

De-duplication systems are adopting De-duplication strategies such as client side or server side De-duplication.Particularly, in the beginning of cloud storage, data De-duplication techniques happen to add importance to store original volumes of data in the cloud. This technique motivates the enterprise and organization to farm out data storage to cloud service providers, as proof of several case studies was done.Block level de-duplication is used to discover, removes redundancies compare with previously stored information. The file will be separated into smaller segment as given by the system size or uneven size blocks or chunks whatever we need. Using predetermined size of blocks, the system can simplify the computations of block limits, although using uneven size blocks provides improved de-duplication. Secure Cloud introduce a new concept to audit entities with continuation of a Map Reduce cloud, which is used to help the client is easy to make data tags before feeding the data and audit with theprobity of data is analysed and stored in the cloud. the subject of previously finished work is fixed so that the computational load at user or auditor has huge amount of tag making. In accumulation, Secure Cloud also enables secure de-duplication. Perceive that the “validity” measured in Secure Cloud is the deterrence of leakage, the side channel information. In order to check the leakage of such side channel data, the work accept the custom and design a proof of rights protocol amid cloud servers and clients, which permit clients to confirm the cloud servers what they exactly own the object information.

Backup Storage Block Level Deduplication with DDUMBFS and BACULA

Backup Storage Block Level Deduplication with DDUMBFS and BACULA

English

English

Detection of File Level and Block Level Deduplication and attacks in Cloud Computing Environment

In cloud computing, security and storage space management techniques are most important factors for improving the performance of cloud computing. Secure deduplication is a technique for eliminating duplicate copies of storage data, and provides security to them. To reduce storage space and upload bandwidth in cloud storage The basic idea in this paper is that we can eliminate duplicate copies of storage data and limit the damage of stolen data if we decrease the value of that stolen information to the attacker. Dekey new construction in which users do not need to manage any keys on their own but instead securely distribute the convergent key shares across multiple servers for insider attacker. User profiling and decoys, then, serve two purposes. First one is validating whether data access is authorized when abnormal information access is detected, and second one is that confusing the attacker with bogus information.

Secure De-Duplication using Convergent Keys (Convergent Cryptography) for Cloud Storage

One very important challenges of today’s cloud storage services is that the management of the ever-increasing amount of data. To create data management scalable, de-duplication has been a widely known technique to condense space for storing and transfer information measure in cloud storage. Rather than keeping multiple data copies with an equivalent content, de-duplication eliminates redundant data by keeping just one physical copy and referring different redundant data to it copy. Convergent Encryption, additionally called content hash keying, could be a cryptosystem that produces identical ciphertext from identical plaintext files. Data de-duplication could be a technique for eliminating duplicate copies of data, and has been wide employed in cloud storage to scale back space for storing and transfer information measure. Promising because it is, associate degree arising challenge is to perform secure de-duplication in cloud storage. Currently daily the foremost arising challenge is to perform secure de-duplication in cloud storage. Though Convergent Encryption has been extensively adopted for secure de-duplication, a essential issue of creating Convergent Encryption sensible is to with efficiency and dependably manage a large range of convergent keys. we have a tendency to initial introduce a baseline approach during which file level de-duplication is performed and it doesn't support on-line approach. For that purpose the current context have a tendency to formally address the matter of achieving economical and reliable key management in secure de-duplication. Within the planned design Dupkey, users don’t ought to manage any keys on their own. It implements de-duplication at block level and provides a lot of security. Dupkey is on-line approach and supply deduplication and reduces DB size on cloud.

Auditable versioned data storage outsourcing

Auditability is crucial for data outsourcing, facilitating accountability and identifying data loss or corruption incidents in a timely manner, reducing in turn the risks from such losses. In recent years, in synch with the growing trend of outsourcing, a lot of progress has been made in designing probabilistic (for efficiency) provable data possession (PDP) schemes. However, even the recent and advanced PDP solutions that do deal with dynamic data, do so in a limited manner, and for only the latest version of the data. A naive solution treating different versions in isolation would work, but leads to tremendous overheads, and is undesirable. In this paper, we present algorithms to achieve full persistence (all intermediate configurations are preserved and are modifiable) for an optimized skip list (known as FlexList) so that versioned data can be audited. The proposed scheme provides deduplication at the level of logical, variable sized blocks, such that only the altered parts of the different versions are kept, while the persistent data-structure facilitates access (read) of any arbitrary version with the same storage and process efficiency that state-of-the-art dynamic PDP solutions provide for only the current version, while commit (write) operations incur around 5% additional time. Furthermore, the time overhead for auditing arbitrary versions in addition to the latest version is imperceptible even on a low-end server. Additionally, the application of our approach opens up the possibility to naturally support block level deduplication. While a naive solution to audit versions would copy the whole data and the data structure for each version, our solution utilizes storage space amounting very close to the most efficient delta-based solutions. Accordingly, we explore how the proposed data structure benefits the system with block level deduplication besides adding auditability property, and how it can be integrated with a state-of-the-art versioning system (Git), and in the process scale the storage efficiency of Git, and thus help scale the size of data to be stored in Git, without compromising the retrieval efficiency of arbitrary versions.

MMSD: a Metadata-Aware Multi-Tiered Source Deduplication Cloud Backup System in the Personal Computing Environment

The deduplication efficiency/overhead ratio of existing source deduplication solutions in cloud backup systems, in despite of their complex deduplication workflows, was not ideal due to their insufficient study of file semantics. In this paper, we present MMSD, a metadata-aware multi-tiered source deduplication cloud backup system in personal computing environment, to obtain an optimal tradeoff between the deduplication efficiency and deduplication metadata storage overhead and finally achieve a shorter backup window than existing approaches. MMSD makes full use of file metadata including file size, type, timestamp, path information, modification frequency, and the advantage of whole file level deduplication compared with chunk level deduplication, and efficiently combines these two deduplication levels. Our experimental results with real world datasets show that, compared with the state-of-art source deduplication methods, MMSD can improve the overall deduplication efficiency from 75% to 91% with only 33.8% of deduplication metadata storage overhead and shorten the backup window by at least 54.2%.

Development and Utilization of Best Practice Operational Guidelines for Immunization Information Systems

To describe collaborative efforts to develop best practice operational guidelines for immunization information systems (IIS) and discuss awareness, acceptance, and utilization of the guidelines by the IIS community. Business analysis and facilitation techniques were used to support collaboration among IIS stakeholders who analyzed existing practices, brainstormed new approaches, and developed consensus-based recommendations. The guideline development process and its impact on IIS were evaluated using a postworkshop questionnaire, the IIS Annual Report, and an online survey for one of the guidelines. Immunization information systems domain in the United States. Staff from IIS; federal agencies; state, regional, and county health departments; privately run registries; and vendors/consultants. These included (1) completion of best practice guidelines; (2) degree of satisfaction among the participating IIS stakeholders with the work process and outcomes, interest, and willingness to implement best practice guidelines in their own IIS; (3) awareness and use of, as well as satisfaction with, the guidelines for patient active/inactive immunization status among IIS; and (4) acceptance and implementation of best practice guidelines across the US IIS. Since 2005, operational guidelines (that contain best practice recommendations) have been developed for 5 IIS functional areas: reminder/recall notifications, data quality assurance, vaccination level deduplication, management of patient active/inactive status, and adverse events reporting. Immunization information systems stakeholders who participated in the development of the recommendations expressed a high level of satisfaction with the process and the recommendations. Based on IIS Annual Report data, from 2007 to 2009 use of the guidelines increased from 46% to 80% of IIS. Best practice recommendations offer practical guidelines on the most challenging operational areas for IIS. Initial evidence indicates adoption of the recommendations and satisfaction with the development process. Additional studies are needed to assess the degree of guidelines use across the IIS community.

Launching enterprise data backup and recovery solutions: the case of Ozonetel

Ozonetel Systems Pvt. Ltd. is a company set up in 2008 by entrepreneur CSN Murthy, as a subsidiary of Rocsys Technology. Ozonetel systems have focused on the data, network and voice security offerings for enterprises. The advent of the internet has brought in several challenges in terms of security of data and network systems. While big companies can develop their own security products, or can buy them, small and medium companies have neither the financial muscle nor technological strength for doing this. Ozonetel wanted to target SMEs across various sectors. Ozonetel developed a solution with features like automatic storage from host computers, incremental backup, block level de-duplication, etc. However, the USP of the product was 'customised data backup and recovery solution at half the price of MNCs'. The company faced problems in reaching different market sectors despite being based in Hyderabad, a software hub. Determined to exploit its first mover advantage, Ozonetel decided to study the buying behaviour for IT usage across sectors and merge the results with an earlier study focusing on lack of market penetration of MNCs. With the help of a few marketing interns from the Indian Institutes of Management (IIMs), Ozonetel carried out this study in and around Hyderabad.