Permanent Data Loss Research Articles

Data recovery methods for DNA storage based on fountain codes

Today's digital data storage systems typically offer advanced data recovery solutions to address the problem of catastrophic data loss, such as software-based disk sector analysis or physical-level data retrieval methods for conventional hard disk drives. However, DNA-based data storage currently relies solely on the inherent error correction properties of the methods used to encode digital data into strands of DNA. Any error that cannot be corrected utilizing the redundancy added by DNA encoding methods results in permanent data loss. To provide data recovery for DNA storage systems, we present a method to automatically reconstruct corrupted or missing data stored in DNA using fountain codes. Our method exploits the relationships between packets encoded with fountain codes to identify and rectify corrupted or lost data. Furthermore, we present file type-specific and content-based data recovery methods for three file types, illustrating how a fusion of fountain encoding-specific redundancy and knowledge about the data can effectively recover information in a corrupted DNA storage system, both in an automatic and in a guided manual manner. To demonstrate our approach, we introduce DR4DNA, a software toolkit that contains all methods presented. We evaluate DR4DNA using both in-silico and in-vitro experiments.

Preservation Strategies for Biodiversity Data

Preservation Strategies for Biodiversity Data

Ransomware prevention using moving target defense based approach

AbstractOver the past decade, there has been a rapidly rising trend of malware (ransomware) that limits user access by encrypting the data and demanding the ransom against the decryption key. In most cases, such encryption may lead to a permanent data loss. In order to prevent this unwanted encryption, we propose a method based on Moving Target Defense (MTD) approach. Our method is based on the alteration of the attack surface to reduce the attack success ratio. We have used multiple layers of MTD. The first layer generates random extensions that hide the existing known file extensions. This will protect user files against those ransomware variants which encrypt files having some specific extensions. Our second layer of protection uses event‐based MTD in which tasks are scheduled to change file extensions at the occurrence of specific events which mostly occur due to the execution of ransomware in the system. As a result of our proposed method, we have successfully protected user files against well‐known ransomware variants such as WannaCry, Cerber, Locky, Tesla, Revil, Bitlocker, Darkside, Ranzy.

Data Repair-Efficient Fault Tolerance for Cellular Networks Using LDPC Codes

The base station-mobile device communication traffic has dramatically increased recently due to mobile data, which in turn heavily overloaded the underlying infrastructure. To decrease Base Station (BS) interaction, intra-cell communication between local devices, known as Device-to-Device, is utilized for distributed data caching. Nevertheless, due to the continuous departure of existing nodes and the arrival of newcomers, the missing cached data may lead to permanent data loss. In this study, we propose and analyze a class of Low-Density Parity Check (LDPC) codes for distributed data caching in cellular networks. Contrary to traditional distributed storage, a novel repair algorithm for LDPC codes is proposed which is designed to exploit the minimal direct BS communication. To assess the versatility of LDPC codes and establish performance comparisons to classic coding techniques, novel theoretical and experimental evaluations are derived. Essentially, the theoretical/numerical results for repair bandwidth cost in presence of BS are presented in a distributed caching setting. Accordingly, when the gap between the cost of downloading a symbol from BS and from other local network nodes is not dramatically high, we demonstrate that LDPC codes can be considered as a viable fault-tolerance alternative in cellular systems with caching capabilities for both low and high code rates.

Storage and Proximity Management for Centralized Personal Health Records Using an IPFS-Based Optimization Algorithm

Centralized Personal Health Records (PHR) are mutable with compromised security as it may lead to a single point of failure. Confidentiality, protection and security are the common issues in clinical record frameworks. Specific security and protection schemes are being used to secure clinical records. Accordingly, using the Interplanetary File System (IPFS), a decentralized PHR can be maintained to allow patients to access their records without delay. Moreover, a Kademlia-based distributed hash table provides fault tolerance and enables patients to keep track of their medical history. However, a significant issue in IPFS is data availability. It is only available on the web until users or hosts of the network request each peer, later it leads to a permanent loss of data. We propose an architecture that aims to provide faster retrieval and constant PHR availability using Blockchain and IPFS. The results show that an optimal node is selected in each iteration amongst all the available adjacent nodes.

Anomaly Detection Model for Predicting Hard Disk Drive Failures

ABSTRACT The electromechanical design of the HDD (Hard Disk Drive) renders it more susceptible to failures than other components of the computer system. The failure of HDD leads to permanent data loss, which is typically more expensive than HDD itself. The SMART (Self-Monitoring, Analysis and Reporting Technology) system warns the user if any HDD parameter has exceeded the predefined threshold value needed for safe HDD operation. Machine learning methods take advantage of dependence between multiple SMART parameters in order to make failure prediction more precise. In this paper, we present a failure prediction model based on the anomaly detection method involving an adjustable decision boundary. SMART parameters are ranked by the importance and the 13 most significant ones are used as the initial feature set in our model. In the following stage, we optimized the feature set by removing those that have no major contribution to the anomaly detection model, forming the final feature set comprising seven features only. The proposed anomaly detection model achieved 96.11% failure detection rate on average, with 0% false detection rate in ten random tests. The proposed model predicted more than 80% of failures 24 hours before their actual occurrence, which enables timely data backup.

Avoiding data loss and corruption for file transfers with Fast Integrity Verification

End-to-end integrity verification is used to avoid silent data corruption in file transfers by comparing the checksum of files at source and destination end points. However, it increases transfer times significantly as checksum computation requires reading files back from the storage and running compute-intensive hash computation. In this paper, we propose Fast Integrity VERification (FIVER) algorithm which alleviates the overhead of end-to-end integrity verification by overlapping checksum computation with transfer operation and enabling I/O sharing between the two. The results obtained from various network and dataset settings show that FIVER is able to bring down the cost of end-to-end integrity verification from up to 120% by the state-of-the-art solutions to below 15%. Moreover, existing implementations of end-to-end integrity verification are vulnerable to permanent data loss in the case of an unexpected power outage due to completing the integrity verification process while data is still on memory. FIVER addresses this issue by enforcing dirty data on memory to be flushed to disk before finishing integrity verification such that power loss during any phase of a transfer would cause integrity verification to fail and transfer application to retransfer lost data.

An Efficient Data Auditing Protocol With a Novel Sampling Verification Algorithm

Currently, data auditing is an important means to check the integrity of the data stored on the cloud. Existing data auditing schemes have done a lot of work in terms of functionality, implementation, and security. As we can see, all of these schemes are based on the auditing framework proposed by Ateniese et al., which uses random sampling to probabilistically check whether the data is integrity or not. But multiple repeated sampling may lead to an intersection between the challenge sets selected each time, reducing the probability of detection, and the time for detecting corrupted blocks will be indefinite. If the corrupted data is not found for a long time, the data remedial measures will become invalid, resulting in permanent data loss. To address the above problem, we designed an efficient sampling verification algorithm. Based on this algorithm, the auditing scheme is further optimized, and a dynamic auditing function has been developed for the scheme to facilitate data owners to update data. Through the security analysis, our scheme is soundness. The experiment shows that our sampling verification can detect corrupted blocks faster than other auditing schemes.

Data rescue in manuscripts: a hydrological modelling study example

ABSTRACTRecently, data statements have been required in hydrological manuscripts. Authors are usually confused by different data management policies. Thus, a frequent response is to write down “data available upon request from the corresponding author”. Such a simplified response may lead to dormancy or permanent loss of data. We present an example of data management from a published hydrological modelling study. We started from figures with complete scientific stories in the publication and traced backwards to find relevant source data through workflows (linking). In this manner, the source data can be identified (archiving) and preserved (hosting). Such simple practices of data linking, archiving and hosting can rescue data and require only limited effort by the authors. We suggest that this data rescue process(es) should become routine in the scientific publication process. The data rescued with workflows can significantly improve the research transparency and reusability, which leads to a fairer and more open culture in hydrology.

Automated SWIR based empirical sun glint correction of Landsat 8-OLI data over coastal turbid water

The optimal exploitation of the oceanic information provided by recent high spatial resolution sensors such as Landsat 8-OLI is strongly conditioned by the quality of the water reflectance signal retrieval. One main issue stands in the ability to correct water pixels for the contamination of the sun glint, which might induce a seasonal or permanent loss of data according to the latitude. The SWIR information now provided for the most recent high spatial resolution sensors was used for evaluating the sun glint level and correcting the radiative signal for its effect. This has been performed transposing historical empirical formalisms based on the NIR signal. An automated SWIR-based sun glint correction procedure was then developed using a 4-year OLI archive gathered over very turbid waters of French Guiana (227 scenes). This procedure allows the practical limitations associated with past similar empirical methods (sensitivity to water turbidity and manual image per image correction) to be overcome. While a satisfactory preservation of the information over sun glint free pixels was observed, comparison exercises based on in situ Rrs data gathered in sun glint affected areas emphasize the relevance of the proposed methodology (correction by a factor of 14 of the averaged bias in the Rrs values after removing sun glint effects). Current limitations in the applicability of this SWIR-based empirical automated method are mainly associated with the presence of high cloud coverage, thin clouds in the OLI scene or highly spatially variable marine or atmospheric signal (around 47%, 42% and 11%, respectively, of the total of 12% of failure over French Guiana OLI archive). The potential large applicability of the procedure developed in this work was eventually demonstrated over contrasted coastal environments.

Global Biotic Interactions: A Catalyst For Integrating Existing Interaction Datasets, Connecting Data Curators And Developing Data Exchange Methods

Since 2013, Global Biotic Interactions (GloBI, globalbioticinteractions.org, Poelen et al. 2014) has taken an opportunistic, decentralized approach to integrating, and make accessible, existing species interaction datasets. Rather than expecting dataset curators to conform to some publication regime, methods were developed to automatically and algorithmically discover, parse and link existing datasets without the need to reformat, relocate, or transfer ownership of, the existing dataset. The automated nature of GloBI helps to: (a) automate propagation of dataset updates (b) quickly detect data integration issues (e.g. outage, change in data format), (c) integrate new datasets without having to contact some central office, (d) avoid permanent data loss due to software integration bugs, and, last but not least, (e) access to datasets even after GloBI goes away. As far back as 1927, Charles Elton, a founding father of modern ecology, realized the importance of linking natural history knowledge stored in professional journals while acknowledging the value of local (amateur) knowledge. Despite technological advances, details on how species interact are only still largely available by studying professional journals, manually inspecting datasets or striking up a conversation with a ecologist, farmer or citizen scientist. The lack of access to species interaction data is known as the Eltonian shortfall (Hortal et al. 2015). GloBI’s mission is to address this shortcoming. By borrowing from software engineering practices such as test driven development and continuous integration, re-purposing freely available platforms such as GitHub, Zenodo, Travis CI and integrating with many existing biodiversity services (e.g. globalnames.org, eol.org, crossref.org, geonames.org), GloBI has grown to include about 2.8M interaction records spanning 100k taxa (source: globalbioticinteractions.org/references, 17 July 2017) and has established bi-directional links to projects including, but not limited to, the NCBI Taxonomy, World Register of Marine Species, Encyclopedia of Life and iNaturalist. As GloBI continues to link existing species interaction datasets, and form a loosely affiliated community of data curators, educators and (citizen) scientists, the data integration platform is well-suited to play an active and experimental role in the development of novel methods to more easily mobilize and integrate species interaction data in an effort to realize Charles Elton's dream to "[...] provide conceptions which can link up into some complete scheme the colossal store of facts about natural history which has accumulated up to date in this rather haphazard manner. [...]" (Elton 1927).

Ransomware attacks: detection, prevention and cure

Over the past three years, ransomware has become one of the biggest cyber scams to hit businesses. Indeed, the FBI estimates that losses incurred in 2016 due to ransomware will top $1bn. Ransomware is malicious software that allows a hacker to restrict access to an individual's or company's vital information in some way and then demand some form of payment to lift the restriction. The most common form of restriction today is encryption of important data on the computer or network, which essentially lets the attacker hold user data or a system hostage.Over the past three years, ransomware has become one of the biggest scams to hit businesses. Indeed, the FBI estimates that losses in 2016 will top $1bn.The ramifications of a successful attack are far more extensive than just the cost of the ransom. Organisations can suffer lost productivity, loss of business, inconvenience to customers and potentially the permanent loss of data. Ross Brewer of LogRhythm details the five phases of a ransomware attack and the five steps you can take to prepare for, handle and recover from it.

Using data mules for sensor network data recovery

In this paper, we study the problem of efficient data recovery using the data mules approach, where a set of mobile sensors with advanced mobility capabilities re-acquire lost data by visiting the neighbors of failed sensors, thereby avoiding permanent data loss in the network. Our approach involves defining the optimal communication graph and mules’ placements such that the overall traveling time and distance is minimized regardless to which sensors crashed. We explore this problem under different practical network topologies such as arbitrary graphs, grids and random linear networks and provide approximation algorithms based on multiple combinatorial techniques. Simulation experiments demonstrate that our algorithms outperform various competitive solutions for different network models, and that they are applicable for practical scenarios.

Effects of Data Scrubbing on Reliability in Storage Systems

Silent data corruptions, which are induced by latent sector errors, phantom writes, DMA parity errors and so on, can be detected by explicitly issuing a read command to a disk controller and comparing the corresponding data with their checksums. Because some of the data stored in a storage system may not be accessed for a long time, there is a high chance of silent data corruption occurring undetected, resulting in data loss. Therefore, periodic checking of the entire data in a storage system, known as data scrubbing, is essential to detect such silent data corruptions in time. The errors detected by data scrubbing will be recovered by the replica or the redundant information maintained to protect against permanent data loss. The longer the period between data scrubbings, the higher the probability of a permanent data loss. This paper proposes a Markov failure and repair model to conservatively analyze the effect of data scrubbing on the reliability of a storage system. We show the relationship between the period of a data scrubbing operation and the number of data replicas to manage the reliability of a storage system by using the proposed model.

Soft Particle-Induced Magnetic Erasure Without Physical Damage to the Media

With ever increasing areal density, interactions of particles with a head-disk interface become an ever more important factor impacting the drive reliability. Although particles trapped between the head and the disk could induce mechanical damage to the media resulting in permanent loss of data, data loss has also been observed without any obvious signs of physical damage to the media. We devised a component-level test to study this mode of data erasure on both glass and aluminium media. Our data indicate that the frictional heating associated with contact force between the particle and the disk could lead to permanent loss of data. In addition, we performed investigations to study the impact of air bearing design features, load/unload mechanism, and particle number density on the head disk interface.

Noninvasive vibration diagnostics to detect head–disk contact in hard disk drives

Reliability of modern hard disk drives (HDDs) is a major concern especially with decreasing head-to-disk spacing in order to attain higher recording densities. One of the reasons for HDD failure is the occurrence of intermittent or sustained head–disk contacts. It is therefore important to be able to detect such head–disk contacts early on so that backup safety schemes can be implemented before catastrophic HDD failure and permanent data loss. In the study reported here, we used noninvasive vibration diagnostics and multiaxial vibration measurements to assess the condition and functionality of HDDs.We chose a miniature triaxial accelerometer for the diagnostic study, given that it is relatively inexpensive and easy to use.We measured contact vibration, induced by a scratch on the top surface of the rotating disk, on the outside of the HDD by attaching an accelerometer on the actuator shaft.We found that contact vibrations between the head and disk inside the HDD can reliably be measured on the outside of the HDD with an accelerometer. We also found that a miniature accelerometer does not cause significant mass loading, as verified by noncontacting laser vibrometry

Phonetic structures of endangered languages: Investigative techniques

The survival of many languages is endangered as the speakers die out or are absorbed into larger communities. Probably over half the 8000 languages in the world will not be spoken by the end of the next century. Their disappearance will result in a permanent loss of data for future phonetic investigations. Given the magnitude of the loss, there is a research problem in deciding which languages should be documented phonetically, and what is the best way of describing their phonetic structures. It was decided to investigate endangered languages that (1) are still actively spoke by a sizable community; (2) are sufficiently well studied so that their phonological structure is known; (3) have features that are of phonetic interest. At least five speakers were recorded saying sets of words and phrases that illustrate the phonological constrasts. Then standardized descriptions were made in quantified form using a set of basic phonetic measures, designed to characterize both specific and universal phonetic features. The data include aerodynamic records that reveal airstream mechanisms and the timing of articulatory gestures, video of lip movements, and palatographic records showing articulatory contacts. Full acoustic analyses are made in the UCLA Phonetics Laboratory. [Work supported by NSF.]

Comparing rebuild algorithms for mirrored and RAID5 disk arrays

Several disk array architectures have been proposed to provide high throughput for transaction processing applications. When a single disk in a redundant array fails, the array continues to operate, albeit in a degraded mode with a corresponding reduction in performance. In addition, the lost data must be rebuilt to a spare disk in a timely manner to reduce the probability of permanent data loss. Several researchers have proposed and examined algorithms for rebuilding the failed disk in a disk array with parity. We examine the use of these algorithms to rebuild a mirrored disk array and compare the rebuild time and performance of the RAID5 and mirrored arrays. Redirection of Reads provides comparable average response times and better rebuild times than Piggybacking for a mirrored array, whereas these two algorithms perform similarly for a RAID5 array. In our experiments comparing the two architectures, a mirrored array has more disks than a RAID5 array and can sustain 150% more I/Os per second during the rebuild process. Even if the size of the RAID5 array is increased to match the mirrored array, the mirrored array reduces response times by up to 60% and rebuild times by up to 45%.