Catastrophic Data Loss Research Articles

Data recovery methods for DNA storage based on fountain codes

Today's digital data storage systems typically offer advanced data recovery solutions to address the problem of catastrophic data loss, such as software-based disk sector analysis or physical-level data retrieval methods for conventional hard disk drives. However, DNA-based data storage currently relies solely on the inherent error correction properties of the methods used to encode digital data into strands of DNA. Any error that cannot be corrected utilizing the redundancy added by DNA encoding methods results in permanent data loss. To provide data recovery for DNA storage systems, we present a method to automatically reconstruct corrupted or missing data stored in DNA using fountain codes. Our method exploits the relationships between packets encoded with fountain codes to identify and rectify corrupted or lost data. Furthermore, we present file type-specific and content-based data recovery methods for three file types, illustrating how a fusion of fountain encoding-specific redundancy and knowledge about the data can effectively recover information in a corrupted DNA storage system, both in an automatic and in a guided manual manner. To demonstrate our approach, we introduce DR4DNA, a software toolkit that contains all methods presented. We evaluate DR4DNA using both in-silico and in-vitro experiments.

Personal Digital Archiving and Catastrophic Data Loss: Is it Getting Better?

Abstract Given the changing landscape of personal computing, this study asks: (1) has the occurrence of personal catastrophic data loss improved with the maturity of digital technology and services, and (2) have the reasons for data loss changed with the maturing of digital technology? These questions are addressed through a qualitative content analysis of N = 136 participant responses from 2013 to 2022 to a question about their personal experience of data loss. The study finds that there is no decline over time in participant’s experience of data loss, and the reasons for data loss continue to be varied. Losing digital information endures as an unpleasant experience for today’s computing users, and personal digital archiving ought to be introduced to young people early on to avoid learning about data loss “the hard way.”

Specifying and Checking File System Crash-Consistency Models

Applications depend on persistent storage to recover state after system crashes. But the POSIX file system interfaces do not define the possible outcomes of a crash. As a result, it is difficult for application writers to correctly understand the ordering of and dependencies between file system operations, which can lead to corrupt application state and, in the worst case, catastrophic data loss. This paper presents crash-consistency models, analogous to memory consistency models, which describe the behavior of a file system across crashes. Crash-consistency models include both litmus tests, which demonstrate allowed and forbidden behaviors, and axiomatic and operational specifications. We present a formal framework for developing crash-consistency models, and a toolkit, called Ferrite, for validating those models against real file system implementations. We develop a crash-consistency model for ext4, and use Ferrite to demonstrate unintuitive crash behaviors of the ext4 implementation. To demonstrate the utility of crash-consistency models to application writers, we use our models to prototype proof-of-concept verification and synthesis tools, as well as new library interfaces for crash-safe applications.

Specifying and Checking File System Crash-Consistency Models

Applications depend on persistent storage to recover state after system crashes. But the POSIX file system interfaces do not define the possible outcomes of a crash. As a result, it is difficult for application writers to correctly understand the ordering of and dependencies between file system operations, which can lead to corrupt application state and, in the worst case, catastrophic data loss. This paper presents crash-consistency models, analogous to memory consistency models, which describe the behavior of a file system across crashes. Crash-consistency models include both litmus tests, which demonstrate allowed and forbidden behaviors, and axiomatic and operational specifications. We present a formal framework for developing crash-consistency models, and a toolkit, called Ferrite, for validating those models against real file system implementations. We develop a crash-consistency model for ext4, and use Ferrite to demonstrate unintuitive crash behaviors of the ext4 implementation. To demonstrate the utility of crash-consistency models to application writers, we use our models to prototype proof-of-concept verification and synthesis tools, as well as new library interfaces for crash-safe applications.

Preservation of Documents and Photographic Images: Long Term Strategies for Future Generations

Upon discovering the unfamiliar object in their grandparents' attic, most people today would be hard-pressed to play back a recording made on a wax cylinder of the late 1800s. What will people do just 50 years from now with an optical disk or magnetic hard drive? Over time, we have recorded our memories in many ways: letters, post cards, photographs, movies, audio and video recordings are a few examples. In earlier days, interpreting those non-time-sequenced (“still”) recordings was independent of the technology used to create them – you could hold and view a hard copy document in your hand. But even in those early days, time-sequenced recordings depended on the technology used to create them to render them back into a usable form. Our success in dealing with technology changes and rendering early recordings for the masses has been mixed at best. Hard copy documents and images, however, were never an issue until recent times. How will future generations deal with those post cards, letters to the family, and photographs that have now been replaced by “Word documents”, email, and digital images on the computer? Consumers need to become aware of long-term storage and preservation issues that relate to the preservation of the data behind digital documents including photographic images, letters, etc. The more obvious issues, such as accidental or catastrophic data loss and hardware format evolution, have been recognized by the archiving community. Consumers need to be alerted to these issues and be prepared to develop preservation strategies as well. However, longer-term issues beyond routine backup and migration of data also need to be considered. The basic solution of preservation via human-readable hardcopy documents and images is one option, but this raises a fundamental question regarding image preservation that transcends even the more complex solutions—the long-term stability of the chosen media, whether digital or analog. This paper provides an update on preservation strategies for the consumer. While the familiar advice to “make a hard copy” provides a solid foundation, we go beyond this recommendation, with the intent to raise awareness of the need to create a long-term preservation plan for documents and images, and the data behind them.

Image storage and permanence considerations in the long-term preservation of photographic images – update 2010

Archivists and consumers, alike, need to become aware of long-term storage and preservation issues that relate to the preservation of the data behind digital photographic images. The more obvious issues, such as accidental or catastrophic data loss and hardware format evolution, are only now being recognized in the archiving community. Consumers need to be alerted to these issues and be prepared to develop preservation strategies as well. However, longer-term issues beyond routine backup and migration of data need to be considered. The very basic solution of preservation via hardcopy images stored in shoeboxes or albums is one option, but this raises a fundamental question regarding image preservation that transcends even the more complex solutions—the long-term stability of the chosen media, whether digital or analog. This paper discusses archiving and preservation as it relates to images, and the data behind those images, along with historical perspectives and an overview of possible longer-term preservation strategies [1–3]. The importance of image permanence standards, as they relate to overall selection of preservation strategies, will also be discussed.

Image Storage and Permanence Considerations in the Long-Term Preservation of Photographic Images – An Update and Additional Considerations

Archivists and consumers, alike, need to become aware of long-term storage and preservation issues that relate to the preservation of the data behind digital photographic images. The more obvious issues, such as accidental or catastrophic data loss and hardware format evolution, are only now being recognized in the archiving community. Consumers need to be alerted to these issues and be prepared to develop preservation strategies as well. However, longer-term issues beyond routine backup and migration of data need to be considered. The very basic solution of preservation via hardcopy images stored in shoeboxes or albums is one option, but this raises a fundamental question regarding image preservation that transcends even the more complex solutions—the long-term stability of the chosen media, whether digital or analog. This paper discusses archiving and preservation as it relates to images, and the data behind those images, along with historical perspectives and an overview of possible longer-term preservation strategies [1–3]. The importance of image permanence standards, as they relate to overall selection of preservation strategies, will also be discussed.

Validating DICOM Content in a Remote Storage Model

Verifying the integrity of DICOM files transmitted between separate archives (eg, storage service providers, network attached storage, or storage area networks) is of critical importance. The software application described in this article retrieves a specified number of DICOM studies from two different DICOM storage applications; the primary picture archiving and communication system (PACS) and an off-site long-term archive. The system includes a query/retrieve (Q/R) module, storage service class provider (SCP), a DICOM comparison module, and a graphical user interface. The system checks the two studies for DICOM 3.0 compliance and then verifies that the DICOM data elements and pixel data are identical. Discrepancies in the two data sets are recorded with the data elements (tag number, value representation, value length, and value field) and pixel data (pixel value and pixel location) in question. The system can be operated automatically, in batch mode, and manually to meet a wide variety of use cases. We ran this program on a 15% statistical sample of 50,000 studies (7500 studies examined). We found 2 pixel data mismatches (resolved on retransmission) and 831 header element mismatches. We subsequently ran the program against a smaller batch of 1000 studies, identifying no pixel data mismatches and 958 header element mismatches. Although we did not find significant issues in our limited study, given other incidents that we have experienced when moving images between systems, we conclude that it is vital to maintain an ongoing, automatic, systematic validation of DICOM transfers so as to be proactive in preventing possibly catastrophic data loss.

The distributed data center: front-end solutions

The need to provide reliable data availability has always driven data center design. Redundant storage systems and backup mechanisms are essential for enterprises to provide 24/7 data access. Using a redundant array of independent disks (RAID) and data server clusters helps prevent catastrophic data loss. However, redundancy within the data center itself represents only a partial solution. Although backup mechanisms preserve data, you have a high-availability solution only if redundancy also protects against a network failure between the center and the outside world. A distributed data center provides such a solution.

Assessing medical residents’ usage and perceived needs for personal digital assistants

Health care professionals need information delivery tools for accessing information at the point of patient care. Personal digital assistants (PDAs), or hand-held devices demonstrate great promise as point of care information devices. An earlier study [The Constellation Project: experience and evaluation of personal digital assistants in the clinical environment, in: Proceedings of the 19th Annual Symposium on Computer Applications in Medical Care, 1995, 678] on the use of PDAs at the point of care found that hardware constraints, such as memory capability limited their usefulness, however, they were used frequently for accessing medical references and drug information [The Constellation Project: experience and evaluation of personal digital assistants in the clinical environment, in: Proceedings of the 19th Annual Symposium on Computer Applications in Medical Care, 1995, 678]. Since this study was completed in 1995, hand-held computer technology has advanced rapidly, and between 26 and 50% of physicians currently use PDAs [Physician’s use of hand-helds increases from 15% in 1999 to 26% in 2001: Harris interactive poll results, Harris Poll. 8-24-2002 (electronic citation); ACP–ASIM survey finds nearly half of U.S. members use hand-held computers: ACP–ASIM press release, American College of Physicians, 9-3-2002 (electronic citation)]. This use appears higher among residents, with one recent study finding that over two-thirds of family practice residencies use hand-held computers in their training programs [J. Am. Med. Inform. Assoc. 9 (1) (2002) 80]. In this study, we systematically evaluate PDA usage by residents in our institution using quantitative and qualitative methods. Our evaluation included a brief on-line survey of 88 residents in seven residency programs including primary care and specialty practices. The surveys were completed between 26 October 2001 and 30 April 2002. Follow-up interviews with 15 of the surveyed residents were then conducted between 24 April 2002 and 13 May 2002. The original contributions of this study are the evaluation of residents in primary and specialty programs and evaluation of both medical application software and the conventional personal organizational software (such as calendars and to-do lists). This evaluation was also conducted using significantly advanced hardware and software compared with previous studies [The Constellation Project: experience and evaluation of personal digital assistants in the clinical environment, in: Proceedings of the 19th Annual Symposium on Computer Applications in Medical Care, 1995, 678]. Results of our survey and follow-up interviews of residents showed most residents use PDAs daily, regardless of practice or whether their program encourages PDAs. Uses include commercial medical references and personal organization software, such as calendars and address books. Concerns and drawbacks mentioned by these residents included physical size of the PDA and the potential for catastrophic data loss. Another issue raised by our results suggests that security and Health Information Portability and Accountability Act (HIPAA) compliance need to be addressed, in part by resident education about securing patient data on PDAs. Overall, PDAs may become even more widely used if two issues can be addressed: (a) providing secure clinical data for the current patients of a given resident, and (b) allaying concerns of catastrophic data loss from their PDAs (e.g. by educating residents about procedures to recover information from PDA backup files).

Human interface with logging operation optimization programs

This paper presents a case study involving the comparison of two programs which were designed for use in the logging industry. Of particular interest in this comparison are the human factors problems likely to arise during field use of such programs. This paper discusses the need for application of consistent sets of human factors design principles. A specific major conclusion related to the field use of these programs is that it is advisable to provide effective redundancy techniques against catastrophic loss of data in an environment where the user has multiple taks to perform. This paper also discusses the consequences of "good" as well as "poor" examples of menu design, display design, data entry error checking, and handling of erroneous keystroke and keystroke sequences.