An OAIS-Compatible Data Repository for the National Digital Newspaper Program: Lessons Learned During Phase I

Abstract

In 2004, an agreement between the Library of Congress and the National Endowment for the Humanities established the National Digital Newspaper Program (NDNP), with a goal to create a digital resource of historical American newspapers. The twenty-year initiative would build upon the success of the United States Newspaper Program (USNP) in two ways. It would provide wider access to USNP-created bibliographic data, which cataloged information on newspaper titles published in the United States from 1690 to the present. NDNP would also fund state projects to digitize, from USNP and other microfilm, their local newspapers, with an eventual aim to make tens of millions of digitized pages freely available.With a mindset toward preservation, NDNP chose to follow the philosophy of the Open Archival Information System (OAIS) Model. An OAIS-compatible repository was envisioned that would interact with discrete groups of actors, including: producers, managers and consumers. Since each group would have different roles with respect to the repository, the repository would need to present different options to each.During its first phase, NDNP created a working prototype, a proof-of-concept repository for implementation of ingestion, storage and dissemination processes. In order to accomplish this, the techniques used to produce, preserve and index the digital objects had to be reexamined and modified several times. Then in the move from a prototype scale to a scale that could encompass an increasing amount of pages, the initial repository assumptions were again reexamined.As the development phase of NDNP neared completion, a wide public release of the web site was scheduled for March 2007. In the process of scaling operations toward that wide public release, aspects of the repository structure were modified again based on the experience gained in building it. This paper offers a case study of OAIS implementation, specifically how OAIS concepts were interpreted, applied and sometimes modified over the development phase of a national digitization program.