Disrupting the library: Digital scholarship and Big Data at the National Library of Scotland

Abstract

With a mass digitisation programme underway and the addition of non-print legal deposit and web archive collections, the National Library of Scotland is now both producing and collecting data at an unprecedented rate, with over 5PB of storage in the Library’s data centres. As well as the opportunities to support large scale analysis of the collections, this also presents new challenges around data management, storage, rights, formats, skills and access. Furthermore, by assuming the role of both creators and collectors, libraries face broader questions about the concepts of ‘collections' and ‘heritage', and the ethical implications of collecting practices. While the ‘collections as data’ movement has encouraged cultural heritage organisations to present collections in machine-readable formats, new services, processes and tools also need to be established to enable these emerging forms of research, and new modes of working need to be established to take into account an increasing need for transparency around the creation and presentation of digital collections. This commentary explores the National Library of Scotland's new digital scholarship service, the implications of this new activity and the obstacles that libraries encounter when navigating a world of Big Data.