NOAA Open Data Dissemination (Formerly NOAA Big Data Project/Program)

Abstract

The National Oceanic and Atmospheric Administration (NOAA) Research to Operations (R2O) experiment called the Big Data Project (BDP) was envisioned as a scalable approach for disseminating exponentially increasing NOAA observation, model, and research data sets to the public using commercial cloud services. At the start of the project, during the concept development phase, it was unclear how the specifics might work so a spiral development approach was adopted. It was expected that the number of data sets would increase, and the data extent would grow to cover complete records of some holdings, and that format experimentation would be needed to determine optimal cloud offerings. This dissemination model would require a new way for the BDP and NOAA to engage with end users, who could range from large enterprises to small businesses and individuals. The BDP was expected to change the game, not just by reaching a broad and diverse set of users but also by encouraging new ones. As Kathryn Sullivan, Ph.D, former NOAA Administrator under whom the BDP began, noted, “The agency's aim is to ‘spur innovation’ and to explore how to create a global economic return on investment” (Konkel, , paragraph 6). This chapter describes the journey of BDP as it developed, transitioned, and evolved from a research experiment experiment to an operational enterprise function for NOAA, now known as NOAA Open Data Dissemination (NODD).