Weather and environmental satellite big data tsunami: What to do with it? (Conference Presentation)

Abstract

Weather and environmental satellites that collect earth imaging and scientific data are the ultimate source of “Big Data”. A 2004 study conducted by a Committee on Environmental Satellite Data Utilization organized by National Research Council (NRC) of National Academies coined the term “The Data Tsunami” to describe the increasing data rates and volumes of global satellite observations. Between 2004 and 2014, upward of 100 new satellites were launched with ever increasing sensing capabilities and concomitant increases in data and information volumes. NOAA, the US agency that plans, operates, collects, archives and distributes national weather and environmental satellite data has, to date, a total archive of more than 13,000 terabytes from year 2015 alone! According to another report from NOAA, NOAA’s digital archives have now grown to almost 20 times their 1999 volume at a growth rate of approximately 1.2 petabytes per year. Note that NOAA’s total archive, at the dawn of the 21st century, was then less than 1.2 petabytes. It has become obvious that knowing about “what to do with Big Data” will be a truly significant challenge. Furthermore, dramatic developments in earth imaging capacity are being delivered by ambitious startups that are launching large fleets of small, interconnected satellites. The number of earth observing imaging satellites nearly doubled during 2014 alone. With views into various aspects of global activity being refreshed at a faster rate and higher resolution than ever before, the rapid increase of this vast volume of imaging data is already changing how we use satellite Big Data in the business world. In this paper, I attempt to show how the emerging field of Big Data Analytics is relevant to the vast and growing volumes of weather and environmental satellite data. How are we to harvest valuable weather and environmental information from “The Big Data Tsunami” yielded by national and international satellite assets?