NASA Earth Science Data Research Articles

Ten Years of VIIRS On-Orbit Geolocation Calibration and Performance

The first innovative Visible Infrared Imaging Radiometer Suite (VIIRS) sensor aboard the Suomi National Polar-orbiting Partnership (SNPP) satellite has been in operation for 10 years since its launch on 28 October 2011. The second VIIRS sensor aboard the first Join Polar Satellite System (JPSS-1) satellite has been in operation for 4 years since its launch on 18 November 2017, which became NOAA-20. Well-geolocated and radiometrically calibrated Level-1 sensor data records (SDRs) from VIIRS are crucial to numerical weather prediction (NWP) and Level-2+ environmental data record (EDR) algorithms and products. The high quality of Level-2+ EDRs is a requirement for the continuity of NASA Earth science data records (ESDRs) and climate data records (CDRs), one of the two objectives of the SNPP mission and one of the three elements in the JPSS mission objective. The other objective of the SNPP mission is risk reduction for the follow-on JPSS missions. This paper summarizes the on-orbit geolocation calibration and validation (Cal/Val) activities for both VIIRS sensors onboard SNPP and NOAA-20 in the past 10 years. These activities include nominal geolocation Cal/Val activities, risk reduction activities, and improvements for the on-orbit VIIRS sensor operations. After these activities, sub-pixel geolocation accuracy is achieved. Nadir equivalent geolocation uncertainty is generally within 75 m (1-σ), or 20% imagery band pixels, in either the along-scan or along-track direction for both SNPP and NOAA-20 VIIRS sensors. The worst 16-day measured geolocation errors (radial, 3-σ) are 280 m and 267 m, respectively, in the latest SNPP and NOAA-20 VIIRS data collections, which are better than the required accuracy of 375 m (radial, 3-σ). The risk reduction activities also improved VIIRS builds for JPSS-3 and JPSS-4 satellites, and provide lessons learned for other VIIRS-like sensor builds.

The evolution of a geoscience standard: An instructive tale of science keyword development and adoption

In 1987, NASA sponsored an international workshop that inspired the Directory Interchange Format or DIF – a metadata format to enable “catalog interoperability”. The DIF formed the basis of the International Directory Network (IDN) and the Global Change Master Directory (GCMD) and included a set of science keywords. The primary intent was to catalog NASA Earth science and related data, but the keywords have been implemented in many different systems and adopted in varying ways by many different organizations around the world. This review provides an ethnographic examination of how the keywords have evolved and been managed and how they have been adopted over the last 35 years. It illustrates how semantic approaches have evolved over time and provides insights on how standards and associated processes can be sustained and adaptable. Ongoing institutional commitment is essential, but so is transparency and technical flexibility. Understanding and empowering the different roles involved in standards creation, maintenance, and use of standards as well as the services that standards enable is also critical. It is apparent that semantic representations need to be mindful of different contexts and carefully define verbs as well as nouns and categories. Understanding and representing relationships is central to interdisciplinary interoperability.

A review of Earth Artificial Intelligence

In recent years, Earth system sciences are urgently calling for innovation on improving accuracy, enhancing model intelligence level, scaling up operation, and reducing costs in many subdomains amid the exponentially accumulated datasets and the promising artificial intelligence (AI) revolution in computer science. This paper presents work led by the NASA Earth Science Data Systems Working Groups and ESIP machine learning cluster to give a comprehensive overview of AI in Earth sciences. It holistically introduces the current status, technology, use cases, challenges, and opportunities, and provides all the levels of AI practitioners in geosciences with an overall big picture and to “blow away the fog to get a clearer vision” about the future development of Earth AI. The paper covers all the majorspheres in the Earth system and investigates representative AI research in each domain. Widely used AI algorithms and computing cyberinfrastructure are briefly introduced. The mandatory steps in a typical workflow of specializing AI to solve Earth scientific problems are decomposed and analyzed. Eventually, it concludes with the grand challenges and reveals the opportunities to give some guidance and pre-warnings on allocating resources wisely to achieve the ambitious Earth AI goals in the future.

PODPAC: open-source Python software for enabling harmonized, plug-and-play processing of disparate earth observation data sets and seamless transition onto the serverless cloud by earth scientists

In this paper, we present the Pipeline for Observational Data Processing, Analysis, and Collaboration (PODPAC) software. PODPAC is an open-source Python library designed to enable widespread exploitation of NASA earth science data by enabling multi-scale and multi-windowed access, exploration, and integration of available earth science datasets to support analysis and analytics; automatic accounting for geospatial data formats, projections, and resolutions; simplified implementation and parallelization of geospatial data processing routines; standardized sharing of data and algorithms; and seamless transition of algorithms and data products from local development to distributed, serverless processing on commercial cloud computing environments. We describe the key elements of PODPAC’s architecture, including Nodes for unified encapsulation of disparate scientific data sources; Algorithms for plug-and-play processing and harmonization of multiple data source Nodes; and Lambda functions for serverless execution and sharing of new data products via the cloud. We provide an overview of our open-source code implementation and testing process for development and deployment of PODPAC. We describe our interactive, JupyterLab-based end-user documentation including quick-start examples and detailed use case studies. We conclude with examples of PODPAC’s application to: encapsulate data sources available on Amazon Web Services (AWS) Open Data repository; harmonize processing of multiple earth science data sets for downscaling of NASA Soil Moisture Active Passive (SMAP) soil moisture data; and deploy a serverless SMAP-based drought monitoring application for use access from mobile devices. We postulate that PODPAC will also be an effective tool for wrangling and standardizing massive earth science data sets for use in model training for machine learning applications.

A Quantitative Framework to Inform Cloud Data System Architecture and Services Requirements Based on User Needs and Expected Demand

The NASA open data policy and the increases in data volume from recent and future missions have resulted in a need to re-evaluate data archive information systems and services. For example, the Surface Water Ocean Topography mission (SWOT) will produce $\sim 15$ TB of data every day that needs to be freely and openly accessible through the NASA Physical Oceanography Distribute Active Archive Center (PO.DAAC). Because of the computational, and distribution challenges associated with making such volumes of Earth Observation data publicly available, the NASA Earth Science Data Information System (ESDIS) is moving their archive systems, like PO.DAAC, to the cloud. To facilitate this migration, system requirements need to accommodate expected use. As such, the objective of this study was to assesses expected user needs and develop a quantitative framework that can be traced to requirements for the cloud-based data archive system architecture and services that enable data discovery, access and utility. We used a two-tier data gathering method that included a short survey of potential users (identified at professional meetings and community listservs) and in-depth interviews. The survey provided a means for scaling findings from the in-depth interviews to a more representative sample, thus enabling assessment of expected impact to various user communities for different services that could be provided. This framework enables traceability for setting priorities for data services development to requirements for a cloud-based data archive system and services that enable data discovery, access and utility.

MONITORING AND FORECASTING OF SEASONAL RICE CROP PRODUCTIVITY FOR PADDY DOMINATED REGIONS OF INDIA

Abstract. The relevance of this study is immense for India. The Indian economy largely depends on agriculture, which is impacted by weather extremes and variability in monsoon. India is more vulnerable to disruption from drought than countries like the United States. While agriculture accounts for just 16 percent of India’s economy, half of its 1.3 billion people work on farms, thus, making agriculture the backbone of the Indian economy. However, in agriculture, rice is India’s most important food crop with nearly 1 billion Indian people reliant on it as their major food source. Most important constraint to rice production is water stress which affects nearly ~40 million ha of rainfed system from the total ~45 million ha area under rice cultivation. Future climate change effects on rainfall timing and amount, and projected increases in temperature are expected to exacerbate existing water stresses and will have a direct impact on agriculture in India, especially rice cultivation. We have developed an integrated system that is successfully implemented in many countries. The integrated system RHEAS (Regional Hydrological Extreme and Assessment System) coupled with M-DSSAT (modified DSSAT crop model) ingests various NASA Earth science data to produce a set of relevant hydrologic products (e.g., drought indices, water excess/stress information) and rice yields nowcasts (current conditions), forecasts, and seasonal projections, and then feed them into the operational agency. The overarching goal of this study is to provide this integrated system to stakeholder to improve decision-making process and mitigate the plights of rice farmers and prepare the country to deal with ground realities based on the forecast of rice production.

NASA EOSDIS Data Identifiers: Approach and System

NASA’s Earth Science Data and Information System (ESDIS) Project began investigating the use of Digital Object Identifiers (DOIs) in 2010 with the goal of assigning DOIs to various data products. These Earth science research data products produced using Earth observations and models are archived and distributed by twelve Distributed Active Archive Centers (DAACs) located across the United States. Each data center serves a different Earth science discipline user community and, accordingly, has a unique approach and process for generating and archiving a variety of data products. These varied approaches present a challenge for developing a DOI solution. To address this challenge, the ESDIS Project has developed processes, guidelines, and several models for creating and assigning DOIs. Initially the DOI assignment and registration process was started as a prototype but now it is fully operational. In February 2012, the ESDIS Project started using the California Digital Library (CDL) EZID for registering DOIs. The DOI assignments were initially labor-intensive. The system is now automated, and the assignments are progressing rapidly. As of February 28, 2017, over 50% of the data products at the DAACs had been assigned DOIs. Citations using the DOIs increased from about 100 to over 370 between 2015 and 2016.

BRIDGING THE GAP BETWEEN NASA EARTH OBSERVATIONS AND DECISION MAKERS THROUGH THE NASA DEVELOP NATIONAL PROGRAM

Abstract. The NASA DEVELOP National Program bridges the gap between NASA Earth Science and society by building capacity in both participants and partner organizations that collaborate to conduct projects. These rapid feasibility projects highlight the capabilities of satellite and aerial Earth observations. Immersion of decision and policy makers in these feasibility projects increases awareness of the capabilities of Earth observations and contributes to the tools and resources available to support enhanced decision making. This paper will present the DEVELOP model, best practices, and two case studies, the Colombia Ecological Forecasting project and the Miami-Dade County Ecological Forecasting project, that showcase the successful adoption of tools and methods for decision making. Through over 90 projects each year, DEVELOP is always striving for the innovative, practical, and beneficial use of NASA Earth science data.

BRIDGING THE GAP BETWEEN NASA EARTH OBSERVATIONS AND DECISION MAKERS THROUGH THE NASA DEVELOP NATIONAL PROGRAM

The NASA DEVELOP National Program bridges the gap between NASA Earth Science and society by building capacity in both participants and partner organizations that collaborate to conduct projects. These rapid feasibility projects highlight the capabilities of satellite and aerial Earth observations. Immersion of decision and policy makers in these feasibility projects increases awareness of the capabilities of Earth observations and contributes to the tools and resources available to support enhanced decision making. This paper will present the DEVELOP model, best practices, and two case studies, the Colombia Ecological Forecasting project and the Miami-Dade County Ecological Forecasting project, that showcase the successful adoption of tools and methods for decision making. Through over 90 projects each year, DEVELOP is always striving for the innovative, practical, and beneficial use of NASA Earth science data.

Optimal Reorganization of NASA Earth Science Data for Enhanced Accessibility and Usability for the Hydrology Community

AbstractA long‐standing “Digital Divide” in data representation exists between the preferred way of data access by the hydrology community and the common way of data archival by earth science data centers. Typically, in hydrology, earth surface features are expressed as discrete spatial objects (e.g., watersheds), and time‐varying data are contained in associated time series. Data in earth science archives, although stored as discrete values (of satellite swath pixels or geographical grids), represent continuous spatial fields, one file per time step. This Divide has been an obstacle, specifically, between the Consortium of Universities for the Advancement of Hydrologic Science, Inc. and NASA earth science data systems. In essence, the way data are archived is conceptually orthogonal to the desired method of access. Our recent work has shown an optimal method of bridging the Divide, by enabling operational access to long‐time series (e.g., 36 years of hourly data) of selected NASA datasets. These time series, which we have termed “data rods,” are pre‐generated or generated on‐the‐fly. This optimal solution was arrived at after extensive investigations of various approaches, including one based on “data curtains.” The on‐the‐fly generation of data rods uses “data cubes,” NASA Giovanni, and parallel processing. The optimal reorganization of NASA earth science data has significantly enhanced the access to and use of the data for the hydrology user community.

Atmospheric Composition Processing System (ACPS)

The Atmospheric Composition Processing System (ACPS) is a Community-Oriented Measurement-based Processing System that builds on the heritage mission-based processing used for MODIS, Total Ozone Mapping Spectrometer, and Ozone Monitoring Instrument missions. The ACPS features key changes in scalability, interfaces, and provenance capture that will increase access to NASA's Earth Science data and processing capabilities as well as improve the overall system's usefulness as compared with its predecessors. The ACPS consolidates measurement-based processing into a highly reusable suite that will also serve as the platform for NASA's evaluation of the operational ozone products from the National Polar-orbiting Operational Environmental Satellite System Preparatory Project satellite planned to launch in 2010.