An In-Memory Data-Cube Aware Distributed Data Discovery Across Clouds for Remote Sensing Big Data

Abstract

Abstract: The recent proliferation of high-resolution Earth observation data, along with the advent of open-data initiatives, has led to the availability of petabyte-scale Earth observation datasets accessible for free. However, this abundance of big data presents significant challenges for regional to global spatio-temporal analysis. The traditional approach of "downloadpreprocess-store-analyze" introduces excessive data downloading overhead and computational barriers, hindering efficient analysis. The Earth observation data cube (EODC) paradigm offers a solution by revolutionizing the storage and management of spatio-temporal RS data. Yet, the presence of multiple EODC solutions has resulted in "information silos," making the sharing and joint use of RS data across EODCs difficult.