On storing and retrieving geospatial big-data in cloud

Abstract

Cloud storage is a kind of external storage which can provide by unlimited storage space with high availability and low cost on maintenance. On the other side, the size of geospatial data is too large and is increasing dramatically which makes such data is hard to be stored in the local data warehouse. Hence following the benefits of Cloud storage, such geospatial data is suitable to be stored in Cloud storage and managed by local data warehouse. However, there is a gap between Cloud storages and data warehouses built on traditional infrastructures, such as the mostly adopted massive parallel processing (MPP) based data warehouse. Therefore, in this paper, we propose a middleware-like architecture to connect MPP data warehouse and Cloud storage. It supports traditional geospatial data retrieving while integrating the Cloud storage lineage by a set of technical designs. Based on the prototype system and practical data, we demonstrate the appreciable performance and the flexibility for other third parties' development. Another major contribution of this paper is that we implement the prototype on open-source data warehouse and we make it open-sourced to public.