Serverless Geospatial Data Processing Workflow System Design

Mete Ercan Pakdil,Rahmi Nurhan Çelik

doi:10.3390/ijgi11010020

Mete Ercan Pakdil, Rahmi Nurhan Çelik

Open Access

https://doi.org/10.3390/ijgi11010020

Copy DOI

Abstract

Geospatial data and related technologies have become an increasingly important aspect of data analysis processes, with their prominent role in most of them. Serverless paradigm have become the most popular and frequently used technology within cloud computing. This paper reviews the serverless paradigm and examines how it could be leveraged for geospatial data processes by using open standards in the geospatial community. We propose a system design and architecture to handle complex geospatial data processing jobs with minimum human intervention and resource consumption using serverless technologies. In order to define and execute workflows in the system, we also propose new models for both workflow and task definitions models. Moreover, the proposed system has new Open Geospatial Consortium (OGC) Application Programming Interface (API) Processes specification-based web services to provide interoperability with other geospatial applications with the anticipation that it will be more commonly used in the future. We implemented the proposed system on one of the public cloud providers as a proof of concept and evaluated it with sample geospatial workflows and cloud architecture best practices.

Highlights

Serverless Geospatial DataOver the last decade, big geospatial data has become a vital trend in the industry as data collection techniques become easier and more accessible, and storage options have increased significantly at the same time
For the sake of brevity, we evaluated the system with two inclusive sample workflows that leveraged all presented features in workflow definition models and challenged the system design over execution models such as parallel and iterative processing
We aimed to provide a system design that can be applied in real-world scenarios on any major cloud provider or on-premises that offers serverless technologies

Summary

Introduction

Serverless Geospatial DataOver the last decade, big geospatial data has become a vital trend in the industry as data collection techniques become easier and more accessible, and storage options have increased significantly at the same time. The new ways of geospatial data collection technologies such as drones, robots, and satellites have brought new data storage and processing requirements that are predicted to become more complex and demanding. These new requirements could be met with cloud computing technologies. These new technologies could have a learning curve for geospatial data scientists and analysts. Classical geospatial data computation systems mostly run on on-premise platforms with limited computational capacity or storage. On the other hand, having such a dedicated infrastructure is an expensive investment for single usage or a limited number of processes. Cloud computing platforms are presented as a solution for these problems; leveraging cloud computing technologies needs expertise in this area at a certain level [1]

Objectives

Methods

Results

Conclusion