Open Source Framework for Enabling HPC and Cloud Geoprocessing Services

Abstract

Geoprocessing is a set of tools that can be used to efficiently address several pressing chal-lenges for the global economy ranging from agricultural productivity, the design of transport networks, to the prediction of climate change and natural disasters. This paper describes an Open Source Framework developed, within three European projects, for Ena-bling High-Performance Computing (HPC) and Cloud geoprocessing services applied to agricultural challenges. The main goals of the European Union projects EUXDAT (EUro-pean e-infrastructure for eXtreme Data Analytics in sustainable developmenT), CYBELE (fostering precision agriculture and livestock farming through secure access to large-scale HPC-enabled virtual industrial experimentation environment empowering scalable big data analytics), and EOPEN (opEn interOperable Platform for unified access and analysis of Earth observatioN data) are to enable the use of large HPC systems, as well as big data management, user-friendly access and visualization of results. In addition, these projects focus on the development of software frameworks, and fuse Earth-observation data, such as Copernicus data, with non-Earth-observation data, such as weather, environmental and social media information. In this paper, we describe the agroclimatic-zones pilot used to validate the framework. Finally, performance metrics collected during the execution (up to 182 times speedup with 256 MPI processes) of the pilot are presented.