GeoQuery: Integrating HPC systems and public web-based geospatial data tools

Abstract

Interdisciplinary use of geospatial data requires the integration of data from a breadth of sources, and frequently involves the harmonization of different methods of sampling, measurement, and technical data types. These integrative efforts are often inhibited by fundamental geocomputational challenges, including a lack of memory efficient or parallel processing approaches to traditional methods such as zonal statistics. GeoQuery (geoquery.org) is a dynamic web application which utilizes a High Performance Computing cluster and novel parallel geospatial data processing methods to overcome these challenges. Through an online interface, GeoQuery users can request geospatial data - which spans categories including geophysical, environmental and social measurements - to be aggregated to user-selected units of analysis (e.g., subnational administrative boundaries). Once a request has been processed, users are provided with permanent links to access their customized data and documentation. Datasets made available through GeoQuery are reviewed, prepared, and provisioned by geospatial data specialists, with processing routines tailored for each dataset. The code used and steps taken while preparing datasets and processing user requests are publicly available, ensuring transparency and replicability of all data and processes. By mediating the complexities of working with geospatial data, GeoQuery reduces the barriers to entry and the related costs of incorporating geospatial data into research across disciplines. This paper presents the technology and methods used by GeoQuery to process and manage geospatial data and user requests.