A Parallel Implementation of an XDraw Viewshed Algorithm with Spark

Abstract

Viewshed analysis is an indispensable part of digital terrain analysis and is widely used in many application domains. High-resolution raster DEM data bring significant computational challenges to the existing viewshed analysis algorithms, which are computationally intensive and require a large memory space and massive computing power. The viewshed analysis process can be accelerated through the use of Apache Spark. In this article, we present both a tile-based raster data storing strategy and an equal-volume computing strategy for distributed viewshed computation using Spark. The parallel implementation of the XDraw algorithm mainly consists of three parts: (1) partitioning a raster DEM file into square tile sets and reorganizing these tile sets to prevent tile overlap across data divisions of HDFS, (2) subdividing the DEM into multiple equal-volume data sectors according to the viewpoint position, and (3) retrieving the corresponding tile sets of each sector to perform the XDraw algorithm independently and efficiently. Experiments with realworld datasets show that the two proposed strategies can achieve higher speed-up and efficiency for XDraw viewshed analysis as the raster DEM data volume is dramatically increased.