Spark-Enabled XDraw Viewshed Analysis

Abstract

Viewshed analysis is an indispensable part of digital terrain analysis and widely used in many application domains. High-resolution raster digital elevation model (DEM) data bring significant computational challenges to the existing viewshed analysis algorithms, which are computationally intensive and require a large memory space and massive computing power. The visibility calculation can be accelerated using Apache Spark. In this article, we present a Spark-based parallel computing approach for the XDraw algorithm, which is composed of a tile-based raster data storing strategy, an equivolume computing strategy, and a stream-merging write-back strategy. The parallel implementation of the XDraw algorithm mainly consists of three parts: partitioning a raster DEM file into square tile sets and reorganizing these tile sets to prevent tile overlap across data divisions of Hadoop Distributed File System, subdividing the DEM into multiple equivolume data sectors according to the viewpoint position, and performing the XDraw algorithm on the corresponding tile sets of each sector independently and writing back the viewshed results efficiently. Experiments on real-world datasets show that the proposed computing approach can achieve higher speedup and efficiency for XDraw viewshed analysis as the raster DEM data volume is dramatically increased. The results also show that the approach has also satisfactory scalability as the number of data nodes in clusters is increased.