Including the Quadtree index in SpatialHadoop

Abstract

SpatialHadoop is a full-fledged MapReduce framework with native support for spatial data. It uses a two-level (global and local) index structure to store the spatial data in a distributed cluster. The global index partitions data across computing nodes, while the local index organizes data inside each computing node. SpatialHadoop supports R-tree as a local index on the contents of each physical partition. The Quadtree is one of the most used hierarchical access methods to index spatial data because of its property to partition the embedding space into regular cells, regardless of the data distribution. Top-k queries, e.g., k Nearest Neighbors Query (kNNQ) and k Closest Pairs Query (kCPQ), are common operations used in spatial applications. In this paper, the Quadtree is included as a local index in SpatialHadoop and, the design of kNNQ and kCPQ MapReduce algorithms, using this spatial index, is also presented. The experiments have demonstrated the efficiency and scalability of the Quadtree in comparison with the R-tree for distributed kNNQ and kCPQ in SpatialHadoop.