Abstract
The rapid growth of big spatial data urged the research community to develop several big spatial data systems. Regardless of their architecture, one of the fundamental requirements of all these systems is to spatially partition the data efficiently across machines. The core challenges of big spatial partitioning are building high spatial quality partitions while simultaneously taking advantages of distributed processing models by providing load balanced partitions. Previous works on big spatial partitioning are to reuse existing index search trees as-is, e.g., the R-tree family, STR, Kd-tree, and Quad-tree, by building a temporary tree for a sample of the input and use its leaf nodes as partition boundaries. However, we show in this paper that none of those techniques has addressed the mentioned challenges completely. This paper proposes a novel partitioning method, termed R*-Grove, which can partition very large spatial datasets into high quality partitions with excellent load balance and block utilization. This appealing property allows R*-Grove to outperform existing techniques in spatial query processing. R*-Grove can be easily integrated into any big data platforms such as Apache Spark or Apache Hadoop. Our experiments show that R*-Grove outperforms the existing partitioning techniques for big spatial data systems. With all the proposed work publicly available as open source, we envision that R*-Grove will be adopted by the community to better serve big spatial data research.
Highlights
The recent few years witnessed a rapid growth of big spatial data collected by different applications such as satellite imagery (Eldawy et al, 2015b), social networks (Magdy et al, 2014), smart phones (Henke et al, 2016), and VGI (Goodchild, 2007)
This paper proposes a novel spatial partitioning technique for big data, termed R*-Grove, which completely addresses all of three aforementioned limitations
The last two groups indicate that R*-Grove significantly outperforms other partitioning techniques in terms of range query and spatial join query performance
Summary
The recent few years witnessed a rapid growth of big spatial data collected by different applications such as satellite imagery (Eldawy et al, 2015b), social networks (Magdy et al, 2014), smart phones (Henke et al, 2016), and VGI (Goodchild, 2007). Traditional Spatial DBMS technology could not scale up to these petabytes of data which led to the birth of many big spatial data management systems such as SpatialHadoop (Eldawy and Mokbel, 2015), GeoSpark (Yu et al, 2015), Simba (Xie et al, 2016), LocationSpark (Tang et al, 2016), and Sphinx (Eldawy et al, 2017), to name a few Regardless of their architecture, all these systems need an essential preliminary step that partitions the data across machines before the execution can be parallelized. A common method that was first introduced in SpatialHadoop (Eldawy and Mokbel, 2015), is the sample-based STR partitioner This method picks a small sample of the input to determine its distribution, packs this sample using the STR packing algorithm (Leutenegger et al, 1997), and uses the boundaries of the leaf nodes to partition the entire data. The SPLITNODE method takes an overflow node with M + 1 records and splits it into two nodes
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
