Spatial Data Index Research Articles

The Framework of GeoSOT-3D Grid Modeling for Spatial Artificial Intelligence

Abstract. With the development of society, spatial artificial intelligence (spatial AI) research is gradually able to play a greater role. However, spatial AI has problems such as data alignment, poor interpretability, and cross domain learning. Therefore, this paper proposes an innovative GeoSOT-3D grid modeling framework for spatial AI research, which enhances the application capabilities of spatial AI. Grid modeling will be able to run through the upstream and downstream of spatial AI research, providing encoding calculations and spatial neighborhood embedding matrices for spatial data. This paper also uses task examples to demonstrate how to effectively organize and index spatial data using GeoSOT-3D grids and conduct spatial AI research. The use of GeoSOT-3D grids for spatial AI analysis has enormous potential and broad application prospects, which will help promote the further development and application of spatial AI.

Geospatial indexing for sea–land navigation based on machine learning

The exponential growth of geospatial data has created a need for efficient data processing and indexing, especially for acquiring the required data around while traveling on land and sea. Machine learning based learned index has brought a new phase in the field of spatial data indexing on land and sea for fast and accurate acquisition of points of interest. Recently proposed spatial learned indexes and traditional spatial indexes are difficult to maintain their performance while traveling on land and sea due to high memory cost and time consumption. Considering the above issue, we propose SLIP, a novel in-memory spatial learned index. SLIP uses an optimized mapping function and a piecewise linear model to handle data updates (e.g., insertion and deletion) efficiently. Based on SLIP, we design a range query algorithm and a faster KNN(K-NearestNeighbor) query algorithm using the optimized radius expansion model to get the results more quickly. Experiments on real geospatial datasets and synthetic datasets used the electronic navigation chart software show that SLIP achieved an average improvement of 10x in query performance over traditional spatial indexes and state-of-the-art learned indexes. Therefore, SLIP provides an innovative idea for high-efficiency geospatial data processing and indexing, and offers an efficient method for acquiring the surrounding geographic data for future navigation on land and sea.

Spatial Index Structures for Modern Storage Devices: A Survey

To optimize the processing of spatial queries, there is an increasing interest in combining <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">spatial index structures</i> with <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">modern storage devices</i> like flash-based Solid State Drives, PCM, and 3D Xpoint. These devices have several advantages compared to classical Hard Disk Drives, such as lower power consumption, and faster reads and writes. However, modern storage devices have changed the paradigm of data management because of their intrinsic characteristics, such as asymmetric read and write costs. Intending to exploit the benefits of modern storage devices, the development of spatial index structures for these devices has been an emerging research topic with recent and constant advances in the literature. This includes the adaptation of existing spatial index structures like the R-tree, or even the design of innovative structures. In this article, we present a comprehensive survey that highlights the key ideas, compares the main characteristics, and discusses the advantages and disadvantages of spatial index structures for modern storage devices. Further, we study how experimental evaluations have been conducted to empirically compare these structures. Finally, we discuss challenges and identify potential future trends when indexing spatial data in this era of modern storage devices.

Adaptive Indexing of Objects with Spatial Extent

Can we quickly explore large multidimensional data in main memory? Adaptive indexing responds to this need by building an index incrementally, in response to queries; in its default form, it indexes a single attribute or, in the presence of several attributes, one attribute per index level. Unfortunately, this approach falters when indexing spatial data objects, encountered in data exploration tasks involving multidimensional range queries. In this paper, we introduce the Adaptive Incremental R-tree (AIR-tree): the first method for the adaptive indexing of non-point spatial objects; the AIR-tree incrementally and progressively constructs an in-memory spatial index over a static array, in response to incoming queries, using a suite of heuristics for creating and splitting nodes. Our thorough experimental study on synthetic and real data and workloads shows that the AIR-tree consistently outperforms prior adaptive indexing methods focusing on multidimensional points and a pre-built static R-tree in cumulative time over at least the first thousand queries.

Uzamsal Verilerde İndeks Kullanımının Sorgu Verimliliği Üzerinde Etkileri

Indexing spatial data is of great importance in terms of accessibility and query performance to this data. In this study, query performances between indexed and non-indexed tables and bulk loading times of different sizes of data into index structures were measured by performing some query experiments on a spatial database created on the Turkey map. In this way, it is aimed to emphasize the importance of choosing the right index type and using indexes on spatial data.

Patterns and Influencing Factors of Express Outlets in China

China’s express delivery industry has developed rapidly in the past decade, and the spatial distribution of express delivery outlets can reflect variations in regional development to a certain extent. Previous studies lacked point of interest (POI) data as the research object to analyze the limitations of express delivery outlets, and also lack a focus on the patterns that specifically occur in China. This study implemented spatial analysis methods such as nearest neighbor index, kernel density, and exploratory spatial data analysis to explore the spatial distribution patterns and characteristics of China’s express outlets, and used geographic detector factor detection to objectively analyze the factors that affect their distribution. The following are the main conclusions of this study: (1) in China, express delivery outlets are more abundant in the east and sparser in the west; (2) the local agglomeration of express outlets presents a concentric structure, and outlier clusters appeared in the northeast and central regions; (3) the distribution of the express outlets resulted from the combined action of multiple factors (e.g., population factors, etc.). Our study not only explores and analyzes the overall situation in China but also broadens the scope of express outlet research.

Applied open-source Discrete Global Grid Systems

Abstract. Discrete Global Grid Systems (DGGS) are spatial reference systems that use a hierarchical tessellation of cells to partition and address the globe and provide alternative spatial data format and indexing methods as compared to traditional vector and raster spatial data. In order to effectively use DGGS, functional software needs to be available and data needs to be indexed into a DGGS. We compare the software APIs of the 5 main open-source DGGS implementations – Uber H3, Google S2, rHEALPix by Landcare Research New Zealand, RiskAware OpenEAGGR, and DGGRID by Southern Oregon University – and present exemplary workflows for converting spatial and vector and raster datasets into DGGS-indexed format. We summarize, that Uber H3 and Google S2 provide more mature software library functionalities and DGGRID provides excellent functionality to construct grids with desired geometric properties and to load point data but does not provide functions for traversal and navigation within a grid after its construction.

Spatial Concept Query Based on Lattice-Tree

As a basic method of spatial data operation, spatial keyword query can provide meaningful information to meet user demands by searching spatial textual datasets. How to accurately understand users’ intentions and efficiently retrieve results from spatial textual big data are always the focus of research. Spatial textual big data and their complex correlation between textual features not only enrich the connotation of spatial objects but also bring difficulties to the efficient recognition and retrieval of similar spatial objects. Because there are a lot of many-to-many relationships between massive spatial objects and textual features, most of the existing research results that employ tree-like and table-like structures to index spatial data and textual data are inefficient in retrieving similar spatial objects. In this paper, firstly, we define spatial textual concept (STC) as a group of spatial objects with the same textual keywords in a limited spatial region in order to present the many-to-many relationships between spatial objects and textual features. Then we attempt to introduce the concept lattice model to maintain a group of related STCs and propose a hybrid tree-like spatial index structure, the lattice-tree, for spatial textual big data. Lattice-tree employs R-tree to index the spatial location of objects, and it embeds a concept lattice structure into specific tree nodes to organize the STC set from a large number of textual keywords of objects and their relationships. Based on this, we also propose a novel spatial keyword query, named Top-k spatial concept query (TkSCQ), to answer STC and retrieve similar spatial objects with multiple textual features. The empirical study is carried out on two spatial textual big data sets from Yelp and Amap. Experiments on the lattice-tree verify its feasibility and demonstrate that it is efficient to embed the concept lattice structure into tree nodes of 3 to 5 levels. Experiments on TkSCQ evaluate lattice from results, keywords, data volume, and so on, and two baseline index structures based on IR-tree and Fp-tree, named the inverted-tree and Fpindex-tree, are developed to compare with the lattice-tree on data sets from Yelp and Amap. Experimental results demonstrate that the Lattice-tree has the better retrieval efficiency in most cases, especially in the case of large amounts of data queries, where the retrieval performance of the lattice-tree is much better than the inverted-tree and Fpindex-tree.

The Brain Observatory Storage Service and Database (BossDB): A Cloud-Native Approach for Petascale Neuroscience Discovery.

Technological advances in imaging and data acquisition are leading to the development of petabyte-scale neuroscience image datasets. These large-scale volumetric datasets pose unique challenges since analyses often span the entire volume, requiring a unified platform to access it. In this paper, we describe the Brain Observatory Storage Service and Database (BossDB), a cloud-based solution for storing and accessing petascale image datasets. BossDB provides support for data ingest, storage, visualization, and sharing through a RESTful Application Programming Interface (API). A key feature is the scalable indexing of spatial data and automatic and manual annotations to facilitate data discovery. Our project is open source and can be easily and cost effectively used for a variety of modalities and applications, and has effectively worked with datasets over a petabyte in size.

An Optimal Framework for Spatial Query Optimization Using Hadoop in Big Data Analytics

Background and Objective: Spatial queries frequently used in Hadoop for significant data process. However, vast and massive size of spatial information makes it difficult to process the spatial inquiries proficiently, so they utilized the Hadoop system for process the Big Data. Boolean Queries &amp; Geometry Boolean Spatial Data for Query Optimization using Hadoop System are used. In this paper, a lightweight and adaptable spatial data index for big data have discussed, which have used to process in Hadoop frameworks. Results demonstrate the proficiency and adequacy of spatial ordering system for various spatial inquiries. Methods: In this section, the different type of approaches are used which helps to understand the procedure to develop an efficient system by involving the methods like efficient and scalable method for processing Top-k spatial Boolean Queries, Efficient query processing in Geographic web search engines. Geographic search engine query processing combines text and spatial data processing technique &amp; Top-k spatial preference Queries. In this work, the implementation of all the methods is done for comparative analysis. Results and Discussion: The execution of algorithm gives results which show the difference of performance over different data types. Three different graphs are presented here based on the different data inputs indexing and data types. Results show that when the number of rows to be executed increases the performance of geohash decreases, while the crucial point for change in performance of execution is not visible due to sudden hike in number of rows returned. Conclusion: The query processing have discussed in geographic web search engines. In this work a general framework for ranking search results based on a combination of textual and spatial criteria, and proposed several algorithms for efficiently executing ranked queries on very large collections have discussed. The integrated of proposed algorithms into an existing high-performance search engine query processor and works on evaluating them on a large data set and realistic geographic queries. The results shows that in many cases geographic query processing can be performed at about the same level of efficiency as text-only queries.

Comparative Analysis of Spark and Ignite for Big Spatial Data Processing

Recently, spatial data became one of the most interesting fields related to big data studies, in which the spatial data have been generated and consumed from different resources. However, the increasing numbers of location-based services and applications such as Google Maps, vehicle navigation, recommendation systems are the main foundation of the idea of spatial data. On the other hand, several researchers started to discover and compared spatial frameworks to understand the requirements for spatial database processing, manipulating, and analysis systems. Apache Spark, Apache Ignite, and Hadoop are the most widely known frameworks for large data processing. However, Apache Spark, Apache Ignite have integrated different spatial data operations and analysis queries, but each system has its advantages and disadvantages when dealing with spatial data. Dealing with a new framework or system that needs to integrate new functionality sometimes becomes a risky decision if we did not examine it well The main aim of this research is to conduct a comprehensive evaluation of big spatial data computing on two well-known data management systems Apache Ignite and Apache Spark. The comparative has been done on four different domains, experimental environment setup, supported features, supported functions and queries, and performance and execution time. The results show that GeoSpark has recorded more flexibility to use than SpatialIgnite. We thoroughly investigated and discovered that multiple factors affect the performance of both frameworks, such as CPU, Main memory, data set size the complexity of data type, and programming environment. spark is more advanced and equipped with several functionalities that made it well suitable with spatial data queries and indexing. such as kNN queries; in which these functionalities are not supported in SpatialIgnite.

Effectively learning spatial indices

Machine learning, especially deep learning, is used increasingly to enable better solutions for data management tasks previously solved by other means, including database indexing. A recent study shows that a neural network can not only learn to predict the disk address of the data value associated with a one-dimensional search key but also outperform B-tree-based indexing, thus promises to speed up a broad range of database queries that rely on B-trees for efficient data access. We consider the problem of learning an index for two-dimensional spatial data. A direct application of a neural network is unattractive because there is no obvious ordering of spatial point data. Instead, we introduce a rank space based ordering technique to establish an ordering of point data and group the points into blocks for index learning. To enable scalability, we propose a recursive strategy that partitions a large point set and learns indices for each partition. Experiments on real and synthetic data sets with more than 100 million points show that our learned indices are highly effective and efficient. Query processing using our indices is more than an order of magnitude faster than the use of R-trees or a recently proposed learned index.

DAPR-tree: a distributed spatial data indexing scheme with data access patterns to support Digital Earth initiatives

ABSTRACT This paper proposes a novel data indexing scheme, the distributed access pattern R-tree (DAPR-tree), for spatial data retrieval in a distributed computing environment. As compared to traditional distributed indexing schemes, the DAPR-tree introduces the data access patterns during the indexing utilization stage so that a more balanced indexing structure can be provided for spatial applications (e.g. Digital Earth data warehouse). In this new indexing scheme, (a) an indexing penalty matrix is proposed by considering the balance of data number, topology and access load between different indexing nodes; (b) an ‘access possibility’ element is integrated to a classic ‘Master-Client’ structure for a distributed indexing environment; and (c) indexing algorithm for the DAPR-tree is provided for index implementations. By using a duplication of official GEOSS Clearinghouse system as a case study, the DAPR-tree was evaluated in a number of scenarios. The results show that our indexing schemes generally outperform (around 9%) traditional distributed indices with the utilization of data access patterns. Finally, we discuss the applicability of the DARP-tree and document DARP-tree shortcomings to encourage researchers pursuing related topics in Big Data indexing for Digital Earth and other geospatial initiatives.

SwapQt: Cloud-based in-memory indexing of dynamic spatial data

The ubiquity of geo-positioning technologies stimulates continuous growth in dynamic spatial datasets that fuels the development of location-based services. These services require tracking and querying a large population of moving objects. High workloads of users’ requests, both location updates and queries, need to be processed concurrently. Current solutions employ an index that is updated incrementally or rebuilt from scratch periodically. Due to the concurrency of updates and queries, current solutions still suffer from query staleness. In this paper, we present swapQt, a novel in-memory cloud-based approach for indexing dynamic spatial data that efficiently processes updates and answers queries. SwapQt consists of two main components, a routing index and local indexes. The routing index maintains the addresses of all the cloud nodes in the system and the boundaries of the data in each cloud node. Two local indexes, one to process updates and another to answer queries, are maintained and swapped periodically in each cloud node to eliminate interference between incoming updates and queries. swapQt outperformed the state-of-the-art approaches in terms of speedup and query staleness. For a workload of 1 million updates, the query staleness in swapQt was around 0.22 s compared to 4.3 s for the state-of-the-art approach. All the experiments were conducted on Microsoft Azure Cloud Computing Platform to provide realistic experimental settings.

An Efficient Parallel Algorithm for Polygons Overlay Analysis

Map overlay analysis is essential for geospatial analytics. Large scale spatial data pressing poses challenges for geospatial map overlay analytics. In this study, we propose an efficient parallel algorithm for polygons overlay analysis, including active-slave spatial index decomposition for intersection, multi-strategy Hilbert ordering decomposition, and parallel spatial union algorithm. Multi-strategy based spatial data decomposition mechanism is implemented, including parallel spatial data index, the Hilbert space-filling curve sort, and decomposition. The results of the experiments showed that the parallel algorithm for polygons overlay analysis achieves high efficiency.

Spatial Data Indexing and Query Processing in GeoCloud

Abstract GeoCloud is essential for spatial data management. This article depicts GeoCloud and SpatialHadoop, both of which are developed for spatial information, indexing, and query processing. It contains traditional spatial indexing that comprises R-tree, Hilbert R-tree, and improved Bloom filter tree. We enhance the query search by utilizing Spatial Join, Range Query, k-nearest neighbor (k-NN), and Max k-NN queries. By doing so, we implement the data structures and query evaluation performance of different spatial datasets in GeoCloud instances with SpatialHadoop. We show that our proposed system is more efficient in terms of data storage and retrieval in GeoCloud.

A QUADTREE SPATIAL INDEX METHOD WITH INCLUSION RELATIONS FOR THE INCREMENTAL UPDATING OF VECTOR LANDCOVER DATABASE

Abstract. In vector landcover database, there are a lot of complex polygons with many holes, even nesting holes. In the incremental updating (i.e., using the change-only information to update the land cover database), a new changed parcel usually has 2-dimensional intersections (e.g., overlap, cover, equal and inside, etc.) with several existing regions, automatic updating operations need to identify the affected objects for the new changes at first. If the existing parcels include complex polygons (i.e., the polygon with holes), it is still needed to determine if there are 2-dimensional intersections between the new changed polygon and each holes of the involved complex polygons. The relation between the complex polygon and its holes has not been presented in the current spatial data indexing methods, only the MBB (Minimum Bounding Box) of the exterior ring of the complex polygon has been stored, the non-involved holes can not be filtered at the first step of spatial access methods. As the refinement geometric operation is costly, therefore the updating process for the complex polygons is very complicated and low efficient using the current spatial data indexing methods. In order to solve this problem, an improved quadtree spatial index method is presented in this paper. In this method, the polygons is divided to two categories according to the relations with the quadrant axes, i.e., disjoint to the axes and intersect with the axes. The intersect polygons are still divided to 5 cases according to the intersection position among the polygons and the different level quadrant axes. The intersection polygons are stored in the different level root nodes in our index tree, and five buckets denoted as XpB, XnB, YpB, YnB, XYB are used to store the polygons intersecting the different level quadrant axes respectively. The polygons disjoint to all quadrant axes are stored in the leaf nodes in this method. The authors developed the spatial index structure with inclusion relations and the algorithms of the corresponding index operations (e.g., insert, delete and query) for the complex polygons. The effectiveness of the improved index is verified by an experiment of land cover data incremental updating. Experimental results show that the proposed index method is significantly more efficient than the traditional quadtree index in terms of spatial query efficiency, and the time efficiency of the incremental updating is increased about 3 times using the proposed index method than that using the traditional quadtree index.

SOLD: A node-Splitting algorithm for R-tree based on Objects’ Locations Distribution

Spatial data indexing methods are of extreme importance as they massively build up as a result of the explosive growth in capturing data with spatial features. No matter how much the data size is, eventually it will reside on disk pages. Disk pages have to be properly indexed to preserve spatial properties of objects, optimise disk space usage and improve objects’ retrieval performance. One of the most popular spatial data indexes is the R-tree which is a height balanced tree data structure, where leaf nodes resemble disk pages and contain pointers to objects’ locations. A single tree node can host up to a maximum number of objects, where any more insertion makes it an overflown node and it has to be split. Better splits lead to better index performance and more utilisation of disk space. In this work, we introduce a new way of finding the most proper split for an overflown node in the R-tree index. The proposed work scans – in a linear cost – the overflown node’s objects once to identify the distribution of objects’ locations (minimum bounding rectangles (MBRs)) in relative to its node’s bounding rectangle (node’s MBR). It uses objects’ locations to calculate – for each main axis – the split quality factors: expected overlap between resulting nodes, objects distribution evenness among resulting nodes and the perimeter of resulting nodes. The axis with better combined quality factors values is selected as the split axis. The Splitting based on Objects’ Locations Distribution (SOLD) algorithm was implemented and tested against two other splitting algorithms, experiments using synthetic and real data files showed good results and it outperformed both algorithms in index creation tests and data retrieval tests.

Efficient query processing on large spatial databases: A performance study

A spatial index, called xBR+-tree, is compared against the R*-tree and the R+-tree.Performance of tree building and processing of common spatial queries is studied.Existing and new algorithms for processing these queries are utilized.Extensive experimental results on medium and large datasets are presented.The xBR+-tree is the winner of execution-time in all cases and of I/0 in most cases. Processing of spatial queries has been studied extensively in the literature. In most cases, it is accomplished by indexing spatial data using spatial access methods. Spatial indexes, such as those based on the Quadtree, are important in spatial databases for efficient execution of queries involving spatial constraints and objects. In this paper, we study a recent balanced disk-based index structure for point data, called xBR+-tree, that belongs to the Quadtree family and hierarchically decomposes space in a regular manner. For the most common spatial queries, like Point Location, Window, Distance Range, Nearest Neighbor and Distance-based Join, the R-tree family is a very popular choice of spatial index, due to its excellent query performance. For this reason, we compare the performance of the xBR+-tree with respect to the R*-tree and the R+-tree for tree building and processing the most studied spatial queries. To perform this comparison, we utilize existing algorithms and present new ones. We demonstrate through extensive experimental performance results (I/O efficiency and execution time), based on medium and large real and synthetic datasets, that the xBR+-tree is a big winner in execution time in all cases and a winner in I/O in most cases.

QUERY SUPPORT FOR GMZ

Abstract. Generic text-based compression models are simple and fast but there are two issues that needs to be addressed. They cannot leverage the structure that exists in data to achieve better compression and there is an unnecessary decompression step before the user can actually use the data. To address these issues, we came up with GMZ, a lossless compression model aimed at achieving high compression ratios. The decision to design GMZ (Khandelwal and Rajan, 2017) exclusively for GML's Simple Features Profile (SFP) seems fair because of the high use of SFP in WFS and that it facilitates high optimisation of the compression model. This is an extension of our work on GMZ. In a typical server-client model such as Web Feature Service, the server is the primary creator and provider of GML, and therefore, requires compression and query capabilities. On the other hand, the client is the primary consumer of GML, and therefore, requires decompression and visualisation capabilities. In the first part of our work, we demonstrated compression using a python script that can be plugged in a server architecture, and decompression and visualisation in a web browser using a Firefox addon. The focus of this work is to develop the already existing tools to provide query capability to server. Our model provides the ability to decompress individual features in isolation, which is an essential requirement for realising query in compressed state. We con - struct an R-Tree index for spatial data and a custom index for non-spatial data and store these in a separate index file to prevent alter - ing the compression model. This facilitates independent use of compressed GMZ file where index can be constructed when required. The focus of this work is the bounding-box or range query commonly used in webGIS with provision for other spatial and non-spatial queries. The decrement in compression ratios due to the new index file is in the range of 1–3 percent which is trivial considering the benefits of querying in compressed state. With around 75 % average compression of the original data, query support in compressed state and decompression support in the browser, GMZ can be a good alternative to GML for WFS-like services.