Large Spatial Databases Research Articles

Prediction of stream water quality in Godavari River Basin, India using statistical and artificial neural network models

Abstract The successful prediction of the stream or river water quality is gaining the attention of various governmental agencies, and pollution control boards worldwide due to its useful applications in determining watershed health, biodiversity, ecology, and suitability of potable water needs of the river basin. The physically based computational water quality models would require large spatial and temporal information databases of climatic, hydrologic, and environmental variables and solutions of nonlinear, partial differential equations at each grid point in a river basin. These models suffer from estimability, convergence, stability, approximation, dispersion, and consistency issues. In such a problematic modeling scenario, an artificial neural network (ANN) modeling of 22 stream water quality parameters (SWQPs) is performed from easily measurable data of precipitation, temperature, and novel land use parameters obtained from Geographic Information System (GIS) analysis for the Godavari River Basin, India. The ANN models are compared with the more traditional, statistical linear, and nonlinear regression models for accuracy and performance statistics. This study obtains regression coefficients of 0.93, 0.78, 0.83, and 0.74 for electrical conductivity, dissolved oxygen, biochemical oxygen demand, and nitrate in testing using feedforward ANNs compared with a maximum of 0.45 using linear and nonlinear regressions. Principal component analysis (PCA) is performed to reduce the input data dimension. The subsequent modeling using radial basis function and ANNs is found to improve the overall regression coefficients slightly for the chosen four water quality parameters (WQPs). A closed form equation for electrical conductivity has been derived from MATLAB simulations. The successful modeling results indicate the effectiveness and potential of ANNs over the statistical regression approaches for estimating the highly nonlinear problem of stream water quality distributions.

Hybrid pre-local skyline and post-global skyline filter Based optimized GPMRS framework using multiple mappers and multiple reducers framework

Due to the rapid and exponential development of spatial databases, traditional spatial relationships and its co-locating measures have become inaccurate and inefficient. Many different spatial co-locating algorithms have been tried to detect nearest neighbor patterns within limited datasets in order to evaluate their spatial patterns. Finding co-locating patterns using skyline computational models is one of the active research areas on large spatial databases. However, most of the conventional spatial skyline co-locating models require high computational memory and time on large spatial databases. Another limitation with traditional skyline models is finding the filtered candidate sets in the local and skyline computational models. In order to improve the accuracy of the traditional skyline models, a hybrid parallel filter based skyline model is developed on the large spatial databases to optimize data filtering along with ranking process. Experimental results describes the efficiency of proposed model on conventional skyline computational models on large databases.

QualiOSM: An Architecture to Improve Data Completeness on OpenStreetMap

OpenStreetMap (OSM) is a large spatial database in which geographic information is voluntarily contributed by thousands of users. In Geographic Information Systems (GIS), and more specifically, in Volunteered Geographic Information (VGI), as in the case of OSM, the issue of data completeness is a constant concern, since users without technical knowledge actively participate in the processes of including, editing and excluding data. Also in thecase of OSM, users can add information to the objects assigning special labels for them. These labels are popularly called tags, and the process of assigning them to objects contributes to improving the attribute completeness, an important metric of data quality. In this context, this article proposes the QualiOSM architecture, which generates an automatic tag adder with the purpose of improving the completeness of address information for OSM objects in Brazil, using the reverse geocoding tools Nominatim, CEP Aberto and the database from Correios. The QualiOSM architecture showed good results for improving the completeness of city, neighborhood and street information in OSM objects, especially in scenarios of large urban centers, where the level of mapping is usually better compared to scenarios in rural or peripheral environments.

Finding exceedance locations in a large spatial database using nonparametric regression

In the era of big data analysis, it is of interest to develop diagnostic tools for preliminary scanning of large spatial databases. One problem is identification of locations where certain characteristics exceed a given norm, e.g. timber volume or mean tree diameter exceeding a user-defined threshold. Some of the challenges are, large size of the database, randomness, complex shape of the spatial mean surface, heterogeneity and others. In a step-by-step procedure, we propose a method for achieving this for large spatial data sets. For illustration, we work through a simulated spatial data set as well as a forest inventory data set from Alaska (source: USDA Forest Services). Working within the framework of nonparametric regression modeling, the proposed method can attain a high degree of flexibility regarding the shape of the spatial mean surface. Taking advantage of the large sample size, we also provide asymptotic formulas that are easy to implement in any statistical software.

Assessment of soil erosion using two spatial approaches: RUSLE and SWAT Model

In this paper, we attempted to review the erosion in the Ouergha watershed by applying two spatial approaches. The Ouergha watershed has an area of around 7300 km² representing approximately 18.2% of the Sebou basin of which it is the main tributary. In order to develop the erosion map using the SWAT model, it was important to prepare a large spatial database describing basin proprieties, furthermore, the daily hydro-climatic data. This model integrates MUSLE equation for the estimation of specific degradation. In addition, the estimation of erosion through SWAT was consolidated by constructing an erosion mapping through RUSLE method. This method was applied following an approach based on the use of remote sensing data and GIS tools to produce the major factors involved in the erosive process and their integration into RUSLE. The results obtained, in cartographic form, make it possible to target areas that require priority action for a larger-scale analysis, with a view to finding appropriate solutions to fight against erosion and protect the natural environment. Soil degradation in the Ouergha watershed is around 27 ton/ha/year (SWAT_MUSLE) and 25 ton/ha/year (RUSLE). Average sediment yield was estimated for Al Wahda dam of 10.4 Million tons.

Outlier Detection at GBIF Using DBSCAN

Geographic outliers at GBIF (Global Biodiversity Information Facility) are a known problem. Outliers can be errors, coordinates with high uncertainty, or simply occurrences from an undersampled region. Often in data cleaning pipelines, outliers are removed (even if they are legitimate points) because the researcher does not have time to verify each record one-by-one. Outlier points are usually occurrences that need attention. Currently, there is no outlier detection implemented at GBIF and it is up to the user to flag outliers themselves. DBSCAN (a density-based algorithm for discovering clusters in large spatial databases with noise) is a simple and popular clustering algorithm. It uses two parameters, (1) distance and (2) a minimum number of points per cluster, to decide if something is an outlier. Since occurrence data can be very patchy, non-clustering distance-based methods will fail often Fig. 1. DBSCAN does not need to know the expected number of clusters in advance. DBSCAN does well using only distance and does not require some additional environmental variables like Bioclim. Advanatages of DBSCAN : Simple Easy to understand Only two parameters to set Scales well No additional data sources needed Users would understand how their data was changed Simple Easy to understand Only two parameters to set Scales well No additional data sources needed Users would understand how their data was changed Drawbacks : Only uses distance Must choose parameter settings Sensitive to sparse global sampling Does not include any other relevant environmental information Can only flag outliers outside of a point blob Only uses distance Must choose parameter settings Sensitive to sparse global sampling Does not include any other relevant environmental information Can only flag outliers outside of a point blob Outlier detection and error detection are different. If your goal is to produce a system with no false positives, it will fail. While more complex environmentally-informed outlier detection methods (like reverse jackknifing (Chapman 2005)) might perform better for certain examples or even in genreal, DBSCAN performs adequately on almost everything despite being very simple. Currently I am using DBSCAN to find errors and assess dataset quality. It is a Spark job written in Scala (github). It does not run on species with lots of (>30K) unique latitude-longitude points, since the current implementation relies on an in-memory distance matrix. However, around 99% of species (plants, animals, fungi) on GBIF have fewer than >30K unique lat-long points (2,283 species keys / 222,993 species keys). There are other implementations ( example) that might scale to many more points. There are no immediate plans to include DBSCAN outliers as a data quality flag on GBIF, but it could be done somewhat easily, since this type of method does not rely on any external environmental data sources and already runs on the GBIF cluster.

Forecasting Worldwide Local Carbon Dioxide Levels Using Spectral Techniques and Big Data Sources

This paper presents a method for the estimation and forecast of worldwide local emission levels based on a space-time extension of a discrete thin-plate smoothing model in the frequency domain. The proposed method can potentially be applied where there is an interest in forecasting the time evolution of large geographical or spatial databases at all global grid locations simultaneously. In particular, the paper illustrates the method in the estimation and forecast of underlying local levels of carbon dioxide around the globe using the 0.1ox 0.1o grid from the EDGAR database. The results highlight the world distribution of carbon dioxide emission levels and their variations during the historic period recorded in the database as well as their forecasts up to 2025. Global carbon dioxide will reach 53,161 mill tons by 2025, representing an increase of 29.4% from 2018 levels and of more than 52% since 2012. Shanghai, China, is the location leader in carbon dioxide emission levels since 2009; reaching 46.13 mill tons in its surrounding 10 km square region by 2025. China-Korea, North America, India, Europe and Japan, in this order, will be the highest emitting regions in the world by 2025, with Africa-Middle East, due to the oil and gas fields industry, and South America rising fast behind. In fact, most developing countries show a high rate of increase in their carbon dioxide levels during the 2012--2025 period. In comparison, North America and Japan show stable levels while Europe may see its levels actually decrease. However, even within these more developed regions it is of environmental interest to identify high emitting locations at a local level.

Use of amplified expenditure expenses using the calculation services scheme

An information analyst can study the ability of social achievement according to countless user site records by evaluating multiple circular range queries. More specifically, the zeros of the previous search, especially dealing with system comparisons or encrypting command-saving files, can be expanded and exploited to allow exploration of spatial data between axes and rectangular parallels. While most search file encryption schemes focus on common SQL queries, for example, keyword queries and logical queries, two searches have been achieved in particular from the search for geometric range to encrypted spatial data. Our main contribution to the fact that our design is in fact a general approach, can support several types of to the axis. With the rapid development of social systems and location-based services and travel using a laptop, the amount of data that people create every day continues to grow dramatically. It is no longer easy or profitable for companies to keep a large amount of data in their area. More specifically, it lacks a general approach, which can accept various types of engineering queries on a large scale in encrypted spatial data, regardless of their specific geometric shapes. Our design has great potential for use and application in large applications, for example, location-based services and spatial databases, requiring the use of sensitive spatial data based on robust privacy.

Problems of Large Spatial Databases

Problems of Large Spatial Databases

PREDICTION BASED WORKLOAD PERFORMANCE EVALUATION FOR DISASTER MANAGEMENT SPATIAL DATABASE

Abstract. This paper discusses a prediction based workload performance evaluation implementation during Disaster Management, especially at the response phase, to handle large spatial data in the event of an eruption of the Merapi volcano in Indonesia. Complexity associated with a large spatial database are not the same with the conventional database. This implies that in coming complex work loads are difficult to be handled by human from which needs longer processing time and may lead to failure and undernourishment. Based on incoming workload, this study is intended to predict the associated workload into OLTP and DSS workload performance types. From the SQL statements, it is clear that the DBMS can obtain and record the process, measure the analysed performances and the workload classifier in the form of DBMS snapshots. The Case-Based Reasoning (CBR) optimised with Hash Search Technique has been adopted in this study to evaluate and predict the workload performance of PostgreSQL. It has been proven that the proposed CBR using Hash Search technique has resulted in acceptable prediction of the accuracy measurement than other machine learning algorithm like Neural Network and Support Vector Machine. Besides, the results of the evaluation using confusion matrix has resulted in very good accuracy as well as improvement in execution time. Additionally, the results of the study indicated that the prediction model for workload performance evaluation using CBR which is optimised by Hash Search technique for determining workload data on shortest path analysis via the employment of Dijkstra algorithm. It could be useful for the prediction of the incoming workload based on the status of the predetermined DBMS parameters. In this way, information is delivered to DBMS hence ensuring incoming workload information that is very crucial to determine the smooth works of PostgreSQL.

Efficient query processing on large spatial databases: A performance study

A spatial index, called xBR+-tree, is compared against the R*-tree and the R+-tree.Performance of tree building and processing of common spatial queries is studied.Existing and new algorithms for processing these queries are utilized.Extensive experimental results on medium and large datasets are presented.The xBR+-tree is the winner of execution-time in all cases and of I/0 in most cases. Processing of spatial queries has been studied extensively in the literature. In most cases, it is accomplished by indexing spatial data using spatial access methods. Spatial indexes, such as those based on the Quadtree, are important in spatial databases for efficient execution of queries involving spatial constraints and objects. In this paper, we study a recent balanced disk-based index structure for point data, called xBR+-tree, that belongs to the Quadtree family and hierarchically decomposes space in a regular manner. For the most common spatial queries, like Point Location, Window, Distance Range, Nearest Neighbor and Distance-based Join, the R-tree family is a very popular choice of spatial index, due to its excellent query performance. For this reason, we compare the performance of the xBR+-tree with respect to the R*-tree and the R+-tree for tree building and processing the most studied spatial queries. To perform this comparison, we utilize existing algorithms and present new ones. We demonstrate through extensive experimental performance results (I/O efficiency and execution time), based on medium and large real and synthetic datasets, that the xBR+-tree is a big winner in execution time in all cases and a winner in I/O in most cases.

A methodology for dealing with Spatial Big Data

Spatial data mining (SDM) refers to the mining of knowledge from spatial data. Recently, location-based services have enabled the gathering of a significant amount of geo-referenced data, i.e., of spatial big data (SBD). Spatial datasets often exceed the ability of current computing systems to manage these data with reasonable effort; therefore, data-intensive computing and data mining techniques are useful tools for conducting an analysis. In this paper, we present an approach to the clustering of high-dimensional data that allows a flexible approach to the statistical modelling of phenomena characterised by unobserved heterogeneity. Numerous clustering algorithms have been developed for large databases; density-based algorithms particularly treat a huge amount of data in large spatial databases. We present the Modified Density-Based Spatial Clustering of Applications with Noise (MDBSCAN) algorithm and compare it to the classical k-means approach. Both applications use synthetic datasets and a dataset of satellite images.

A methodology for dealing with spatial big data

Spatial data mining (SDM) refers to the mining of knowledge from spatial data. Recently, location-based services have enabled the gathering of a significant amount of geo-referenced data, i.e., of spatial big data (SBD). Spatial datasets often exceed the ability of current computing systems to manage these data with reasonable effort; therefore, data-intensive computing and data mining techniques are useful tools for conducting an analysis. In this paper, we present an approach to the clustering of high-dimensional data that allows a flexible approach to the statistical modelling of phenomena characterised by unobserved heterogeneity. Numerous clustering algorithms have been developed for large databases; density-based algorithms particularly treat a huge amount of data in large spatial databases. We present the Modified Density-Based Spatial Clustering of Applications with Noise (MDBSCAN) algorithm and compare it to the classical k-means approach. Both applications use synthetic datasets and a dataset of satellite images.

Managing uncertainty of large spatial databases

Spatial data are prevalent in location-based services (LBS), sensor networks, and RFID monitoring systems. Data readings collected in these applications are often imprecise. The uncertainty in the data can arise from multiple sources, including measurement errors due to the sensing instrument and discrete sampling of the measurements. It is often important to record the imprecision and also to take it into account when processing the spatial data. The challenges of handling the uncertainty in spatial data includes modeling, semantics, query operators and types, efficient execution, and user interfaces. Probabilistic models have been proposed for handling the uncertainty. In this paper, we examine the modeling and querying issues of this kind of databases.

Spatial co-location pattern mining of facility points-of-interest improved by network neighborhood and distance decay effects

ABSTRACTThe aim of mining spatial co-location patterns is to find the corresponding subsets of spatial features that have strong spatial correlation in the real world. This is an important technology for the extraction and comprehension of implicit knowledge in large spatial databases. However, existing methods of co-location mining consider events as taking place in a homogeneous and isotropic context in Euclidean space, whereas the physical movement in an urban space is usually constrained by a road network. Furthermore, previous works do not take the ‘distance decay effect’ of spatial interactions into account, which may reduce the effectiveness of the result. Here we propose an improved spatial co-location pattern mining method, including the network-constrained neighborhood and addition of a distance-decay function, to find the spatial dependence between network phenomena (e.g. urban facilities). The underlying idea is to utilize a model function in the interest measure calculation to weight the contribution of a co-location to the overall interest measure instance inversely proportional to the separation distance. Our approach was evaluated through extensive experiments using facility points-of-interest data sets. The results show that the network-constrained approach is a more effective method than the traditional one in network-structured space. The proposed approach can also be applied to other human activities (e.g. traffic accidents) constrained by a street network.

Towards an Automated Comparison of OpenStreetMap with Authoritative Road Datasets

AbstractOpenStreetMap (OSM) is an extraordinarily large and diverse spatial database of the world. Road networks are amongst the most frequently occurring spatial content within the OSM database. These road network representations are usable in many applications. However the quality of these representations can vary between locations. Comparing OSM road networks with authoritative road datasets for a given area or region is an important task in assessing OSM's fitness for use for applications like routing and navigation. Such comparisons can be technically challenging and no software implementation exists which facilitates them easily and automatically. In this article we develop and propose a flexible methodology for comparing the geometry of OSM road network data with other road datasets. Quantitative measures for the completeness and spatial accuracy of OSM are computed, including the compatibility of OSM road data with other map databases. Our methodology provides users with significant flexibility in how they can adjust the parameterization to suit their needs. This software implementation is exclusively built on open source software and a significant degree of automation is provided for these comparisons. This software can subsequently be extended and adapted for comparison between OSM and other external road datasets.

UK Biobank Urban Morphometric Platform (UKBUMP) – a nationwide resource for evidence-based healthy city planning and public health interventions

The built environment (BE) has emerged as one of the ‘first causes’ of chronic disease, capable of explaining its socio-spatial variation. There is an increasing need for objective, detailed and precise measurements of attributes of BE that may influence our lifestyle, behaviour and hence physical and mental health.In this paper, we report the UK Biobank Urban Morphometric Platform (UKBUMP), the first ever very large sample size high resolution spatial database of urban morphological metrics (morphometrics), being developed for half-a-million participants of the UK Biobank Prospective study spatially distributed across 22 UK cities. Large-scale objective assessment of the BE was conducted employing state-of-the-art spatial and network analyses upon multiple national-level spatial data-sets.Prospective large-scale objective assessment of the BE enables the development of BE-health modelling studies that have the potential to identify causal pathways from specific attributes of the BE to various complex chronic health outcomes as well as well-being. The UKBUMP will act as a national resource, providing a platform for evidence-based healthy city planning and interventions for the first half of the twenty-first century.

Approximate String Search in Large Spatial Database

Applications are featured with both text and location information, which leads to a search like: spatial approximate string search (SAS). Mainly four issues are identified in the general area of SAS. They are: (i) Spatial approximate string search in Euclidean space (Esas); (ii) Spatial approximate string search on road networks (RSAS); (iii) Selectivity Estimation for Esas Range Queries; (iv) Multi-Approximate-Keyword Routing query on road networks. For efficiently answering spatial approximate string queries in Euclidean space, SAS propose a novel index structure, IR2 -tree, which is based on the R-tree augmented with the min-wise signature and the linear hashing technique. Extensive experiments on large real data sets demonstrate the efficiency and effectiveness of the proposed approach.

Design of computationally efficient density-based clustering algorithms

The basic DBSCAN (Density-Based Spatial Clustering of Applications with Noise) algorithm uses minimum number of input parameters, very effective to cluster large spatial databases but involves more computational complexity. The present paper proposes a new strategy to reduce the computational complexity associated with the DBSCAN by efficiently implementing new merging criteria at the initial stage of evolution of clusters. Further new density based clustering (DBC) algorithms are proposed considering correlation coefficient as similarity measure. These algorithms though computationally not efficient, found to be effective when there is high similarity between patterns of dataset. The computations associated with DBC based on correlation algorithms are reduced with new cluster merging criteria. Test on several synthetic and real datasets demonstrates that these computationally efficient algorithms are comparable in accuracy to the traditional one. An interesting application of the proposed algorithm has been demonstrated to identify the regional hazard regions present in the seismic catalog of Japan.

Critical Analysis of Density-based Spatial Clustering of Applications with Noise (DBSCAN) Techniques

Clustering is the most used technique in data mining. Clustering maximize the intra-cluster similarity and minimize the inter clusters similarity. DBSCAN is the basic density based clustering algorithm. Cluster is defined as regions of high density are separated from regions that are less dense. DBSCAN algorithm can discover clusters of arbitrary shapes and size in large spatial databases. Beside its popularity, DBSCAN has drawbacks that its worst time complexity reaches to O (n2). Similarly, it cannot deal with varied densities. It is hard to know the initial value of input parameters. In this study, we have studied and discussed some significant enhancement of DBSCAN algorithm to tackle with these problems. We analysed all the enhancements to computational time and output to the original DBSCAN. Majority of variations adopted hybrid techniques and use partitioning to overcome the limitations of DBSCAN algorithm. Some of which performs better and some have their own usefulness and characteristics.