Volunteered Geographic Information Quality Research Articles

Validating the Quality of Volunteered Geographic Information (VGI) for Flood Modeling of Hurricane Harvey in Houston, Texas

The primary objective of this study was to examine the quality of volunteered geographic information (VGI) data for flood mapping of Hurricane Harvey. As a crowdsourcing platform, the U-Flood project mapped flooded streets in the Houston metro area. This research examines the following: (1) If there are any significant differences in water depth (WD) among the hydraulic and hydrologic (H&H) model, the Federal Emergency Management Agency (FEMA) reference floodplain map, and the VGI? (2) Are there any significant differences in the inundated areas between the floodplain modeled by the VGI and hydraulic simulation? This study used HEC-RAS to simulate flood inundation maps and validated the results with high water marks (HWM) and the FEMA-modeled floodplain after Hurricane Harvey. The statistical results showed that there were significant differences in the WD, the inundated road count, and the length inside/outside of HEC-RAS-modeled floodplain. The results also showed that a less consistent decreasing trend between the U-Flood data and the modeled floodplain over time and space. This study empirically evaluated the data quality of the VGI based on observed and modeled data in flood monitoring. The findings from this study fill the gaps in the literature by assessing the uncertainty and data quality of VGI, providing insights into using supplementary data in flood mapping research.

Quality assessment of volunteered geographic information for outdoor activities: an analysis of OpenStreetMap data for names of peaks in Japan

ABSTRACT Geographical studies of outdoor activities have increased in recent years with the rise in popularity of these activities worldwide, including in Japan. Volunteered geographic information (VGI) is a key tool for organizing outdoor activities as it offers a means to determine the locational information and names of places. To evaluate the quality of VGI, geospatial data generated by land survey agencies and other VGI are often utilized as reference data. However, since these reference data may not be available, other methods are necessary to assure the quality of VGI. In this study, we examined five trust indicators based on the inherent characteristics of VGI through an empirical case study. We used mountain names extracted from OpenStreetMap in Japan as data because there were almost no other VGI in the vicinity. As a result, we isolated three trust indicators, namely versions, users, and tag corrections, to examine the thematic accuracy of VGI because these were the only statistically significant indicators. However, we found that the prediction rate of thematic accuracy was very low. To improve thematic accuracy, this study recommends using the most accurate versions, applying correctly given tags, and considering the motivations and characteristics of the VGI contributors.

A Risk-Based Approach for Enhancing the Fitness of Use of VGI

Volunteered Geographic Information (VGI) phenomena offer an alternative or supplement to the authoritative mechanism of geospatial data acquisition. It aims to allow people without professional geospatial skills or knowledge to participate in the geospatial data collection. VGI has been boosted by recent advances in geospatial technology and applications. VGI applications have shown great potential in various areas such as disaster management and public health. However, VGI suffers from a lack of quality assurance, because VGI contributors may lack knowledge of the geospatial domain and credibility. Moreover, VGI data may have different levels of detail and precision, and may have been collected for different purposes. Appropriate VGI data for a specific application may be less appropriate for another application. End-users may use VGI data without being aware of its appropriateness to their requirements. This may cause a risk that arises when end-users inappropriately use, or be uncertain about using VGI data in their applications. This risk may undermine the VGI project, in which end-users are involved. This paper proposes an approach that aims to enhance VGI quality assurance by measuring the spatio-semantic similarity between user requirements and provided VGI, and to evaluate the fitness-for-use of VGI. The proposed approach is based on an algorithm to help VGI end-users to deal with the risks related to the quality of VGI data. The approach helps end-user make appropriate decisions about VGI data (e.g., considering or not considering VGI data and being careful when using VGI data).

Leveraging Road Characteristics and Contributor Behaviour for Assessing Road Type Quality in OSM

Volunteered Geographic Information (VGI) is often collected by non-expert users. This raises concerns about the quality and veracity of such data. There has been much effort to understand and quantify the quality of VGI. Extrinsic measures which compare VGI to authoritative data sources such as National Mapping Agencies are common but the cost and slow update frequency of such data hinder the task. On the other hand, intrinsic measures which compare the data to heuristics or models built from the VGI data are becoming increasingly popular. Supervised machine learning techniques are particularly suitable for intrinsic measures of quality where they can infer and predict the properties of spatial data. In this article we are interested in assessing the quality of semantic information, such as the road type, associated with data in OpenStreetMap (OSM). We have developed a machine learning approach which utilises new intrinsic input features collected from the VGI dataset. Specifically, using our proposed novel approach we obtained an average classification accuracy of 84.12%. This result outperforms existing techniques on the same semantic inference task. The trustworthiness of the data used for developing and training machine learning models is important. To address this issue we have also developed a new measure for this using direct and indirect characteristics of OSM data such as its edit history along with an assessment of the users who contributed the data. An evaluation of the impact of data determined to be trustworthy within the machine learning model shows that the trusted data collected with the new approach improves the prediction accuracy of our machine learning technique. Specifically, our results demonstrate that the classification accuracy of our developed model is 87.75% when applied to a trusted dataset and 57.98% when applied to an untrusted dataset. Consequently, such results can be used to assess the quality of OSM and suggest improvements to the data set.

Towards Development of a Real-Time Point Feature Quality Assessment Method for Volunteered Geographic Information Using the Internet of Things

One of the most important challenges of volunteered geographic information (VGI) is the quality assessment. Existing methods of VGI quality assessment, either assess the quality by comparing a reference map with the VGI map or deriving the quality from the metadata. The first approach does not work for a real-time scenario and the latter delivers approximate values of the quality. Internet of Things (IoT) networks provide real-time observations for environment monitoring. Moreover, they publish more precise information than VGI. This paper introduces a method to assess the quality of VGI in real-time using IoT observations. The proposed method filters sensor observation outliers in the first step. Then it matches sensors and volunteers’ relationships in terms of location, time, and measurement type similarity using a hypergraph model. Then the quality of matched data is assessed by calculating positional and attribute accuracy. To evaluate the method, VGI data of the water level and quality in Tarashk–Bakhtegan–Maharlou water basin is studied. A VGI quality map of the data is assessed by a referenced authoritative map. The output of this step is a VGI quality map, which was used as a reference to check the proposed method quality. Then this reference VGI quality map and the proposed method VGI quality map are compared to assess positional and attribute accuracy. Results demonstrated that 76% of the method results have less than 20 m positional error (i.e., difference with the reference VGI quality map). Additionally, more than 92% of the proposed method VGI data have higher than 90% attribute accuracy in terms of similarity with the reference VGI quality map. These findings support the notion that the proposed method can be used to assess VGI quality in real-time.

Ranking of VGI contributor reputation using an evaluation‐based weighted pagerank

AbstractThe quality of volunteered geographic information (VGI) is questionable as it emerges through a diversity of contributors. The reputation of a contributor is increasingly applied to VGI quality assessment and its assurance. Research on how to measure and validate reputation, however, is still required. This study proposes an evaluation‐based weighted PageRank (EWPR) algorithm to provide a ranking metric of reputation. Ranking is established on the basis of the assumptions that: (a) there is an evaluation relationship between VGI contributors; (b) the reputation of a contributor is movable within the VGI community; and (c) highly active contributors are more likely to have high reputation. By means of case studies using OpenStreetMap, two existing methods are compared with the EWPR algorithm. The results show that the algorithms used for web pages and social network participants are applicable to understand the reputation of VGI contributors. This study also applies credibility, based on reputation, as a popular indicator of VGI quality. The high correlation between credibility and VGI characteristics indicates that reputation is useful for dealing with quality variability in VGI.

OpenStreetMap quality assessment using unsupervised machine learning methods

AbstractThe reliability and quality of volunteered geographic information (VGI) continue to be pressing concerns. Many VGI projects lack standard geospatial data quality assurance procedures, and the reliability of contributors remains in question. Traditional approaches rely on comparing VGI to an “authoritative” or “gold standard” dataset to assess quality. This study investigates VGI quality by analysing the OpenStreetMap (OSM) database in Ottawa‐Gatineau, focusing on historical map features and contributor data to gain an understanding of how users are contributing to the database, and their ability to do so accurately. Unsupervised machine learning analyses expose a cluster of experienced contributors classified as “OSM validators/experts”, which are then further used to attribute data quality. They are identified through a combination of strong contribution loadings associated with the use and experience of advanced OSM editors, and weaker loadings associated with feature creation and frequency of contributions leading to further correction. Limitations are discussed with implications for future work.

Using Kappa methodology to consider volunteered geographic information in official land administration systems in developing countries

Developing countries have ineffective and outdated land administration systems (LAS) which make it a challenge to properly register, manage and secure tenure for local communities. Moreover, limited maintenance budgets prevalent in these countries make it difficult for organizations to conduct regular systematic updates of geographic information. For a LAS to remain useful, it must reflect realities on the ground, and this can only be achieved if land information is reported regularly. However, if changes in the land are not captured in properly administered land registers, LAS lose societal relevance and are eventually replaced by informal systems. Volunteered Geographic Information (VGI) can address these challenges by providing a timely, affordable, flexible, inclusive, up-to-date, and fit for purpose land information to support the limited current systems. However, the consideration and integration of such external sources relies on the quality and credibility of contributed datasets by the public. The public whom in most cases are non-geographic experts can contribute datasets of varying quality and detail, resulting in the reluctance of officials to utilize VGI in their databases. Therefore, this study proposes a Kappa statistical measure methodology: a consensus-based measure of agreement to establish ‘proxy’ quality and credibility of VGI. The methodology was successfully implemented in a case study of Botswana to demonstrate its effectiveness in establishing trust and confidence of contributed datasets by the public. The innovative approach to Kappa presented here, minimizes the need for ground truthing of VGI data.

VGI contributors’ awareness of geographic information quality and its effect on data quality: a case study from Japan

ABSTRACTIn many countries, geospatial data are typically provided by public institutions. Cities have been mapped using such public data. On the other hand, the demand for geospatial data has been diversifying, given the requirements for mapping cities. To respond to demands for new geospatial data, creation of citizen-generated open data and volunteered geographic information (VGI) have recently become popular. However, the quality of such open data and VGI are not always guaranteed. The number of studies on quality assessments of VGI have increased in recent years. The present study aimed to identify OpenStreetMap (OSM), one type of VGI, as well as contributors’ awareness of data quality, and the relationships between their awareness and the positional accuracy of the OSM data contributed by them. The results showed that awareness or lack of the positional accuracy did not affect the quality of the OSM data created by the contributors. These findings suggest that the crowdsourcing approach might not guarantee the data quality of VGI.

An Approach to Control the Positional Accuracy of Point Features in Volunteered Geographic Information Systems

Volunteered geographic information (VGI) is a huge source of user-generated geographic information. There is an enormous potential to use VGI in different mapping activities due to its significant advantages. VGI is found to be richer and more up-to-date than authoritative geographic information. However, VGI quality is an obvious challenge that needs to be addressed in order to get the full potential of VGI. Positional accuracy is one of the important aspects of VGI quality. Although VGI positional accuracy can be high in some contexts, VGI datasets are characterized by a large spatial heterogeneity. This paper proposes an approach for controlling positional accuracy as well as decreasing the spatial heterogeneity of point features in VGI systems. A case study has been conducted in order to ensure the applicability and effectiveness of the proposed approach.

Exploring the use of crowdsourced geographic information in defence: challenges and opportunities

Geographic data are used by United Kingdom (UK) defence for purposes including peacekeeping, humanitarian aid and disaster relief, and fighting wars. The geographic extent of defence data covers the world, with greater focus directed towards areas considered to be of current interest. Traditionally, these data have been officially sourced, e.g. via National Mapping Agencies, but there is now increasing interest in the potential of crowdsourced geographic data to supplement authoritative data where they are not available, outdated or incomplete. Volunteered geographic information (VGI) and social media have the potential to provide this needed missing information. This paper presents initial work carried out in identifying the potential of crowdsourced geographic information in defence. We first provide a short description of the role of UK defence and review the existing literature on crowdsourced geographic information in defence, as well as generic VGI quality assessment methods. We then explore the potential of crowdsourced data in real-world applications: the conflation of VGI and social media with official data for effective decision-making in war zones, and the potential for crowdsourcing to increase effective collaboration between machines and humans in disaster situations. Based on our review, we outline specific research challenges for deploying crowdsourced geographic information in defence, focussing on data quality and fitness-for-purpose assessment. Defence-specific constraints include the need for rapid quality assessment processes and the need to communicate high-quality information effectively in situations where rapid decision-making is required. Ethical issues are also of fundamental importance.

THE QUALITY OF OPENSTREETMAP IN MALAYSIA: A PRELIMINARY ASSESSMENT

Abstract. Over recent years, the phenomena, Web 2.0 has led to the growth of volunteered geographic information (VGI). The emergence of VGI has played an important role in providing timely data when the costs and its availability is a major concern particularly during emergency and humanitarian efforts. The worldwide crowdsourcing efforts through OpenStreetMap (OSM), the most successful open platform for collaborative mapping have managed to assist authorities such as during the 2017 Mexico earthquake and Hurricane Irma and Maria that impacted several countries in America continent. However, there are lots of arguments on the quality of VGI, particularly in regard to OpenStreetMap (OSM). Therefore, this study was carried out to assess the quality of OSM against authoritative sources using a dataset of Putrajaya, Malaysia. This study assessed the quality of OSM, including completeness, positional and thematic accuracy. From the preliminary assessment, the results showed that the OSM data was good in terms of relative positioning accuracy, particularly in road feature, but still poor in terms of completeness and thematic correctness against the reference dataset. This study is significant with an expected contribution to the assessment of quality of VGI in developing countries that commonly facing slow-paced progress in mapping the OSM. The findings could be used as a basis for various parties that plan to use OSM in Malaysia, particularly Putrajaya as a supplementary data to authoritative sources, including data supplied by the professional surveyors.

Improvement of a location-aware recommender system using volunteered geographic information

Recommender systems (RS), as supportive tools, filter information from a massive amount of data based on the determined preferences. Most of the RS require information about the context of users such as their locations. In such cases, location-aware recommender systems (LARS) can be employed to suggest more personalized items to the users. The most current research projects on LARS focus on the development of algorithms, evaluation methods and applications. However, the role of up-to-date spatial databases in LARS is not a well-researched area. The up-to-date spatial information would potentially improve the accuracy of items which are recommended by LARS. Volunteered geographic information (VGI) could be a low-cost source of up-to-date spatial information for LARS. This article proposes an approach to enrich spatial databases of LARS by VGI. Since not all records of VGI are fitted for use in LARS, a mechanism is developed to identify useful information. Some VGI data sets refer to existing spatial data in the database while other VGI data sets are shared for the first time. Therefore, the proposed method assessed the quality of VGI with reference source (for VGI which is existed in the database) and VGI without reference source (for VGI which is shared for the first time). To demonstrate the feasibility of the proposed approach, a mobile application has been developed to recommend suitable restaurants to the users based on their geospatial locations. The evaluation of the method indicates that VGI can potentially enhance the functionality of the LARS in predicting the users’ interests.

Modeling aggregated expertise of user contributions to assess the credibility of OpenStreetMap features

AbstractThe emergence of volunteered geographic information (VGI) during the past decade has fueled a wide range of research and applications. The assessment of VGI quality and fitness‐of‐use is still a challenge because of the non‐standardized and crowdsourced data collection process, as well as the unknown skill and motivation of the contributors. However, the frequent approach of assessing VGI quality against external data sources using ISO quality standard measures is problematic because of a frequent lack of available external (reference) data, and because for certain types of features, VGI might be more up‐to‐date than the reference data. Therefore, a VGI‐intrinsic measure of quality is highly desirable. This study proposes such an intrinsic measure of quality by developing the concept of aggregated expertise based on the characteristics of a feature's contributors. The article further operationalizes this concept and examines its feasibility through a case study using OpenStreetMap (OSM). The comparison of model OSM feature quality with information from a field survey demonstrates the successful implementation of this novel approach.

Volunteered geographic information quality assessment using trust and reputation modelling in land administration systems in developing countries

ABSTRACTThis article presents an innovative approach to establish the quality and credibility of Volunteered Geographic Information (VGI) such that it can be considered in Land Administration Systems (LAS) on a Fit for Purpose (FFP) basis. A participatory land information system can provide affordable and timely FFP information about land and its resources. However, the establishment of such a system involves more than just technical solutions and administrative procedures: many social, economic and political aspects must be considered. Innovative approaches like VGI can help address the lack of accurate, reliable and FFP land information for LAS, but integration of such sources relies on the quality and credibility of VGI. Verifying volunteer efforts can be difficult without reference to ground truth: a novel Trust and Reputation Modelling methodology is proposed as a suitable technique to effect such VGI data set validation. This method has been applied to successfully demonstrate that VGI can produce accurate and reliable data sets which can be used to conduct regular systematic updates of geographic information in official systems. It relies on a view that the public can police themselves in establishing proxy measures of VGI quality thus facilitating VGI to be used on a FFP basis in LAS.

Trust as a proxy indicator for intrinsic quality of Volunteered Geographic Information in biodiversity monitoring programs

In this article, we present a fuzzy model for intrinsic quality assessment of Volunteered Geographic Information (VGI) on species occurrences obtained by Citizen Science (CS) biodiversity monitoring programs. The proposed VGI quality assurance approach evaluates the thematic and positional quality of the crowdsourced biodiversity observation in terms of the trustworthiness of the observation by combining three indicators of consistency with habitat, consistency with surroundings, and reputation of contributor, that characterize the geographical and social aspects of trust in VGI. To evaluate the performance and usability of the proposed approach for evaluating the trustworthiness of crowdsourced observations and detecting thematic and positional errors in crowdsourced observations, the developed approach was applied to the crowdsourced observations on Acer macrophyllum collected through the CS biodiversity monitoring projects of E-Flora BC and iNaturalist. The result of a conformity test at the optimal acceptance threshold (sensitivity = 0.99, specificity = 0.8, and Cohen’s kappa = 0.79), the achieved area under the curve (AUC) value (AUC = 0.98), and the results of the complementary investigation on the predictions of the proposed model indicated that the proposed fuzzy trust model exhibited promising predictive performance and was able to flag the majority of attribute and positional errors in the crowdsourced biodiversity observations.

Coverage inequality and quality of volunteered geographic features in Chinese cities: Analyzing the associated local characteristics using geographically weighted regression

The volunteered geographic information (VGI) gains increasing popularity with the general public and scientific community. However, the optimism about the VGI has been tempered by two critical issues: inequality in data coverage (social justice) and data quality. It therefore requires a better understanding of the mechanism driving VGI contributions and content quality. With a case of China, this paper demonstrates one potential avenue, examining the associations between VGI coverage/quality and local demographic and socioeconomic characteristics. In particular, VGI data are harvested from the OpenStreetMap for 333 cities in China. VGI coverage is measured by the total volume of different geographic features (point, line and polygon); and VGI quality is described from two aspects: completeness and accuracy. Geographically weighted regression (GWR) shows that both demographic and socioeconomic factors have statistically significant influences on VGI coverage and quality. More specifically, densely populous cities with more young, educated and non-agricultural people enjoy higher VGI coverage and quality. Cities with lower VGI coverage and quality are primarily located in the western and southwestern regions where the ethnic minorities concentrate. High VGI coverage and quality are possibly observed in economically developed cities with high marketization degree. Besides, possibility of high VGI coverage and quality occurs in cities with more labor in scientific research and greater percentage of employers in the tertiary industry. The GWR also demonstrates that the strength and nature of the obtained relationships vary across the 333 cities. The spatial non-stationary relationships may partially answer for the controversial empirical conclusions in earlier case studies at different scales. Quantitative analysis (Gini index, Lorenz curve and Moran's I index) further evidences the great inequality in VGI coverage and quality. It can be safely inferred that the differences in engagement and use of VGI, as a new digital divide, can raise troubling concerns on the social justice implications.

Guided Classification System for Conceptual Overlapping Classes in OpenStreetMap

The increased development of Volunteered Geographic Information (VGI) and its potential role in GIScience studies raises questions about the resulting data quality. Several studies address VGI quality from various perspectives like completeness, positional accuracy, consistency, etc. They mostly have consensus on the heterogeneity of data quality. The problem may be due to the lack of standard procedures for data collection and absence of quality control feedback for voluntary participants. In our research, we are concerned with data quality from the classification perspective. Particularly in VGI-mapping projects, the limited expertise of participants and the non-strict definition of geographic features lead to conceptual overlapping classes, where an entity could plausibly belong to multiple classes, e.g., lake or pond, park or garden, marsh or swamp, etc. Usually, quantitative and/or qualitative characteristics exist that distinguish between classes. Nevertheless, these characteristics might not be recognizable for non-expert participants. In previous work, we developed the rule-guided classification approach that guides participants to the most appropriate classes. As exemplification, we tackle the conceptual overlapping of some grass-related classes. For a given data set, our approach presents the most highly recommended classes for each entity. In this paper, we present the validation of our approach. We implement a web-based application called Grass&Green that presents recommendations for crowdsourcing validation. The findings show the applicability of the proposed approach. In four months, the application attracted 212 participants from more than 35 countries who checked 2,865 entities. The results indicate that 89% of the contributions fully/partially agree with our recommendations. We then carried out a detailed analysis that demonstrates the potential of this enhanced data classification. This research encourages the development of customized applications that target a particular geographic feature.

Utilizing fuzzy set theory to assure the quality of volunteered geographic information

This paper presents a fuzzy system to assure the quality of volunteered geographic information (VGI) collected for the purposes of species surveillances. The system uses trust as a proxy of quality. It defines the trust using both the provenance of user expertise and the fitness of geographic context and quantifies it using fuzzy set theory. The system was applied to a specific scenario—VGI-based crop pest surveillance—to demonstrate its usefulness in handling VGI quality. A case study was conducted in Jiangxi province of China, where location-based rice pest surveillance reports generated by the local farmers were collected. A field pest survey was conducted by the local pest management experts to verify the farmer-generated reports, and the survey results were used as ground truth data. The quality of the farmer-generated reports were also assessed through the fuzzy system and compared to the pest survey results. It was observed that the degree to which these two sets of results agreed to each other was satisfactory.

A VGI Quality Assessment Method for VGI based on Trustworthiness

Recent advances in technology have greatly transformed how geographic information is produced and have led to the phenomenon of Volunteered Geographic Information (VGI). VGI allows people with little geographical knowledge to contribute in the creation of maps and other kinds of geographic information. Because VGI is gathered by individuals who often have no formal training, the credibility and reliability of VGI is challenging. In this paper, we study what kinds of things might contribute to an assessment of the trustworthiness of data and the reputation of contributors in a VGI system. We present a model for analysing these characteristics and a method for automatically creating trustworthiness and reputation scores in order to assess the quality of VGI features.