Abstract
The utility of land cover maps for natural resources management relies on knowing the uncertainty associated with each map. The continuous advances typical of remote sensing, including the increasing availability of higher spatial and temporal resolution satellite data and data analysis capabilities, have created both opportunities and challenges for improving the application of accuracy assessment. There are well established accuracy assessment methods, but their underlying assumptions have not changed much in the last couple decades. Consequently, revisiting how map error and accuracy have been performed and reported over the last two decades is timely, to highlight areas where there is scope for better utilization of emerging opportunities. We conducted a quantitative literature review on accuracy assessment practices for mapping via remote sensing classification methods, in both terrestrial and marine environments. We performed a structured search for land and benthic cover mapping, limiting our search to journals within the remote sensing field, and papers published between 1998–2017. After an initial screening process, we assembled a database of 282 papers, and extracted and standardized information on various components of their reported accuracy assessments. We discovered that only 56% of the papers explicitly included an error matrix, and a very limited number (14%) reported overall accuracy with confidence intervals. The use of kappa continues to be standard practice, being reported in 50.4% of the literature published on or after 2012. Reference datasets used for validation were collected using a probability sampling design in 54% of the papers. For approximately 11% of the studies, the sampling design used could not be determined. No association was found between classification complexity (i.e. number of classes) and measured accuracy, independent from the size of the study area. Overall, only 32% of papers included an accuracy assessment that could be considered reproducible; that is, they included a probability-based sampling scheme to collect the reference dataset, a complete error matrix, and provided sufficient characterization of the reference datasets and sampling unit. Our findings indicate that considerable work remains to identify and adopt more statistically rigorous accuracy assessment practices to achieve transparent and comparable land and benthic cover maps.
Highlights
Maps are an important tool for natural resources management, for activities like spatial planning, monitoring ecosystem changes, ecosystem services evaluation, and habitat conservation
In the past two decades, the remote sensing community has undertaken important efforts in developing a set of guiding principles for accuracy assessment [1,18,19,20,21,22,23,24], which were initially focused on land cover mapping, but have been more widely adopted to other applications
The studies varied widely in the level of detail provided about the map validation process, as well as on the metrics used to report its accuracy
Summary
Maps are an important tool for natural resources management, for activities like spatial planning, monitoring ecosystem changes, ecosystem services evaluation, and habitat conservation. Any map is a model—a simplification of reality—[1,2], and as such it has an associated error Estimating this error or assessing the accuracy of a map is important to evaluate classification methodologies and sources of uncertainty, but is key to assess if a map is fit for purpose [3]. In the past two decades, the remote sensing community has undertaken important efforts in developing a set of guiding principles for accuracy assessment [1,18,19,20,21,22,23,24], which were initially focused on land cover mapping, but have been more widely adopted to other applications (e.g. coral reefs, change detection). “good practice” guidelines recommend the use of an error matrix and relevant per-class metrics [20,25] to estimate accuracy, and confidence intervals to quantify precision
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
