Mapping per-pixel predicted accuracy of classified remote sensing images

Abstract

The traditional approach of map accuracy assessment based on an error matrix does not capture the spatial variation in classification accuracy. Here, per-pixel accuracy prediction methods are proposed based on interpolating accuracy values from a test sample in which the reference class of each sampled pixel has been determined. Different accuracy prediction methods were developed based on four factors: predictive domain (spatial versus spectral), interpolation function (constant, linear, Gaussian, and logistic), incorporation of class information (interpolating each class separately versus grouping them together), and sample size. Developing accuracy maps using the spectral domain is a new approach in contrast to previous efforts based on the spatial domain. Performance of the prediction methods was evaluated using 26 test blocks, with 10km×10km dimensions, dispersed throughout the United States. Each block had complete coverage reference data manually extracted by interpreters and a land-cover map produced from Landsat imagery using a decision tree classification. The full scene maps were then compared to the corresponding reference maps to produce complete coverage accuracy information for each block. The predicted accuracy maps were produced from a sample of the reference data (i.e., the test dataset). The performance of the sample-based accuracy predictions was evaluated using the area under the curve (AUC) of the receiver operating characteristic. Relative to existing accuracy prediction methods, our proposed methods resulted in improvements of AUC of 0.15 or greater. Evaluation of the four factors comprising the accuracy prediction methods demonstrated that: i) interpolations should be done separately for each class instead of grouping all classes together; ii) if an all-classes approach is used, the spectral domain will result in substantially greater AUC than the spatial domain; iii) for the smaller sample size and per-class predictions, the spectral and spatial domain yielded similar AUC; iv) for the larger sample size (i.e., very dense spatial sample) and per-class predictions, the spatial domain yielded larger AUC; v) increasing the sample size improved accuracy predictions with a greater benefit accruing to the spatial domain; and vi) the function used for interpolation had the smallest effect on AUC. To conclude, the ability to produce per-pixel accuracy predictions yielding simple to understand accuracy maps opens up new possibilities for error propagation of remotely sensed products in a variety of disciplines.