Reference-free method for investigating classification uncertainty in large-scale land cover datasets

Abstract

In the last decade, land cover products are produced at a global scale and updated with an unprecedent speed with the development of earth observation and mapping techniques. However, assessing large-scale land cover datasets is always a challenging task because of lack of ground truth and high dependency on manual inspection. To promote the efficiency of assessment, this study introduces a generic framework that identifies potential land cover classification errors without traditional reference data. Depending on rich features in remote sensing data, the overall procedure of can be regarded as a multi-class anomaly detection problem. To improve the performance in dealing with complex classification scheme, a pairing strategy is firstly proposed for pairwise analysis on each two classes, so that the multi-class problem is decomposed into multiple smaller binary-class problems. Secondly, the proximity matrix of each class pair is generated on the basis of extra trees model, which is applicable to mixed numerical and categorical data and provides robust proximity measurement by including multiple levels of randomness. Finally, an improved proximity-based anomaly detection algorithm is applied, and the detection results of each class pair are ensembled to obtain the final anomaly score of each land instance. Experiments show that the overall area under the receiver operating characteristic curve score reaches approximately 0.9 for synthetic datasets and 0.8 for real-life datasets. The proposed method is expected to provide users valuable data for a more efficient assessment in the absence of ground truth in practice.