The true false ground truths: What interest?

Abstract

The existence of a few unreliable ground truth (GT) data sets which are often used as reference by the remote sensing community for the assessment and comparison of classification results is really problematic and poses a number of questions. Two of these ground truth data sets can be cited: Pavia University and Indian Pine. A rigorous analysis of spectral signatures of pixels in these images shows that some classes which are considered as homogeneous from the ground truth are clearly not, since the pixels which belong to the same classes have different spectral signatures, and probably do not belong to the same category. The persistence in using data sets from a biased ground truth does not allow objective comparisons between classification methods and does not contribute to providing explanation of physical phenomena that images are supposed to reflect. In this communication, we present a fine and complete analysis of the spectral signatures of pixels within each class for the two ground truth data sets mentioned above. The metrics used show some incoherence and inaccuracy of these data which wrongly serve as references in several classification comparative studies.