Superpixel-based multi-scale multi-instance learning for hyperspectral image classification

Abstract

Superpixels can define meaningful local regions within a hyperspectral image (HSI) and have become the building blocks of various HSI classification methods. The superpixels in HSIs consist of multiple spectral pixels, sharing a similar structure with the data in multi-instance learning (MIL). However, the potential of MIL methods in the field of HSI classification has been rarely explored. In this paper, we propose the superpixel-based multi-scale multi-instance learning (MSMIL) framework, enhancing the superpixel representation with MIL for the first time. Segmenting the HSIs with superpixels, MSMIL converts the HSI classification into MIL problems and extracts superpixel representations via the MIL method, namely multi-instance factor analysis (MIFA). Compared with the existing methods focusing exclusively on the local information, MIFA utilizes the deviations from an overall generative model to describe the superpixels, retaining both the local and the global information. Moreover, MSMIL introduces multi-scale superpixels and a spectral–spatial decision fusion strategy for further refinement, where the results of multi-scale superpixel maps are weighted according to prediction certainty and spatial consistency. The proposed method is evaluated on four benchmark datasets and achieves competitive results. For example, MSMIL outperforms the comparison methods with a margin of 5% overall accuracy on the Indian Pines dataset, when 2% pixels are selected as the training set.