Low-Shot Learning for the Semantic Segmentation of Remote Sensing Imagery

Abstract

Recent advances in computer vision using deep learning with RGB imagery (e.g., object recognition and detection) have been made possible thanks to the development of large annotated RGB image data sets. In contrast, multispectral image (MSI) and hyperspectral image (HSI) data sets contain far fewer labeled images, in part due to the wide variety of sensors used. These annotations are especially limited for semantic segmentation, or pixelwise classification, of remote sensing imagery because it is labor intensive to generate image annotations. Low-shot learning algorithms can make effective inferences despite smaller amounts of annotated data. In this paper, we study low-shot learning using self-taught feature learning for semantic segmentation. We introduce: 1) an improved self-taught feature learning framework for HSI and MSI data and 2) a semisupervised classification algorithm. When these are combined, they achieve the state-of-the-art performance on remote sensing data sets that have little annotated training data available. These low-shot learning frameworks will reduce the manual image annotation burden and improve semantic segmentation performance for remote sensing imagery.