Multi-view Deep Representations with Cross-Dataset Transfer for Remote Sensing Image Retrieval and Classification

Abstract

Transfer learning is a challenging task in computer vision, due to the differences of data distribution. Although convolutional neural network (CNN) could learn different levels of image abstraction, the single-view features extracted from the final layer of a pre-trained or fine-tuned CNN may result in insufficient image description over different datasets. To address this issue, we focus on deep representations with multi-view analysis for remote sensing image (RSI) retrieval and classification tasks using five recently released large-scale datasets. First, cross-dataset transfer learning is presented by fine-tuning a pre-trained CNN on one dataset and testing the fine-tuned network on another one. Second, multi-view image representations are explored in terms of different activation vectors as well as CNNs. Finally, a multi-view fusion and a random projection (RP) strategy are proposed to improve the accuracies and computational cost of both RSI tasks, respectively. Extensive experiments are conducted to demonstrate the effectiveness of the proposed method.