Classification of Remote Sensing scenes using Semi-Supervised Domain Adaptation based on Entropy Adversarial Optimization

Abstract

In this paper, we present a new method for semi-supervised domain adaptation in remote sensing scene classification. The method is based on a pre-trained Convolutional Neural Network (CNN) model for the extraction of highly discriminative features, followed by a fully connected layer with softmax activation function that is responsible for the classification task. The weights of the fully connected layer represent prototype feature vectors for each class. These weights are divide by a temperature parameter for normalization. The whole network is trained on both the labeled and unlabeled target samples. First, the whole network is trained on the labeled source and target samples using the standard cross entropy loss to predict their correct classes. At the same time, the model is trained to learn domain invariant features using another loss function based on entropy computed over the unlabeled target samples. Unlike the standard cross entropy loss, the novel entropy loss function is computed on the predicted probabilities of the model and does not need the true labels. The proposed model combines the standard cross entropy loss and the new unlabeled samples entropy loss and optimizes them jointly. However, the new entropy loss function needs to be maximized with respect to the classification layer to learn features that are domain invariant (hence removing the data shift), and at the same time, it should be minimized with respect to the CNN feature extractor to learn discriminative feature that are clustered around the class prototypes (in other words reducing intra-class variance). To accomplish this maximization and minimization processes at the same time, we use an adversarial training approach, where we alternate between the two processes. This type of approach is called minmax entropy and the new proposed method is called Domain Adaptation CNN with MinMax Entropy (DACNN-MME). The proposed method is tested on three RS scene datasets, namely UC Merced, AID, and NWPU. The preliminary experimental results demonstrate the potential of the proposed method. Its performance is already better than several state-of-the-art methods including RevGard, ADDA, Siamese-GAN, and MSCN. With more analysis and fine-tuning of the method even better results can be achieved in the future.