Focus on Hard Categories and Hard Examples: Remote Sensing Image Scene Classification via Expert Model and Hard Example Mining

Abstract

Deep learning has seen dramatic improvements in remote-sensing image scene classification. However, hard categories and hard examples widely exist in the data sets, due to the intraclass diversity and interclass similarity. In this letter, we propose a novel framework to address these issues. Specifically, our method first trains a general model to obtain the confusion matrix and select the hard categories. Then a sampling strategy is proposed to restructure the training set and an expert model is trained to focus on the hard categories. Finally, the knowledge of the expert model is distilled into the student model through a novel loss function, which encourages the student model to predict the hard label provided by manual annotation. Thus, the model can match the soft label provided by the expert model and pay more attention to the hard examples simultaneously. With this method, the student model cannot only deal with hard categories but also hard examples existing in other easy categories. To empirically demonstrate the effectiveness of the proposed method, we comprehensively evaluate the method on three publicly available benchmark data sets, the obtained results show that the proposed method outperforms the existing baseline methods and achieves superior results on all three data sets.