Multi-Structure Joint Decision-Making Approach for Land Use Classification of High-Resolution Remote Sensing Images Based on CNNs

Lu Xu,Yiyun Chen,Aji Gao,Jiawei Pan

doi:10.1109/access.2020.2976484

Abstract

Land use classification of high-resolution remote sensing (HRRS) images is a challenging and prominent problem in which pretrained convolutional neural networks (CNNs) have made amazing achievements. However, single-structured pretrained CNNs have limitations to obtain high classification accuracy. Besides, each pretrained CNNs has different classification ability to classify land use. Therefore, taking advantages of different pretrained CNNs is essential for land use classification. In this study, we propose a novel classification approach based on multi-structure joint decision-making strategy and pretrained CNNs. The basic idea is to apply three CNNs to classify land use separately with the final classification results achieved by joint decision-making strategy. The proposed approach comprises of three steps. First, we create a new fully connected layer and Softmax classification layer. We combine them with the convolutional layers of AlexNet, Inception-v3, and ResNet18. AlexNet also includes the first two layers of fully connected layers. Secondly, we train these designed CNNs to converge by momentum-driven stochastic gradient descent. Thirdly, we utilize joint decision-making strategy to obtain the final prediction results by combining the prediction results of these designed CNNs. The performance of the proposed approach is evaluated on the UC Merced land use, AID, NWPU-45, OPTIMAL-31 datasets and further compared with the state-of-the-art methods. Results demonstrate that the proposed approach outperforms other methods. The benefits of the proposed approach are threefold. First, the multi-structure network maximizes different pretrained CNN structures to extract rich convolution features. Secondly, it could remarkably improve the classification accuracy of indistinguishable land use types of the HRRS images. Thirdly, it has great potential on small dataset with more land use types. The proposed CNN based on multi-structure joint decision approach achieves accurate and reliable land use classification with these benefits.

Highlights

Land is an indispensable component of human production and life [1]
We propose a multi-structure joint decision approach based on convolutional neural networks (CNNs) (MJDCNN) and evaluate its performance in four highresolution remote sensing (HRRS) image datasets of land use
We aim to (1) establish MJDCNN for HRRS image classification based on three different pretrained CNNs (i.e. AlexNet, Inception-v3 and ResNet18) and joint decision-making strategy; (2) explore the effect of network training iterations on the MJDCNN’s classification performance; (3) evaluate the performance of MJDCNN and the three pretrained CNNs in terms of the overall classification accuracy, F1-score, and the classification accuracy of single land use type; and (4) compare the overall classification accuracies of MJDCNN and other state-of-the-art classification methods

Summary

Introduction

Precise and refined land use classification is essential for land resource management [2], [3], urban planning [4]–[6], precision agriculture [7], [8], environmental protection [9], and sustainable development [10]. The same land use types have complex spatial and structural patterns, whereas different land use types may have similar reflectance. The precise classification of land use types has become a difficult issue due to high intraclass heterogeneity and low interclass diversity [13]–[15]. HRRS images cover substantial land use information, which provides opportunities for land use classification and brings new challenges

Objectives

Methods

Results

Discussion

Conclusion