Semantic Segmentation of Remote Sensing Images Using Transfer Learning and Deep Convolutional Neural Network With Dense Connection

Binge Cui,Yan Lu,Xin Chen

doi:10.1109/access.2020.3003914

Abstract

Semantic segmentation is an important approach in remote sensing image analysis. However, when segmenting multiobject from remote sensing images with insufficient labeled data and imbalanced data classes, the performances of the current semantic segmentation models were often unsatisfactory. In this paper, we try to solve this problem with transfer learning and a novel deep convolutional neural network with dense connection. We designed a UNet-based deep convolutional neural network, which is called TL-DenseUNet, for the semantic segmentation of remote sensing images. The proposed TL-DenseUNet contains two subnetworks. Among them, the encoder subnetwork uses a transferring DenseNet pretrained on three-band ImageNet images to extract multilevel semantic features, and the decoder subnetwork adopts dense connection to fuse the multiscale information in each layer, which can strengthen the expressive capability of the features. We carried out comprehensive experiments on remote sensing image datasets with 11 classes of ground objects. The experimental results demonstrate that both transfer learning and dense connection are effective for the multiobject semantic segmentation of remote sensing images with insufficient labeled data and imbalanced data classes. Compared with several other state-of-the-art models, the kappa coefficient of TL-DenseUNet is improved by more than 0.0752. TL-DenseUNet achieves better performance and more accurate segmentation results than the state-of-the-art models.

Highlights

With the rapid development of remote sensing technology, a massive number of remote sensing images are becoming available every day [1]
RELATED WORKS we briefly describe the modern structure of semantic segmentation and two deep learning techniques: transfer learning and dense connection
For the performance of segmenting classes with limited data, such as arbor forest, TLDenseUNet’s F1 score was improved by at least 4.74% and IoU was improved by at least 5.73%, indicating that the transferring DenseNet-121 improved multiscale feature extraction from remote sensing images. These findings demonstrate the superiority of TL-DenseUNet in the semantic segmentation of remote sensing images with insufficient and imbalanced labeled data

Summary

Introduction

With the rapid development of remote sensing technology, a massive number of remote sensing images are becoming available every day [1]. Semantic segmentation is one of the fundamental ways to analyze remote sensing images. This approach can and quickly obtain the land cover information of the area of interest, thereby providing data support for applications such as precision agriculture, desertification. Deep convolutional neural networks have achieved great success in many fields, and have proven their excellent performance in many applications [10] This trend has attracted many researchers to apply deep convolutional neural networks to the field of remote sensing image semantic segmentation [11]–[13]. Sherrah [15] used an FCN-based network without any downsampling to semantically segment high-resolution aerial images. Their method used dilated convolution in DeepLab [16] which can maintain the full resolution of the images in each layer of the network

Methods

Findings

Discussion

Conclusion