Deep Learning Approach for Multi-class Semantic Segmentation of UAV Images

Abstract

Image understanding plays a very crucial role in remote sensing applications. For this, image semantic segmentation is one of the approaches where each pixel of an image is assigned to particular classes based on various features. Aerial semantic segmentation suffers from the class imbalance problem. Proper differentiation of least represented categories is challenging and a goal for the state-of-art approach. In this work, we present a novel deep learning method to perform this task. We proposed a lightweight encoder-decoder network residual depth separable UNet (RDS-UNet) and conditional random field for effective segmentation on very high-resolution aerial images. We proposed patch-with-multi-class sampling to handle the class imbalance problem without increasing the computational overhead during the training process. We created a semi-precise annotated UAV dataset named NESAC UAV Seg for the aerial semantic segmentation task. We demonstrated the efficacy of our model using the publicly available benchmark Drone Deploy dataset and our NESAC UAV Seg dataset. Our model required approximately half the number of trainable parameters and floating point operations compared to other methods. A detailed ablation study is presented to showcase the effectiveness of various modules utilized in our network.