Multi-Task Learning and Multimodal Fusion for Road Segmentation

Abstract

The road segmentation task is to extract the road surface from the image at pixel level. In road segmentation for remote sensing images, deep learning-based methods have shown high-quality results in various scenarios. However, existing segmentation methods usually produce discontinuous roads, which is not beneficial to applying practical scenarios. We propose a multi-task learning method of road segmentation, direction estimation and road edge learning to make our model connect roads reasonably. Moreover, we use the initial road segmentation results and the direction estimation results to make cascade inference to improve the model’s performance. We adopt the Canny operator to extract the edge information of images as the auxiliary modality fusion. We demonstrate our method’s effectiveness on two large-scale road segmentation datasets DeepGlobe and SpaceNet.