Polynomial approximation based spectral dual graph convolution for scene parsing and segmentation

Abstract

Semantic segmentation requires both a large receptive field and accurate spatial information. Although existing methods based on the FCN have greatly improved the accuracy, it still does not show satisfactory results on complex scene parsing and tiny object identification. The convolution operation in FCN suffers from a restricted receptive field, while global modeling is fundamental to dense prediction tasks. In this work, we apply graph convolution into the semantic segmentation task and propose a spectral dual graph convolution module to solve the above problems. Moreover, the semantic segmentation task can be divided into two directions, one of which is to get a large receptive field and consider the global context information; the other is to focus on extracting spatial and contour clues, such as sharply changing curves and tiny objects. From the spectral-domain, it is supposed that low-frequency information is critical to the former task, while high-frequency information is vital to the latter task. Accordingly, high-frequency and low-frequency biased graph convolutions are proposed to process the above information separately. Experiments on Cityscapes, COCO Stuff, PASCAL Context, and PASCAL VOC demonstrate the effectiveness of our methods on semantic segmentation. The proposal achieves comparable performance with advantages in computational and memory overhead.