CBFNet: Constraint balance factor for semantic segmentation

Abstract

The deep neural network model based on spatial pyramid pooling for multi-scale context information has been widely adopted in image semantic segmentation. However, this structure reduces the feature resolution during the inference process, which should be recovered by bilinear interpolation. We find that the performance of boundary location heavily depends on the interpolation operation. Assuming that the foreground object has a boundary with uneven curvature changes, it will be difficult to preserve details in the lower resolution feature map. In this case, if the model achieves near-optimal fitting, the prediction of the boundary point needs to satisfy strict constraints. But this condition is very difficult to meet for the model. For this purpose, we propose a constraint balance factor (CBF) relaxation method to improve the fitting ability of the model. The constraint balance factor is learned end-to-end by a sub-network. This sub-network can be easily embedded into a semantic segmentation network that requires interpolation without too many extra parameter increments. The experimental results illustrate that the proposed CBFNet performs well on PASCAL VOC 2012 and Cityscapes datasets.