Weakly-supervised semantic segmentation with saliency and incremental supervision updating

Abstract

Weakly-supervised semantic segmentation aims at tackling the dense labeling task using weak supervision so as to reduce human annotation efforts. For weakly-supervised semantic segmentation using only image-level annotation, we propose a novel model of Learning with Saliency and Incremental Supervision Updating (LSISU), in which both the guidances of saliency prior and class information are jointly used and the segmentation supervision is dynamically updated. In the proposed LSISU, we present an image saliency objective complementary to classification loss, by which the trained weakly-supervised deep network can effectively deal with object co-occurrence problem. Meanwhile, we make full use of the class-wise pooling strategy to generate initial mask estimation of high quality. Given an initial annotation, a segmentation network is learned along with incremental supervision updating, which plays a role of region expansion and corrects the falsely estimated supervision for training images. The incremental supervision updating is performed on the fly and involves repeated usage of a fully connected conditional random field algorithm. LSISU achieves superior segmentation performance in terms of mIoU metric on benchmark datasets, which are 62.5% on the PASCAL VOC 2012 test set and 30.1% on the COCO val set.