Weakly supervised semantic segmentation via self-supervised destruction learning

Abstract

Currently, weakly supervised semantic segmentation approaches adopt the Class Activation Map (CAM) to generate the initial attention maps from the standard classification backbone network, with only image-level class labels as training supervision. In this paper, we propose a novel “destruction learning” method via self-supervised manner, producing the CAM attention maps better covering the whole object rather than only the most discriminative regions as previous approaches. Region destruction mechanism is proposed to deliberately “destruct” the global structure in both Mid-Level and Low-Level feature learning following jigsaw puzzle operation, for better local feature extraction of the classification network. Specifically, a Mid-Level Distribution-and-Collection Module is proposed to firstly independently process local patches in parallel and then aggregate them, achieving enhanced local part sensitivity. For low-level “destruction learning”, a Low-Level Destruction Module is proposed to partition the original image into patches and shuffle them randomly to obtain a destructed image, which enforces the network to find weak information from the scattered discriminative small parts. The two destruction modules can be conveniently embedded into arbitrary backbone networks and trained in an end-to-end manner, without any extra annotations. Our proposed method achieves outstanding performance on the PASCAL VOC 2012 dataset (e.g., with 69.1% mIoU on the testing set), which surpasses existing state-of-the-art weakly supervised semantic segmentation methods.