Multi-source weakly supervised salient object detection via boosting weak-annotation source and constraining object structure

Abstract

Training under multi-source weak annotations can reduce the performance gap between weakly supervised and fully supervised salient object detection (SOD) methods. However, it is challenging to train multiple weak-annotation sources since it may lose important structural information of salient regions and generate unnecessary noise when utilizing weak-annotation sources and fusing multi-source weak annotations. In this study, we propose a novel weakly supervised saliency detection framework which contains a multi-source fused refinement network (MFRN) and a salient region prediction network (SRPN), to make full use of the attributive advantages of diverse weak-annotation sources to generate a refined saliency map. In MFRN, we design a weak-supervision boosting module to integrate pixel-wise boosted annotations into weak annotations, so as to eliminate the risk of missing salient information when using only weak annotation for learning saliency and provided strengthened weak supervision for training SRPN. As unnecessary noise is produced in the training process, we propose a structure-constraint loss for SRPN to eliminate noise from the scope of salient objects. The structural consistency between the detected saliency regions and the real salient objects that comply with human perception can be preserved by the awareness of the structural constraint. Moreover, an edge enhancement branch is designed to generate feature maps with rich structures for SRPN to sharpen the edge of the predicted salient region. Experimental results on several large-scale datasets demonstrate that the proposed method can outperform other weakly supervised SOD methods; furthermore, the proposed method even outperformed some fully supervised methods.