Robust and Refined Salient Object Detection Based on Diffusion Model

Abstract

Salient object detection (SOD) networks are vulnerable to adversarial attacks. As adversarial training is computationally expensive for SOD, existing defense methods instead adopt a noise-against-noise strategy that disrupts adversarial perturbation and restores the image either in input or feature space. However, their limited learning capacity and the need for network modifications limit their applicability. In recent years, the popular diffusion model coincides with the existing defense idea and exhibits excellent purification performance, but there still remains an accuracy gap between the saliency results generated from the purified images and the benign images. In this paper, we propose a Robust and Refined (RoRe) SOD defense framework based on the diffusion model to simultaneously achieve adversarial robustness as well as improved accuracy for benign and purified images. Our proposed RoRe defense consists of three modules: purification, adversarial detection, and refinement. The purification module leverages the powerful generation capability of the diffusion model to purify perturbed input images to achieve robustness. The adversarial detection module utilizes the guidance classifier in the diffusion model for multi-step voting classification. By combining this classifier with a similarity condition, precise adversarial detection can be achieved, providing the possibility of regaining the original accuracy for benign images. The refinement module uses a simple and effective UNet to enhance the accuracy of purified images. The experiments demonstrate that RoRe achieves superior robustness over state-of-the-art methods while maintaining high accuracy for benign images. Moreover, RoRe shows good results against backward pass differentiable approximation (BPDA) attacks.