RAPID: Robust multi-pAtch masker using channel-wise Pooled varIance with two-stage patch Detection

Abstract

Recently, adversarial patches have become frequently used in adversarial attacks in real-world settings, evolving into various shapes and numbers. However, existing defense methods often exhibit limitations in addressing specific attacks, datasets, or conditions. This underscores the demand for versatile and robust defenses capable of operating across diverse scenarios. In this paper, we propose the RAPID (Robust multi-pAtch masker using channel-wise Pooled varIance with two-stage patch Detection) framework, a stable solution to restore detection efficacy in the presence of multiple patches. The RAPID framework excels in defending against attacks regardless of patch number or shape, offering a versatile defense adaptable to diverse adversarial scenarios. RAPID employs a two-stage strategy to identify and mask coordinates associated with patch attacks. In the first stage, we propose the ‘channel-wise pooled variance’ to detect candidate patch regions. In the second step, upon detecting these regions, we identify dense areas as patches and mask them accordingly. This framework easily integrates into the preprocessing stage of any object detection model due to its independent structure, requiring no modifications to the model itself. Evaluation indicates that RAPID enhances robustness by up to 60% compared to other defenses. RAPID achieves mAP50 and mAP@50-95 values of 0.696 and 0.479, respectively.