Aggregation of attention and erasing for weakly supervised object localization

Abstract

In weakly supervised object localization (WSOL), objects are localized using image-level labels only. Most previous WSOL methods erase the most discriminative region using a fixed threshold to cover the entire region of the object, or they use random selection between attention and erasing. However, these methods have several limitations. First, the previous erasing method does not guarantee the erasing of adjacent pixels in the most discriminative region. Because adjacent pixels have similar information, this property may decrease the effect of the erasing. Next, random selection does not ensure the consistent use of attention and erasing, and it can be less effective to utilize both of them. To overcome these limitations, we propose a new sequential block, namely, a spatial attention branch followed by a refinement branch. Initially, the spatial attention branch generates a feature map with enhanced spatial information. Subsequently, the refinement branch helps the network capture the entire area of the object using attention and erasing modules. The attention module enhances the positional information by considering the spatial relationships. The erasing module erases all values within the specified region using various erasing sizes to increase the erasing effect. Finally, to make full use of the advantages of attention and erasing, these modules are combined through element-wise addition. Using such approaches, the proposed method achieves a state-of-the-art localization performance on the CUB-200-2011 and ILSVRC 2012 datasets. Furthermore, the proposed block requires only lightweight trainable parameters, demonstrating the efficiency of the proposed approach. • We propose a novel block that both attention and erasing coexist. • Randomly choose the size of erasing to be more effective erasing. • A simple and effective combination of attention and erasing