Learning Deep Asymmetric Tolerant Part Representation

Abstract

Categorization objects at a sub-ordinate level inevitably poses a significant challenge, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e.</i> , inter-class difference is very subtle and only exists in a few key parts. Therefore, how to localize these key parts for discriminative visual categorization without requiring expensive pixel-level annotations becomes a core question. To that end, this paper introduces a novel asymmetry tolerant part segmentation network (ATP-Net). ATP-Net simultaneously learns to segment parts and identify objects in an end-to-end manner using only image-level category labels. Given the intrinsic asymmetry property of part alignment, a desirable learning of part segmentation should be capable of incorporating such property. Despite the efforts towards regularizing weakly supervised part segmentation, none of them consider this vital and intrinsic property, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e.</i> , the spatial asymmetry of part alignment. Our work, for the first time, proposes to explicitly characterize the spatial asymmetry of part alignment for visual tasks. We propose a novel asymmetry loss function to guide the part segmentation by encoding the spatial asymmetry of part alignment, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e.</i> , restricting the upper bound of how asymmetric those self-similar parts are to each other in the network learning. Via a comprehensive ablation study, we verify the effectiveness of the proposed ATP-Net in driving the network learning towards semantically meaningful part segmentation and discriminative visual categorization. Consistently superior/competitive performance are reported on 12 datasets covering crop cultivar classification, plant disease classification, bird/butterfly species classification, large-scale natural image classification, attribute recognition and landmark localization.