Region interaction and attribute embedding for zero-shot learning

Abstract

Zero-shot learning (ZSL) aims to recognize unseen class images only by using seen class images for training. In ZSL tasks, unseen classes share local attribute semantics with seen classes. However, most ZSL methods directly learn an embedding from global feature space to semantic space, which may fail to discover local attribute semantics and cause the strong bias problem. In this paper, a novel method called region interaction and attribute embedding (RIAE) is proposed. RIAE consists of two modules: the region graph network (RGN) and the attribute feature embedding (AFE). RGN constructs a region graph where each region (image patch) is regarded as a graph node for region interaction. That is, through the graph-convolution operations of RGN, the node features can aggregate the information from their neighboring node features and update themselves. In order to learn the compatibility between the updated node features and local attribute semantics, AFE is designed to assign high attention weights to the node features which have large compatibilities with attribute semantic vectors. Relying on attribute semantic vectors rather than human annotations, RIAE can adaptively extract and update local attribute features, and further learn an embedding from attribute feature space to semantic space. Extensive experiments on the benchmark datasets show that RIAE gets state-of-the-art or competitive performances.