Zero-Shot Embedding via Regularization-Based Recollection and Residual Familiarity Processes

Abstract

The goal of zero-shot learning (ZSL) is to transfer knowledge learned from seen classes during training to unseen classes for testing, with the help of auxiliary information, such as attributes and descriptions. Most of the existing methods view ZSL as a label-embedding problem, in which class and image representations are embedded to a common space. However, many methods either show a bias toward seen classes caused by the projection domain-shift problem, or sacrifice the performance of seen classes to generalize to unseen ones. In this article, we present an embedding approach for ZSL, which is motivated by human recognition memory, namely, recollection and familiarity (R&F). We propose a decoder to regularize the nonlinear mapping between the semantic space and the visual space, which represents the reasonable recollection process, and use a residual block to refine the recognition ability for seen classes, which indicates the familiarity process. R&F can generalize well to unseen classes, while retaining the discriminative ability for the seen classes. Extensive experiments are conducted on Animals with Attribute (AwA1), Animals with Attributes 2 (AwA2), Attribute Pascal&Yahoo (aPY), SUN Attribute (SUN), Caltech-UCSD-Birds 200-2011 (CUB), and ImageNet databases. As qualitative and quantitative results show, the proposed approach outperforms state-of-the-art embedding-based methods by a large margin and significantly alleviates the projection domain-shift problem.