CS-GAN: Centrosymmetric Generative Adversarial Network for Cross-Media Retrieval

Abstract

Cross-Media retrieval has become necessary to satisfy the urgent needs of multimedia economic and cultural value. The heterogeneous gap makes it difficult to directly retrieve media of different modalities. Existing common space-based Cross-Media retrieval methods usually focus on increasing the inter-class distance while paying little attention to decreasing the intra- class distance, they neglect to correlate the mapping process from original space to common space, which limits the narrowing of intra-class distance. To address this problem, we propose Centrosymmetric Generative Adversarial Network (CS-GAN) for Cross-Media retrieval, which consists of two generative models and our proposed Heterogeneous Input Discriminative Model (HIDM), forming a centrosymmetric structure. Our proposed HIDM conjoins the mapping processes of each media type by dimensionality reduction and knowledge transfer, which ensures the unification of internal processing and well retains the underlying manifold structure information. Besides, it discriminates the distribution of common representations for both image and text at the same time, which can interrelate the generative mapping, preserve the distribution of the original features and narrow the intra-class distances of common embeddings. The experimental results on PKU XMediaNet and Pascal Sentences show the effectiveness of our proposed CS-GAN.