Exploiting geometrical structures using Autoencoders and click data for image re-ranking

Abstract

Image re-ranking is effective in improving text-based image retrieval performance. However, to achieve such a target is often constrained by two important issues: one is that any one visual feature of images is usually too superficial to represent the whole information of images; the other is that the corresponding textual information often mismatches semantics of the images. In this paper, we utilize Autoencoders to extract deep features from artificial features of images and exploit click data to bridge the semantic gap between query words and image semantics. A graph-based algorithm, Multi-modal Image Re-ranking with Autoencoders and Click Semantics (MIR-AC), is proposed to adaptively integrate geometrical structures based on feature spaces from Autoencoders and click data by constructing two manifolds with updating weights. In particular, MIR-AC achieves image re-ranking by conducting an iterative optimization process in which image ranking scores and weights of manifolds are updated alternatively. Experiments are conducted on a realworld dataset and results demonstrate that MIR-AC outperforms given state-of-the-arts in image re-ranking.