Exploring feature space with semantic attributes

Abstract

Indexing is a critical step for searching digital images in a large database. To date, how to design discriminative and compact indexing strategy still remains a challenging issue, partly due to the well-known semantic gap between user queries and rich semantics in the large scale dataset. In this paper, we propose to construct a novel joint semantic-visual space by leveraging visual descriptors and semantic attributes, which aims to narrow down the semantic gap by taking both attribute and indexing into one framework. Such a joint space embraces the flexibility of conducting Coherent Semantic-visual Indexing, which employs binary codes to boost the retrieval speed with satisfying accuracy. To solve the proposed model effectively, three contributions are made in this submission. First, we propose an interactive optimization method to find the joint space of semantic and visual descriptors. Second, we prove the convergence property of our optimization algorithm, which guarantees our system will find a good solution in certain rounds. At last, we integrate the semantic-visual joint space system with spectral hashing, which can find an efficient solution to search up to million scale datasets. Experiments on two standard retrieval datasets i.e., Holidays1M and Oxford5K, show that the proposed method presents promising performance compared with the state-of-the-arts.