Relevance feedback for Content-based Image Retrieval using deep learning

Abstract

The “semantic gap” issue which exists the low-level image pixels captured by machines and high-level semantic concepts perceived by human has always been a key challenge for lots of applications, such as computer vision, pattern recognition, Content-based Image Retrieval (CBIR). The recent success of deep learning researches bring a hope for bridging the semantic gap. Many researchers have attempted to explore deep learning techniques with applying to CBIR tasks. However, it still has some restrictions for CBIR tasks, because the machine could not well comprehend the semantic concepts from low-level pixels to high-level semantic concepts. In this paper, we introduced the Relevance Feedback (RF) Model , which captures the users' feedback information. When the system returned the initial retrieval results to the user and the initial retrieval results meet the demand of the user, the retrieval process is called off. Generally, relevance feedback is usually added due to the poor results of retrieval. Our CBIR system asks the user to feedback the information related to query image. So the system obtains the information of semantically similar and dissimilar images for the positive and negative feedback samples. Then the initial retrieval results are resorted based the updated user's feedback information. When the resorted results meet the demand of the user, RF model is stop and return the final optimal results to users. the RF is performed iteratively until the user is satisfied with the refined results. Our proposed feedback model is optimized for each user's results. From our empirical studies, deep learning combining with RF model significantly outperforms only applying deep learning for CBIR tasks.