An implicit relevance feedback method for CBIR with real-time eye tracking

Abstract

Relevance feedback is an efficient approach to improve the performance of content-based image retrieval systems, and implicit relevance feedback approaches, which gather users' feedback by biometric devices (e.g. eye tracker), have extensively investigated in recent years. This paper proposes a novel image retrieval system with implicit relevance feedback, named eye tracking based relevance feedback system (ETRFs). ETRFs is composed of three main modules: image retrieval subsystem based on bag-of-word architecture; user relevance assessment that implicitly acquires relevant images with the help of a modern eye tracker; and relevance feedback module that applies a weighted query expansion method to fuse users' relevance feedback. ETRFs is implemented online and real-time, which makes it remarkably distinguish from other offline systems. Ten subjects participate our experiments on the dataset of Oxford buildings and UKBench. The experimental results demonstrate that ETRFs achieves notable improvement for image retrieval performance.