A Novel Image Retrieval System with Real-Time Eye Tracking

Abstract

Relevance feedback is one of approach to improve the performance of content-based image retrieval system, and implicit feedback approaches, which gather users' feedback by biometric devices (e.g. eye tracker), are extensively investigated in recent years. This paper proposes a novel image retrieval system with eye tracking (IRSET). IRSET is composed of three modules: image retrieval module based on standard bag-of-words, eye tracking module to obtain a user's fixation data and to infer feedback information, and query expansion module that fuses the user's feedback and the input query to form a richer latent query. The implicit feedback of IRSET is implemented online and real-time, which makes IRSET remarkably distinguish from other systems with implicit feedback. We conduct experiments on the dataset of Oxford building for ten participants. The experimental results demonstrate that IRSET is an attractive interface to image retrieval and improves the retrieval performance.