An intelligent recommendation system using gaze and emotion detection

Abstract

Recently, recommendation system has become popular in many e-commerce websites. It helps users by suggesting products which they could buy. Existing work till now uses past feedback of user, similarity of other users’ buying pattern, or a hybrid approach in which both type of information is used. But the pitfall of these approaches is that there is a need to collect and process huge amount of data for good recommendation. This paper is aimed at developing an efficient recommendation system by incorporating user’s emotion and interest to provide good recommendations. The proposed system does not require any of aforementioned data and works without the continuous and interminable attention of the user. In this framework, we capture user’s eye-gaze and facial expression while exploring websites through inexpensive, visible light “webcam”. The eye-gaze detection method uses pupil-center extraction of both eyes and calculates the reference point through a joint probability. The facial expression uses landmark points of face and analyzes the emotion of the user. Both methods work in approximate real time and the proposed framework thus provides intelligent recommendations on-the-fly without requirement of feedback and buying patterns of users.