The impact of hesitation, a social signal, on a user’s quality of experience in multimedia content retrieval

Abstract

The social signal (SS) of hesitation is commonly manifested through a multiplicity of nonverbal behavioural cues when a user is faced with a variety of decision choices. The aim of this study is to show that the utilization of the SS of hesitation in a conversational recommender system (RS) can improve the user quality of experience (QoE) when interacting with a video-on-demand system. An appropriate experimental design was modelled to detect the impact of the SS. The experimental scenario was a manual video-on-demand system with a conversational RS where the user selected one video clip among several presented on the screen. The system adjusted the list of the video items to be recommended according to the extracted SS class {hesitation, no hesitation}. To detect if the user was hesitating, we used hand movements, eye behaviour and time between two selections. Two user groups were tested to allow realistic estimation of the impact of the SS. In the user test group, the SS of hesitation was considered, while in the control group it was not. The evaluation of impact of the SS on QoE was based on pre- and post-interaction questionnaires. Our results showed a significant difference in user satisfaction with the system between those two groups, indicating that the use of SS of hesitation in conversational RS improves the QoE when the user interacts with a video-on-demand system.