@ICT: attention-based virtual content insertion

Abstract

In this paper, we propose an attention-based virtual content insertion solution, called @ICT. Virtual content insertion (VCI) is an emerging application of video analysis and has been used in video augmentation and advertisement insertion. An ideal VCI solution should make the inserted virtual content being noticed by audiences and at the same time should not interfere with audiences’ viewing experience on the original content. To balance these two conflicting issues, meaning high attention and low intrusiveness, we choose higher attentive shots as insertion time while determine insertion place and content interdependently by considering lower attention together with visual consistency. We also propose a measurement of intrusiveness from the viewpoint of visual attention. Furthermore, @ICT includes an in-scene insertion module, which embeds the virtual content into the videos with higher vividness and lower intrusiveness. @ICT is able to obtain an optimal balance between the noticing of the virtual content by audiences and disruption of viewing experience to the original content. It needs little prior knowledge and is applied to general videos. Extensive quantitative and qualitative evaluations on the VCI result have verified the effectiveness of the solution.