Abstract
A few-shot personalized saliency prediction based on adaptive image selection considering object and visual attention is presented in this paper. Since general methods predicting personalized saliency maps (PSMs) need a large number of training images, the establishment of a theory using a small number of training images is needed. To tackle this problem, although finding persons who have visual attention similar to that of a target person is effective, all persons have to commonly gaze at many images. Thus, it becomes difficult and unrealistic when considering their burden. On the other hand, this paper introduces a novel adaptive image selection (AIS) scheme that focuses on the relationship between human visual attention and objects in images. AIS focuses on both a diversity of objects in images and a variance of PSMs for the objects. Specifically, AIS selects images so that selected images have various kinds of objects to maintain their diversity. Moreover, AIS guarantees the high variance of PSMs for persons since it represents the regions that many persons commonly gaze at or do not gaze at. The proposed method enables selecting similar users from a small number of images by selecting images that have high diversities and variances. This is the technical contribution of this paper. Experimental results show the effectiveness of our personalized saliency prediction including the new image selection scheme.
Highlights
Many researchers have attempted to predict a saliency map that indicates image components that are more attractable than their neighbors [1,2,3,4]
This paper presents a few-shot personalized saliency maps (PSMs) prediction (FPSP) method using a small amount of training data based on adaptive image selection (AIS) considering object and visual attention
Xn ∈ Rd1 ×d2 ×d3 (n = 1, 2, . . . , N; N being the number of training images, d1 × d2 being the number of pixels, and d3 being the number of color channels) and Universal Saliency Map (USM) SUSM ( Xn ) ∈ Rd1 ×d2 are used for training the multi-task Convolutional Neural Network (CNN)
Summary
Many researchers have attempted to predict a saliency map that indicates image components that are more attractable than their neighbors [1,2,3,4]. For predicting a PSM by this method for a new person not included in the PSM dataset, a large amount of gaze data, which involve a heavy burden, must be obtained for retraining the multi-task CNN. This paper presents a few-shot PSM prediction (FPSP) method using a small amount of training data based on adaptive image selection (AIS) considering object and visual attention. The person similarity is calculated by using selected images included in the PSM dataset These images are chosen by AIS focusing on the diversity of images and the variance of PSMs. For guaranteeing the high diversity of the selected images, AIS focuses on the kinds of objects included in the training images in the PSM dataset by using a deep learning-based object detection method. FPSP of a target image for the new target person is realized on the basis of the person similarity and PSMs predicted by the multi-task CNN trained for the persons in the PSM dataset. We newly introduce the AIS into the above PSM prediction approach
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
