Crowd saliency prediction with optimal feature combinations

Abstract

Crowd saliency prediction refers to predicting where people look at in crowd scene. Humans have remarkable ability to rapidly direct their gaze to select visual information of interest when looking at a visual scene. Until now, research efforts are still focused on that which type of feature is representative for crowd saliency, and which type of learning model is the robust one for crowd saliency prediction. In this paper, we propose a Random Forest (RF) based crowd saliency prediction approach with optimal feature combination, i.e., the Feature Combination Selection for Crowd Saliency (FCSCS) framework. More specifically, we first define two representative crowd saliency features: FaceSizeDiff and FacePoseDiff. Next, we adopt the Random Forest (RF) algorithm to construct our saliency learning model. Then, we evaluate the performance of crowd saliency prediction classifiers with different feature combinations (fifteen combinations in our experiments). Those selected features include low-level features (i.e., color, intensity, orientation), four existing crowd features (i.e., face size, face density, frontal face, profile face) and two new defined features (i.e., FaceSizeDiff and FacePoseDiff). Finally, we obtain the optimal feature combination that is most suitable for crowd saliency prediction. We conduct extensive experiments and empirical evaluation to demonstrate the satisfactory performance of our approach.