Semi-Supervised Multiple Feature Fusion for Video Preference Estimation

Abstract

This paper presents a new method to estimate user preferences for videos based on multiple feature fusion via semi-supervised Multiview Local Fisher Discriminant Analysis (sMvLFDA). The proposed method first extracts multiple visual features from videos and functional near-infrared spectroscopy (fNIRS) features from fNIRS signals recorded during watching videos. Next, we apply Locality Preserving Canonical Correlation Analysis (LPCCA) to each visual feature and fNIRS features and project each visual feature to the new feature spaces (fNIRS-based visual feature spaces). Consequently, since the correlation between each visual feature and fNIRS features which reflect user preferences is maximized, we can transform visual features into features which also reflect user preferences. In addition, we newly introduce sMvLFDA and fuse multiple fNIRS-based visual features via sMvLFDA. sMvLFDA fuses features while using labeled samples and unlabeled samples simultaneously to reduce overfitting to the labeled samples. Furthermore, sMvLFDA adequately uses complementary properties in multiple features. Therefore, it can be expected that the fused features are more effective for estimation of user preferences than each fNIRS-based visual feature. The main contribution of this paper is the new derivation of sMvLFDA. Consequently, by using the fused features, it becomes feasible to estimate user preferences for videos successfully.