Video Domain Adaptation based on Optimal Transport in Grassmann Manifolds

Abstract

Domain adaptation is a fundamental research field, which focuses on transforming knowledge between different domains. With the massive growth of video data, the video domain adaptation problem becomes increasingly significant for practical tasks. Motivated by the excellent performance of Grassmann manifolds representation in video recognition tasks, we propose an optimal transport based video domain adaptation model on Grassmann manifolds. The proposed model reduces the discrepancy between different domains for the frame and video level features. First, the frame level discrepancy is reduced by extracting domain consistency features. At the video level, a fixed number of frame features are formed and represented as points on Grassmann manifolds. These points are fused with predicted labels to form fusion features. Finally, the video level discrepancy is reduced by minimizing the distribution discrepancy of the fusion features between two domains. Cross-domain video recognition experiments demonstrate the validity of the proposed model. The experimental results demonstrate the excellent performance of the proposed algorithm compared with the state-of-art video domain adaptation models.