Multi-view Instance Attention Fusion Network for classification

Abstract

Multi-view learning for classification has achieved a remarkable performance compared with the single-view based methods. Inspired by the instance based learning which directly regards the instance as the prior and well preserves the valuable information in different instances, a Multi-view Instance Attention Fusion Network (MvIAFN) is proposed to efficiently exploit the correlation across both instances and views. Specifically, a small number of instances from different views are first sampled as the set of templates. Given an additional instance and based on the similarities between it and the selected templates, it can be re-presented by following an attention strategy. Thanks for this strategy, the given instance is capable of preserving the additional information from the selected instances, achieving the purpose of extracting the instance-correlation. Additionally, for each sample, we not only perform the instance attention in each single view but also get the attention across multiple views, allowing us to further fuse them to obtain the fused attention for each view. Experimental results on datasets substantiate the effectiveness of our proposed method compared with state-of-the-arts.