On Multi-modal Fusion Learning in constraint propagation

Abstract

Constraint propagation methods demonstrate splendid performance in constrained clustering tasks. Although some multi-modal constraint propagation methods have been proposed in recent years, a feasible and robust approach to multi-modal feature fusion in pairwise constraint propagation is still in demand. This paper presents a novel multi-modal fusion approach in order to cope with the constraint propagation on multi-modal datasets, called Multi-modal Fusion Learning (MFL). The proposed method can reach a multi-modal fusion based on the observed constraint information and the propagation process. It is capable of handling any number of modalities without any prior knowledge of each modality. We merge the fusion learning and constraint propagation into one unified problem and solve it by a bound-constrained quadratic optimization. Our proposed method has been tested in clustering tasks on two publicly available multi-modal datasets to show its superior performance.