Source Separation of Multimodal Data: A Second-Order Approach Based on a Constrained Joint Block Decomposition of Covariance Matrices

Abstract

Blind source separation aims at extracting unknown sources from mixtures of them. When multimodal data are considered (i.e. multi-set or multi-kind), some joint analysis are needed, for instance multi-set canonical correlation analysis or independent vector analysis. However, these methods only consider unidimensional sources in each set/modality. In this letter, an approach for dealing with multidimensional sources in each modality is derived. It assumes that the underlying dimensions in each modality for each source are known and it is based on a piecewise second order stationary model. Based on the likelihood, a contrast function is derived for the Gaussian case and is shown to be a constrained joint block decomposition of covariance matrices. Numerical simulations exhibit the merit of using a few number of modalities: it improves the quality of the separation and reduces the variance on the estimates. Finally, the proposed method outperforms the multi-set canonical correlation analysis and the independent component analysis applied to each individual modality followed by a clustering.