A Bottleneck Network with Light Attention for Multimodal Clustering

Abstract

Multimodal Clustering (MMC) aims to boost clustering performance by utilizing complementary information from multimodal data. However, the heterogeneous gap between various modalities and the interference of superfluous information in multimodal data makes it challenging to comprehend the complex dependencies between them. To address these issues, we present a Bottleneck Network with Light Attention for Multimodal Clustering (LABMC) to understand the dependencies between different modalities and explore complementary information while maximally eliminating superfluous information. Specifically, Light Attention (LA) is proposed as a concise multimodal interaction mechanism that introduces a convolutional high-level semantic layer to understand the common high-level semantics among different modalities and explores related complementary information to obtain a multimodal fusion representation. It can efficiently comprehend the latent dependencies between diverse modalities to bridge the heterogeneous gap. Subsequently, we construct a Superfluity Elimination (SE) function to erase superfluous information from the modalities. We divide the mutual information between the fusion representation and multimodal data into superfluous information and complementary information. Under the premise of preserving complementary information with an approximation, we utilize a variational optimization method to minimize the mutual information to safely remove superfluity. The combination of LA and SE naturally forms a bottleneck, which enables the extraction of complementary information while reducing superfluous information. Finally, we measure the quality of the modalities to dynamically assign the optimal weights to SE function, which significantly enhances the rationality of SE. Extensive experiment results on five real-world datasets show that LABMC outperforms the state-of-the-art MMC methods.