Multi-instance Multi-label Learning Based on Parallel Attention and Local Label Manifold Correlation

Abstract

Real-world applications always contain complex data objects with multiple semantics. The machine learning tasks generalized from these applications can be formulated as multi-instance multi-label learning (MIML) problems. Each object is represented by a bag of multiple instances and can be associated with multiple labels simultaneously. Extensive studies on the MIML have been conducted during the past few years. However, many of them do not consider local label correlations, which is critical in multi-label scenario. To address this problem, we propose the multi-instance multi-label learning based on parallel attention and local label manifold correlation (MIML-LLMC) algorithm. First, parallel multiple attention mechanisms convert each bag into a number of fusion vectors, one for each label. They are also utilized to find out which instances in the bag trigger respective labels to improve explanability. Second, local label manifold correlation vectors (LLMCs) are constructed based on label manifold generation and clustering. They provide a high level of representation of the label vectors. Third, each fusion vector is concatenated to the LLMC to predict each label independently. With the utilization of attention and LLMCs, MIML-LLMC is able to discover instance-label relations and exploit local label correlations simultaneously. Experiments reveal that our approach is highly competitive to the state-of-the-art MIML algorithms and yields reasonable results in understanding the relations between input patterns and output label semantics.