Multi-instance Multi-label Research Articles

Single-Stage Broad Multi-Instance Multi-Label Learning (BMIML) With Diverse Inter-Correlations and Its Application to Medical Image Classification

Single-Stage Broad Multi-Instance Multi-Label Learning (BMIML) With Diverse Inter-Correlations and Its Application to Medical Image Classification

A multi-instance multi-label learning algorithm based on radial basis functions and multi-objective particle swarm optimization

Radial basis function (RBF) neural networks for Multi-Instance Multi-Label (MIML) directly can exploit the connections between instances and labels so that they can preserve useful prior information, but they only adopt Gaussian radial basis function as their RBF whose parameters are difficult to determine. In this paper, parameters can be obtained by multi-objective optimization methods with multi performance measures treated as objectives, specifically, parameter estimation of different RBFs by an improved multi-objective particle swarm optimization (MOPSO) is proposed where Recall rate and Precision rate are chosen to obtain the most desirable Pareto optimal solution set. Furthermore, share-learning factor is proposed to modify the particle velocity in standard MOPSO to improve the global search ability and group cooperative ability. It is experimentally demonstrated that the proposed method can estimate the reliable parameters of different RBFs, and it is also very competitive with the state of art MIML methods.

Nearest neighbor-based approaches for multi-instance multi-label classification

Nearest neighbor-based methods are classic techniques that, due to their efficiency, still are widely used today. However, they have not been broadly applied to solve the multi-instance multi-label (MIML) problem, a supervised learning paradigm that combines multi-instance (MI) and multi-label (ML) learning. This work presents new neighbor-based approaches for solving MIML problems. On the one hand, MIML data are transformed into ML data and ML nearest neighbor algorithms are used. On the other hand, algorithms that directly address MIML data and use a bag-based distance are proposed.A comprehensive study and an overall comparison have been conducted to study the performance of these methods using different configurations. Experiments included 16 datasets and 8 performance metrics. The results and statistical tests showed that the problem transformation applied and the distance function used impacted the performance and that the approaches that do not transform the problem obtained the best predictive results. Furthermore, most of the proposed algorithms outperformed the MIMLkNN algorithm, the state-of-art algorithm for MIML learning based on nearest-neighbor. Therefore, the relevance and capabilities of neighbor-based approaches to obtain competitive results in MIML learning are shown. Finally, all the algorithms developed in this paper have been included in the MIML library to facilitate the comparison with other future proposals.

Meta Multi-Instance Multi-Label learning by heterogeneous network fusion

Multi-Instance Multi-Label Learning (MIML) models complex objects (bags), each of which is composed with a set of instances and associated with a set of labels. Current MIML solutions still focus on a single-type of bags and assume an independent and identically distribution (IID) of training data. But these bags are linked with objects of other types, which also encode the semantics of bags. In addition, they generally need abundant labeled data for training. To effectively mine MIML objects linked with objects of other types, we propose a heterogeneous network embedding and meta learning based approach (MetaMIML). MetaMIML introduces the context learner with network embedding to learn context representations of bags for structure information extraction, the task learner to extract the meta knowledge for fast adapting to new tasks with scarce training objects, and finally fuses the structural and attribute information to predict the labels of bags/instances. In this way, MetaMIML can not only naturally deal with MIML objects at data level improving, but also exploit the power of meta-learning at the model enhancing. Experiments on benchmark datasets demonstrate that MetaMIML achieves a significantly better performance than state-of-the-art algorithms.

MIML-GAN: A GAN-Based Algorithm for Multi-Instance Multi-Label Learning on Overlapping Signal Waveform Recognition

Existing studies for automatic waveform recognition of overlapping signals have mostly been conducted in a supervised manner. Although demonstrating superior performance in recent years, supervised methods rely heavily on sufficient labeled samples, rely but the acquisition of annotated data is expensive, time-consuming, and sometimes rely infeasible. This shortage drives the need for semi-supervised learning methods, where the unlabeled samples can be fully exploited in the training stage. In addition, multi-instance multi-label (MIML) learning is essentially another weakly supervised learning protocol, and precisely fits the form of the time-frequency images TFIs obtained from the transformation of overlapping signals. In this paper, delving into the MIML learning problem, we leverage the advantage of adversarial training to formulate an effective algorithm MIML-GAN, which is tailored to the MIML problem of overlapping signal waveform recognition. After feeding the TFIs into the network, MIML-GAN approximates the distribution of the training data using the adversarial learning principle. Subsequently, the baglevel prediction can be derived from the instance-level prediction upon the MIML discriminator through adaptive threshold calibration. Specifically, we elaborately studied the global optimality of the MIML-GAN objective function, and extensive simulations are carried out with overlapping signal dataset, validating the ascendancy of the proposed method. Comparative experiments demonstrate that the proposed algorithm possesses promising feature representation capability, and outperforms the existing semi-supervised and supervised signal waveform recognition approaches.

Residual Attention-Aided U-Net GAN and Multi-Instance Multilabel Classifier for Automatic Waveform Recognition of Overlapping LPI Radar Signals

Automatic waveform recognition of overlapping low probability of intercept (LPI) radar signals is an important and challenging task in electronic reconnaissance of the increasingly complicated spectrum environment. In this article, an overlapping LPI waveform recognition processing framework incorporating residual attention-aided U-net generative adversarial network (GAN) and multiinstance multilabel (MIML) classifier is proposed. This framework includes five cascade modules and can achieve satisfactory recognition performance by training with single type of signals only. First, the training signals are transformed into time–frequency images. Then, a residual attention U-net GAN (RAUGAN) with residual learning is employed to reconstruct signal images from noise-contaminated images and with the supervision of the high-quality ones. After that, an instance generation module with asymmetric convolutions generates instance representations, which are then fed into the subsequent residual attention MIML classifier (RAMIML). Finally, an adaptive threshold calibration module is implemented to obtain appropriate thresholds for the final decision. Besides, two loss functions are elaborately designed for the RAUGAN and RAMIML, respectively. Extensive experimental results validate the recognition performance of the proposed framework with a recognition accuracy of 80$\%$ at signal-to-noise ratio $>-18$ dB and show higher robustness on the power ratios and generative performance compared with other state-of-the-art methods.

W44. OXTR DNA METHYLATION DIFFERENTIATES MEN ON THE OBESITY SPECTRUM WITH AND WITHOUT AN EATING DISORDER

The facts show that multi-instance multi-label (MIML) learning plays a pivotal role in Artificial Intelligence studies. Evidently, the MIML learning introduces a framework in which data is described by a bag of instances associated with a set of labels. In this framework, the modeling of the connection is the challenging problem for MIML. The RBF neural network can explain the complex relations between the instances and labels in the MIMLRBF. The parameters estimation of the RBF network is a difficult task. In this paper, the computational convergence and the modeling accuracy of the RBF network has been improved. The present study aimed to investigate the impact of a novel hybrid algorithm consisting of Gases Brownian Motion optimization (GBMO) algorithm and the gradient based fast converging parameter estimation method on multi-instance multi-label learning. In the current study, a hybrid algorithm was developed to estimate the RBF neural network parameters (the weights, widths and centers of the hidden units) simultaneously. The algorithm uses the robustness of the GBMO to search the parameter space and the efficiency of the gradient. For this purpose, two real-world MIML tasks and a Corel dataset were utilized within a two-step experimental design. In the first step, the GBMO algorithm was used to determine the widths and centers of the network nodes. In the second step, for each molecule with fixed inputs and number of hidden nodes, the parameters were optimized by a structured nonlinear parameter optimization method (SNPOM). The findings demonstrated the superior performance of the hybrid algorithmic method. Additionally, the results for training and testing the dataset revealed that the hybrid method enhances RBF network learning more efficiently in comparison with other conventional RBF approaches. The results obtain better modeling accuracy than some other algorithms.

Towards Discovering What Patterns Trigger What Labels

In many real applications, especially those involving data objects with complicated semantics, it is generally desirable to discover the relation between patterns in the input space and labels corresponding to different semantics in the output space. This task becomes feasible with MIML (Multi-Instance Multi-Label learning), a recently developed learning framework, where each data object is represented by multiple instances and is allowed to be associated with multiple labels simultaneously. In this paper, we propose KISAR, an MIML algorithm that is able to discover what instances trigger what labels. By considering the fact that highly relevant labels usually share some patterns, we develop a convex optimization formulation and provide an alternating optimization solution. Experiments show that KISAR is able to discover reasonable relations between input patterns and output labels, and achieves performances that are highly competitive with many state-of-the-art MIML algorithms.

Cost‐effective multi‐instance multilabel active learning

Multi-instance multi-label (MIML) Active Learning (M2AL) aims to improve the learner while reducing the cost as much as possible by querying informative labels of complex bags composed of diverse instances. Existing M2AL solutions suffer high query costs for scrutinizing all relevant labels of MIML samples, querying excessive bag–label or instance–label pairs. To address these issues, a Cost-effective M2AL solution (CM2AL) is presented. CM2AL first selects the most informative bag–label pairs by leveraging uncertainty, label correlations, label space sparsity, and informativeness from queried instances of the bag, and thus avoids scrutinizing all labels. Next, it queries the most probably positive instance–label pairs of the selected bag–label pair. Particularly, if the feedback is positive, the bag is positively annotated with the label. For negative feedback, it further leverages the label of the neighborhood bags and the label of the nearby instances of queried instances of this bag, if the suggested labels from bag- and instance-levels disagree, CM2AL temporally gives up querying this bag–label pair and moves to another most informativeness one; otherwise, it takes the agreed label to annotate the bag, which further saves the cost by avoiding the excessive query. Extensive experiments on MIML data sets from diverse domains show that CM2AL can more reduce the cost while managing a better performance than state-of-the-art methods, the collaboration between bags and instances contributes to the saved cost.

Smart Handheld Based Human Activity Recognition Using Multiple Instance Multiple Label Learning

Human activity recognition (HAR) and monitoring is beneficial for many medical applications, such as eldercare and post-trauma rehabilitation after surgery. HAR models based on smartphone’s accelerometer data could provide a convenient and ubiquitous solution to this problem. However, such models are mostly concerned with identifying basic activities such as ‘stand’/‘walk’ and thus the high-level context such as ‘walk in a queue’ for which a set of specific activities is performed remain unnoticed. Consequently, in this paper, we design a HAR framework that can identify a group of activities (rather than a single basic activity) being performed in a time window, thus, enables us to extract more meaningful information about the subject’s overall context. An algorithm is designed to formulate HAR as a multi-instance multi-label (MIML) learning problem. The procedure of generating feature bags of consecutive activity traces having multiple labels is formulated. In this work, the temporal relationship among activities is exploited to obtain a more comprehensive HAR model. Interestingly, the framework is found to completely/partially identify activity sequences that may not even be present in the training dataset. The framework is implemented and found to be working adequately when tested with real dataset collected from 8 users for 12 different activity combinations. MIML-kNN is found to provide maximum average precision (around 90%) even for an unseen test data-set.

Saliency based multiple object cosegmentation by ensemble MIML learning

As an interesting and emerging topic, multiple foreground cosegmentation (MFC) aims at extracting a finite number of common objects from an image collection, which is useful to variety of visual media applications. Although a number of approaches have been proposed to address this problem, many of them are designed with the misleading consistent information, suboptimal image representation, or inefficient segmentation assist and thus still suffer from certain limitations, which reduces their capability in the real-world scenarios. To alleviate these limitations, we propose a novel unsupervised MFC framework, which is composed of three components: unsupervised label generation, saliency based pseudo-annotation and cosegmentation by MIML learning. Specifically, we combine the high-level and low-level feature to represent the proposal objects, and adopt a novel SPAP clustering scheme to obtain more accurate consistent information of common objects. Then the saliency based pseudo-annotation help us reformulate the MFC problem as a Multi-Instance Multi-Label (MIML) learning problem by label propagation. Finally, by introducing a novel ensemble MIML learning scheme, the consistent information of common objects can more efficiently assist the segmentation of the images and get the more accurate segmentation results. We evaluate our framework on widely used public databases including the ICoseg dataset, MSRC dataset and FlickrMFC dataset for single and multiple common object cosegmentation respectively. Comparison results show that the proposed methods reach advanced and efficient performance.

Multi-task MIML learning for pre-course student performance prediction

In higher education, the initial studying period of each course plays a crucial role for students, and seriously influences the subsequent learning activities. However, given the large size of a course’s students at universities, it has become impossible for teachers to keep track of the performance of individual students. In this circumstance, an academic early warning system is desirable, which automatically detects students with difficulties in learning (i.e., at-risk students) prior to a course starting. However, previous studies are not well suited to this purpose for two reasons: 1) they have mainly concentrated on e-learning platforms, e.g., massive open online courses (MOOCs), and relied on the data about students’ online activities, which is hardly accessed in traditional teaching scenarios; and 2) they have only made performance prediction when a course is in progress or even close to the end. In this paper, for traditional classroom-teaching scenarios, we investigate the task of pre-course student performance prediction, which refers to detecting at-risk students for each course before its commencement. To better represent a student sample and utilize the correlations among courses, we cast the problem as a multi-instance multi-label (MIML) problem. Besides, given the problem of data scarcity, we propose a novel multi-task learning method, i.e., MIML-Circle, to predict the performance of students from different specialties in a unified framework. Extensive experiments are conducted on five real-world datasets, and the results demonstrate the superiority of our approach over the state-of-the-art methods.

An Explainable Multi-Instance Multi-Label Classification Model for Full Slice Brain CT Images

Abstract Brain CT is the first choice for diagnosing intracranial diseases. However, the doctors who can accurate diagnosis is insufficient with the increasing number of patients. Nowadays, many computer-aided diagnosis algorithms were developed to help doctors diagnose and reduce time. However, most of the research classifies each slice isolated, regard this as an image-level classification problem. It is not comprehensive enough because many conditions can only be diagnosed by considering adjacent slices and the relationships between diseases. In order to better fit the characteristics of this task, we formal it as a Multi-instance Multi-label (MIML) learning problem at sequence-level. In this paper, we analyze the difficulties in the brain CT images classification domain. And we propose an efficient model that can improve performance, reduce the number of parameters and give model explanations. In our model, the convolution neural network (CNN) extracts the feature vector from each image of a set of full slice brain CT. The multi-instance detect module focuses on the key images which could assist doctors quickly locate suspicious images and avoid mistakes. We evaluated our model on two datasets: CQ500 and RSNA. The F1 scores are 0.897 and 0.854 respectively. The proposed model outperforms the previous sequence-level model SDLM with only a quarter of the parameters. Low computation and high performance make the model have clinical applicability.

A Local Weighted Multi-instance Multi-label Network for Fault Diagnosis of Rolling Bearings Using Encoder Signal

Rolling bearings are the key components of modern machinery, and thus, the diagnosis of bearing faults plays a crucial role in ensuring the reliable operation of the equipment. Due to the limitation of cost and environmental conditions, the traditional fault diagnosis method based on the vibration signal is not always suitable for practical use. An encoder is widely used in the mechanical transmission system, and its signal contains rich information about health status and working conditions. However, the encoder signals are mostly used to monitor the rotational speed in the industrial and not considered as an alternative to the vibration signal. In this article, a framework based on a local weighted multi-instance multilabel (LWMIML) network is proposed to make full use of the information contained in the encoder to achieve the purpose of bearing fault diagnosis. This framework first constructs the instance generator to acquire feature vectors. On this basis, a local weighted strategy based on the MIML network is used to enhance the potential association between features and labels. Finally, the superiority of this method is verified by the data of the fault simulation test rig. The results indicate that the proposed method offers a promising tool, which will help to reduce the cost of future intelligent fault diagnosis systems.

Automatic Waveform Recognition of Overlapping LPI Radar Signals Based on Multi-Instance Multi-Label Learning

In an ever-increasingly complex electromagnetic environment, multiple low probability of intercept (LPI) radar emitters may transmit their own signals simultaneously on similar bands, resulting in overlapping receiving signals in both time and frequency domain. In this letter, a novel Multi-Instance Multi-Label learning framework based on Deep Convolutional Neural Network (MIML-DCNN) is proposed to automatically recognize the overlapping LPI radar signals,which is trained by single type of signals only. The framework handles signals in an end-to-end manner that is integrated with a well-designed instance generation module, a sophisticated MIML classifier, and an adaptive threshold calibration. Through comprehensive experiments on simulated overlapping signals with four different modulation types, we prove that the proposed framework identifies each individual signal type precisely in the presence of overlapping signals, and is also robust to variation of the signal-to-noise ratio (SNR) and power ratio conditions.

Multi-View Multi-Instance Multi-Label Learning Based on Collaborative Matrix Factorization

Multi-view Multi-instance Multi-label Learning (M3L) deals with complex objects encompassing diverse instances, represented with different feature views, and annotated with multiple labels. Existing M3L solutions only partially explore the inter or intra relations between objects (or bags), instances, and labels, which can convey important contextual information for M3L. As such, they may have a compromised performance.\
 In this paper, we propose a collaborative matrix factorization based solution called M3Lcmf. M3Lcmf first uses a heterogeneous network composed of nodes of bags, instances, and labels, to encode different types of relations via multiple relational data matrices. To preserve the intrinsic structure of the data matrices, M3Lcmf collaboratively factorizes them into low-rank matrices, explores the latent relationships between bags, instances, and labels, and selectively merges the data matrices. An aggregation scheme is further introduced to aggregate the instance-level labels into bag-level and to guide the factorization. An empirical study on benchmark datasets show that M3Lcmf outperforms other related competitive solutions both in the instance-level and bag-level prediction.

Metric Learning-Based Multi-Instance Multi-Label Classification With Label Correlation

In multi-instance multi-label learning (MIML) problems, predicting the labels of unseen bags becomes difficult when the labels of their instances are not provided directly. Therefore, it is necessary to exploit the label correlations to enhance the accuracy of the MIML classification. This paper presents the metric learning-based MIML-kNN (MI(ML)2kNN) method, which is composed of three parts. First, the label Laplacian matrix can be learned to obtain the label correlations by minimizing the label manifold regularizer. Then, based on label correlations, a novel objective function for the MIML is proposed where Mahalanobis distances between positively correlated labels and bags are minimized. Moreover, this objective function can be optimized by employing the Rayleigh-Ritz theorem and gradient descent (GD) alternately. Finally, the average Hausdorff distances of bag-bag pairs and bag-label pairs are calculated to construct the MIML-kNN classifier. Multiple classification experiments on three image and text benchmarks show the practicability and validity of our proposed method by comparing with the baseline methods.

A Randomized Clustering Forest Approach for Efficient Prediction of Protein Functions

With the advances in genetic sequencing technology, the automated assignment of protein function has become a key challenge in bioinformatics and computational biology. In nature, many kinds of proteins consist of a variety of structural domains, and each domain almost holds its own function independently or implements a new function in cooperation with neighbors. Thus, a multi-domain protein function prediction problem can be converted into multi-instance multi-label (MIML) learning tasks. In this paper, we propose a novel ensemble MIML algorithm called multi-instance multi-label randomized clustering forest (MIMLRC-Forest) for protein function prediction. In MIMLRC-Forest, we develop a set of hierarchical clustering trees and conduct a label transfer mechanism to identify the relevant function labels in learning process. The clustering tree with a hierarchical structure can handle the multi-label problem by exploiting more discriminable label concepts at higher-level nodes and by transferring less discriminable labels into the lower-level nodes. Then, the label dependency can be computed by aggregating tree labels for protein function prediction. Extensive experiments on five real-world protein data sets show the effectiveness of the proposed algorithm compared with several state-of-the-art baselines, including MIMLSVM, MIMLNN, MIML-kNN, EnMIMLNN, and M3MIML.

Semi-Supervised Multi-Modal Multi-Instance Multi-Label Deep Network with Optimal Transport

Complex objects are usually with multiple labels, and can be represented by multiple modal representations, e.g., the complex articles contain text and image information as well as multiple annotations. Previous methods assume that the homogeneous multi-modal data are consistent, while in real applications, the raw data are disordered, e.g., the article constitutes with variable number of inconsistent text and image instances. Therefore, Multi-modal Multi-instance Multi-label (M3) learning provides a framework for handling such task and has exhibited excellent performance. However, M3 learning is facing two main challenges: 1) how to effectively utilize label correlation and 2) how to take advantage of multi-modal learning to process unlabeled instances. To solve these problems, we first propose a novel Multi-modal Multi-instance Multi-label Deep Network (M3DN), which considers M3 learning in an end-to-end multi-modal deep network and utilizes consistency principle among different modal bag-level predictions. Based on the M3DN, we learn the latent ground label metric with the optimal transport. Moreover, we introduce the extrinsic unlabeled multi-modal multi-instance data, and propose the M3DNS, which considers the instance-level auto-encoder for single modality and modified bag-level optimal transport to strengthen the consistency among modalities. Thereby M3DNS can better predict label and exploit label correlation simultaneously. Experiments on benchmark datasets and real world WKG Game-Hub dataset validate the effectiveness of the proposed methods.

A Deep Multi-Modal CNN for Multi-Instance Multi-Label Image Classification.

Deep convolutional neural networks (CNNs) have shown superior performance on the task of single-label image classification. However, the applicability of CNNs to multi-label images still remains an open problem, mainly because of two reasons. First, each image is usually treated as an inseparable entity and represented as one instance, which mixes the visual information corresponding to different labels. Second, the correlations amongst labels are often overlooked. To address these limitations, we propose a deep multi-modal CNN for multi-instance multi-label image classification, called MMCNN-MIML. By combining CNNs with multi-instance multi-label (MIML) learning, our model represents each image as a bag of instances for image classification and inherits the merits of both CNNs and MIML. In particular, MMCNN-MIML has three main appealing properties: 1) it can automatically generate instance representations for MIML by exploiting the architecture of CNNs; 2) it takes advantage of the label correlations by grouping labels in its later layers; and 3) it incorporates the textual context of label groups to generate multi-modal instances, which are effective in discriminating visually similar objects belonging to different groups. Empirical studies on several benchmark multi-label image data sets show that MMCNN-MIML significantly outperforms the state-of-the-art baselines on multi-label image classification tasks.