Multi-instance Learning Algorithm Research Articles

A multi-instance multi-label learning algorithm based on radial basis functions and multi-objective particle swarm optimization

Radial basis function (RBF) neural networks for Multi-Instance Multi-Label (MIML) directly can exploit the connections between instances and labels so that they can preserve useful prior information, but they only adopt Gaussian radial basis function as their RBF whose parameters are difficult to determine. In this paper, parameters can be obtained by multi-objective optimization methods with multi performance measures treated as objectives, specifically, parameter estimation of different RBFs by an improved multi-objective particle swarm optimization (MOPSO) is proposed where Recall rate and Precision rate are chosen to obtain the most desirable Pareto optimal solution set. Furthermore, share-learning factor is proposed to modify the particle velocity in standard MOPSO to improve the global search ability and group cooperative ability. It is experimentally demonstrated that the proposed method can estimate the reliable parameters of different RBFs, and it is also very competitive with the state of art MIML methods.

Real-World Anomaly Detection in Video Using Spatio-Temporal Features Analysis for Weakly Labelled Data with Auto Label Generation

Detecting anomalies in videos is a complex task due to diverse content, noisy labeling, and a lack of frame-level labeling. To address these challenges in weakly labeled datasets, we propose a novel custom loss function in conjunction with the multi-instance learning (MIL) algorithm. Our approach utilizes the UCF Crime and ShanghaiTech datasets for anomaly detection. The UCF Crime dataset includes labeled videos depicting a range of incidents such as explosions, assaults, and burglaries, while the ShanghaiTech dataset is one of the largest anomaly datasets, with over 400 video clips featuring three different scenes and 130 abnormal events. We generated pseudo labels for videos using the MIL technique to detect frame-level anomalies from video-level annotations, and to train the network to distinguish between normal and abnormal classes. We conducted extensive experiments on the UCF Crime dataset using C3D and I3D features to test our model's performance. For the ShanghaiTech dataset, we used I3D features for training and testing. Our results show that with I3D features, we achieve an 84.6% frame-level AUC score for the UCF Crime dataset and a 92.27% frame-level AUC score for the ShanghaiTech dataset, which are comparable to other methods used for similar datasets.

Human activity recognition based on multi‐instance learning

AbstractHuman activity recognition (HAR) is the process of classifying a person's actions, and it is an essential task for many human‐centered applications. Multi‐instance learning (MIL) is a special case of machine learning where the training examples are bags containing many instances, and a single class label is assigned for an entire bag of instances. In this study, we integrated these two concepts by introducing a novel approach: “human activity recognition based on multi‐instance learning”, called HAR‐MIL. Unlike previous studies, the proposed HAR‐MIL method represents human activities differently: as a bag of various wearable sensors (gyroscope, magnetometer, accelerometer, and linear acceleration). HAR‐MIL presents an applicable and flexible model by providing multi‐instance representation and eliminating the restrictions of traditional single‐instance representation. Therefore, the adverse effects of missing data, defective sensors, and biased measurement on activity classification performance were minimized. This study is the first to investigate the performance of two MIL algorithms (SimpleMI and MIWrapper) on HAR. In this study, we explored the effect of four main factors (sensor positions, sensor types, base learners, and single or multiple participants) on the multi‐instance representation of relevant human daily activities. The effectiveness of the proposed HAR‐MIL method was demonstrated on 50 participant‐based and sensor‐position‐based activity recognition datasets. The experimental results showed that HAR‐MIL is effective for wearable sensor‐based HAR with high classification accuracy (99.32%). Furthermore, the results showed that the proposed method outperformed the state‐of‐the‐art methods by 10% on average on the same dataset.

Robust bag classification approach for multi-instance learning via subspace fuzzy clustering

Multi-instance learning (MIL) allows predictive algorithms to use complex data representation. The data in MIL is organized in the form of labeled bags of instances, and the labels of instances are not available in the training phase. The processing and classification of complex bag representation result in a complicated hypothesis space. Moreover, identifying essential instances inside is also important, as these instances trigger positive labels for the bag and play a vital role in the model interpretation and bag classification. The recent MIL algorithms are not robust to tackle hypothesis space complication. Additionally, the existing instance selection algorithms are based on explicit assumptions regarding the relationship of instances to the bag label. However, these assumptions may hold for a specific bag in the dataset but may not apply to the whole dataset. To deal with the hypothesis space complication and robust instance selection without any prior assumption, this paper proposes a fuzzy subspace clustering approach for robust instance selection and ensemble-based variant for Fisher vector (FV) encoding to solve MIL problems, named (FCBE-miFV). Specifically, the proposed algorithm uses a subspace fuzzy clustering approach to compute instance selection probabilities, selects essential instances from the bag, transforms the input bag into FV, and classifies the generated bag encodings using a stacking-based ensemble approach to obtain improved bag level classification performance. The proposed FCBE-miFV improves model performance by incorporating essential instances in the bag encoding process. The experimental results show that the FCBE-miFV obtained comparable performance to the state-of-the-art MIL problems.

Multiple instance convolutional neural network for gallbladder assessment from laparoscopic images.

We present an artificial intelligence framework for vascularity classification of the gallbladder (GB) wall from intraoperative images of laparoscopic cholecystectomy (LC). A two-stage Multiple Instance Convolutional Neural Network is proposed. First, a convolutional autoencoder is trained to extract feature representations from 4585 patches of GB images. The second model includes a multi-instance encoder that fetches random patches from a GB region and outputs an equal number of embeddings that feed a multi-input classification module, which employs pooling and self-attention mechanisms, to perform prediction. The evaluation was performed on 234GB images of low and high vascularity from 68 LC videos. Thorough comparison with various state-of-the-art multi-instance and single-instance learning algorithms was performed for two experimental tasks: image- and video-level classification. The proposed framework shows the best performance with accuracy 92.6%-93.2% and F1 93.5%-93.9%, close to the agreement of two expert evaluators (94%). The proposed technique provides a novel approach to classify LC operations with respect to the vascular pattern of the GB wall.

Cross-scale Attention Guided Multi-instance Learning for Crohn's Disease Diagnosis with Pathological Images.

Multi-instance learning (MIL) is widely used in the computer-aided interpretation of pathological Whole Slide Images (WSIs) to solve the lack of pixel-wise or patch-wise annotations. Often, this approach directly applies "natural image driven" MIL algorithms which overlook the multi-scale (i.e. pyramidal) nature of WSIs. Off-the-shelf MIL algorithms are typically deployed on a single-scale of WSIs (e.g., 20× magnification), while human pathologists usually aggregate the global and local patterns in a multi-scale manner (e.g., by zooming in and out between different magnifications). In this study, we propose a novel cross-scale attention mechanism to explicitly aggregate inter-scale interactions into a single MIL network for Crohn's Disease (CD), which is a form of inflammatory bowel disease. The contribution of this paper is two-fold: (1) a cross-scale attention mechanism is proposed to aggregate features from different resolutions with multi-scale interaction; and (2) differential multi-scale attention visualizations are generated to localize explainable lesion patterns. By training ~250,000 H&E-stained Ascending Colon (AC) patches from 20 CD patient and 30 healthy control samples at different scales, our approach achieved a superior Area under the Curve (AUC) score of 0.8924 compared with baseline models. The official implementation is publicly available at https://github.com/hrlblab/CS-MIL.

HUMAN DETECTION AND MOTION RECOVERY BASED ON MONOCULAR VISION

ABSTRACT Objective: Provides interactive games and human animation real motion data and technical options. Therefore, how to complete the position, attitude detection, and motion recovery under monocular vision has become an important research direction. Methods: This paper improves the part-based human detection algorithm and uses the AdaBoost multi-instance learning algorithm to train the part detector. Results: The results show that obtaining blood pressure waveform based on monocular vision pulse wave is feasible and has generalization. Conclusions: The results show the feasibility and accuracy of the gait motion detection, motion recovery and analysis system for human lower limbs based on monocular vision. Level of evidence II; Therapeutic studies - investigation of treatment results.

Multi-Instance Learning Algorithm Based on LSTM for Chinese Painting Image Classification

Aiming at the problem of weakly supervised learning in traditional Chinese painting image classification, a novel multi-instance learning algorithm based on Long and Short-Term Memory neural network with attention mechanism (ALSTM-MIL) is proposed. Firstly, by using the Pyramid Overlapping Grid Division (POGP), a multi-instance modeling scheme is designed to convert Chinese painting images into multi-instance bag, thereby transforming the problem of Chinese painting image classification into a MIL problem. Secondly, an efficient sequence generator is designed. It selects discriminative instances from the positive bags, construct a discriminative instance set (DIS), and convert multi-instance bags into equal-length ordered sequences. Thirdly, an LSTM network model with an attention mechanism is designed to perform semantic analysis on multi-instance bags to obtain their memory coding features, and then combined with the Softmax classifier to achieve semantic classification of traditional Chinese painting images. Experimental results on the Chinese painting (CP) image set show that the LSTM network built on the visual feature set is feasible, and the performance of the proposed MIL algorithm is also superior to other classification algorithms.

Multiple Instance Learning with Emerging Novel Class

Diverse applications involving complicated data objects such as proteins and images are solved by applying multi-instance learning (MIL) algorithms. However, few MIL algorithms can deal with problems in an open and dynamic environment, where new categories of samples emerge. In this type of emerging novel class setting, algorithms should be able to not only classify the samples from the observed classes accurately, but also recognize the samples from the novel class. In this paper, we focus on the Multi-Instance learning with Emerging Novel class (MIEN) problem, and formulate MIEN from a metric learning perspective. We extract key instances to form the “super-bag” for each observed class, and non-key instances from all the observed classes to form a “meta super-bag”. Based on these super-bags, we propose the MIEN-metric method to learn discriminative metrics for classifying MIL bags from the observed classes and recognizing bags from the novel class. Experimental results of diverse domains, e.g., biological function annotation, text categorization, and object-centric/scene-centric image classification, show MIEN-metric outperforms other baseline methods significantly when the novel class emerges. Meanwhile, MIEN-metric is comparable with state-of-the-art MIL algorithms for binary classification in the traditional MIL setting.

Dimension reduction and classification of hyperspectral images based on neural network sensitivity analysis and multi-instance learning

Hyperspectral remote image sensing is a rapidly developing integrated technology used widely in numerous areas. The rich spectral information from hyperspectral images aids in recognition and classification of many types of objects, but the high dimensionality of these images leads to information redundancy. In this paper, we used sensitivity analysis for dimension reduction. However, another challenge is that hyperspectral images identify objects as either a "different body with the same spectrum" or "same body with a different spectrum." Therefore, it is difficult to maintain the correct correspondence between ground objects and samples, which hinders classification of the images. This issue can be addressed using multi-instance learning for classification. In our proposed method, we combined neural network sensitivity analysis with a multiinstance learning algorithm based on a support vector machine to achieve dimension reduction and accurate classification for hyperspectral images. Experimental results demonstrated that our method provided strong overall classification effectiveness when compared with prior methods.

Multi-Instance Dimensionality Reduction via Sparsity and Orthogonality.

We study a multi-instance (MI) learning dimensionality-reduction algorithm through sparsity and orthogonality, which is especially useful for high-dimensional MI data sets. We develop a novel algorithm to handle both sparsity and orthogonality constraints that existing methods do not handle well simultaneously. Our main idea is to formulate an optimization problem where the sparse term appears in the objective function and the orthogonality term is formed as a constraint. The resulting optimization problem can be solved by using approximate augmented Lagrangian iterations as the outer loop and inertial proximal alternating linearized minimization (iPALM) iterations as the inner loop. The main advantage of this method is that both sparsity and orthogonality can be satisfied in the proposed algorithm. We show the global convergence of the proposed iterative algorithm. We also demonstrate that the proposed algorithm can achieve high sparsity and orthogonality requirements, which are very important for dimensionality reduction. Experimental results on both synthetic and real data sets show that the proposed algorithm can obtain learning performance comparable to that of other tested MI learning algorithms.

Multi-instance transfer metric learning by weighted distribution and consistent maximum likelihood estimation

Abstract Multi-Instance learning (MIL) aims to predict labels of unlabeled bags by training a model with labeled bags. The usual assumption of existing MIL methods is that the underlying distribution of training data is the same as that of the testing data. However, this assumption may not be valid in practice, especially when training data from a source domain and testing data from a target domain are drawn from different distributions. In this paper, we put forward a novel algorithm Multi-Instance Transfer Metric Learning (MITML). Specially, MITML first attempts to bridge the distributions of different domains by using the bag weighting method. Then a consistent maximum likelihood estimation method is learned to construct an optimal distance metric and exploited to classify testing bags. Comprehensive experimental results on benchmark datasets have demonstrated that the learning performance of the proposed MITML algorithm is better than those of other state-of-the-art MIL algorithms.

Predicting human gaze with multi-level information

Eye fixation models, which try to quantitatively predict human eye attended areas in visual fields, have received increasing interest in recent years. In this paper, a novel framework is proposed for the detection of eye fixations. First, a multi-channel detection module, which extracts information of color contrast, salient object proposals and center bias from input image, is conducted to introduce various useful information into the subsequent fixations detection. In salient object detection channel, we employ the multi-instance learning (MIL) algorithm to determine which object proposal can attract attention, which avoids the fuzzyness of positive sample selection. Second, an adaptive weighted fusion method achieved by deep learning framework is proposed to fuse the multi-level information (i.e., contrast, objective, center bias) together for the detection of fixations, so that the integration of information between each level becomes more scientific. Finally, the detection result is optimized by embedding semantic information. Experimental results show that the algorithm has achieved competitive results in MIT1003, MIT300 and Toronto120 dataset.

An improved multi-instance multi-label learning algorithm based on representative instances selection and label correlations

Multi-Instance Multi-Label Learning (MIML) has been successfully used in image and text classification problems. It is noteworthy that few of the previous studies consider the pattern-label relations. Inevitably, there are some useless instances in a bag which will reduce the accuracy of the annotation. In this paper we focus on this problem. Firstly, an instance selection method via joint l2,1-norms constraint is employed to eliminate the useless instances and select the representative instances by modelling the instance correlation. Then, bags are mapped to these representative instances. Finally, the classifier is trained by an optimisation algorithm based on label correlations. Experimental results on image data set, text data sets and bird song audio data set show that the proposed algorithm significantly improves the performance of MIML classifier compared with the state-of-the-art methods.

An improved multi-instance multi-label learning algorithm based on representative instances selection and label correlations

An improved multi-instance multi-label learning algorithm based on representative instances selection and label correlations

Multi-Instance Multi-Label Learning for Multi-Class Classification of Whole Slide Breast Histopathology Images.

Digital pathology has entered a new era with the availability of whole slide scanners that create the high-resolution images of full biopsy slides. Consequently, the uncertainty regarding the correspondence between the image areas and the diagnostic labels assigned by pathologists at the slide level, and the need for identifying regions that belong to multiple classes with different clinical significances have emerged as two new challenges. However, generalizability of the state-of-the-art algorithms, whose accuracies were reported on carefully selected regions of interest (ROIs) for the binary benign versus cancer classification, to these multi-class learning and localization problems is currently unknown. This paper presents our potential solutions to these challenges by exploiting the viewing records of pathologists and their slide-level annotations in weakly supervised learning scenarios. First, we extract candidate ROIs from the logs of pathologists' image screenings based on different behaviors, such as zooming, panning, and fixation. Then, we model each slide with a bag of instances represented by the candidate ROIs and a set of class labels extracted from the pathology forms. Finally, we use four different multi-instance multi-label learning algorithms for both slide-level and ROI-level predictions of diagnostic categories in whole slide breast histopathology images. Slide-level evaluation using 5-class and 14-class settings showed average precision values up to 81% and 69%, respectively, under different weakly labeled learning scenarios. ROI-level predictions showed that the classifier could successfully perform multi-class localization and classification within whole slide images that were selected to include the full range of challenging diagnostic categories.

Two Birds With One Stone: A Unified Approach to Saliency and Co-Saliency Detection via Multi-Instance Learning

Saliency detection on an individual image as well as co-saliency detection from a group of images are currently popular topics or reflect future trends more recently due to their importance and challenging roles in computer vision. In many cases, co-saliency detection is usually dependent on the single-image saliency detection results. Nevertheless, most efforts have been made to tackle them separately and not much attention has been paid to tackling them together in a unified idea. Being aware of these two tasks are highly related, the difference from previous surveys is that this paper applies a unified framework by employing a multi-instance learning (MIL) algorithm to resolve both the issues, and formulating single-image saliency and co-saliency detection as top-down weakly supervised learning paradigm. Specifically, for single-image saliency detection, we utilize the evidence confidence-support vector machine algorithm to learn a discriminant model to predict the saliency on test images. For co-saliency detection from image group, we concatenate the EC values and saliency scores to obtain the final results of co-saliency detection. By observing the importance of the selection of negative bags in the MIL framework, we also introduce a novel selection strategy of negative bags to improve the robustness of the proposed method. Experimental results on publicly available image benchmark data sets have demonstrated that the proposed unified framework can achieve competitive performances as compared with the state-of-the-art algorithms in terms of accuracy and effectiveness.

Weakly supervised vehicle detection in satellite images via multi-instance discriminative learning

Vehicle detection in satellite images has attracted extensive research interest with widespreading application potentials. The main challenge lies in the difficulty of labeling sufficient training instances (vehicle rectangles) across all resolutions and imaging conditions of satellite images, which degenerates the performance of vehicle detectors trained correspondingly. To tackle this challenge, in this paper we propose an intelligent and labor-light scheme for large-scale training of vehicle detectors. Our scheme only requires region-level group annotation, i.e. whether this region contains vehicle(s) or not, without explicitly labeling the bounding boxes of vehicles. To this end, a novel weakly supervised, multi-instance learning algorithm is designed to learn instance-wise vehicle detectors from such “weak labels”. In particular, a density estimator is firstly adopted to estimate the density map of vehicle instances from the positive regions. Then, a multi-instance SVM is trained to classify and locate vehicle instances from this map. We have carried out extensive experiments on a large-scale satellite image collection that contains various resolutions and imaging conditions. We have demonstrated that the proposed scheme has achieved superior performance by comparing to a set of state-of-the-art and alternative approaches.

Multi-instance learning based on representative instance and feature mapping

In this paper, based on the representative instances and feature mapping, we propose two Multi-Instance Learning (MIL) algorithms, i.e. Representative Instance and Feature Mapping for instances (RIFM-I) and Representative Instance and Feature Mapping for bags (RIFM-B). These two algorithms firstly select representative positive and negative instances from positive and negative bags, respectively, and then map selected instances and bags to the feature space, in which MIL problem is converted into conventional single-instance learning problem. Finally, Support Vector Data Description (SVDD) method is introduced to solve the converted problem. The experiment on the MUSK dataset shows that RIFM-I performs better than RIFM-B and provides highest classification accuracies compared with the best results obtained among all the methods, and RIFM-B achieves a competitive average accuracy performance. Furthermore, RIFM-I is applied on COREL image repository for the content-based image retrieval. The experimental results show that RIFM-I outperforms the other image retrieval methods, such as MILES and MissSVM, and is able to distinguish two easily confused categories, Beach and Mountains, quite well. In addition, The results in ten data sets commonly used in MIL also show that RIFM-I can achieve better results in most cases.

A multi-instance multi-label learning algorithm based on instance correlations

Existing multi-instance multi-label learning algorithms generally assume that instances in a bag are independent of each other, which is difficult to be guaranteed in practical applications. A nove...