Multiple Instance Classification Research Articles

Bayesian multiple instance classification based on hierarchical probit regression

Bayesian multiple instance classification based on hierarchical probit regression

A deep multiple-instance text binary classification for topic relevant content extraction on social media

Social media platforms have rich text data, which can be used in data mining and analysis. However, given the fact, the evolution speed of natural languages is rapid in social media, and data on social media is very noisy. This is a great challenge to the accuracy of data analysis. To overcome this problem, we propose a topic-relevant content extraction (TRCE) based on deep multiple instance classification, leveraging existing information and hierarchical relationships among texts under a thread on social media as weak supervision to extract topic-strong-relevant data and filter out noise accurately without manually labeling data. The proposed method introduces latent variables, Bernoulli distribution, and variational inference into multiple-instance learning (MIL) to generate pseudo labels. Then we employ a dual-stream neural network with a 3-stage training process to achieve training MIL end-to-end. Experimental results show TRCE has a significant improvement compared with other MIL methods. Meanwhile, it only has a little decrease compared with supervised text classification on accuracy and F1 score. Given the fact TRCE does not need manually labeled data at all, while supervised classification relies heavily on labeled data, TRCE is a competitive method to extracting topic-relevant data and filtering out noise on social media.

Ambient noise-based weakly supervised manhole localization methods over deployed fiber networks.

We present a manhole localization method based on distributed fiber optic sensing and weakly supervised machine learning techniques. For the first time to our knowledge, ambient environment data is used for underground cable mapping with the promise of enhancing operational efficiency and reducing field work. To effectively accommodate the weak informativeness of ambient data, a selective data sampling scheme and an attention-based deep multiple instance classification model are adopted, which only requires weakly annotated data. The proposed approach is validated on field data collected by a fiber sensing system over multiple existing fiber networks.

Deep semi-supervised multiple instance learning with self-correction for DME classification from OCT images.

Supervised deep learning has achieved prominent success in various diabetic macular edema (DME) recognition tasks from optical coherence tomography (OCT) volumetric images. A common problematic issue that frequently occurs in this field is the shortage of labeled data due to the expensive fine-grained annotations, which increases substantial difficulty in accurate analysis by supervised learning. The morphological changes in the retina caused by DME might be distributed sparsely in B-scan images of the OCT volume, and OCT data is often coarsely labeled at the volume level. Hence, the DME identification task can be formulated as a multiple instance classification problem that could be addressed by multiple instance learning (MIL) techniques. Nevertheless, none of previous studies utilize unlabeled data simultaneously to promote the classification accuracy, which is particularly significant for a high quality of analysis at the minimum annotation cost. To this end, we present a novel deep semi-supervised multiple instance learning framework to explore the feasibility of leveraging a small amount of coarsely labeled data and a large amount of unlabeled data to tackle this problem. Specifically, we come up with several modules to further improve the performance according to the availability and granularity of their labels. To warm up the training, we propagate the bag labels to the corresponding instances as the supervision of training, and propose a self-correction strategy to handle the label noise in the positive bags. This strategy is based on confidence-based pseudo-labeling with consistency regularization. The model uses its prediction to generate the pseudo-label for each weakly augmented input only if it is highly confident about the prediction, which is subsequently used to supervise the same input in a strongly augmented version. This learning scheme is also applicable to unlabeled data. To enhance the discrimination capability of the model, we introduce the Student-Teacher architecture and impose consistency constraints between two models. For demonstration, the proposed approach was evaluated on two large-scale DME OCT image datasets. Extensive results indicate that the proposed method improves DME classification with the incorporation of unlabeled data and outperforms competing MIL methods significantly, which confirm the feasibility of deep semi-supervised multiple instance learning at a low annotation cost.

Multiple instance classification via quadratic programming

Multiple instance classification via quadratic programming

Multiple Instance Classification for Gastric Cancer Pathological Images Based on Implicit Spatial Topological Structure Representation

Gastric cancer is a malignant tumor with high incidence. Computer-aided screening systems for gastric cancer pathological images can contribute to reducing the workload of specialists and improve the efficiency of disease diagnosis. Due to the high resolution of images, it is common to divide the whole slide image (WSI) into a set of image patches with overlap before utilizing deep neural networks for further analysis. However, not all patches split from the same cancerous WSI contain information of cancerous issues. This restriction naturally satisfies the assumptions of multiple instance learning (MIL). Moreover, the spatial topological structure relationships between local areas in a WSI are destroyed in the process of patch partitioning. Most existing multiple instance classification (MIC) methods fail to take into account the topological relationships between instances. In this paper, we propose a novel multiple instance classification framework based on graph convolutional networks (GCNs) for gastric microscope image classification. Firstly, patch embeddings were generated by feature extraction. Then, a graph structure was introduced to model the spatial topological structure relationships between instances. Additionally, a graph classification model with hierarchical pooling was constructed to achieve this multiple instance classification task. To certify the effectiveness and generalization of our method, we conducted comparative experiments on two different modes of gastric cancer pathological image datasets. The proposed method achieved average fivefold cross-validation precisions of 91.16% and 98.26% for gastric cancer classification on the two datasets, respectively.

Multiple instance classification: Bag noise filtering for negative instance noise cleaning

Data in the real world is far from being perfect. The appearance of noise is a common issue that arises from the limitations of data acquisition mechanisms and human knowledge. In classification, label noise will hinder the performance of almost all classifiers, inducing a bias in the built model. While label noise has recently attracted researchers’ attention in standard classification, it has only recently begun to be studied in multiple instance classification. In this work, we propose the usage of filtering algorithms for multiple instance classification that are able to reduce the impact of negative instances within the bags. In order to do so, we decompose the bags to form a standard classification problem that can be efficiently treated by a specialized noise filter. Such a decomposition is tackled in different ways, with the aim of exploiting the knowledge offered by the examples from opposite bags. The bags are then rebuilt, without the identified noise instances. In our experiments, we show that by applying our approach we can diminish the impact of noise and even obtain better results at 0% noise level for several classifiers. Our approach sets out a promising approach to dealing with noise in the bags of multiple instance datasets and further improve the classification rate of the built models.

SVM-Based Multiple Instance Classification via DC Optimization

A multiple instance learning problem consists of categorizing objects, each represented as a set (bag) of points. Unlike the supervised classification paradigm, where each point of the training set is labeled, the labels are only associated with bags, while the labels of the points inside the bags are unknown. We focus on the binary classification case, where the objective is to discriminate between positive and negative bags using a separating surface. Adopting a support vector machine setting at the training level, the problem of minimizing the classification-error function can be formulated as a nonconvex nonsmooth unconstrained program. We propose a difference-of-convex (DC) decomposition of the nonconvex function, which we face using an appropriate nonsmooth DC algorithm. Some of the numerical results on benchmark data sets are reported.

An embarrassingly simple approach to neural multiple instance classification

Multiple Instance Learning (MIL) is a weak supervision learning paradigm that allows modeling of machine learning problems in which labels are not available for individual examples but only for groups of examples called bags. A positive bag may contain one or more positive examples but it is not known which examples in the bag are positive. All examples in a negative bag belong to the negative class. Such problems arise frequently in fields of computer vision, medical image processing and bioinformatics. Many neural network-based solutions have been proposed in the literature for MIL. However, almost all of them rely on introducing specialized blocks and connectivity in their architectures. In this paper, we present a simple and effective approach to Multiple Instance Learning in neural networks. We propose a simple bag-level ranking loss function that allows Multiple Instance Classification in any neural architecture. We have demonstrated the effectiveness of our proposed method for popular MIL benchmark datasets. Additionally, we have also tested the performance of our method in convolutional neural networks used to model an MIL problem derived from the well-known MNIST dataset. Results show that despite being simpler, our proposed scheme is comparable or better than existing methods in the literature in practical scenarios. Python code files for all the experiments can be found at https://github.com/amina01/ESMIL

A Lagrangian Relaxation Approach for Binary Multiple Instance Classification.

In the standard classification problems, the objective is to categorize points into different classes. Multiple instance learning (MIL), instead, is aimed at classifying bags of points, each point being an instance. The main peculiarity of a MIL problem is that, in the learning phase, only the label of each bag is known whereas the labels of the instances are unknown. We discuss an instance-level learning approach for a binary MIL classification problem characterized by two classes of instances, positive and negative, respectively. In such a problem, a negative bag is constituted only by negative instances, while a bag is positive if it contains at least one positive instance. We start from a mixed integer nonlinear optimization model drawn from the literature and the main result we obtain is to prove that a Lagrangian relaxation approach, equipped with a dual ascent scheme, allows us to obtain an optimal solution of the original problem. The relaxed problem is tackled by means of a block coordinate descent (BCD) algorithm. We provide, finally, the results of our implementation on some benchmark data sets.

Supervised aggregated feature learning for multiple instance classification

This paper introduces a novel algorithm, called Supervised Aggregated FEature learning or SAFE, which combines both (local) instance level and (global) bag level information in a joint framework to address the multiple instance classification task. In this realm, the collective assumption is used to express the relationship between the instance labels and the bag labels, by means of taking the sum as aggregation rule. The proposed model is formulated within a least squares support vector machine setting, where an unsupervised core model (either kernel PCA or kernel spectral clustering) at the instance level is combined with a classification loss function at the bag level. The corresponding dual problem consists of solving a linear system, and the bag classifier is obtained by aggregating the instance scores. Synthetic experiments suggest that SAFE is advantageous when the instances from both positive and negative bags can be naturally grouped in the same cluster. Moreover, real-life experiments indicate that SAFE is competitive with the best state-of-the-art methods.

Single- vs. multiple-instance classification

In multiple-instance (MI) classification, each input object or event is represented by a set of instances, named a bag, and it is the bag that carries a label. MI learning is used in different applications where data is formed in terms of such bags and where individual instances in a bag do not have a label. We review MI classification from the point of view of label information carried in the instances in a bag, that is, their sufficiency for classification. Our aim is to contrast MI with the standard approach of single-instance (SI) classification to determine when casting a problem in the MI framework is preferable. We compare instance-level classification, combination by noisy-or, and bag-level classification, using the support vector machine as the base classifier. We define a set of synthetic MI tasks at different complexities to benchmark different MI approaches. Our experiments on these and two real-world bioinformatics applications on gene expression and text categorization indicate that depending on the situation, a different decision mechanism, at the instance- or bag-level, may be appropriate. If the instances in a bag provide complementary information, a bag-level MI approach is useful; but sometimes the bag information carries no useful information at all and an instance-level SI classifier works equally well, or better.

A theoretical and empirical analysis of support vector machine methods for multiple-instance classification

The standard support vector machine (SVM) formulation, widely used for supervised learning, possesses several intuitive and desirable properties. In particular, it is convex and assigns zero loss to solutions if, and only if, they correspond to consistent classifying hyperplanes with some nonzero margin. The traditional SVM formulation has been heuristically extended to multiple-instance (MI) classification in various ways. In this work, we analyze several such algorithms and observe that all MI techniques lack at least one of the desirable properties above. Further, we show that this tradeoff is fundamental, stems from the topological properties of consistent classifying hyperplanes for MI data, and is related to the computational complexity of learning MI hyperplanes. We then study the empirical consequences of this three-way tradeoff in MI classification using a large group of algorithms and datasets. We find that the experimental observations generally support our theoretical results, and properties such as the labeling task (instance versus bag labeling) influence the effects of different tradeoffs.

SMILe: Shuffled Multiple-Instance Learning

Resampling techniques such as bagging are often used in supervised learning to produce more accurate classifiers. In this work, we show that multiple-instance learning admits a different form of resampling, which we call "shuffling." In shuffling, we resample instances in such a way that the resulting bags are likely to be correctly labeled. We show that resampling results in both a reduction of bag label noise and a propagation of additional informative constraints to a multiple-instance classifier. We empirically evaluate shuffling in the context of multiple-instance classification and multiple-instance active learning and show that the approach leads to significant improvements in accuracy.

Robust support vector machines for multiple instance learning

This paper presents the multiple instance classification problem that can be used for drug and molecular activity prediction, text categorization, image annotation, and object recognition. In order to model a more robust representation of outliers, hard margin loss formulations that minimize the number of misclassified instances are proposed. Although the problem is $\mathcal{NP}$ -hard, computational studies show that medium sized problems can be solved to optimality in reasonable time using integer programming and constraint programming formulations. A three-phase heuristic algorithm is proposed for larger problems. Furthermore, different loss functions such as hinge loss, ramp loss, and hard margin loss are empirically compared in the context of multiple instance classification. The proposed heuristic and robust support vector machines with hard margin loss demonstrate superior generalization performance compared to other approaches for multiple instance learning.

Relaxing support vectors for classification

We introduce a novel modification to standard support vector machine (SVM) formulations based on a limited amount of penalty-free slack to reduce the influence of misclassified samples or outliers. We show that free slack relaxes support vectors and pushes them towards their respective classes, hence we use the name relaxed support vector machines (RSVM) for our method. We present theoretical properties of the RSVM formulation and develop its dual formulation for nonlinear classification via kernels. We show the connection between the dual RSVM and the dual of the standard SVM formulations. We provide error bounds for RSVM and show it to be stable, universally consistent and tighter than error bounds for standard SVM. We also introduce a linear programming version of RSVM, which we call RSVMLP. We apply RSVM and RSVMLP to synthetic data and benchmark binary classification problems, and compare our results with standard SVM classification results. We show that relaxed influential support vectors may lead to better classification results. We develop a two-phase method called RSVM2 for multiple instance classification (MIC) problems, where RSVM formulations are used as classifiers. We extend the two-phase method to the linear programming case and develop RSVMLP2. We demonstrate the classification characteristics of RSVM2 and RSVMLP2, and report our classification results compared to results obtained by other SVM-based MIC methods on public benchmark datasets. We show that both RSVM2 and RSVMLP2 are faster and produce more accurate classification results.

Fast Bundle Algorithm for Multiple-Instance Learning

We present a bundle algorithm for multiple-instance classification and ranking. These frameworks yield improved models on many problems possessing special structure. Multiple-instance loss functions are typically nonsmooth and nonconvex, and current algorithms convert these to smooth nonconvex optimization problems that are solved iteratively. Inspired by the latest linear-time subgradient-based methods for support vector machines, we optimize the objective directly using a nonconvex bundle method. Computational results show this method is linearly scalable, while not sacrificing generalization accuracy, permitting modeling on new and larger data sets in computational chemistry and other applications. This new implementation facilitates modeling with kernels.

Multiple Instance Classification Based on Least Squares Twin Support Vector Machine

Multiple Instance Classification Based on Least Squares Twin Support Vector Machine

Maximum Margin Multiple Instance Clustering With Applications to Image and Text Clustering

In multiple instance learning problems, patterns are often given as bags and each bag consists of some instances. Most of existing research in the area focuses on multiple instance classification and multiple instance regression, while very limited work has been conducted for multiple instance clustering (MIC). This paper formulates a novel framework, maximum margin multiple instance clustering (M(3)IC), for MIC. However, it is impractical to directly solve the optimization problem of M(3)IC. Therefore, M(3)IC is relaxed in this paper to enable an efficient optimization solution with a combination of the constrained concave-convex procedure and the cutting plane method. Furthermore, this paper presents some important properties of the proposed method and discusses the relationship between the proposed method and some other related ones. An extensive set of empirical results are shown to demonstrate the advantages of the proposed method against existing research for both effectiveness and efficiency.

Aggregate Homotopy Method for Min-Max-Min Programming Satisfying a Weak-Normal Cone Condition

Min-max-min programming is an important but difficult nonsmooth programming. An aggregate homotopy method was given for solving min-max-min programming by Bo Yu el al. However, the method requires a difficult to verify weak-normal cone condition. Moreover, the method is only theoretically with no algorithmic implementation. In this paper, the weak normal cone condition is discussed first. A class of min-max-min programming satisfying the condition is introduced. A detailed algorithm to implement the method is presented. Models arising from some applications such as support vector machine for multiple-instance classification in data mining, can be included in the problem. In the end, the aggregate homotopy method is given to solve the multiple-instance support vector machine model.