Semi-supervised Multi-label Classification Research Articles

Semi-Supervised Multi-Label Classification of Land Use/Land Cover in Remote Sensing Images With Predictive Clustering Trees and Ensembles

Semi-Supervised Multi-Label Classification of Land Use/Land Cover in Remote Sensing Images With Predictive Clustering Trees and Ensembles

Semi-supervised multi-label classification using an extended graph-based manifold regularization

Graph-based algorithms are known to be effective approaches to semi-supervised learning. However, there has been relatively little work on extending these algorithms to the multi-label classification case. We derive an extension of the Manifold Regularization algorithm to multi-label classification, which is significantly simpler than the general Vector Manifold Regularization approach. We then augment our algorithm with a weighting strategy to allow differential influence on a model between instances having ground-truth vs. induced labels. Experiments on four benchmark multi-label data sets show that the resulting algorithm performs better overall compared to the existing semi-supervised multi-label classification algorithms at various levels of label sparsity. Comparisons with state-of-the-art supervised multi-label approaches (which of course are fully labeled) also show that our algorithm outperforms all of them even with a substantial number of unlabeled examples.

A new Feature Reduction Algorithm Based on Fuzzy Rough Relation for the Multi-label Classification

The paper aims to improve the multi-label classification performance using the feature reduction technique. According to the determination of the dependency among features based on fuzzy rough relation, features with the highest dependency score will be retained in the reduction set. The set is subsequently applied to enhance the performance of the multi-label classifier. We investigate the effectiveness of the proposed model againts the baseline via time complexity.
 Keywords: 
 Fuzzy rough relation, label-specific feature, feature reduction set
 References
 [1] Richard Jensen, Chris Cornelis, Fuzzy-Rough Nearest Neighbor Classification and Prediction. Proceedings of the 6th International Conference on Rough Sets and Current Trends in Computing, 2011, 310-319.
 [2] Y.H. Qian, Q. Wang, H.H. Cheng, J.Y. Liang, C.Y. Dang, Fuzzy-Rough feature selection accelerator, Fuzzy Sets Syst. 258 (2014) 61-78.
 [3] Quang-Thuy Ha, Thi-Ngan Pham, Van-Quang Nguyen, Minh-Chau Nguyen, Thanh-Huyen Pham, Tri-Thanh Nguyen, A New Text Semi-supervised Multi-label Learning Model Based on Using the Label-Feature Relations, International Conference on Computational Collective Intelligence, LNAI 11055, Springer, 2018, pp. 403-413.
 [4] Daniel Kostrzewa, Robert Brzeski, The data Dimensionality Reduction and Feature Weighting in the Classification Process Using Forest Optimization Algorithm, ACIIDS, 2019, pp. 97-108.
 [5] Nele Verbiest, Fuzzy Rough and Evolutionary Approaches to Instance Selection, PhD Thesis, Ghent University, 2014.
 [6] Y. Yu, W. Pedrycz, D.Q. Miao, Multi-label classification by exploiting label correlations, Expert syst, Appl. 41 (2014) 2989-3004.
 [7] M.L. Zhang, LIFT: Multi-label learning with label-specific features, IEEE Trans, Pattern Anal, Mach, Intell 37 (2015) 107-120.
 [8] Suping Xu, Xibei Yang, Hualong Yu, Dong-Jun Yu, Jingyu Yang, Eric CC Tsang, Multi-label learning with label-specific feature reduction, Knowledge-Based Systems 104 (2016) 52-61. https://doi.org/10.1080/24751839.2017.1364925.
 [9] Thi-Ngan Pham, Van-Quang Nguyen, Van-Hien Tran, Tri-Thanh Nguyen, Quang-Thuy Ha, A Semi-supervised multi-label classification framework with feature reduction and enrichment, Journal of Information and Telecommunication 1(4) (2017) 305-318.
 [10] M. Ghaemi, M.R. Feizi-Derakhshi, Feature selection using forest optimization algorithm, Pattern Recognition 60 (2016) 121-129.
 [11] M.L. Zhang, Z.H. Zhou, ML-KNN: A lazy learning approach to multi-label learning, Pattern Recognition 40 (2007) 2038-2048.
 [12] M.Z. Ahmad, M.K. Hasan, A New Approach for Computing Zadeh's Extension Principle, MATEMATIKA. 26(1) (2010) 71-81.
 [13] Richard Jensen, Neil Mac Parthaláin and Qiang Shen. Fuzzy-rough data mining (using the Weka data mining suite), A Tutorial, IEEE WCCI 2014, Beijing, China, July 6, 2014.
 [14] D. Dubois, H. Prade, Rough fuzzy sets and fuzzy rough sets, Int. J. Gen. Syst. 17 (1990) 191-209.

A semi-supervised multi-label classification framework with feature reduction and enrichment

ABSTRACTMulti-label classification (MLC) has drawn much attention thanks to its usefulness and omnipresence in real-world applications in which objects may be characterized by more than one label as in the traditional approach. Getting multi-label examples is costly and time-consuming; therefore, semi-supervised learning approach should be considered to take advantages of both labelled and unlabelled data. In this work, we propose a semi-supervised MLC algorithm exploiting the specific features of the prominent class label(s) chosen by a greedy approach as an extension of the LIFT algorithm, and unlabelled data consumption mechanism from the TESC algorithm. We also make a semi-supervised MLC application framework for Vietnamese texts with several feature enrichment steps including (a) a stage of enriching features by adding hidden topic features and (b) a stage of dimensional reduction for subtracting irrelevant features. Experimental results on a dataset of hotel reviews (for tourism) indicate that a reasonable amount of unlabelled data helps to increase the F1 score. Interestingly, with a small amount of labelled data, our algorithm can reach a comparative performance to the case of using a larger amount of labelled data.

Semi-Supervised Multi-Label Classification Through Topological Active Learning

Multi-label classification is becoming increasingly widespread as a data mining technique. Its objective is to categorize models in several non-exclusive groups, and is applied in such areas as news categorization, image labeling and music classification, among others. Our contribution is to use the paradigm of active learning with the topological power of the Act-SOM for semi-supervised multi-label classification, taking into account the multi-label information, and selecting unlabeled data which can lead to the largest reduction of the expected model loss. This paper deals with various multi-label datasets by presenting in active learning, a set of results ranging from: 1) Transductive classifier TSVM with relevance sampling methods for multi-labeled data in various application domains; 2) Proposed semi-supervised classifier Act-SOM in multi-label active learning, adopting a strategy relative to the evaluation by the uncertainty of the labels. Act-SOM based on active learning selects the most uncertain data while clearly improving the test rate with less than 30% of labeled instances added, which is our main contribution. We present the results from the statistical tests using critical diagrams. Thus, potential of the proposed multi-label classification method is demonstrate, due mainly to the competitive properties with global consistency of the semi-supervised Act-SOM through topological active learning.

Semi-supervised multi-label classification using incomplete label information

Classifying multi-label instances using incompletely labeled instances is one of the fundamental tasks in multi-label learning. Most existing methods regard this task as supervised weak-label learning problem and assume sufficient partially labeled instances are available. However, collecting or annotating such instances is expensive and time-consuming. In contrast, abundant unlabeled instances are easy to accumulate. Recently, some methods move toward exploiting unlabeled instances and performing transductive multi-label classification. However, these methods can not directly apply to new instances, which are not available during training process. In this paper, we proposed an approach called Semi-supervised multi-label classification using incomplete label information (SMILE for short). SMILE first estimates label correlation from partially labeled instances and replenishes missing labels of these instances. Then, it takes advantage of labeled and unlabeled instances to construct a neighborhood graph. Next, the known labels and replenished ones of labeled instances, along with unlabeled instances are exploited to train a graph based semi-supervised linear classifier. SMILE can further replenish the missing labels of training instances based on the adopted neighborhood graph. In addition, it can directly predict the labels of completely unlabeled new instances. The empirical study on multi-label datasets shows that SMILE performs significantly better than other related methods across various evaluation criteria and it is important to leverage unlabeled data with label correlation for multi-label classification.

A semi-supervised multi-label classification framework with feature reduction and enrichment

ABSTRACTMulti-label classification (MLC) has drawn much attention, thanks to its usefulness and omnipresence in real-world applications in which objects may be characterized by more than one label as in the traditional approach. Getting multi-label examples is costly and time-consuming; therefore, semi-supervised learning approach should be considered to take advantages of both labelled and unlabelled data. In this work, we propose a semi-supervised MLC algorithm exploiting the specific features of the prominent class label(s) chosen by a greedy approach as an extension of the LIFT algorithm, and unlabelled data consumption mechanism from the TExt classification using Semi-supervised Clustering algorithm. We also make a semi-supervised MLC application framework for Vietnamese texts with several feature enrichment steps including (a) a stage of enriching features by adding hidden topic features and (b) a stage of dimensional reduction for subtracting irrelevant features. Experimental results on a data set of hotel reviews (for tourism) indicate that a reasonable amount of unlabelled data helps to increase the F1 score. Interestingly, with a small amount of labelled data, our algorithm can reach a comparative performance to the case of using a larger amount of labelled data.

Semisupervised Multilabel Learning With Joint Dimensionality Reduction

Mutlilabel classification arises in various domains including computer vision and machine learning. Given a single instance, multilabel classification aims to learn a set of labels simultaneously. However, existing methods fail to address two key problems: 1) exploiting correlations among instances and 2) reducing computational complexity. In this letter, we propose a new semisupervised multilabel classification algorithm with joint dimensionality reduction. First, an elaborate matrix is designed for evaluating instance similarity; thus, it can take both labeled and unlabeled instances into consideration. Second, a linear dimensionality reduction matrix is added into the framework of multilabel classification. Besides, the dimensionality reduction matrix and the objective function can be optimized simultaneously. Finally, we design an efficient algorithm to solve the dual problem of the proposed model. Experiment results demonstrate that the proposed method is effective and promising.

Semi-supervised multi-label collective classification ensemble for functional genomics.

BackgroundWith the rapid accumulation of proteomic and genomic datasets in terms of genome-scale features and interaction networks through high-throughput experimental techniques, the process of manual predicting functional properties of the proteins has become increasingly cumbersome, and computational methods to automate this annotation task are urgently needed. Most of the approaches in predicting functional properties of proteins require to either identify a reliable set of labeled proteins with similar attribute features to unannotated proteins, or to learn from a fully-labeled protein interaction network with a large amount of labeled data. However, acquiring such labels can be very difficult in practice, especially for multi-label protein function prediction problems. Learning with only a few labeled data can lead to poor performance as limited supervision knowledge can be obtained from similar proteins or from connections between them. To effectively annotate proteins even in the paucity of labeled data, it is important to take advantage of all data sources that are available in this problem setting, including interaction networks, attribute feature information, correlations of functional labels, and unlabeled data.ResultsIn this paper, we show that the underlying nature of predicting functional properties of proteins using various data sources of relational data is a typical collective classification (CC) problem in machine learning. The protein functional prediction task with limited annotation is then cast into a semi-supervised multi-label collective classification (SMCC) framework. As such, we propose a novel generative model based SMCC algorithm, called GM-SMCC, to effectively compute the label probability distributions of unannotated protein instances and predict their functional properties. To further boost the predicting performance, we extend the method in an ensemble manner, called EGM-SMCC, by utilizing multiple heterogeneous networks with various latent linkages constructed to explicitly model the relationships among the nodes for effectively propagate the supervision knowledge from labeled to unlabeled nodes.ConclusionExperimental results on a yeast gene dataset predicting the functions and localization of proteins demonstrate the effectiveness of the proposed method. In the comparison, we find that the performances of the proposed algorithms are better than the other compared algorithms.

Kernelized Bayesian Matrix Factorization.

We extend kernelized matrix factorization with a full-Bayesian treatment and with an ability to work with multiple side information sources expressed as different kernels. Kernels have been introduced to integrate side information about the rows and columns, which is necessary for making out-of-matrix predictions. We discuss specifically binary output matrices but extensions to realvalued matrices are straightforward. We extend the state of the art in two key aspects: (i) A full-conjugate probabilistic formulation of the kernelized matrix factorization enables an efficient variational approximation, whereas full-Bayesian treatments are not computationally feasible in the earlier approaches. (ii) Multiple side information sources are included, treated as different kernels in multiple kernel learning which additionally reveals which side sources are informative. We then show that the framework can also be used for supervised and semi-supervised multilabel classification and multi-output regression, by considering samples and outputs as the domains where matrix factorization operates. Our method outperforms alternatives in predicting drug-protein interactions on two data sets. On multilabel classification, our algorithm obtains the lowest Hamming losses on 10 out of 14 data sets compared to five state-of-the-art multilabel classification algorithms. We finally show that the proposed approach outperforms alternatives in multi-output regression experiments on a yeast cell cycle data set.

Predict Subcellular Locations of Singleplex and Multiplex Proteins by Semi-Supervised Learning and Dimension-Reducing General Mode of Chou's PseAAC

Predicting protein subcellular location is one of major challenges in Bioinformatics area since such knowledge helps us understand protein functions and enables us to select the targeted proteins during drug discovery process. While many computational techniques have been proposed to improve predictive performance for protein subcellular location, they have several shortcomings. In this work, we propose a method to solve three main issues in such techniques; i) manipulation of multiplex proteins which may exist or move between multiple cellular compartments, ii) handling of high dimensionality in input and output spaces and iii) requirement of sufficient labeled data for model training. Towards these issues, this work presents a new computational method for predicting proteins which have either single or multiple locations. The proposed technique, namely iFLAST-CORE, incorporates the dimensionality reduction in the feature and label spaces with co-training paradigm for semi-supervised multi-label classification. For this purpose, the Singular Value Decomposition (SVD) is applied to transform the high-dimensional feature space and label space into the lower-dimensional spaces. After that, due to limitation of labeled data, the co-training regression makes use of unlabeled data by predicting the target values in the lower-dimensional spaces of unlabeled data. In the last step, the component of SVD is used to project labels in the lower-dimensional space back to those in the original space and an adaptive threshold is used to map a numeric value to a binary value for label determination. A set of experiments on viral proteins and gram-negative bacterial proteins evidence that our proposed method improve the classification performance in terms of various evaluation metrics such as Aiming (or Precision), Coverage (or Recall) and macro F-measure, compared to the traditional method that uses only labeled data.

Semi-supervised multi-label classification algorithm based on local learning

Semi-supervised multi-label classification algorithm based on local learning

Semi-supervised multi-label image classification based on nearest neighbor editing

Semi-supervised multi-label classification has been applied to many real-world applications such as image classification, document classification and so on. In semi-supervised learning, unlabeled samples are added to the training set for enhancing the classification performance, however, noises are introduced simultaneously. In order to reduce this negative effect, the nearest neighbor data editing technique is introduced to semi-supervised multi-label classification, and thus an algorithm named Multi-Label Self-Training with Editing (MLSTE) is proposed in this work. The proposed algorithm is able to solve the uncertainty problem in semi-supervised multi-label classification to some extent, by improving the performance of determining the label number and selecting confident samples during the course of semi-supervised learning. Extensive experimental results on several benchmark datasets have been carried out to verify the effectiveness of the proposed MLSTE algorithm.