Semi-supervised Classification Framework Research Articles

Robust semi-supervised support vector machines with Laplace kernel-induced correntropy loss functions

The insufficiency and contamination of supervision information are the main factors affecting the performance of support vector machines (SVMs) in real-world applications. To address this issue, novel correntropy loss functions and Laplacian SVM (LapSVM) are utilized for robust semi-supervised classification. It is known that correntropy loss functions have been used in robust learning and achieved promising results. However, the potential for more diverse priors has not been extensively explored. In this paper, a correntropy loss function induced from Laplace kernel function, called LK-loss, is applied to LapSVM for the construction of robust semi-supervised classifier. Properties of LK-loss are demonstrated including robustness, symmetry, boundedness, Fisher consistency and asymptotic approximation behaviors. Moreover, the asymmetric version of LK-loss is introduced to further improve the performance. Concave-convex procedure (CCCP) technique is used to handle the non-convexity of Laplace kernel-induced correntropy loss functions iteratively. Experimental results show that in most cases, the proposed methods have better generalization performance than the comparing ones, which demonstrate the feasibility and effectiveness of the proposed semi-supervised classification framework.

Variational Information Bottleneck for Semi-Supervised Classification.

In this paper, we consider an information bottleneck (IB) framework for semi-supervised classification with several families of priors on latent space representation. We apply a variational decomposition of mutual information terms of IB. Using this decomposition we perform an analysis of several regularizers and practically demonstrate an impact of different components of variational model on the classification accuracy. We propose a new formulation of semi-supervised IB with hand crafted and learnable priors and link it to the previous methods such as semi-supervised versions of VAE (M1 + M2), AAE, CatGAN, etc. We show that the resulting model allows better understand the role of various previously proposed regularizers in semi-supervised classification task in the light of IB framework. The proposed IB semi-supervised model with hand-crafted and learnable priors is experimentally validated on MNIST under different amount of labeled data.

Pointwise manifold regularization for semi-supervised learning

Manifold regularization (MR) provides a powerful framework for semi-supervised classification using both the labeled and unlabeled data. It constrains that similar instances over the manifold graph should share similar classification outputs according to the manifold assumption. It is easily noted that MR is built on the pairwise smoothness over the manifold graph, i.e., the smoothness constraint is implemented over all instance pairs and actually considers each instance pair as a single operand. However, the smoothness can be pointwise in nature, that is, the smoothness shall inherently occur “everywhere” to relate the behavior of each point or instance to that of its close neighbors. Thus in this paper, we attempt to develop a pointwise MR (PW_MR for short) for semi-supervised learning through constraining on individual local instances. In this way, the pointwise nature of smoothness is preserved, and moreover, by considering individual instances rather than instance pairs, the importance or contribution of individual instances can be introduced. Such importance can be described by the confidence for correct prediction, or the local density, for example. PW_MR provides a different way for implementing manifold smoothness. Finally, empirical results show the competitiveness of PW_MR compared to pairwise MR.

Hyper-Heuristic Framework for Sequential Semi-Supervised Classification Based on Core Clustering

Existing stream data learning models with limited labeling have many limitations, most importantly, algorithms that suffer from a limited capability to adapt to the evolving nature of data, which is called concept drift. Hence, the algorithm must overcome the problem of dynamic update in the internal parameters or countering the concept drift. However, using neural network-based semi-supervised stream data learning is not adequate due to the need for capturing quickly the changes in the distribution and characteristics of various classes of the data whilst avoiding the effect of the outdated stored knowledge in neural networks (NN). This article presents a prominent framework that integrates each of the NN, a meta-heuristic based on evolutionary genetic algorithm (GA) and a core online-offline clustering (Core). The framework trains the NN on previously labeled data and its knowledge is used to calculate the error of the core online-offline clustering block. The genetic optimization is responsible for selecting the best parameters of the core model to minimize the error. This integration aims to handle the concept drift. We designated this model as hyper-heuristic framework for semi-supervised classification or HH-F. Experimental results of the application of HH-F on real datasets prove the superiority of the proposed framework over the existing state-of-the art approaches used in the literature for sequential classification data with evolving nature.

Semi-Supervised Text Classification Framework: An Overview of Dengue Landscape Factors and Satellite Earth Observation.

In recent years there has been an increasing use of satellite Earth observation (EO) data in dengue research, in particular the identification of landscape factors affecting dengue transmission. Summarizing landscape factors and satellite EO data sources, and making the information public are helpful for guiding future research and improving health decision-making. In this case, a review of the literature would appear to be an appropriate tool. However, this is not an easy-to-use tool. The review process mainly includes defining the topic, searching, screening at both title/abstract and full-text levels and data extraction that needs consistent knowledge from experts and is time-consuming and labor intensive. In this context, this study integrates the review process, text scoring, active learning (AL) mechanism, and bidirectional long short-term memory (BiLSTM) networks, and proposes a semi-supervised text classification framework that enables the efficient and accurate selection of the relevant articles. Specifically, text scoring and BiLSTM-based active learning were used to replace the title/abstract screening and full-text screening, respectively, which greatly reduces the human workload. In this study, 101 relevant articles were selected from 4 bibliographic databases, and a catalogue of essential dengue landscape factors was identified and divided into four categories: land use (LU), land cover (LC), topography and continuous land surface features. Moreover, various satellite EO sensors and products used for identifying landscape factors were tabulated. Finally, possible future directions of applying satellite EO data in dengue research in terms of landscape patterns, satellite sensors and deep learning were proposed. The proposed semi-supervised text classification framework was successfully applied in research evidence synthesis that could be easily applied to other topics, particularly in an interdisciplinary context.

A new Feature Reduction Algorithm Based on Fuzzy Rough Relation for the Multi-label Classification

The paper aims to improve the multi-label classification performance using the feature reduction technique. According to the determination of the dependency among features based on fuzzy rough relation, features with the highest dependency score will be retained in the reduction set. The set is subsequently applied to enhance the performance of the multi-label classifier. We investigate the effectiveness of the proposed model againts the baseline via time complexity.
 Keywords: 
 Fuzzy rough relation, label-specific feature, feature reduction set
 References
 [1] Richard Jensen, Chris Cornelis, Fuzzy-Rough Nearest Neighbor Classification and Prediction. Proceedings of the 6th International Conference on Rough Sets and Current Trends in Computing, 2011, 310-319.
 [2] Y.H. Qian, Q. Wang, H.H. Cheng, J.Y. Liang, C.Y. Dang, Fuzzy-Rough feature selection accelerator, Fuzzy Sets Syst. 258 (2014) 61-78.
 [3] Quang-Thuy Ha, Thi-Ngan Pham, Van-Quang Nguyen, Minh-Chau Nguyen, Thanh-Huyen Pham, Tri-Thanh Nguyen, A New Text Semi-supervised Multi-label Learning Model Based on Using the Label-Feature Relations, International Conference on Computational Collective Intelligence, LNAI 11055, Springer, 2018, pp. 403-413.
 [4] Daniel Kostrzewa, Robert Brzeski, The data Dimensionality Reduction and Feature Weighting in the Classification Process Using Forest Optimization Algorithm, ACIIDS, 2019, pp. 97-108.
 [5] Nele Verbiest, Fuzzy Rough and Evolutionary Approaches to Instance Selection, PhD Thesis, Ghent University, 2014.
 [6] Y. Yu, W. Pedrycz, D.Q. Miao, Multi-label classification by exploiting label correlations, Expert syst, Appl. 41 (2014) 2989-3004.
 [7] M.L. Zhang, LIFT: Multi-label learning with label-specific features, IEEE Trans, Pattern Anal, Mach, Intell 37 (2015) 107-120.
 [8] Suping Xu, Xibei Yang, Hualong Yu, Dong-Jun Yu, Jingyu Yang, Eric CC Tsang, Multi-label learning with label-specific feature reduction, Knowledge-Based Systems 104 (2016) 52-61. https://doi.org/10.1080/24751839.2017.1364925.
 [9] Thi-Ngan Pham, Van-Quang Nguyen, Van-Hien Tran, Tri-Thanh Nguyen, Quang-Thuy Ha, A Semi-supervised multi-label classification framework with feature reduction and enrichment, Journal of Information and Telecommunication 1(4) (2017) 305-318.
 [10] M. Ghaemi, M.R. Feizi-Derakhshi, Feature selection using forest optimization algorithm, Pattern Recognition 60 (2016) 121-129.
 [11] M.L. Zhang, Z.H. Zhou, ML-KNN: A lazy learning approach to multi-label learning, Pattern Recognition 40 (2007) 2038-2048.
 [12] M.Z. Ahmad, M.K. Hasan, A New Approach for Computing Zadeh's Extension Principle, MATEMATIKA. 26(1) (2010) 71-81.
 [13] Richard Jensen, Neil Mac Parthaláin and Qiang Shen. Fuzzy-rough data mining (using the Weka data mining suite), A Tutorial, IEEE WCCI 2014, Beijing, China, July 6, 2014.
 [14] D. Dubois, H. Prade, Rough fuzzy sets and fuzzy rough sets, Int. J. Gen. Syst. 17 (1990) 191-209.

A Semi-Supervised and Incremental Modeling Framework for Wafer Map Classification

Wafer map analysis provides critical information for quality control and yield improvement tasks in semiconductor manufacturing. In particular, wafer patterns of gross failing areas (GFA) are important clues to identify the causes of relevant failures during the manufacturing process. In this work, a semi-supervised classification framework is proposed for wafer map analysis, and its application to wafer bin maps with GFA patterns classification is demonstrated. The Ladder network and the semi-supervised variational autoencoder are adopted to classify wafer bin maps in comparison with a standard convolutional neural network (CNN) model on two real-world datasets. The results have illustrated that two semi-supervised models are consistently and substantially better than the CNN model across various training data percentages by effective utilization of the unlabeled data. Active learning and pseudo labeling are also utilized to accelerate the learning curve.

Collaborative learning of lightweight convolutional neural network and deep clustering for hyperspectral image semi-supervised classification with limited training samples

Deep learning provides excellent potentials for hyperspectral images (HSIs) classification, but it is infamous for requiring large amount of labeled samples while the collection of high-quality labels for HSIs is extremely expensive and time-consuming. Furthermore, when the limited training samples are available, deep learning methods may suffer from over-fitting. In this work, we propose a novel collaborative learning framework for semi-supervised HSI classification with joint deep convolutional neural networks and deep clustering. Specifically, a lightweight 3D convolutional neural network (CNN) with much less parameters compared with classical 3D CNNs is designed for deep discriminative feature learning and classification. Then a deep clustering method, that is approximate rank-order clustering (AROC) algorithm, is applied to cluster deep features to generate pseudo labels for abundant unlabeled samples. Finally, we fine-tune the lightweight 3D CNN by minimizing a dual-loss (softmax loss and center loss) using both true and pseudo labels. Experimental results on three challenging HSI datasets demonstrate that the proposed method can achieve better performance than other state-of-the-art deep learning based methods and traditional HSI classification methods methods.

A Novel Robust Semisupervised Classification Framework for Quality-Related Coupling Faults in Manufacturing Industries

An imbalanced number of faulty and normal samples make the traditional supervised classification methods difficult to ensure their classification performance. Accordingly, semisupervised learning methods have recently become hotspots both in academic research and practical application domains. Different from previous schemes, this paper dedicates on the correlations, common features, and specific features among quality-related coupling faults in manufacturing industries. The main innovations are as follows: first, it is the first time to develop a robust semisupervised classification framework for quality-related coupling faults, which integrates semisupervised multitask feature selection and manifold learning; second, manifold structures and local discriminant information of unlabeled and limited labeled faulty samples are sufficiently explored to improve the classification performance; and third, correlations among quality-related coupling faults are accurately captured, which are crucial for understanding the uniqueness and relationships of them at the feature level. The proposed method is finally validated in a representative manufacturing industry, i.e., hot strip mill process, where detailed simulation processes are presented and better classification performance is shown compared with the existing approaches.

Hierarchical clustering and stochastic distance for indirect semi-supervised remote sensing image classification

Usually, image classification methods have supervised or unsupervised learning paradigms. While unsupervised methods do not need training data, the meanings behind the classified elements are not explicitly know. Conversely, supervised methods are able to provide classification results with an intrinsic meaning, since a labeled dataset is available for training, which may be a limitation in some cases. The semi-supervised learning paradigm, which simultaneously exploits both labeled and unlabeled data, may be an alternative to this dilemma. This work proposes a semi-supervised classification framework through the combination of the Hierarchical Divisive Algorithm and stochastic distance concepts, where the former is adopted to automatically determine clusters in the data and the latter is used to label such clusters in a supervised way. In order to verify the potential of the proposed framework, two case studies about land use and land cover classification were carried out in an Amazonian area using synthetic aperture radar and multispectral data acquired by ALOS PALSAR and LANDSAT-5 TM sensors. Supervised methods based on statistical concepts were also included in these studies as baselines. The results show that when very small training sets are available, the proposed method provides results up to 14.6% and 3.8% more accurate than the baselines with respect to the classification of TM and PALSAR images, respectively.

A Highly Accurate Framework for Self-Labeled Semisupervised Classification in Industrial Applications

Self-labeled technique, a paradigm of semisupervised classification (SSC), is highly effective in alleviating the shortage of labeled data in classification tasks via an iterative self-labeling process. Although existing self-labeled SSC models show great prospect in industrial applications, they suffer from performance degeneration caused by false-positive label-predictions of unlabeled data during the iterative self-labeling process. For addressing this issue, this paper proposes a novel SSC framework, which is highly compatible with most existing self-labeled SSC models. The main idea of this framework is to incorporate a differential-evolution-based positioning optimization algorithm for classification into the iterative self-labeling process, aiming at optimizing the positioning of newly labeled data. Specifically, five representative self-labeled SSC models with different characteristics are modified based on the proposed framework to check their performances. Experimental results on 45 benchmark datasets demonstrate that the proposed framework is highly compatible with tested self-labeled SSC models, and significantly effective in improving their performances.

Semi-supervised manifold regularization with adaptive graph construction

Manifold regularization (MR) provides a powerful framework for semi-supervised classification (SSC) learning. It imposes the smoothness constraint over a constructed manifold graph, and its performance largely depends on such graph. However, 1) The manifold graph is usually pre-constructed before classification, and fixed during the classification learning process. As a result, independent with the subsequent classification, the graph does not necessarily benefit the classification performance. 2) There are parameters needing tuning in the graph construction, while parameter selection in semi-supervised learning is still an open problem currently, which sets up another barrier for constructing a “well-performing” manifold graph benefiting the performance. To address those issues, we develop a novel semi-supervised manifold regularization with adaptive graph (AGMR for short) in this paper by integrating the graph construction and classification learning into a unified framework. In this way, the manifold graph along with its parameters will be optimized in learning rather than pre-defined, consequently, it will be adaptive to the classification, and benefit the performance. Further, by adopting the entropy and sparse constraints respectively for the graph weights, we derive two specific methods called AGMR_entropy and AGMR_sparse, respectively. Our empirical results show the competitiveness of those AGMRs compared to MR and some of its variants.

A semi-supervised multi-label classification framework with feature reduction and enrichment

ABSTRACTMulti-label classification (MLC) has drawn much attention thanks to its usefulness and omnipresence in real-world applications in which objects may be characterized by more than one label as in the traditional approach. Getting multi-label examples is costly and time-consuming; therefore, semi-supervised learning approach should be considered to take advantages of both labelled and unlabelled data. In this work, we propose a semi-supervised MLC algorithm exploiting the specific features of the prominent class label(s) chosen by a greedy approach as an extension of the LIFT algorithm, and unlabelled data consumption mechanism from the TESC algorithm. We also make a semi-supervised MLC application framework for Vietnamese texts with several feature enrichment steps including (a) a stage of enriching features by adding hidden topic features and (b) a stage of dimensional reduction for subtracting irrelevant features. Experimental results on a dataset of hotel reviews (for tourism) indicate that a reasonable amount of unlabelled data helps to increase the F1 score. Interestingly, with a small amount of labelled data, our algorithm can reach a comparative performance to the case of using a larger amount of labelled data.

Self-training semi-supervised classification based on density peaks of data

Abstract Having a multitude of unlabeled data and few labeled ones is a common problem in many practical applications. A successful methodology to tackle this problem is self-training semi-supervised classification. In this paper, we introduce a method to discover the structure of data space based on find of density peaks. Then, a framework for self-training semi-supervised classification, in which the structure of data space is integrated into the self-training iterative process to help train a better classifier, is proposed. A series of experiments on both artificial and real datasets are run to evaluate the performance of our proposed framework. Experimental results clearly demonstrate that our proposed framework has better performance than some previous works in general on both artificial and real datasets, especially when the distribution of data is non-spherical. Besides, we also find that the support vector machine is particularly suitable for our proposed framework to play the role of base classifier.

Semisupervised Hyperspectral Image Classification Using Small Sample Sizes

Hyperspectral image classification is a challenging task when only a small number of labeled samples are available due to the difficult, expensive, and time-consuming ground campaigns required to collect the ground-truth information. It is also known that the classification performance is highly dependent on the size of the labeled data. In this letter, a semisupervised learning-based hyperspectral image classification framework is proposed as a solution to these problems. One of the contributions of this letter is the selection of the initial labeled training samples with a subtractive clustering-based approach, which provides the most informative samples for graph-based self-training. Another contribution is the decision-level combination of results obtained by support vector machines and kernel sparse representation classifiers. Additionally, a combination of the spatial and spectral information by creating a window structure is also proposed via integrating contextual information from the neighboring pixels. The explanatory experiments confirm that the proposed framework offers better and more promising results, even using a small number of initial labeled samples.

Semi-supervised classification by discriminative regularization

One basic assumption in graph-based semi-supervised classification is manifold assumption, which assumes nearby samples should have similar outputs (or labels). However, manifold assumption may not always hold for samples lying nearby but across the boundary of different classes. As a consequence, samples close to the boundary are quite likely to be misclassified. In this paper, we introduce an approach called semi-supervised classification by discriminative regularization (SSCDR for short) to address this problem. SSCDR first constructs a k nearest neighborhood graph to capture the local manifold structure of samples, and a discriminative graph to encode the discriminative information derived from constrained clustering on labeled and unlabeled samples. Next, it separately treats the discriminative graph and the neighborhood graph in a discriminative regularization framework for semi-supervised classification, and forces nearby samples across the boundary to have different labels. Experimental results on various datasets collected from UCI, LibSVM and facial image datasets demonstrate that SSCDR achieves better performance than other related methods, and it is also robust to the input values of parameter k.

Semi-supervised classification framework of hyperspectral images based on the fusion evidence entropy

Increasing attention is being paid to the classification of ground objects using hyperspectral spectrometer images. A key challenge of most hyperspectral classifications is the cost of training samples. It is difficult to acquire enough effective marked label sets using classification model frameworks. In this paper, a semi-supervised classification framework of hyperspectral images is proposed to better solve problems associated with hyperspectral image classification. The proposed method is based on an iteration process, making full use of the small amount of labeled data in a sample set. In addition, a new unlabeled data trainer in the self-training semi-supervised learning framework is explored and implemented by estimating the fusion evidence entropy of unlabeled samples using the minimum trust evaluation and maximum uncertainty. Finally, we employ different machine learning classification methods to compare the classification performance of different hyperspectral images. The experimental results indicate that the proposed approach outperforms traditional state-of-the-art methods in terms of low classification errors and better classification charts using few labeled samples.

A semi-supervised multi-label classification framework with feature reduction and enrichment

ABSTRACTMulti-label classification (MLC) has drawn much attention, thanks to its usefulness and omnipresence in real-world applications in which objects may be characterized by more than one label as in the traditional approach. Getting multi-label examples is costly and time-consuming; therefore, semi-supervised learning approach should be considered to take advantages of both labelled and unlabelled data. In this work, we propose a semi-supervised MLC algorithm exploiting the specific features of the prominent class label(s) chosen by a greedy approach as an extension of the LIFT algorithm, and unlabelled data consumption mechanism from the TExt classification using Semi-supervised Clustering algorithm. We also make a semi-supervised MLC application framework for Vietnamese texts with several feature enrichment steps including (a) a stage of enriching features by adding hidden topic features and (b) a stage of dimensional reduction for subtracting irrelevant features. Experimental results on a data set of hotel reviews (for tourism) indicate that a reasonable amount of unlabelled data helps to increase the F1 score. Interestingly, with a small amount of labelled data, our algorithm can reach a comparative performance to the case of using a larger amount of labelled data.

Robust Semi-Supervised Classification for Noisy Labels Based on Self-Paced Learning

Data labeling is a tedious and subjective task that can be time consuming and error-prone; however, most learning algorithms are sensitive to noisy labels. This problem raises the need to develop algorithms that can exploit large amount of unlabeled data and also be robust to noisy label information. In this letter, we propose a novel semi-supervised classification framework that is robust to noisy labels, named self-paced manifold regularization. The proposed framework naturally integrates self-paced learning regime into the manifold regularization framework for selecting labeled training samples in a theoretically sound manner, and utilizes locally linear reconstructions to control the smoothness of the classifier with respect to the manifold structure of data. Finally, the alternative search strategy is adopted for the proposed framework to obtain the classifier. The proposed method can not only suppress the negative effect of noisy initial labels in semi-supervised learning, but also obtain an explicit multiclass classifier for newly coming data points. Experimental results demonstrate the effectiveness of the proposed method.

Enhanced manifold regularization for semi-supervised classification.

Manifold regularization (MR) has become one of the most widely used approaches in the semi-supervised learning field. It has shown superiority by exploiting the local manifold structure of both labeled and unlabeled data. The manifold structure is modeled by constructing a Laplacian graph and then incorporated in learning through a smoothness regularization term. Hence the labels of labeled and unlabeled data vary smoothly along the geodesics on the manifold. However, MR has ignored the discriminative ability of the labeled and unlabeled data. To address the problem, we propose an enhanced MR framework for semi-supervised classification in which the local discriminative information of the labeled and unlabeled data is explicitly exploited. To make full use of labeled data, we firstly employ a semi-supervised clustering method to discover the underlying data space structure of the whole dataset. Then we construct a local discrimination graph to model the discriminative information of labeled and unlabeled data according to the discovered intrinsic structure. Therefore, the data points that may be from different clusters, though similar on the manifold, are enforced far away from each other. Finally, the discrimination graph is incorporated into the MR framework. In particular, we utilize semi-supervised fuzzy c-means and Laplacian regularized Kernel minimum squared error for semi-supervised clustering and classification, respectively. Experimental results on several benchmark datasets and face recognition demonstrate the effectiveness of our proposed method.