Semi-supervised Classification Approach Research Articles

Hyper-Laplacian Regularized Multilinear Multiview Self-Representations for Clustering and Semisupervised Learning.

In this paper, we address the multiview nonlinear subspace representation problem. Traditional multiview subspace learning methods assume that the heterogeneous features of the data usually lie within the union of multiple linear subspaces. However, instead of linear subspaces, the data feature actually resides in multiple nonlinear subspaces in many real-world applications, resulting in unsatisfactory clustering performance. To overcome this, we propose a hyper-Laplacian regularized multilinear multiview self-representation model, which is referred to as HLR-M2VS, to jointly learn multiple views correlation and a local geometrical structure in a unified tensor space and view-specific self-representation feature spaces, respectively. In unified tensor space, a well-founded tensor low-rank regularization is adopted to impose on the self-representation coefficient tensor to ensure global consensus among different views. In view-specific feature space, hypergraph-induced hyper-Laplacian regularization is utilized to preserve the local geometrical structure embedded in a high-dimensional ambient space. An efficient algorithm is then derived to solve the optimization problem of the established model with theoretical convergence guarantee. Furthermore, the proposed model can be extended to semisupervised classification without introducing any additional parameters. An extensive experiment of our method is conducted on many challenging datasets, where a clear advance over state-of-the-art multiview clustering and multiview semisupervised classification approaches is achieved.

A novel semi supervised approach for text classification

Text categorization, also known as text classification is a supervised classification problem. It aims to assign a predefined class label or group to a new or unknown text document. Most of the time we need a collection of large data from each class to train the classifier. It may be noted that, it is very hard or expensive to collect labelled text data. In most cases we assign the label manually which is neither cost effective nor efficient. In this paper, we have introduced a semi-supervised classification approach where the learner needs very small amount of labelled data with a large amount of unlabeled data to assign a class label to a new or unknown text document. The proposed method uses Kohonen self organizing map (SOM) for labelling the unlabeled data and three classifiers namely support vector machine (SVM), Naive Bayes (NB), and decision tree (DT): classification and regression tree (CART) for observing the accuracy of classification. The experimental results obtained show the effectiveness of our proposed method.

A Collective Learning Approach for Semi-Supervised Data Classification

A Collective Learning Approach for Semi-Supervised Data Classification

Divide and conquer approach for semi-supervised multi-category classification through localized kernel spectral clustering

In this paper, we propose ‘divide-and-conquer approach for multi-category semi-supervised’ (DAC-MSS) classification and a novel semi-supervised binary classifier termed as ‘twin support vector machine with localized kernel spectral clustering’ (TW-LKSC). DAC-MSS builds a multi-category classifier model organized in the form of a tree of binary classifiers. The tree consists of several TW-LKSC classifiers which use a training set consisting of few labeled samples and rest unlabeled samples to generate a pair of hyperplanes, by solving a system of linear equations. The propagation of labels to unlabeled patterns is achieved through localized kernel spectral clustering (LKSC) which is the core clustering model embedded in TW-LKSC. TW-LKSC also employs cluster prototype to localize the generation of hyperplanes and prevents them from extending infinitely. The strength of DAC-MSS is its better classification accuracy and improved learning time, due to divide and conquer approach, as compared to one-against-all based semi-supervised classification algorithms. This is proved experimentally for benchmark UCI datasets. We have applied DAC-MSS for color image segmentation of images from Berkley Segmentation Dataset.

Multi-attribute and relational learning via hypergraph regularized generative model

Abstract The real-world networking data may contain different types of attribute views and relational view. Hence, it is desirable to collectively use available attribute views and relational view in order to build effective learning models. We call this framework multi-attribute and relational learning. Collective classification is one of the popular approaches that can handle both attribute and relational information for network data. However, in collective classification only one type of attribute and relational view is involved and little attention is received for multi-attribute and relational learning. In this paper, we propose a new semi-supervised collective classification approach, called hypergraph regularized generative model (HRGM), for multi-attribute and relational learning. In the approach, a generative model based on the Probabilistic Latent Semantic Analysis (PLSA) method is developed to leverage attribute information, and a hypergraph regularizer is incorporated to effectively exploit higher-order relational information among the data samples. Experimental results on various data sets have demonstrated the effectiveness of the proposed HRGM, and revealed that our approach outperforms existing collective classification methods and multi-view classification methods in terms of accuracy.

Incremental Semi-Supervised classification of data streams via self-representative selection

Incremental learning has been developed for supervised classification, where knowledge is accumulated incrementally and represented in the learning process. However, labeling sufficient samples in each data chunk is of high cost, and incremental technologies are seldom discussed in the semi-supervised paradigm. In this paper we advance an Incremental Semi-Supervised classification approach via Self-Representative Selection (IS3RS) for data streams classification, by exploring both the labeled and unlabeled dynamic samples. An incremental self-representative data selection strategy is proposed to find the most representative exemplars from the sequential data chunk. These exemplars are incrementally labeled to expand the training set, and accumulate knowledge over time to benefit future prediction. Extensive experimental evaluations on some benchmarks have demonstrated the effectiveness of the proposed framework.

NMFE-SSCC: Non-negative matrix factorization ensemble for semi-supervised collective classification

Collective classification (CC) is a task to jointly classifying related instances of network data. Enabling CC usually improves the performance of predictive models on fully-labeled training networks with large amount of labeled data. However, acquiring such labels can be difficult and costly, and learning a CC classifier with only a few labeled data can lead to poor performance. On the other hand, there are usually large amount of unlabeled data available in practical. This naturally motivates semi-supervised collective classification (SSCC) approaches for leveraging the unlabeled data to improve CC from a sparsely-labeled network. In this paper, we propose a novel non-negative matrix factorization (NMF) based SSCC algorithm, called NMF-SSCC, to effectively learn a data representation by exploiting both labeled and unlabeled data on the network. Our idea is to use matrix factorization to obtain a compact representation of network data which uncovers the class discrimination of the data inferred from the labeled instances and simultaneously respects the intrinsic network structure. To achieve this, we design a new matrix factorization objective function and incorporate a label matrix factorization term as well as a network regularization term into it. An efficient optimization algorithm using the multiplicative updating rules is then developed to solve the new objective function. To further boost the predicting performance, we extend the proposed NMF-SSCC method into an ensemble scheme, called NMFE-SSCC, in terms of building a classification ensemble with a set of NMF-SSCC collective classifiers using different constructed latent graphs. Each NMF-SSCC classifier is learnt from one latent graph generated with various latent linkages for effectively label propagation. Experimental results on real-world data sets have demonstrated the effectiveness of the new methods.

A multi-label, semi-supervised classification approach applied to personality prediction in social media

Social media allow web users to create and share content pertaining to different subjects, exposing their activities, opinions, feelings and thoughts. In this context, online social media has attracted the interest of data scientists seeking to understand behaviours and trends, whilst collecting statistics for social sites. One potential application for these data is personality prediction, which aims to understand a user’s behaviour within social media. Traditional personality prediction relies on users’ profiles, their status updates, the messages they post, etc. Here, a personality prediction system for social media data is introduced that differs from most approaches in the literature, in that it works with groups of texts, instead of single texts, and does not take users’ profiles into account. Also, the proposed approach extracts meta-attributes from texts and does not work directly with the content of the messages. The set of possible personality traits is taken from the Big Five model and allows the problem to be characterised as a multi-label classification task. The problem is then transformed into a set of five binary classification problems and solved by means of a semi-supervised learning approach, due to the difficulty in annotating the massive amounts of data generated in social media. In our implementation, the proposed system was trained with three well-known machine-learning algorithms, namely a Naïve Bayes classifier, a Support Vector Machine, and a Multilayer Perceptron neural network. The system was applied to predict the personality of Tweets taken from three datasets available in the literature, and resulted in an approximately 83% accurate prediction, with some of the personality traits presenting better individual classification rates than others.

A Comparison of Semi-Supervised Classification Approaches for Software Defect Prediction

AbstractPredicting the defect-prone modules when the previous defect labels of modules are limited is a challenging problem encountered in the software industry. Supervised classification approaches cannot build high-performance prediction models with few defect data, leading to the need for new methods, techniques, and tools. One solution is to combine labeled data points with unlabeled data points during learning phase. Semi-supervised classification methods use not only labeled data points but also unlabeled ones to improve the generalization capability. In this study, we evaluated four semi-supervised classification methods for semi-supervised defect prediction. Low-density separation (LDS), support vector machine (SVM), expectation-maximization (EM-SEMI), and class mass normalization (CMN) methods have been investigated on NASA data sets, which are CM1, KC1, KC2, and PC1. Experimental results showed that SVM and LDS algorithms outperform CMN and EM-SEMI algorithms. In addition, LDS algorithm performs much better than SVM when the data set is large. In this study, the LDS-based prediction approach is suggested for software defect prediction when there are limited fault data.

ONLINE SEMI-SUPERVISED GROWING NEURAL GAS

In this paper we introduce online semi-supervised growing neural gas (OSSGNG), a novel online semi-supervised classification approach based on growing neural gas (GNG). Existing semi-supervised classification approaches based on GNG require that the training data is explicitly stored as the labeling is performed a posteriori after the training phase. As main contribution, we present an approach that relies on online labeling and prediction functions to process labeled and unlabeled data uniformly and in an online fashion, without the need to store any of the training examples explicitly. We show that using on-the-fly labeling strategies does not significantly deteriorate the performance of classifiers based on GNG, while circumventing the need to explicitly store training examples. Armed with this result, we then present a semi-supervised extension of GNG (OSSGNG) that relies on the above mentioned online labeling functions to label unlabeled examples and incorporate them into the model on-the-fly. As an important result, we show that OSSGNG performs as good as previous semi-supervised extensions of GNG which rely on offline labeling strategies. We also show that OSSGNG compares favorably to other state-of-the-art semi-supervised learning approaches on standard benchmarking datasets.

Integrating Spectral Kernel Learning and Constraints in Semi-Supervised Classification

Recently, integrating new knowledge sources such as pairwise constraints into various classification tasks with insufficient training data has been actively studied in machine learning. In this paper, we propose a novel semi-supervised classification approach, called semi-supervised classification with enhanced spectral kernel, which can simultaneously handle both sparse labeled data and additional pairwise constraints together with unlabeled data. Specifically, we first design a non-parameter spectral kernel learning model based on the squared loss function. Then we develop an efficient semi-supervised classification algorithm which takes advantage of Laplacian spectral regularization: semi-supervised classification with enhanced spectral kernel under the squared loss (ESKS). Finally, we conduct many experiments on a variety of synthetic and real-world data sets to demonstrate the effectiveness of the proposed ESKS algorithm.

A probabilistic approach for semi-supervised nearest neighbor classification

In supervised classification, we learn from a training set of labeled observations to form a decision rule for classifying all unlabeled test cases. But if the training sample is small, one may fail to extract sufficient information from that sample to develop a good classifier. Because of the statistical instability of nonparametric methods, this problem becomes more evident in the case of nonparametric classification. In such cases, if one can extract useful information also from unlabeled test cases and use that to modify the classification rule, the performance of the resulting classifier can be improved substantially. In this article, we use a probabilistic framework to develop such methods for nearest neighbor classification. The resulting classifiers, called semi-supervised or transductive classifiers, usually perform better than supervised methods, especially when the training sample is small. Some benchmark data sets are analyzed to show the utility of these proposed methods.

Improving ChIP-seq peak-calling for functional co-regulator binding by integrating multiple sources of biological information

BackgroundChromatin immunoprecipitation coupled with massively parallel sequencing (ChIP-seq) is increasingly being applied to study genome-wide binding sites of transcription factors. There is an increasing interest in understanding the mechanism of action of co-regulator proteins, which do not bind DNA directly, but exert their effects by binding to transcription factors such as the estrogen receptor (ER). However, due to the nature of detecting indirect protein-DNA interaction, ChIP-seq signals from co-regulators can be relatively weak and thus biologically meaningful interactions remain difficult to identify.ResultsIn this study, we investigated and compared different statistical and machine learning approaches including unsupervised, supervised, and semi-supervised classification (self-training) approaches to integrate multiple types of genomic and transcriptomic information derived from our experiments and public database to overcome difficulty of identifying functional DNA binding sites of the co-regulator SRC-1 in the context of estrogen response. Our results indicate that supervised learning with naïve Bayes algorithm significantly enhances peak calling of weak ChIP-seq signals and outperforms other machine learning algorithms. Our integrative approach revealed many potential ERα/SRC-1 DNA binding sites that would otherwise be missed by conventional peak calling algorithms with default settings.ConclusionsOur results indicate that a supervised classification approach enables one to utilize limited amounts of prior knowledge together with multiple types of biological data to enhance the sensitivity and specificity of the identification of DNA binding sites from co-regulator proteins.

Semi-supervised Learning Using Local Regularizer and Unit Circle Class Label Representation

Semi-supervised learning, which aims to learn from partially labeled data and mostly unlabeled data, has been attracted more and more attention in machine learning and pattern recognition. A novel semi-supervised classification approach is proposed, which can not only handle semi-supervised binary classification problem but also deal with semi-supervised multi-class classification problem. The approach is based on local regularizer and unit circle class label representation. The former is minimized so as to cause the class labels to have the desired properties. The latter utilizes two-dimensional vector evenly distributed in circumference of unit circle to represent class label, so multi-class classification can be performed only once. Comparative classification experiments on some benchmark datasets validate the effectiveness of the presented approach.

Tri-Training and Data Editing Based Semi-Supervised Clustering Algorithm

In this paper, a algorithm named DE-Tri-training semi-supervised K-means is proposed, which could get a seeds set of larger scale and less noise. In detail, prior to using the seeds set to initialize cluster centroids, the training process of a semi-supervised classification approach named Tri-training is used to label unlabeled data and add them into the initial seeds set to enlarge the scale. Meanwhile, to improve the quality of the enlarged seeds set, a nearest neighbor rule based data editing technique named Depuration is introduced into Tri-training process to eliminate and correct the mislabeled noise data in the enlarged seeds. Experimental results show that the novel semi-supervised clustering algorithm could effectively improve the cluster centroids initialization and enhance clustering performance.