Graph-based Semi-supervised Learning Method Research Articles

Quantum annealing for semi-supervised learning

Recent advances in quantum technology have led to the development and the manufacturing of programmable quantum annealers that promise to solve certain combinatorial optimization problems faster than their classical counterparts. Semi-supervised learning is a machine learning technique that makes use of both labeled and unlabeled data for training, which enables a good classifier with only a small amount of labeled data. In this paper, we propose and theoretically analyze a graph-based semi-supervised learning method with the aid of the quantum annealing technique, which efficiently utilizes the quantum resources while maintaining good accuracy. We illustrate two classification examples, suggesting the feasibility of this method even with a small portion (30%) of labeled data involved.

Predicting essential genes of 41 prokaryotes by a semi-supervised method

Essential genes are vitally important to the survival and reproduction of organisms. Many machine learning methods have been widely employed to predict essential genes and have obtained satisfactory results. However, most of these methods are supervised methods and may not obtain the desired result when the labeled data are insufficient. In this paper, we proposed a learning with local and global consistency (LGC) method-based classifier, which was employed to predict the essential genes of 41 prokaryotes. LGC is a graph-based semi-supervised learning method that can construct a prediction model using finite label and constraint information. The performance of the proposed classifier was evaluated by employing intra-organism prediction and leave-one-species-out validation. The average AUC value of 41 organisms in intra-organisms prediction was 0.723 when the labeled sample ratio was 0.5. The results of this study indicate that the proposed method can achieve acceptable prediction performance with limited labeled data. Additionally, the results demonstrate that this method has good universality.

Semi-Supervised Learning with Auto-Weighting Feature and Adaptive Graph

Traditional graph-based Semi-Supervised Learning (SSL) methods usually contain two separate steps. First, constructing an affinity matrix. Second, inferring the unknown labels. While such a two-step method has been successful, it cannot take full advantage of the correlation between affinity matrix and label information. In order to address the above problem, we propose a novel graph-based SSL method. It can learn the affinity matrix and infer the unknown labels simultaneously. Moreover, feature selection with auto-weighting is introduced to extract the effective and robust features. Further, the proposed method learns the data similarity matrix by assigning the adaptive neighbors for each data point based on the local distance. We solve the unified problem via an alternative minimization algorithm. Extensive experimental results on synthetic data and benchmark data show that the proposed method consistently outperforms the state-of-the-art approaches.

Graph-based semi-supervised learning: A review

Considering the labeled samples may be difficult to obtain because they require human annotators, special devices, or expensive and slow experiments. Semi-supervised learning (SSL) has tremendous practical value. Moreover, graph-based SSL methods have received more attention since their convexity, scalability and effectiveness in practice. The convexity of graph-based SSL guarantees that the optimization problems become easier to obtain local solution than the general case. The scalable graph-based SSL methods are convenient to deal with large-scale dataset for big data. Graph-based SSL methods aim to learn the predicted function for the labels of those unlabeled samples by exploiting the label dependency information reflected by available label information. The main purpose of this paper is to provide a comprehensive study of graph-based SSL. Specifically, the concept of the graph is first given before introducing graph-based semi-supervised learning. Then, we build a framework that divides the corresponding works into transductive graph-based SSL, inductive graph-based SSL, and scalable graph-based SSL. The core idea of these models is to impose graph constraints to the optimal function, which guarantees the smoothness over the graph. Next, several representative graph-based SSL methods are conducted on the three data sets, including two face data sets and a natural image data set. Finally, we outlook several directions for future work of graph-based SSL, and hope our review on graph-based SSL will offer insights for further research.

Mixture distribution modeling for scalable graph-based semi-supervised learning

Graph-based semi-supervised learning (SSL) has been widely investigated in recent works considering its powerful ability to naturally incorporate the diverse types of information and measurements. However, traditional graph-based SSL methods have cubic complexities and leading to low scalability. In this paper, we propose to perform graph-based SSL on mixture distribution components, named Mixture-distribution based Graph Smoothing (MGS), to address this challenge. Specifically, the intrinsic distributions of data are captured by a mixture density estimation model. A novel mixture-distribution based objective energy function is further proposed to incorporate few available annotations, which ensures the model complexity is irrelevant to the number of raw instances. The energy function can be simplified and effectively solved by viewing the instances and mixture components as the point clouds. Experiments on large datasets demonstrate the remarkable performance improvements and scalability of the proposed model, which proves the superiority of the MGS model.

Graph Convolution Networks with manifold regularization for semi-supervised learning

In recent times, Graph Convolution Networks (GCN) have been proposed as a powerful tool for graph-based semi-supervised learning. In this paper, we introduce a model that enhances label propagation of Graph Convolution Networks (GCN). More precisely, we propose GCNs with Manifold Regularization (GCNMR). The objective function of the proposed GCNMR is composed by a supervised term and an unsupervised term. The supervised term enforces the fitting term between the predicted labels and the known labels. The unsupervised term imposes the smoothness of the predicted labels of the whole data samples. By learning a Graph Convolution Network with the proposed objective function, we are able to derive a more powerful semi-supervised learning. The proposed model retains the advantages of the classic GCN, yet it can improve it with no increase in time complexity. Experiments on three public image datasets show that the proposed model is superior to the GCN and several competing existing graph-based semi-supervised learning methods.

A GA-Based Multi-View, Multi-Learner Active Learning Framework for Hyperspectral Image Classification

This paper introduces a novel multi-view multi-learner (MVML) active learning method, in which the different views are generated by a genetic algorithm (GA). The GA-based view generation method attempts to construct diverse, sufficient, and independent views by considering both inter- and intra-view confidences. Hyperspectral data inherently owns high dimensionality, which makes it suitable for multi-view learning algorithms. Furthermore, by employing multiple learners at each view, a more accurate estimation of the underlying data distribution can be obtained. We also implemented a spectral-spatial graph-based semi-supervised learning (SSL) method as the classifier, which improved the performance of the classification task in comparison with supervised learning. The evaluation of the proposed method was based on three different benchmark hyperspectral data sets. The results were also compared with other state-of-the-art AL-SSL methods. The experimental results demonstrated the efficiency and statistically significant superiority of the proposed method. The GA-MVML AL method improved the classification performances by 16.68%, 18.37%, and 15.1% for different data sets after 40 iterations.

A Novel Adaptive Multi-View Non-Negative Graph Semi-Supervised ELM

This paper represents a semi-supervised learning framework, which integrates multi-view learning, extreme learning machine (ELM) and graph-based semi-supervised learning. The aim is to expand the scope of adaptation of non-negative sparse graph (NNSG) framework, under a multi-view condition and a non-linear relationship. The proposed multi-view learning method will be adaptive since when data is single-view the framework will degenerate into an embedded framework for NNSG framework. The proposed ELM method also will be adaptive since the number of hidden layer neuron will change with different number of input and output layer neuron. The combination of both proposed methods outperforms traditional graph-based semi-supervised learning, such as flexible manifold embedding (FME) and NNSG framework, which can not establish an affinity matrix for multi-view and can not establish a non-liner model for unknown data. Unlike traditional graph-based semi-supervised learning methods, which only can label propagation and build linear regression models for single or multi-view data, our proposed method has an obvious advantage that is applicable to any single or multi-view data, and builds linear or non-linear models. We provides extensive experiments on four public database in order to evaluate the performance of the proposed method. These experiments demonstrate significant improvement over the state-of-the-art algorithms in label propagation and processing of new data.

Image-based face beauty analysis via graph-based semi-supervised learning

Automatic facial beauty analysis has become an emerging research topic. Despite some achieved advances, current methods and systems suffer from at least two limitations. Firstly, many developed systems rely on the use of ad-hoc hand-crafted features that were designed for generic pattern recognition problems. Secondly, while Deep Convolutional Neural Nets (DCNN) have been recently demonstrated to be a promising area of research in statistical machine learning, their use for automatic face beauty analysis may not guarantee optimal performances due to the use of a limited amount of face images with beauty scores. In this paper, we attempt to overcome these two main limitations by jointly exploiting two tricks. First, instead of using hand-crafted face features we use deep features of a pre-trained DCNN able to generate a high-level representation of a face image. Second, we exploit manifold learning theory and deploy three graph-based semi-supervised learning methods in order to enrich model learning without the need of additional labeled face images. These schemes perform graph-based score propagation. The proposed schemes were tested on three public datasets for beauty analysis: SCUT-FBP, M2B, and SCUT-FBP5500. These experiments, as well as many comparisons with supervised schemes, show that the scheme coined Kernel Flexible Manifold Embedding compares favorably with many supervised schemes. They also show that its performances in terms of error prediction and Pearson Correlation are better than those reported for the used datasets.

A splitting method for the locality regularized semi-supervised subspace clustering

Graph-based semi-supervised learning (G-SSL) methods play an increasingly important role in machine learning systems. Recently, latent low-rank representation (LatLRR) graph has gained great success in subspace clustering. However, LatLRR only considers the global structure, while the local geometric information, which is often important to many real applications, is ignored. In this paper, we propose a locality regularized LatLRR model (LR-LatLRR) for semi-supervised subspace clustering problems. This model incorporates two regularization terms into LatLRR by taking the local structure of data into account. Then, we develop an efficient splitting algorithm for solving LR-LatLRR. In addition, we also prove the global convergence of the proposed algorithm. Furthermore, we extend the LR-LatLRR model to a case of including the non-negative constraint. Finally, we conduct experiments on a synthetic data and several real data sets for the semi-supervised clustering problems. Experimental results show that our method can obtain high classification accuracy and outperforms several state-of-the-art G-SSL methods.

Accelerating Flexible Manifold Embedding for Scalable Semi-Supervised Learning

In this paper, we address the problem of large-scale graph-based semi-supervised learning for multi-class classification. Most existing scalable graph-based semi-supervised learning methods are based on the hard linear constraint or cannot cope with the unseen samples, which limits their applications and learning performance. To this end, we build upon our previous work flexible manifold embedding (FME) [1] and propose two novel linear-complexity algorithms called fast flexible manifold embedding (f-FME) and reduced flexible manifold embedding (r-FME). Both of the proposed methods accelerate FME and inherit its advantages. Specifically, our methods address the hard linear constraint problem by combining a regression residue term and a manifold smoothness term jointly, which naturally provides the prediction model for handling unseen samples. To reduce computational costs, we exploit the underlying relationship between a small number of anchor points and all data points to construct the graph adjacency matrix, which leads to simplified closed-form solutions. The resultant f-FME and r-FME algorithms not only scale linearly in both time and space with respect to the number of training samples but also can effectively utilize information from both labeled and unlabeled data. Experimental results show the effectiveness and scalability of the proposed methods.

ALG: Adaptive low-rank graph regularization for scalable semi-supervised and unsupervised learning

Graph-based semi-supervised learning (SSL) is one of the most popular topics in the past decades. Most graph-based SSL methods utilize two stage-approach to infer the class labels of the unlabeled data, where the first stage is to construct a graph in an unsupervised way for characterizing the geometry of data manifold while the second stage is to perform SSL for classification. However, the graph construction and SSL stages are usually separate. They do not share any common information to enhance the performance of classification. In this paper, we aim to solve the above problem by proposing a unified framework for SSL. In detail, we first adopt an adaptive low-rank model for graph construction, where the coefficient matrix is calculated through an efficient procedure so that the corresponding constructed graph can capture the global structure of data manifold. Meriting from such a graph, we then propose a unified framework for SSL, where we have involved the graph construction, the SSL strategy into the whole objective function. As a result, the class labels learned by SSL can provide more discriminative information for graph construction, while the updated graph can further enhance the classification accuracies of SSL. Simulation indicates that the proposed method can achieve better classification performance compared with other state-of-the-art graph-based methods.

PKGCN: prior knowledge enhanced graph convolutional network for graph-based semi-supervised learning

Graph is a widely existed data structure in many real world scenarios, such as social networks, citation networks and knowledge graphs. Recently, Graph Convolutional Network (GCN) has been proposed as a powerful method for graph-based semi-supervised learning, which has the similar operation and structure as Convolutional Neural Networks (CNNs). However, like many CNNs, it is often necessary to go through a lot of laborious experiments to determine the appropriate network structure and parameter settings. Fully exploiting and utilizing the prior knowledge that nearby nodes have the same labels in graph-based neural network is still a challenge. In this paper, we propose a model which utilizes the prior knowledge on graph to enhance GCN. To be specific, we decompose the objective function of semi-supervised learning on graphs into a supervised term and an unsupervised term. For the unsupervised term, we present the concept of local inconsistency and devise a loss term to describe the property in graphs. The supervised term captures the information from the labeled data while the proposed unsupervised term captures the relationships among both labeled data and unlabeled data. Combining supervised term and unsupervised term, our proposed model includes more intrinsic properties of graph-structured data and improves the GCN model with no increase in time complexity. Experiments on three node classification benchmarks show that our proposed model is superior to GCN and seven existing graph-based semi-supervised learning methods.

Matrix Completion for Graph-Based Deep Semi-Supervised Learning

Convolutional Neural Networks (CNNs) have provided promising achievements for image classification problems. However, training a CNN model relies on a large number of labeled data. Considering the vast amount of unlabeled data available on the web, it is important to make use of these data in conjunction with a small set of labeled data to train a deep learning model. In this paper, we introduce a new iterative Graph-based Semi-Supervised Learning (GSSL) method to train a CNN-based classifier using a large amount of unlabeled data and a small amount of labeled data. In this method, we first construct a similarity graph in which the nodes represent the CNN features corresponding to data points (labeled and unlabeled) while the edges tend to connect the data points with the same class label. In this graph, the missing label of unsupervised nodes is predicted by using a matrix completion method based on rank minimization criterion. In the next step, we use the constructed graph to calculate triplet regularization loss which is added to the supervised loss obtained by initially labeled data to update the CNN network parameters.

Fast Semisupervised Learning With Bipartite Graph for Large-Scale Data.

As the captured information in our real word is very scare and labeling sample is time cost and expensive, semisupervised learning (SSL) has an important application in computer vision and machine learning. Among SSL approaches, a graph-based SSL (GSSL) model has recently attracted much attention for high accuracy. However, for most traditional GSSL methods, the large-scale data bring higher computational complexity, which acquires a better computing platform. In order to dispose of these issues, we propose a novel approach, bipartite GSSL normalized (BGSSL-normalized) method, in this paper. This method consists of three parts. First, the bipartite graph between the original data and the anchor points is constructed, which is parameter-insensitive, scale-invariant, naturally sparse, and simple operation. Then, the label of the original data and anchors can be inferred through the graph. Besides, we extend our algorithm to handle out-of-sample for large-scale data by the inferred label of anchors, which not only retains good classification result but also saves a large amount of time. The computational complexity of BGSSL-normalized can be reduced to O(ndm+nm2) , which is a significant improvement compared with traditional GSSL methods that need O(n2d+n3) , where n , d , and m are the number of samples, features, and anchors, respectively. The experimental results on several publicly available data sets demonstrate that our approaches can achieve better classification accuracy with less time costs.

Low‐rank representation for semi‐supervised software defect prediction

Software defect prediction based on machine learning is an active research topic in the field of software engineering. The historical defect data in software repositories may contain noises because automatic defect collection is based on modified logs and defect reports. When the previous defect labels of modules are limited, predicting the defect-prone modules becomes a challenging problem. In this study, the authors propose a graph-based semi-supervised defect prediction approach to solve the problems of insufficient labelled data and noisy data. Graph-based semi-supervised learning methods used the labelled and unlabelled data simultaneously and consider them as the nodes of the graph at the training phase. Therefore, they solve the problem of insufficient labelled samples. To improve the stability of noisy defect data, a powerful clustering method, low-rank representation (LRR), and neighbourhood distance are used to construct the relationship graph of samples. Therefore, they propose a new semi-supervised defect prediction approach, named low-rank representation-based semi-supervised software defect prediction (LRRSSDP). The widely used datasets from NASA projects and noisy datasets are employed as test data to evaluate the performance. Experimental results show that (i) LRRSSDP outperforms several representative state-of-the-art semi-supervised defect prediction methods; and (ii) LRRSSDP can maintain robustness in noisy environments.

Fast semi-supervised learning with anchor graph for large hyperspectral images

As the labeled samples of hyperspectral image (HSI) are very scarce and labeling sample costs too much time and is expensive, semi-supervised learning (SSL) has an important application in hyperspectral image (HSI) classification. Among SSL approaches, graph-based SSL (GSSL) model has recently attracted much attention. However, most GSSL methods still can not deal with the large HSI as their high computational complexity. In this letter, we propose a novel approach, called fast semi-supervised learning with anchor graph (FSSLAG) to solve the large HSI classification problem. In the proposed FSSLAG algorithm, the anchor graph, which is parameter-free, naturally sparse and scale invariant, is first constructed. Then the label of samples can be inferred through the graph. The computational complexity of FSSLAG can be reduced to O(ndm), which is a significant improvement compared with traditional graph-based SSL methods that need O(n3), where n, d and m are the number of samples, features and anchors, respectively. Several experiments have demonstrated the effectiveness and efficiency of FSSLAG in terms of computational speed and classification accuracy.

Spatial and class structure regularized sparse representation graph for semi-supervised hyperspectral image classification

Constructing a good graph that can capture intrinsic data structures is critical for graph-based semi-supervised learning methods, which are widely applied for hyperspectral image (HSI) classification with small amount of labeled samples.Among the existing graph construction methods, sparse representation (SR)-based methods have shown impressive performance on semi-supervised HSI classification tasks. However, most SR-based algorithms fail to consider the rich spatial information of HSI, which has been shown beneficial for classification tasks.In this paper, we propose a spatial and class structure regularized sparse representation (SCSSR) graph for semi-supervised HSI classification. Specifically, spatial information has been incorporated into SR model via the graph Laplacian regularization, it assumes that the spatial neighbors should have similar representation coefficients, the obtained coefficient matrix thus can reflect the similarity between samples more accurately. Besides, we also incorporate probabilistic class structure, which implies the probabilistic relationship between each sample and each class, into SR model to further improve discriminability of graph.The experimental results on Hyperion and AVIRIS hyperspectral data show that our method outperforms state of the art methods.

Robust Graph-Based Semisupervised Learning for Noisy Labeled Data via Maximum Correntropy Criterion.

Semisupervised learning (SSL) methods have been proved to be effective at solving the labeled samples shortage problem by using a large number of unlabeled samples together with a small number of labeled samples. However, many traditional SSL methods may not be robust with too much labeling noisy data. To address this issue, in this paper, we propose a robust graph-based SSL method based on maximum correntropy criterion to learn a robust and strong generalization model. In detail, the graph-based SSL framework is improved by imposing supervised information on the regularizer, which can strengthen the constraint on labels, thus ensuring that the predicted labels of each cluster are close to the true labels. Furthermore, the maximum correntropy criterion is introduced into the graph-based SSL framework to suppress labeling noise. Extensive image classification experiments prove the generalization and robustness of the proposed SSL method.

Instance selection method for improving graph-based semi-supervised learning

Graph-based semi-supervised learning is an important semi-supervised learning paradigm. Although graph-based semi-supervised learning methods have been shown to be helpful in various situations, they may adversely affect performance when using unlabeled data. In this paper, we propose a new graph-based semi-supervised learning method based on instance selection in order to reduce the chances of performance degeneration. Our basic idea is that given a set of unlabeled instances, it is not the best approach to exploit all the unlabeled instances; instead, we should exploit the unlabeled instances that are highly likely to help improve the performance, while not taking into account the ones with high risk. We develop both transductive and inductive variants of our method. Experiments on a broad range of data sets show that the chances of performance degeneration of our proposed method are much smaller than those of many state-of-the-art graph-based semi-supervised learning methods.