Graph-based Semi-supervised Algorithm Research Articles

Multi-Label Learning on Tensor Product Graph

A large family of graph-based semi-supervised algorithms have been developed intuitively and pragmatically for the multi-label learning problem. These methods, however, only implicitly exploited the label correlation, as either part of graph weight or an additional constraint, to improve overall classification performance. Despite their seemingly quite different formulations, we show that all existing approaches can be uniformly referred to as a Label Propagation (LP) or Random Walk with Restart (RWR) on a Cartesian Product Graph (CPG). Inspired by this discovery, we introduce a new framework for multi-label classification task, employing the Tensor Product Graph (TPG) — the tensor product of the data graph with the class (label) graph — in which not only the intra-class but also the inter-class associations are explicitly represented as weighted edges among graph vertices. In stead of computing directly on TPG, we derive an iterative algorithm, which is guaranteed to converge and with the same computational complexity and the same amount of storage as the standard label propagation on the original data graph. Applications to four benchmark multi-label data sets illustrate that our method outperforms several state-of-the-art approaches.

A Constrained Graph-Based Semi-Supervised Algorithm Combined with Particle Cooperation and Competition for Hyperspectral Image Classification

Semi-supervised learning (SSL) focuses on the way to improve learning efficiency through the use of labeled and unlabeled samples concurrently. However, recent research indicates that the classification performance might be deteriorated by the unlabeled samples. Here, we proposed a novel graph-based semi-supervised algorithm combined with particle cooperation and competition, which can improve the model performance effectively by using unlabeled samples. First, for the purpose of reducing the generation of label noise, we used an efficient constrained graph construction approach to calculate the affinity matrix, which is capable of constructing a highly correlated similarity relationship between the graph and the samples. Then, we introduced a particle competition and cooperation mechanism into label propagation, which could detect and re-label misclassified samples dynamically, thus stopping the propagation of wrong labels and allowing the overall model to obtain better classification performance by using predicted labeled samples. Finally, we applied the proposed model into hyperspectral image classification. The experiments used three real hyperspectral datasets to verify and evaluate the performance of our proposal. From the obtained results on three public datasets, our proposal shows great hyperspectral image classification performance when compared to traditional graph-based SSL algorithms.

Semi-supervised classification via simultaneous label and discriminant embedding estimation

Graph-based semi-supervised learning has recently proved to be a powerful paradigm for processing and mining large datasets. The main advantage relies on the fact that these methods can be useful in propagating a small set of known labels to a large set of unlabeled data. However, the lack of labeled data in real applications may affect the semi-supervised learning’s performance. This paper introduces a new semi-supervised framework for simultaneous linear feature extraction and label propagation. The proposed method simultaneously estimates a discriminant transformation, as well as the unknown label, by exploiting both labeled and unlabeled data. The learning model’s unknowns are to be estimated by integrating two types of graph-based smoothness constraints. The resulting semi-supervised model is expected to learn more discriminative information. Experiments are conducted on nine public image datasets. These experimental results show that the proposed method’s performance can be better than that of many state-of-the-art graph-based semi-supervised algorithms.

Transductive Bounds for the Multi-Class Majority Vote Classifier

In this paper, we propose a transductive bound over the risk of the majority vote classifier learned with partially labeled data for the multi-class classification. The bound is obtained by considering the class confusion matrix as an error indicator and it involves the margin distribution of the classifier over each class and a bound over the risk of the associated Gibbs classifier. When this latter bound is tight and, the errors of the majority vote classifier per class are concentrated on a low margin zone; we prove that the bound over the Bayes classifier’ risk is tight. As an application, we extend the self-learning algorithm to the multi-class case. The algorithm iteratively assigns pseudo-labels to a subset of unlabeled training examples that have their associated class margin above a threshold obtained from the proposed transductive bound. Empirical results on different data sets show the effectiveness of our approach compared to the same algorithm where the threshold is fixed manually, to the extension of TSVM to multi-class classification and to a graph-based semi-supervised algorithm.

Quantile Regression Based on Laplacian Manifold Regularizer with the Data Sparsity in <i>l</i>1 Spaces

In this paper, we consider the regularized learning schemes based on l1-regularizer and pinball loss in a data dependent hypothesis space. The target is the error analysis for the quantile regression learning. There is no regularized condition with the kernel function, excepting continuity and boundness. The graph-based semi-supervised algorithm leads to an extra error term called manifold error. Part of new error bounds and convergence rates are exactly derived with the techniques consisting of l1-empirical covering number and boundness decomposition.

The graph based semi-supervised algorithm with ℓ1-regularizer

In this paper a new graph-based semi-supervised algorithm for regression problem is proposed. An excess generalization error bound is established. It evaluates the learning performance of the proposed method and has a fast convergence rate with O ( l ? - 1 ) decay. An example is given to show that the proposed method uses a small portion of the labeled and unlabeled data to represent the target function, which illustrates the sparsity of the algorithm, and can efficiently reduce the computational complexity of the semi-supervised learning. Moreover, some experiments are performed to validate the sparsity and learning performance of the formulation.

Graph-based semi-supervised method for image classification in combination with mean shift

There are some disadvantages of graph-based semi-supervised manifold regularization image classification algorithm, such as high space complexity and time complexity, and all of the labeled and unlabeled samples are involved in training. Therefore, it is hard to classify large-scale images. And high error rate often occursin images with complex background or target. In order to deal with these problems, a graph-based semi-supervised algorithm combining mean shift for image classification was proposed. The improvement of the method lay in two aspects: Firstly, mean shift method was used to smooth the image and the result replaced the original image as the image to be classified. Secondly, only a small number of unlabeled samples were used instead of all the unlabeled samples. The experimental results indicate that the proposed method can improve the classification accuracy and largely reduce the complexity. This algorithm makes it possible for graph-based semi-supervised classification algorithms to classify large-scale images.

Semi-supervised learning with local and global consistency

In a lot of practical machine learning applications, such as web page classification, protein shape classification, unlabelled instances are easy to obtain, but labelled instances are rather too expensive to get. Thus, recently, semi-supervised learning (SSL) methods including graph-based algorithms have attracted many interests from researchers. However, most of these algorithms used the Gaussian function to calculate weights of the edge of the graph. In this paper, we proposed a novel weight for graph-based semi-supervised algorithms. In this new algorithm, the label information is added from problem into SSL algorithm, and the geodesic distance is utilized instead of Euclidean distance to calculate the distance between two instances. Furthermore, class prior knowledge is also added from problem into the target function. In this paper, we focus on learning with local and global consistency. We found that the effect of class prior knowledge maybe different between under low-label rate and high-label rate. Experiments on two University of California Irvine (UCI) data sets and United States Postal Service handwritten digit recognition show that our proposed algorithm is really effective.