Semi-supervised Classification Problem Research Articles

Binarization With Boosting and Oversampling for Multiclass Classification.

Using a set of binary classifiers to solve multiclass classification problems has been a popular approach over the years. The decision boundaries learnt by binary classifiers (also called base classifiers) are much simpler than those learnt by multiclass classifiers. This paper proposes a new classification framework, termed binarization with boosting and oversampling (BBO), for efficiently solving multiclass classification problems. The new framework is devised based on the one-versus-all (OVA) binarization technique. Unlike most previous work, BBO employs boosting for solving the hard-to-learn instances and oversampling for handling the class-imbalance problem arising due to OVA binarization. These two features make BBO different from other existing works. Our new framework has been tested extensively on several multiclass supervised and semi-supervised classification problems using five different base classifiers, including neural networks, C4.5, k -nearest neighbor, repeated incremental pruning to produce error reduction, support vector machine, random forest, and learning with local and global consistency. Experimental results show that BBO can exhibit better performance compared to its counterparts on supervised and semi-supervised classification problems.

Active semi-supervised learning method with hybrid deep belief networks.

In this paper, we develop a novel semi-supervised learning algorithm called active hybrid deep belief networks (AHD), to address the semi-supervised sentiment classification problem with deep learning. First, we construct the previous several hidden layers using restricted Boltzmann machines (RBM), which can reduce the dimension and abstract the information of the reviews quickly. Second, we construct the following hidden layers using convolutional restricted Boltzmann machines (CRBM), which can abstract the information of reviews effectively. Third, the constructed deep architecture is fine-tuned by gradient-descent based supervised learning with an exponential loss function. Finally, active learning method is combined based on the proposed deep architecture. We did several experiments on five sentiment classification datasets, and show that AHD is competitive with previous semi-supervised learning algorithm. Experiments are also conducted to verify the effectiveness of our proposed method with different number of labeled reviews and unlabeled reviews respectively.

Semisupervised Classification Through the Bag-of-Paths Group Betweenness

This paper introduces a novel and well-founded betweenness measure, called the bag-of-paths (BoP) betweenness, as well as its extension, the BoP group betweenness, to tackle semisupervised classification problems on weighted directed graphs. The objective of semisupervised classification is to assign a label to unlabeled nodes using the whole topology of the graph and the labeled nodes at our disposal. The BoP betweenness relies on a BoP framework, assigning a Boltzmann distribution on the set of all possible paths through the network such that long (high-cost) paths have a low probability of being picked from the bag, while short (low-cost) paths have a high probability of being picked. Within that context, the BoP betweenness of node $j$ is defined as the sum of the a posteriori probabilities that node $j$ lies in between two arbitrary nodes $(i,k)$ when picking a path starting in $i$ and ending in $k$ . Intuitively, a node typically receives a high betweenness if it has a large probability of appearing on paths connecting two arbitrary nodes of the network. This quantity can be computed in closed form by inverting an $n\times n$ matrix where $n$ is the number of nodes. For the group betweenness, the paths are constrained to start and end in nodes within the same class, thereby defining a within-class group betweenness for each class. Unlabeled nodes are then classified according to the class showing the highest group betweenness. Experiments on various real-world datasets show that the BoP group betweenness performs competitively compared to all the tested state-of-the-art methods. The benefit of the BoP betweenness is particularly noticeable when only a few labeled nodes are available.

Laplacian least squares twin support vector machine for semi-supervised classification

The recently proposed Laplacian twin support vector machine (Lap-TSVM) is an excellent nonparallel-based kernel tool for semi-supervised classification problems, where its optimal decision hyperplane is determined by solving two quadratic programming problems (QPPs) with matrix inversion operations. In order to reduce its computation cost, in this paper, we formulate a least squares version of Lap-TSVM, termed as Lap-LSTSVM, leading to an extremely fast approach for generating semi-supervised classifiers. Besides, a meaningful regularization parameter is introduced for each problem in Lap-LSTSVM to balance the regularization terms between the reproducing kernel Hilbert spaces (RHKS) term and the manifold regularization (MR) term, instead of two parameters used in Lap-TSVM. In addition, an efficient conjugate gradient (CG) algorithm is further developed for solving the systems of linear equations (LEs) appeared to speed up the training procedure. Experimental results on both several synthetic and real-world datasets confirm the feasibility and the effectiveness of the proposed method.

NONPARALLEL HYPERPLANES PROXIMAL CLASSIFIERS BASED ON MANIFOLD REGULARIZATION FOR LABELED AND UNLABELED EXAMPLES

In this paper, we propose two Laplacian nonparallel hyperplane proximal classifiers (LapNPPCs) for semi-supervised and full-supervised classification problem respectively by adding manifold regularization terms. Due to the manifold regularization terms, our LapNPPCs are able to exploit the intrinsic structure of the patterns of the training set. Furthermore, our classifiers only need to solve two systems of linear equations rather than two quadratic programming (QP) problems as needed in Laplacian twin support vector machine (LapTSVM) (Z. Qi, Y. Tian and Y. Shi, Neural Netw.35 (2012) 46–53). Numerical experiments on toy and UCI benchmark datasets show that the accuracy of our LapNPPCs is comparable with other classifiers, such as the standard SVM, TWSVM and LapTSVM, etc. It is also the case that based on our LapNPPCs, some other TWSVM type classifiers with manifold regularization can be constructed by choosing different norms and loss functions to deal with semi-supervised binary and multi-class classification problems.

Laplacian twin support vector machine for semi-supervised classification

Semi-supervised learning has attracted a great deal of attention in machine learning and data mining. In this paper, we have proposed a novel Laplacian Twin Support Vector Machine (called Lap-TSVM) for the semi-supervised classification problem, which can exploit the geometry information of the marginal distribution embedded in unlabeled data to construct a more reasonable classifier and be a useful extension of TSVM. Furthermore, by choosing appropriate parameters, Lap-TSVM degenerates to either TSVM or TBSVM. All experiments on synthetic and real data sets show that the Lap-TSVM's classifier combined by two nonparallel hyperplanes is superior to Lap-SVM and TSVM in both classification accuracy and computation time.

Semi-supervised Learning Using Local Regularizer and Unit Circle Class Label Representation

Semi-supervised learning, which aims to learn from partially labeled data and mostly unlabeled data, has been attracted more and more attention in machine learning and pattern recognition. A novel semi-supervised classification approach is proposed, which can not only handle semi-supervised binary classification problem but also deal with semi-supervised multi-class classification problem. The approach is based on local regularizer and unit circle class label representation. The former is minimized so as to cause the class labels to have the desired properties. The latter utilizes two-dimensional vector evenly distributed in circumference of unit circle to represent class label, so multi-class classification can be performed only once. Comparative classification experiments on some benchmark datasets validate the effectiveness of the presented approach.

Stochastic Optimization Algorithms for Support Vector Machines Classification

In this paper, we consider the problem of semi-supervised binary classification by Support Vector Machines (SVM). This problem is explored as an unconstrained and non-smooth optimization task when part of the available data is unlabelled. We apply non-smooth optimization techniques to classification where the objective function considered is non-convex and non-differentiable and so difficult to minimize. We explore and compare the properties of Simulated Annealing and of Simultaneous Perturbation Stochastic Approximation (SPSA) algorithms (SPSA with the Lipschitz Perturbation Operator, SPSA with the Uniform Perturbation Operator, Standard Finite Difference Approximation) for semi-supervised SVM classification. Numerical results are given, obtained by running the proposed methods on several standard test problems drawn from the binary classification literature. The performance of the classifiers were evaluated by analyzing Receiver Operating Characteristics (ROC).

Optimization Techniques for Semi-Supervised Support Vector Machines

Due to its wide applicability, the problem of semi-supervised classification is attracting increasing attention in machine learning. Semi-Supervised Support Vector Machines (S3VMs) are based on applying the margin maximization principle to both labeled and unlabeled examples. Unlike SVMs, their formulation leads to a non-convex optimization problem. A suite of algorithms have recently been proposed for solving S3VMs. This paper reviews key ideas in this literature. The performance and behavior of various S3VMs algorithms is studied together, under a common experimental setting.

Content-Based Information Fusion for Semi-Supervised Music Genre Classification

In this paper, we propose an information fusion framework for the semi-supervised distance-based music genre classification problem. We make use of the regularized least-square framework as the basic classifier, which only involves the similarity scores among different music tracks. We present a similarity score that multiplies different scores based on different distance measures. Particularly the distance measures are not restricted to the Euclidean distance. By adding a weight to each single distance based score, we propose an expectation-maximization (EM) algorithm to adaptively learn the fusion scores. Experiments on real music data set show that our approach can give promising results.

Email answering assistance by semi-supervised text classification

Many individuals, organizations, and companies have to answer large amounts of emails. Often, many of these emails contain variations of relatively few frequently asked questions. We address the problem of predicting which of several frequently used answers a user will choose to respond to an email. We map the problem to a semi-supervised text classification problem. In a case study with emails that have been sent to a corporate customer service department, we investigate the ability of the naive Bayesian and support vector classifier to identify the appropriate answers to emails. We study how effectively the transductive Support Vector Machine and the co-training algorithm utilize unlabeled data and investigate why co-training is only beneficial when very few labeled data are available. In addition, we describe a practical assistance system.

Contributions from the physical laboratory of the university of Pennsylvania. No. 1.—A new vertical-lantern galvanometer

Multi-view semi-supervised learning has gained much attention since a great number of unlabeled multi-view data are easy to obtain while few labeled data are available. Accordingly, how to utilize the relationship between labeled and unlabeled data is of significance in multi-view semi-supervised learning. In this paper, we propose an auto-weighted manifold embedding model to address multi-view semi-supervised classification problems, where only a small percentage of labeled data points are used for model training. In the proposed model, data points close to each other in the feature space will be assigned to similar classes in the label space through manifold embedding. Accordingly, the class information of labeled data will be employed in the prediction process for unlabeled data. Moreover, an optimal weight for each view of multi-view data is learned automatically, which enables an encouraging fusion quality for multi-view manifold embedding. To further speed up the calculation process and reduce the computational complexity of the proposed model, an effective accelerated auto-weighted manifold embedding scheme is developed. Besides, theoretical analyses are then provided to indicate a tight approximation bound for the primary manifold embedding method, while the accelerated method is designed by an order decrease of magnitude of computational complexity. Finally, comprehensive experiments on eight publicly available data sets demonstrate the superiority of the proposed models over compared state-of-the-art semi-supervised methods and fully supervised classifiers. Furthermore, the experimental results are suggestive of positive robustness and promising generalization capacity of the proposed methods.