Graph-based Semi-supervised Learning Research Articles

A hybrid safe semi-supervised learning method

Within the past few years, Safe Semi-Supervised Learning (S3L) has become a hot topic in the machine learning field and many related S3L methods have been proposed to safely exploit the unlabeled information. However, these methods only considered the risk from a single level, such as the instance or model level. They can not reduce the adverse effects of both the risky unlabeled instances and inappropriate learning models. Therefore, it is important to investigate a novel effective S3L method. In this paper, we present a hybrid S3L method which can inherit the merits of both the instance-level and model-level approaches. In our algorithm, multiple Graph-based SSL (GSSL) classifiers are firstly trained and used to predict the unlabeled instances. The risk degrees of the unlabeled instances and the qualities of the constructed graphs are then estimated through the predictions of multiple GSSL classifiers. Finally, we build two regularization terms to constrain the predictions of the unlabeled instances and adaptively select the graphs with high qualities. These regularization terms aim at reducing the negative effect of both the risky unlabeled instances and inappropriate learning models with low-quality graphs. Experimental results on different real-world datasets verify the effectiveness of our algorithm by comparisons to the state-of-the-art Supervised Learning (SL), SSL and S3L methods. In conclusion, our algorithm can not only enrich the research of S3L, but enlarge the practical scope of SSL in the expert and intelligent systems to a certain extent.

A GA-Based Multi-View, Multi-Learner Active Learning Framework for Hyperspectral Image Classification

This paper introduces a novel multi-view multi-learner (MVML) active learning method, in which the different views are generated by a genetic algorithm (GA). The GA-based view generation method attempts to construct diverse, sufficient, and independent views by considering both inter- and intra-view confidences. Hyperspectral data inherently owns high dimensionality, which makes it suitable for multi-view learning algorithms. Furthermore, by employing multiple learners at each view, a more accurate estimation of the underlying data distribution can be obtained. We also implemented a spectral-spatial graph-based semi-supervised learning (SSL) method as the classifier, which improved the performance of the classification task in comparison with supervised learning. The evaluation of the proposed method was based on three different benchmark hyperspectral data sets. The results were also compared with other state-of-the-art AL-SSL methods. The experimental results demonstrated the efficiency and statistically significant superiority of the proposed method. The GA-MVML AL method improved the classification performances by 16.68%, 18.37%, and 15.1% for different data sets after 40 iterations.

Detecting a Risk Signal in Stock Investment Through Opinion Mining and Graph-Based Semi-Supervised Learning

The objective of this study is to develop an algorithm to support a decision-making process in stock investment through opinion mining and graph-based semi-supervised learning. For this purpose, this research addresses the following core processes: (1) filtering fake information, (2) assessing credit risk and detecting risk signals, and (3) predicting future occurrences of credit events through sentiment analysis, word2vec, and graph-based semi-supervised learning. First, financial data, including news, texts in social network services, and financial statements, were collected. Among these data, fake information such as rumors and fake news was filtered by author analysis and a rule-based approach. Second, credit risk was assessed by opinion mining and sentiment analysis for both social data and news in the form of a sentiment score and the trend of documents for each stock. A signal for a credit event was then detected by the degree of assessed risk. Consequently, the possibility of credit events such as delisting and bankruptcy in the near future was forecast based on the risk signal using logistic regression. This research illustrated the real case of a company to validate the applicability of the proposed approach. The results of this study can help investors monitor a large amount of historically accumulated data and detect hidden signals of risk events ahead of time.

Aspect Term Extraction using Graph-based Semi-Supervised Learning

Aspect based Sentiment Analysis is a major subarea of sentiment analysis. Many supervised and unsupervised approaches have been proposed in the past for detecting and analyzing the sentiment of aspect terms. In this paper, a graph-based semi-supervised learning approach for aspect term extraction is proposed. In this approach, every identified token in the review document is classified as aspect or non-aspect term from a small set of labeled tokens using label spreading algorithm. The k-Nearest Neighbor (kNN) for graph sparsification is employed in the proposed approach to make it more time and memory efficient. The proposed work is further extended to determine the polarity of the opinion words associated with the identified aspect terms in review sentence to generate visual aspect-based summary of review documents. The experimental study is conducted on benchmark and crawled datasets of restaurant and laptop domains with varying value of labeled instances. The results depict that the proposed approach could achieve good result in terms of Precision, Recall and Accuracy with limited availability of labeled data.

Greedy-Gradient Max Cut-Based Fault Diagnosis for Direct Online Induction Motors

In this paper, a graph-based semi-supervised learning (GSSL) algorithm, greedy-gradient max cut (GGMC), based fault diagnosis method for direct online induction motors is proposed. Two identical 0.25 HP three-phase squirrel-cage induction motors under healthy, single- and multi-fault conditions were tested in the lab. Three-phase stator currents and three-dimensional vibration signals of the two motors were recorded simultaneously in each test, and used as datasets in this study. Features for machine learning are extracted from experimental stator currents and vibration data by the discrete wavelet transform (DWT). To validate the effectiveness of the proposed GGMC-based fault diagnosis method, its classification accuracy using binary classification and multiclass classification for faults of the two motors are compared with other two GSSL algorithms, local and global consistency (LGC) and Gaussian field and harmonic function (GFHF). In this study, the performance of stator currents and vibration as a monitoring signal is evaluated, it is found that stator currents perform much better than vibration signals for multiclass classification, while they both perform well for binary classification.

A Novel Adaptive Multi-View Non-Negative Graph Semi-Supervised ELM

This paper represents a semi-supervised learning framework, which integrates multi-view learning, extreme learning machine (ELM) and graph-based semi-supervised learning. The aim is to expand the scope of adaptation of non-negative sparse graph (NNSG) framework, under a multi-view condition and a non-linear relationship. The proposed multi-view learning method will be adaptive since when data is single-view the framework will degenerate into an embedded framework for NNSG framework. The proposed ELM method also will be adaptive since the number of hidden layer neuron will change with different number of input and output layer neuron. The combination of both proposed methods outperforms traditional graph-based semi-supervised learning, such as flexible manifold embedding (FME) and NNSG framework, which can not establish an affinity matrix for multi-view and can not establish a non-liner model for unknown data. Unlike traditional graph-based semi-supervised learning methods, which only can label propagation and build linear regression models for single or multi-view data, our proposed method has an obvious advantage that is applicable to any single or multi-view data, and builds linear or non-linear models. We provides extensive experiments on four public database in order to evaluate the performance of the proposed method. These experiments demonstrate significant improvement over the state-of-the-art algorithms in label propagation and processing of new data.

Graph-Based Semi-Supervised Learning for Activity Labeling in Health Smart Home

Health Smart Home (HSH) is an important part of smart city. This technology provides a new kind of remote medical treatment, and can effectively alleviate the shortage of medical resources caused by aging population and help elderly people live at home more safely and independently. Activity recognition is the core of Health Smart Home. However, constructing activity recognition models usually requires a large amount of labeled data, which imposes a heavy burden on manual labeling. In this article, the authors propose an activity labeling approach based on a graph-based semi-supervised learning algorithm. This approach can divide the raw sensor event sequence without any label information into appropriate segments. Consecutive sensor events that occurred in a same activity are grouped into a same segment. In addition, this approach requires only a small number of manually labeled segments to complete the labeling of the remaining large number of unlabeled segments, thereby greatly reducing the burden of manual labeling. After that, all the labeled data can be further used for activity recognition in smart homes. Finally, a series of comprehensive experiments are conducted on freely available data sets to validate the effectiveness of the proposed activity labeling approach.

Optimized Combination of Multiple Graphs With Application to the Integration of Brain Imaging and (epi)Genomics Data.

With the rapid development of high-throughput technologies, a growing amount of multi-omics data are collected, giving rise to a great demand for combining such data for biomedical discovery. Due to the cost and time to label the data manually, the number of labelled samples is limited. This motivated the need for semi-supervised learning algorithms. In this work, we applied a graph-based semi-supervised learning (GSSL) to classify a severe chronic mental disorder, schizophrenia (SZ). An advantage of GSSL is that it can simultaneously analyse more than two types of data, while many existing models focus on pairwise data analysis. In particular, we applied GSSL to the analysis of single nucleotide polymorphism (SNP), functional magnetic resonance imaging (fMRI) and DNA methylation data, which accounts for genetics, brain imaging (endophenotypes), and environmental factors (epigenomics) respectively. While parameter selection has been an open challenge for most models, another key contribution of this work is that we explored the parameter space to interpret their meaning and established practical guidelines. Based on the practical significance of each hyper-parameter, a relatively small range of candidate values can be determined in a data-driven way to both optimize and speed up the parameter tuning process. We validated the model through both synthetic data and a real SZ dataset of 184 subjects from the Mental Illness and Neuroscience Discovery (MIND) Clinical Imaging Consortium. In comparison to several existing approaches, our algorithm achieved better performance in terms of classification accuracy. We also confirmed the significance of several brain regions associated with SZ.

Image-based face beauty analysis via graph-based semi-supervised learning

Automatic facial beauty analysis has become an emerging research topic. Despite some achieved advances, current methods and systems suffer from at least two limitations. Firstly, many developed systems rely on the use of ad-hoc hand-crafted features that were designed for generic pattern recognition problems. Secondly, while Deep Convolutional Neural Nets (DCNN) have been recently demonstrated to be a promising area of research in statistical machine learning, their use for automatic face beauty analysis may not guarantee optimal performances due to the use of a limited amount of face images with beauty scores. In this paper, we attempt to overcome these two main limitations by jointly exploiting two tricks. First, instead of using hand-crafted face features we use deep features of a pre-trained DCNN able to generate a high-level representation of a face image. Second, we exploit manifold learning theory and deploy three graph-based semi-supervised learning methods in order to enrich model learning without the need of additional labeled face images. These schemes perform graph-based score propagation. The proposed schemes were tested on three public datasets for beauty analysis: SCUT-FBP, M2B, and SCUT-FBP5500. These experiments, as well as many comparisons with supervised schemes, show that the scheme coined Kernel Flexible Manifold Embedding compares favorably with many supervised schemes. They also show that its performances in terms of error prediction and Pearson Correlation are better than those reported for the used datasets.

HesGCN: Hessian graph convolutional networks for semi-supervised classification

Manifold or local geometry of samples have been recognized as a powerful tool in machine learning areas, especially in the graph-based semi-supervised learning (GSSL) problems. Over recent decades, plenty of manifold assumption-based SSL algorithms (MSSL) have been proposed including graph embedding and graph regularization models, where the objective is to utilize the local geometry of data distributions. One of most representative MSSL approaches is graph convolutional networks (GCN), which effectively generalizes the convolutional neural networks to deal with the graphs with the arbitrary structures by constructing and fusing the Laplacian-based structure information. However, the null space of the Laplacian remains unchanged along the underlying manifold, it causes the poor extrapolating ability of the model. In this paper, we introduce a variant of GCN, i.e. Hessian graph convolutional networks (HesGCN). In particularly, we get a more efficient convolution layer rule by optimizing the one-order spectral graph Hessian convolutions. In addition, the spectral graph Hessian convolutions is a combination of the Hessian matrix and the spectral graph convolutions. Hessian gets a richer null space by the existence of its two-order derivatives, which can describe the intrinsic local geometry structure of data accurately. Thus, HesGCN can learn more efficient data features by fusing the original feature information with its structure information based on Hessian. We conduct abundant experiments on four public datasets. Extensive experiment results validate the superiority of our proposed HesGCN compared with many state-of-the-art methods.

Decoding EEG in Motor Imagery Tasks with Graph Semi-Supervised Broad Learning

In recent years, the accurate and real-time classification of electroencephalogram (EEG) signals has drawn increasing attention in the application of brain-computer interface technology (BCI). Supervised methods used to classify EEG signals have gotten satisfactory results. However, unlabeled samples are more frequent than labeled samples, so how to simultaneously utilize limited labeled samples and many unlabeled samples becomes a research hotspot. In this paper, we propose a new graph-based semi-supervised broad learning system (GSS-BLS), which combines the graph label propagation method to obtain pseudo-labels and then trains the GSS-BLS classifier together with other labeled samples. Three BCI competition datasets are used to assess the GSS-BLS approach and five comparison algorithms: BLS, ELM, HELM, LapSVM and SMIR. The experimental results show that GSS-BLS achieves satisfying Cohen’s kappa values in three datasets. GSS-BLS achieves the better results of each subject in the 2-class and 4-class datasets and has significant improvements compared with original BLS except subject C6. Therefore, the proposed GSS-BLS is an effective semi-supervised algorithm for classifying EEG signals.

그래프 기반 준지도 학습을 이용한 속성값 전파 결측치 추정

그래프 기반 준지도 학습을 이용한 속성값 전파 결측치 추정

Technology opportunity discovery by structuring user needs based on natural language processing and machine learning.

Discovering technology opportunities from the opinion of users can promote successful technological development by satisfying the needs of users. However, although previous approaches using opinion mining only have classified various needs of users into positive or negative categories, they cannot derive the main reasons for their opinion. To solve this problem, this research proposes an approach to exploring technology opportunity by structuring user needs with a concept of opinion trigger of objects and functions of the technology-based products. To discover technology opportunity, first, an opinion trigger is identified from review data using Naïve Base classifier and natural language processing. Second, the opinion triggers and patent keywords that have a similar meaning in context are clustered to discover the needs of the user and need-related technology. Then, the sentimental values of needs are calculated through graph-based semi-supervised learning. Finally, the needs of the user are classified in resolving the problem of vacant technology to discover technology opportunity. Then, an R&D strategy of each opportunity is suggested based on opinion triggers, patent keywords, and their property. Based on the concept of opinion trigger-based methodology, a case study is conducted on automobile—related reviews, extracting the customer needs and presenting important R&D projects such as an extracted need (cargo transportation) and its R&D strategy (resolving contradiction). The proposed approach can analyze the needs of user at a functional level to discover new technology opportunities.

Multi-view adaptive semi-supervised feature selection with the self-paced learning

In recent decades, semi-supervised feature selection (SSFS) has gained extensive research in the field of machine learning and computer vision. Most SSFS algorithms are based on graph-based semi-supervised learning (GSSL), and their performance depend heavily on the quality of the Laplacian weight graph. However, the Laplacian weight graph can't be changed once it is constructed, which greatly restricts the performance of SSFS. To address this defect, in this paper we propose a novel Multi-view Adaptive Semi-supervised Feature Selection (MASFS) algorithm, which introduces the self-paced learning (SPL) into SSFS to make the Laplacian weight graph adaptively change according to the current predicted information. Meanwhile, the MASFS algorithm utilizes the multi-view learning to effectively explore the complementary and related information contained in different views to enhance SSFS performance. We propose a valid iterative algorithm for optimizing the objective function, followed by the convergence analysis and the complexity analysis. To illustrate the effectiveness of the MASFS algorithm, some experiments are carried out on NUS-WIDE dataset and MSRA-MM2.0 dataset and the experimental results indicate that MASFS has better performance than other SSFS algorithms based on GSSL.

A splitting method for the locality regularized semi-supervised subspace clustering

Graph-based semi-supervised learning (G-SSL) methods play an increasingly important role in machine learning systems. Recently, latent low-rank representation (LatLRR) graph has gained great success in subspace clustering. However, LatLRR only considers the global structure, while the local geometric information, which is often important to many real applications, is ignored. In this paper, we propose a locality regularized LatLRR model (LR-LatLRR) for semi-supervised subspace clustering problems. This model incorporates two regularization terms into LatLRR by taking the local structure of data into account. Then, we develop an efficient splitting algorithm for solving LR-LatLRR. In addition, we also prove the global convergence of the proposed algorithm. Furthermore, we extend the LR-LatLRR model to a case of including the non-negative constraint. Finally, we conduct experiments on a synthetic data and several real data sets for the semi-supervised clustering problems. Experimental results show that our method can obtain high classification accuracy and outperforms several state-of-the-art G-SSL methods.

Toward graph-based semi-supervised face beauty prediction

Assessing beauty using facial images analysis is an emerging computer vision problem. To the best of our knowledge, all existing methods for automatic facial beauty scoring rely on fully supervised schemes. In this paper, we introduce the use of semi-supervised learning schemes for solving the problem of face beauty scoring when the image descriptor is holistic and the score is given by a real number. The paper has two main contributions. Firstly, we introduce the use of graph-based semi-supervised learning for face beauty scoring. The proposed method is based on texture and utilizes continuous scores in a full range. Secondly, we adapt and kernelize an existing linear Flexible Manifold Embedding scheme (that works with discrete classes) to the case of real scores propagation. The resulting model can be used for transductive and inductive settings. The proposed semi-supervised schemes were evaluated on three recent public datasets for face beauty analysis: SCUT-FBP, M2B, and SCUT-FBP5500. The obtained experimental results, as well as many comparisons with fully supervised methods, demonstrate that the nonlinear semi-supervised scheme compares favorably with many supervised schemes. The proposed semi-supervised scoring framework paves the way to virtually all applications to adopt continuous scores instead of the usual discrete labels.

Accelerating Flexible Manifold Embedding for Scalable Semi-Supervised Learning

In this paper, we address the problem of large-scale graph-based semi-supervised learning for multi-class classification. Most existing scalable graph-based semi-supervised learning methods are based on the hard linear constraint or cannot cope with the unseen samples, which limits their applications and learning performance. To this end, we build upon our previous work flexible manifold embedding (FME) [1] and propose two novel linear-complexity algorithms called fast flexible manifold embedding (f-FME) and reduced flexible manifold embedding (r-FME). Both of the proposed methods accelerate FME and inherit its advantages. Specifically, our methods address the hard linear constraint problem by combining a regression residue term and a manifold smoothness term jointly, which naturally provides the prediction model for handling unseen samples. To reduce computational costs, we exploit the underlying relationship between a small number of anchor points and all data points to construct the graph adjacency matrix, which leads to simplified closed-form solutions. The resultant f-FME and r-FME algorithms not only scale linearly in both time and space with respect to the number of training samples but also can effectively utilize information from both labeled and unlabeled data. Experimental results show the effectiveness and scalability of the proposed methods.

Label Propagation Ensemble for Hyperspectral Image Classification

The imbalance between limited labeled pixels and high dimensionality of hyperspectral data can easily give rise to Hughes phenomenon. Semisupervised learning (SSL) methods provide promising solutions to address the aforementioned issue. Graph-based SSL algorithms, also called label propagation methods, have obtained increasing attention in hyperspectral image (HSI) classification. However, the graphs constructed by utilizing the geometrical structure similarity of samples are unreliable due to the high dimensionality and complexity of the HSIs, especially for the case of very limited labeled pixels. Our motivation is to construct label propagation ensemble (LPE) model, then use the decision fusion of multiple label propagations to obtain pseudolabeled pixels with high classification confidence. In LPE, random subspace method is introduced to partition the feature space into multiple subspaces, then several label propagation models are constructed on corresponding subspaces, finally the results of different label propagation models are fused at decision level, and only the unlabeled pixels whose label propagation results are the same will be assigned with pseudolabels. Meanwhile extreme learning machine classifiers are trained on the labeled and pseudolabeled samples during the iteration. Compared with traditional label propagation methods, our proposed method can deal with the situation of very limited labeled samples by providing pseudolabeled pixels with high classification confidence, consequently, the accurate base classifiers are obtained. To demonstrate the effectiveness of the proposed method, LPE is compared with several state-of-the-art methods on four hyperspectral datasets. In addition, the method that only use label propagation is investigated to show the importance of ensemble technique in LPE. The experimental results demonstrate that the proposed method can provide competitive solution for HSI classification.

ALG: Adaptive low-rank graph regularization for scalable semi-supervised and unsupervised learning

Graph-based semi-supervised learning (SSL) is one of the most popular topics in the past decades. Most graph-based SSL methods utilize two stage-approach to infer the class labels of the unlabeled data, where the first stage is to construct a graph in an unsupervised way for characterizing the geometry of data manifold while the second stage is to perform SSL for classification. However, the graph construction and SSL stages are usually separate. They do not share any common information to enhance the performance of classification. In this paper, we aim to solve the above problem by proposing a unified framework for SSL. In detail, we first adopt an adaptive low-rank model for graph construction, where the coefficient matrix is calculated through an efficient procedure so that the corresponding constructed graph can capture the global structure of data manifold. Meriting from such a graph, we then propose a unified framework for SSL, where we have involved the graph construction, the SSL strategy into the whole objective function. As a result, the class labels learned by SSL can provide more discriminative information for graph construction, while the updated graph can further enhance the classification accuracies of SSL. Simulation indicates that the proposed method can achieve better classification performance compared with other state-of-the-art graph-based methods.

PKGCN: prior knowledge enhanced graph convolutional network for graph-based semi-supervised learning

Graph is a widely existed data structure in many real world scenarios, such as social networks, citation networks and knowledge graphs. Recently, Graph Convolutional Network (GCN) has been proposed as a powerful method for graph-based semi-supervised learning, which has the similar operation and structure as Convolutional Neural Networks (CNNs). However, like many CNNs, it is often necessary to go through a lot of laborious experiments to determine the appropriate network structure and parameter settings. Fully exploiting and utilizing the prior knowledge that nearby nodes have the same labels in graph-based neural network is still a challenge. In this paper, we propose a model which utilizes the prior knowledge on graph to enhance GCN. To be specific, we decompose the objective function of semi-supervised learning on graphs into a supervised term and an unsupervised term. For the unsupervised term, we present the concept of local inconsistency and devise a loss term to describe the property in graphs. The supervised term captures the information from the labeled data while the proposed unsupervised term captures the relationships among both labeled data and unlabeled data. Combining supervised term and unsupervised term, our proposed model includes more intrinsic properties of graph-structured data and improves the GCN model with no increase in time complexity. Experiments on three node classification benchmarks show that our proposed model is superior to GCN and seven existing graph-based semi-supervised learning methods.