Graph Semi-supervised Learning Research Articles

Manifold-based multi-graph embedding for semi-supervised classification

Graph semi-supervised learning (GSSL) plays an important role in semi-supervised classification by leveraging the similarity of graph topology and convex optimization with Laplacian-based regularization. Current approaches mainly focus on data representation in the Euclidean space and solely rely on a single graph structure, ignoring the underlined manifold-valued structure of the data. To this end, we propose a novel manifold-based multi-graph embedding (M2GE), which explicitly learns representations on both Riemannian manifold and Euclidean space in the embedding learning process. Incorporating multi-graph structure, the rich and varied features are used by M2GE to more fully capture categorical information, where the heterogeneous spatial representations from manifold are complementary. Extensive experimental results on three popular benchmarks verify demonstrate the superiority of our method over state-of-the art competitors.

Toward Robust Graph Semi-Supervised Learning Against Extreme Data Scarcity.

The success of graph neural networks (GNNs) in graph-based web mining highly relies on abundant human-annotated data, which is laborious to obtain in practice. When only a few labeled nodes are available, how to improve their robustness is key to achieving replicable and sustainable graph semi-supervised learning. Though self-training is powerful for semi-supervised learning, its application on graph-structured data may fail because 1) larger receptive fields are not leveraged to capture long-range node interactions, which exacerbates the difficulty of propagating feature-label patterns from labeled nodes to unlabeled nodes and 2) limited labeled data makes it challenging to learn well-separated decision boundaries for different node classes without explicitly capturing the underlying semantic structure. To address the challenges of capturing informative structural and semantic knowledge, we propose a new graph data augmentation framework, augmented graph self-training (AGST), which is built with two new (i.e., structural and semantic) augmentation modules on top of a decoupled GST backbone. In this work, we investigate whether this novel framework can learn a robust graph predictive model under the low-data context. We conduct comprehensive evaluations on semi-supervised node classification under different scenarios of limited labeled-node data. The experimental results demonstrate the unique contributions of the novel data augmentation framework for node classification with few labeled data.

Scalable Semi-Supervised Graph Learning Techniques for Anti Money Laundering

Scalable Semi-Supervised Graph Learning Techniques for Anti Money Laundering

CC-GNN: A clustering contrastive learning network for graph semi-supervised learning

CC-GNN: A clustering contrastive learning network for graph semi-supervised learning

Incremental semi-supervised graph learning NMF with block-diagonal

Non-negative matrix factorization (NMF), as a good data dimensionality reduction method, is widely used in the field of image recognition. Incremental non-negative matrix factorization (INMF) as an improvement solves the problem of inefficiency caused by repeated running of data samples during online learning. However, in the traditional incremental non-negative matrix factorization algorithm, the newly added training samples do not contain label information. Some samples may be marked in both the initial sample and the new training sample in the real application scenario. In order to make full use of the label information carried by the dataset, in this paper, we propose a semi-supervised non-negative matrix factorization model for batch incremental data, incremental semi-supervised graph learning NMF with block diagonal (ISGDNMF). The model is divided into three cases according to the degree of label-carrying for the new batch data: all label-carrying, no label-carrying, and partial label-carrying. The label information is also used to add a diagonal structure to the coefficient matrix, which makes it possible to have stronger discriminatory ability and to distinguish different classes of images more easily. And graph regularization information is added in order to maintain the spatial-geometric structure of the data. Experiments on six image datasets show that this algorithm has superior performance relative to the other seven NMF-based algorithms.

Semi-Supervised Heterogeneous Graph Learning with Multi-Level Data Augmentation

In recent years, semi-supervised graph learning with data augmentation (DA) has been the most commonly used and best-performing method to improve model robustness in sparse scenarios with few labeled samples. However, most existing DA methods are based on the homogeneous graph, but none are specific for the heterogeneous graph. Differing from the homogeneous graph, DA in the heterogeneous graph faces greater challenges: heterogeneity of information requires DA strategies to effectively handle heterogeneous relations, which considers the information contribution of different types of neighbors and edges to the target nodes. Furthermore, over-squashing of information is caused by the negative curvature formed by the non-uniformity distribution and the strong clustering in a complex graph. To address these challenges, this article presents a novel method named HG-MDA (Semi-Supervised Heterogeneous Graph Learning with Multi-Level Data Augmentation). For the problem of heterogeneity of information in DA, node and topology augmentation strategies are proposed for the characteristics of the heterogeneous graph. Additionally, meta-relation-based attention is applied as one of the indexes for selecting augmented nodes and edges. For the problem of over-squashing of information, triangle-based edge adding and removing are designed to alleviate the negative curvature and bring the gain of topology. Finally, the loss function consists of the cross-entropy loss for labeled data and the consistency regularization for unlabeled data. To effectively fuse the prediction results of various DA strategies, sharpening is used. Existing experiments on public datasets (i.e., ACM, DBLP, and OGB) and the industry dataset MB show that HG-MDA outperforms current SOTA models. Additionally, HG-MDA is applied to user identification in internet finance scenarios, helping the business to add 30% key users, and increase loans and balances by 3.6%, 11.1%, and 9.8%.

GPENs: Graph Data Learning With Graph Propagation-Embedding Networks.

Compact representation of graph data is a fundamental problem in pattern recognition and machine learning area. Recently, graph neural networks (GNNs) have been widely studied for graph-structured data representation and learning tasks, such as graph semi-supervised learning, clustering, and low-dimensional embedding. In this article, we present graph propagation-embedding networks (GPENs), a new model for graph-structured data representation and learning problem. GPENs are mainly motivated by 1) revisiting of traditional graph propagation techniques for graph node context-aware feature representation and 2) recent studies on deeply graph embedding and neural network architecture. GPENs integrate both feature propagation on graph and low-dimensional embedding simultaneously into a unified network using a novel propagation-embedding architecture. GPENs have two main advantages. First, GPENs can be well-motivated and explained from feature propagation and deeply learning architecture. Second, the equilibrium representation of the propagation-embedding operation in GPENs has both exact and approximate formulations, both of which have simple closed-form solutions. This guarantees the compactivity and efficiency of GPENs. Third, GPENs can be naturally extended to multiple GPENs (M-GPENs) to address the data with multiple graph structures. Experiments on various semi-supervised learning tasks on several benchmark datasets demonstrate the effectiveness and benefits of the proposed GPENs and M-GPENs.

Designing and planning sustainable customized bus service for departing attendees of planned special events: A two-phase methodology integrating data-driven and demand-responsive

Planned special events (PSE), such as concerts and sporting events, will seriously affect the normal operations of the transportation system, especially at the egress stage. The huge taxi and ride-hailing trips of departing attendees is one of the prime reasons for traffic congestion, meanwhile, insufficient vehicles and traffic congestion around the venue also lead to excessive passenger waiting time. Customized bus (CB) service is an emerging green sustainable transportation mode, and its characteristics are suitable for solving corresponding problems. This paper designs a novel CB service mode and constructs a corresponding two-phase planning methodology integrating data-driven and demand-responsive. The first phase includes a potential demand identification model based on graph semi-supervised learning and an integrated algorithm to deploy CB stops. The second phase is CB route planning with heterogeneous vehicles based on attendee choices. A concert held in Tianjin, China was selected as case study to evaluate the designed CB service. The results show that compared with the corresponding taxis and ride-hailing trips, the generated CB services can save 37.71% of the average travel time, 36.56% of the expenses per passenger and 89.58% of the spatial road occupancy, which can greatly alleviate traffic congestion, reduce carbon emissions and improve attendee travel.

Laplacian-based semi-Supervised learning in multilayer hypergraphs by coordinate descent

Graph Semi-Supervised learning is an important data analysis tool, where given a graph and a set of labeled nodes, the aim is to infer the labels to the remaining unlabeled nodes. In this paper, we start by considering an optimization-based formulation of the problem for an undirected graph, and then we extend this formulation to multilayer hypergraphs. We solve the problem using different coordinate descent approaches and compare the results with the ones obtained by the classic gradient descent method. Experiments on synthetic and real-world datasets show the potential of using coordinate descent methods with suitable selection rules.

Semi-Supervised Graph Learning for Brain Disease Identification

Semi-Supervised Graph Learning for Brain Disease Identification

Semi-supervised graph learning framework for apicomplexan parasite classification

Apicomplexan parasites cause diseases including malaria, toxoplasmosis, and babesiosis, affecting large parts of the world and hampering economic development to a considerable severity. Early recognition of pathogenic parasites can effectively constrain the spread of these infectious diseases. Recently, deep-learning-based computer-aided parasite recognition systems provide many practical tools for automated apicomplexan parasite recognition. However, existing research works are limited by the complicated and expensive manual annotating workload in supervised deep learning methods, which require many experts’ manually annotated samples. Moreover, they also ignore the significance of correlations among different parasite cells, which retain helpful information for identification. Therefore, this paper is the first to employ a semi-supervised learning strategy to reduce the manual labeling workload and introduce a hybrid graph learning algorithm into parasite recognition. Our approach has a superior parasite recognition capability with semi-supervised techniques and the graph-based model. Specifically, this research proposes a Semi-Supervised Graph Learning (SSGL) framework, composing CNN (Convolutional Neural Networks) feature embedding, learnable graph building, and graph feature learning components. To verify the performance of the SSGL model, the paper applies multiple evaluation indicators to appraise our model, and the approach can achieve high accuracy of 91.75%, AUC of 91.83%, the sensitivity of 91.75% and specificity of 97.25% with only a small amount of labeled data (20%). In conclusion, SSGL model provides high parasite recognition capability with minimal labels, greatly reduces the workload of relevant experts, and promotes the popularization of computer-aided parasite diagnosis tools.

A unified deep semi-supervised graph learning scheme based on nodes re-weighting and manifold regularization

In recent years, semi-supervised learning on graphs has gained importance in many fields and applications. The goal is to use both partially labeled data (labeled examples) and a large amount of unlabeled data to build more effective predictive models. Deep Graph Neural Networks (GNNs) are very useful in both unsupervised and semi-supervised learning problems. As a special class of GNNs, Graph Convolutional Networks (GCNs) aim to obtain data representation through graph-based node smoothing and layer-wise neural network transformations. However, GCNs have some weaknesses when applied to semi-supervised graph learning: (1) it ignores the manifold structure implicitly encoded by the graph; (2) it uses a fixed neighborhood graph and focuses only on the convolution of a graph, but pays little attention to graph construction; (3) it rarely considers the problem of topological imbalance.To overcome the above shortcomings, in this paper, we propose a novel semi-supervised learning method called Re-weight Nodes and Graph Learning Convolutional Network with Manifold Regularization (ReNode-GLCNMR). Our proposed method simultaneously integrates graph learning and graph convolution into a unified network architecture, which also enforces label smoothing through an unsupervised loss term. At the same time, it addresses the problem of imbalance in graph topology by adaptively reweighting the influence of labeled nodes based on their distances to the class boundaries. Experiments on 8 benchmark datasets show that ReNode-GLCNMR significantly outperforms the state-of-the-art semi-supervised GNN methods.11The code is available at https://github.com/BiJingjun/ReNode-GLCNMR.

Semi-supervised multi-scale attention-aware graph convolution network for intelligent fault diagnosis of machine under extremely-limited labeled samples

Labeled data are generally scarce in engineering practice, while data-driven methods fail to mine the correlations between samples to utilize the rich unlabeled data, so they cannot achieve satisfactory performance under limited labeled data. To address this problem, a semi-supervised multi-scale attention-aware graph convolution network (MSA-GCN) is proposed for fault diagnosis under extremely-limited labeled samples. First, available labeled data are transformed with unlabeled data into a graph via determining the k-nearest neighbors in frequency domain to construct the neighbor relations. To obtain the useful structural and feature information of unlabeled samples from different neighborhoods, multi-scale graph convolution is implemented to aggregate multi-scale information for labeled samples. Besides, attention mechanism is utilized and a novel adaptive feature fusing layer is designed to achieve cross-scale information fusion of different neighborhoods. With semi-supervised graph learning, the proposed method can fully utilize topological and feature information from unlabeled samples, resulting in a powerful classifier using only few labeled samples. The proposed method is fully verified on three bearing datasets, experimental results show that MSA-GCN can reach an identification accuracy of above 95 % with even as few as 5 labeled training samples each class, which demonstrates its effectiveness under low-label-ratio data.

A Weakly Supervised Learning-Based Oversampling Framework for Class-Imbalanced Fault Diagnosis

With the lack of failure data, class imbalance has become a common challenge in the fault diagnosis of industrial systems. The oversampling methods can tackle the class-imbalanced problem by generating the minority samples to balance the training set. However, one of the main challenges of the existing oversampling methods is how to generate high-quality minority samples. Traditional oversampling methods regard all synthetic samples as minority ones to be added to the training set without filtering. The low-quality synthetic samples would distort the distribution of the dataset and worsen the classification performance. In this article, we propose a weakly supervised oversampling method that treats all synthetic samples as unlabeled samples and develops a graph semisupervised learning algorithm to select high-quality synthetic samples, adding into the final training set as minority samples. To improve the quality of synthetic samples, we propose a cost-sensitive neighborhood component analysis dimensionality reduction method to enhance domain information validity in high-dimensional datasets. Finally, combining a boosting-based ensemble framework, we propose a new imbalanced learning framework suitable for high dimensionality and highly imbalanced fault diagnosis in industrial systems. The experimental validation is performed on five real-world wind turbine blade cracking failure datasets and compared to 15 benchmark methods. The experimental results show that average performances and robustness of the proposed framework are significantly better than those of the benchmark methods.

Single image shadow detection via uncertainty analysis and GCN-based refinement strategy

Learning-based shadow detection methods have achieved an impressive performance, while these works still struggle on complex scenes, especially ambiguous soft shadows. To tackle this issue, this work proposes an efficient shadow detection network (ESDNet) and then applies uncertainty analysis and graph convolutional networks for detection refinement. Specifically, we first aggregate global information from high-level features and harvest shadow details in low-level features for obtaining an initial prediction. Secondly, we analyze the uncertainty of our ESDNet for an input shadow image and then take its intensity, expectation, and entropy into account to formulate a semi-supervised graph learning problem. Finally, we solve this problem by training a graph convolution network to obtain the refined detection result for every training image. To evaluate our method, we conduct extensive experiments on several benchmark datasets, i.e., SBU, UCF, ISTD, and even on soft shadow scenes. Experimental results demonstrate that our strategy can improve shadow detection performance by suppressing the uncertainties of false positive and false negative regions, achieving state-of-the-art results.

Semi-Supervised Superpixel-Based Multi-Feature Graph Learning for Hyperspectral Image Data

Graphs naturally lend themselves to model the complexities of Hyperspectral Image (HSI) data as well as to serve as semi-supervised classifiers by propagating given labels among nearest neighbours. In this work, we present a novel framework for the classification of HSI data in light of a very limited amount of labelled data, inspired by multi-view graph learning and graph signal processing. Given an a priori superpixel-segmented hyperspectral image, we seek a robust and efficient graph construction and label propagation method to conduct semi-supervised learning (SSL). Since the graph is paramount to the success of the subsequent classification task, particularly in light of the intrinsic complexity of HSI data, we consider the problem of finding the optimal graph to model such data. Our contribution is two-fold: firstly, we propose a multi-stage edge-efficient semi-supervised graph learning framework for HSI data which exploits given label information through pseudo-label features embedded in the graph construction. Secondly, we examine and enhance the contribution of multiple superpixel features embedded in the graph on the basis of pseudo-labels in an extension of the previous framework, which is less reliant on excessive parameter tuning. Ultimately, we demonstrate the superiority of our approaches in comparison with state-of-the-art methods through extensive numerical experiments.

A distributed algorithm for graph semi-supervised learning

Graph semi-supervised learning (GSSL) plays an important role in data classification by leveraging the similarity across the graph topology and convex optimization with Laplacian-based regularization. However, the current algorithm to solve the problem is centralized approach calling for heavy computational cost, particularly when the data is of large volume. In this paper, an innovate distributed algorithm is proposed to solve the problem, which is based on the decomposition of the similar graph. Contrary to the centralized approach, the distributed algorithm only requires the neighboring information for solving the optimization. It is proved that difference between the solutions of the distributed algorithm and centralized counterpart is upper bounded. We apply the proposed algorithm to both the synthetic and real-world datasets. The numerical results verify the effectiveness of the proposed distributed algorithm.

Cyclic label propagation for graph semi-supervised learning

Graph neural networks (GNNs) have emerged as effective approaches for graph analysis, especially in the scenario of semi-supervised learning. Despite its success, GNN often suffers from over-smoothing and over-fitting problems, which affects its performance on node classification tasks. We analyze that an alternative method, the label propagation algorithm (LPA), avoids the aforementioned problems thus it is a promising choice for graph semi-supervised learning. Nevertheless, the intrinsic limitations of LPA on feature exploitation and relation modeling make propagating labels become less effective. To overcome these limitations, we introduce a novel framework for graph semi-supervised learning termed as Cyclic Label Propagation (CycProp for abbreviation), which integrates GNNs into the process of label propagation in a cyclic and mutually reinforcing manner to exploit the advantages of both GNNs and LPA. In particular, our proposed CycProp updates the node embeddings learned by GNN module with the augmented information by label propagation, while fine-tunes the weighted graph of label propagation with the help of node embedding in turn. After the model converges, reliably predicted labels and informative node embeddings are obtained with the LPA and GNN modules respectively. Extensive experiments on various real-world datasets are conducted, and the experimental results empirically demonstrate that the proposed CycProp model can achieve relatively significant gains over the state-of-the-art methods.

LaPOLeaF: Label propagation in an optimal leading forest

This paper presents an efficient graph semisupervised learning (GSSL) method that meets the criterion of optimization without iterations. Most existing GSSL methods require iterative optimization to achieve a preset objective because they consider data points to be in peer-to-peer relationships. Additionally, existing GSSL methods must learn from scratch for unseen data because graph structures are specifically built for a given dataset. By leveraging the partial order relationships induced by the local density and distances between data, we developed a novel label propagation algorithm based on the data structure of an optimal leading forest (OLeaF). The time complexity of our method is O(N) for both labeling unclassified data and labeling new data from a dataset after an OLeaF is constructed. Therefore, the two main weaknesses of traditional GSSL are addressed. Additionally, the constructed leading forest offers good interpretability for learning results. We scale the proposed method to accommodate big data by utilizing the block distance matrix technique and locality-sensitive hashing. Extensive experiments on datasets with different characteristics demonstrate the superior efficiency and competitive accuracy of the proposed method.

An Uncertainty-Driven GCN Refinement Strategy for Organ Segmentation

Organ segmentation in CT volumes is an important pre-processing step in many computer assisted intervention and diagnosis methods. In recent years, convolutional neural networks have dominated the state of the art in this task. However, since this problem presents a challenging environment due to high variability in the organ’s shape and similarity between tissues, the generation of false negative and false positive regions in the output segmentation is a common issue. Recent works have shown that the uncertainty analysis of the model can provide us with useful information about potential errors in the segmentation. In this context, we proposed a segmentation refinement method based on uncertainty analysis and graph convolutional networks. We employ the uncertainty levels of the convolutional network in a particular input volume to formulate a semi-supervised graph learning problem that is solved by training a graph convolutional network. To test our method we refine the initial output of a 2D U-Net. We validate our framework with the NIH pancreas dataset and the spleen dataset of the medical segmentation decathlon. We show that our method outperforms the state-of-the-art CRF refinement method by improving the dice score by 1% for the pancreas and 2% for spleen, with respect to the original U-Net’s prediction. Finally, we perform a sensitivity analysis on the parameters of our proposal and discuss the applicability to other CNN architectures, the results, and current limitations of the model for future work in this research direction. For reproducibility purposes, we make our code publicly available at https://github.com/rodsom22/gcn_refinement