Semi-supervised Classification Research Articles

Discrimination of high impedance fault in microgrid power network using semi-supervised machine learning algorithm

This work proposes a semi-supervised classification approach for discriminating high-impedance (HI) faults and other transients in a photovoltaic (PV) interconnected microgrid (MG) network. The suggested classifier combines unsupervised K-means clustering with the supervised multi-layer perceptron neural network algorithm. The K-means clustering technique is utilized in the first phase to detect and remove irrelevant instances from multiple events in the data set. To obtain the final predictions of targeted labels, clustered cases from the first phase are utilized to learn the multi-layer perceptron neural network classifier in the next phase. The suggested method outperforms stand-alone classifiers (K-means clustering and multi-layer perceptron) by providing enhanced accuracy and success rate of discriminating HI fault under standard test conditions and weather intermittency of PV. Furthermore, the results of the performance study clearly show that the suggested model is more resilient and offers superior performance than the stand-alone classifiers under the standard test condition and uncertainty of PV in MG networks.

An efficient framework based on local multi-representatives and noise-robust synthetic example generation for self-labeled semi-supervised classification

An efficient framework based on local multi-representatives and noise-robust synthetic example generation for self-labeled semi-supervised classification

An Iterative Pseudo Label Generation framework for semi-supervised hyperspectral image classification using the Segment Anything Model

Hyperspectral image classification in remote sensing often encounters challenges due to limited annotated data. Semi-supervised learning methods present a promising solution. However, their performance is heavily influenced by the quality of pseudo labels. This limitation is particularly pronounced during the early stages of training, when the model lacks adequate prior knowledge. In this paper, we propose an Iterative Pseudo Label Generation (IPG) framework based on the Segment Anything Model (SAM) to harness structural prior information for semi-supervised hyperspectral image classification. We begin by using a small number of annotated labels as SAM point prompts to generate initial segmentation masks. Next, we introduce a spectral voting strategy that aggregates segmentation masks from multiple spectral bands into a unified mask. To ensure the reliability of pseudo labels, we design a spatial-information-consistency-driven loss function that optimizes IPG to adaptively select the most dependable pseudo labels from the unified mask. These selected pseudo labels serve as iterative point prompts for SAM. Following a suitable number of iterations, the resultant pseudo labels can be employed to enrich the training data for the classification model. Experiments conducted on the Indian Pines and Pavia University datasets demonstrate that even a simple 2D CNN based classification model trained with our generated pseudo labels significantly outperforms eight state-of-the-art hyperspectral image classification methods.

Contrastive multi-graph learning with neighbor hierarchical sifting for semi-supervised text classification

Contrastive multi-graph learning with neighbor hierarchical sifting for semi-supervised text classification

IGCN: A Provably Informative GCN Embedding for Semi-Supervised Learning With Extremely Limited Labels.

Graph Neural Networks (GNNs) have gained much more attention in the representation learning for the graph-structured data. However, the labels are always limited in the graph, which easily leads to the overfitting problem and causes the poor performance. To solve this problem, we propose a new framework called IGCN, short for Informative Graph Convolutional Network, where the objective of IGCN is designed to obtain the informative embeddings via discarding the task-irrelevant information of the graph data based on the mutual information. As the mutual information for irregular data is intractable to compute, our framework is optimized via a surrogate objective, where two terms are derived to approximate the original objective. For the former term, it demonstrates that the mutual information between the learned embeddings and the ground truth should be high, where we utilize the semi-supervised classification loss and the prototype based supervised contrastive learning loss for optimizing it. For the latter term, it requires that the mutual information between the learned node embeddings and the initial embeddings should be high and we propose to minimize the reconstruction loss between them to achieve the goal of maximizing the latter term from the feature level and the layer level, which contains the graph encoder-decoder module and a novel architecture GCN Info. Moreover, we provably show that the designed GCN Info can better alleviate the information loss and preserve as much useful information of the initial embeddings as possible. Experimental results show that the IGCN outperforms the state-of-the-art methods on 7 popular datasets.

Fast semi-supervised classification based on anchor graph

Fast semi-supervised classification based on anchor graph

Semantic graph neural network with multi-measure learning for semi-supervised classification

Graph Neural Networks (GNNs) have attracted increasing attention in recent years and have achieved excellent performance in semi-supervised node classification tasks. The success of most GNNs is attributed to the availability of the original graph structure. However, recent studies have shown that GNNs are vulnerable to the complex underlying structure of the graph, making it necessary to learn comprehensive and robust graph structures for downstream tasks, rather than relying only on the raw graph structure. In light of this, we seek to learn optimal graph structures for downstream tasks and propose a novel framework for semi-supervised classification. Specifically, based on the structural context information of graph and node representations, we encode the complex interactions in semantics and generate semantic graphs to preserve the global structure. Moreover, we develop a novel multi-measure attention layer to optimize the similarity rather than prescribing it a priori, so that the similarity can be adaptively evaluated by integrating measures. These graphs are fused and optimized together with GNN towards semi-supervised classification objective. Extensive experiments and ablation studies on six real-world datasets clearly demonstrate the effectiveness of our proposed model and the contribution of each component. The proposed model not only addresses the inherent vulnerabilities of GNNs to complex graph structures, but also introduces a pioneering approach to learning comprehensive and robust graph representations for semi-supervised classification tasks.

Robust graph mutual-assistance convolutional networks for semi-supervised node classification tasks

Graph convolutional networks (GCNs) have made substantial progress in semi-supervised learning. However, GCNs are sensitive to graph noise, making the reconstruction of imperfect graph structures essential for adapting GCNs to complex scenarios. Prevalent GCNs employ graph augmentation to generate reliable augmented graphs, thereby mining potential graph information. Since traditional GCN layers can only perform message passing independently within each graph, GCNs primarily concentrate on fusing information from the node embeddings independently learned by the reliable augmented graphs. Nevertheless, the neglect of cross-graph information exchange during message passing limits the model's inter-graph interactions ability. In this paper, we propose a GCN framework called Robust graph mutual-assistance convolutional networks (RamNet), which introduces cross-graph information exchange mechanisms into the GCN layers. First, RamNet employs a multi-metric topology augmentation strategy to generate a synthesized graph, thereby exploring more reliable information. Second, it learns inter-graph interaction information by achieving information exchange between the raw graph and the synthesized graph during message passing. Finally, in addition to utilizing a small amount of supervised information, it also captures rich self-supervision information by maximizing consistency relationship between input graphs. We conduct comprehensive experiments in both standard and noisy scenarios, highlighting the superior performance of RamNet.

Scalable and Adaptive Graph Neural Networks with Self-Label-Enhanced Training

Although GNNs have achieved success in semi-supervised graph learning tasks, common GNNs suffer from expensive message passing during each epoch and the exponentially growing receptive field occupying too much memory, especially on large graphs. Neighbor sampling techniques can reduce GNNs’ memory footprints, but they encounter either redundant computation or incomplete edges. Some simplified GNNs decouple graph convolutions and feature transformations to reduce computation in training. However, only a part of them can scale to large graphs without neighbor sampling techniques, which can be concluded as decoupled GNNs. Nevertheless, they either only utilize the last convolution output or simply add multi-hop features with uniform weights, which limits their expressiveness. In this paper, we refine the pipeline of decoupled GNNs and propose Scalable and Adaptive Graph Neural Networks (SAGN), which effectively leverages multi-hop information with a scalable attention mechanism. Moreover, we generalize the input of decoupled GNNs to view another classical technique, label propagation, as a special case of decoupled GNNs and propose decoupled label trick (DecLT) to incorporate label information into decoupled GNNs. Furthermore, by incorporating self-training technique, we further propose the Self-Label-Enhanced (SLE) training framework, leveraging pseudo labels to simultaneously augment the training set and improve label propagation. Extensive experiments show that SAGN outperforms other baselines, and that DecLT and SLE can consistently and significantly improve all types of models on semi-supervised node classification tasks. Many top-ranked models on Open Graph Benchmark (OGB) leaderboard adopt our methods as the main backbone.

Semi-supervised convolutional generative adversarial networks for dynamic fault classification with manifold regularization

Data-driven fault classification identifies and categorizes faults or anomalies in industrial systems, to ensure efficient and safe operations. This paper addresses the challenges of semi-supervised learning for dynamic fault classification in industrial processes. The key problems involve utilizing three unique characteristics of typical industrial process dataset: 1) the presence of unlabeled samples due to high labeling costs and required expert knowledge, 2) the local manifold structure arising from strong variable coupling, and 3) time correlation inherent in time-series sensing data of dynamic processes. To tackle these challenges, a semi-supervised fault classification model, SSDCGAN, is proposed, based on deep convolutional Generative Adversarial Networks. The model captures dynamic temporal information through a moving window technique and leverages manifold regularization to maintain classifier consistency along the manifold’s tangent direction. Evaluations on the Tennessee Eastman (TE) benchmark demonstrate that SSDCGAN enhances fault classification accuracy, outperforming current methods.

I2HGNN: Iterative Interpretable HyperGraph Neural Network for semi-supervised classification

Learning on hypergraphs has garnered significant attention recently due to their ability to effectively represent complex higher-order interactions among multiple entities compared to conventional graphs. Nevertheless, the majority of existing methods are direct extensions of graph neural networks, and they exhibit noteworthy limitations. Specifically, most of these approaches primarily rely on either the Laplacian matrix with information distortion or heuristic message passing techniques. The former tends to escalate algorithmic complexity, while the latter lacks a solid theoretical foundation. To address these limitations, we propose a novel hypergraph neural network named I2HGNN, which is grounded in an energy minimization function formulated for hypergraphs. Our analysis reveals that propagation layers align well with the message-passing paradigm in the context of hypergraphs. I2HGNN achieves a favorable trade-off between performance and interpretability. Furthermore, it effectively balances the significance of node features and hypergraph topology across a diverse range of datasets. We conducted extensive experiments on 15 datasets, and the results highlight the superior performance of I2HGNN in the task of hypergraph node classification across nearly all benchmarking datasets.

HAIS-IDS:A Hybrid Artificial Immune System Model for Intrusion Detection in IoT

The application of the Internet of Things(IoT) is increasing exponentially, the dynamic data flow and distributive operation over low resource devices possesses huge threat to sensitive human data. This paper introduces an artificial immune system (AIS) based approach to intrusion detection in IoT network ecosystems, the proposed approach implements dual-layered AIS; which is robust to zero-day attacks and designed to adapt new types of attack classes in the form of antibodies.In this paper, a Hybrid method has been presented which uses Hybrid of Clonal Selection using Variation auto-encoders as Innate Immune Layer and Apaptive Dentritic Model for identifying intrusions over IoT Specific Datasets.Moreover we present extensive empirical analysis over six IoT network benchmark datasets for semi-supervised multi-class classification task and obtain superior performance compared to five state-of-the-art baselines. Finally, VC-ADIS achieves 99.83% accuracy over MQTT-set dataset.

Contrastive message passing for robust graph neural networks with sparse labels

Graph Neural Networks (GNNs) have achieved great success in semi-supervised learning. Existing GNNs typically aggregate the features via message passing with the aid of rich labels. However, real-world graphs have limited labels, and overfitting weakens the classification ability when labels are insufficient. Besides, traditional message passing is sensitive to structure noise, such as perturbations on edges. The performance of GNNs would drop sharply when trained on such graphs. To mitigate these issues, we present a noise-resistant framework via contrastive message passing. Except for the limited labelled nodes as supervision widely used in GNNs, we model the topology structure by graph likelihood as extra supervision. Specifically, we first propose contrastive graph likelihood, which is defined as a product of the edge likelihood on all connected node pairs. Then, we apply two unfolding updated steps via descent iterations. The first step updates the features in a single view with the aid of initializing the edge probability. Then the second step applies a binary edge for homophily and heterophily views, respectively. The homophily view applies attractive force to pull the positive-connected nodes close; otherwise, the heterophily view utilizes repulsive force to push away the negative-connected nodes. Extensive experiments show that our method has superior performance on semi-supervised node classification tasks with sparse labels and excellent robustness under perturbations in structure.

Multi-Level Cross-Modal Interactive-Network-Based Semi-Supervised Multi-Modal Ship Classification.

Ship image classification identifies the type of ships in an input image, which plays a significant role in the marine field. To enhance the ship classification performance, various research focuses on studying multi-modal ship classification, which aims at combining the advantages of visible images and infrared images to capture complementary information. However, the current methods simply concatenate features of different modalities to learn complementary information, which neglects the multi-level correlation between different modalities. Moreover, the existing methods require a large amount of labeled ship images to train the model. How to capture the multi-level cross-modal correlation between unlabeled and labeled data is still a challenge. In this paper, a novel semi-supervised multi-modal ship classification approach is proposed to solve these issues, which consists of two components, i.e., multi-level cross-modal interactive network and semi-supervised contrastive learning strategy. To learn comprehensive complementary information for classification, the multi-level cross-modal interactive network is designed to build local-level and global-level cross-modal feature correlation. Then, the semi-supervised contrastive learning strategy is employed to drive the optimization of the network with the intra-class consistency constraint based on supervision signals of unlabeled samples and prior label information. Extensive experiments on the public datasets demonstrate that our approach achieves state-of-the-art semi-supervised classification effectiveness.

Enhanced Semi-Supervised Medical Image Classification Based on Dynamic Sample Reweighting and Pseudo-Label Guided Contrastive Learning (DSRPGC)

In semi-supervised learning (SSL) for medical image classification, model performance is often hindered by the scarcity of labeled data and the complexity of unlabeled data. This paper proposes an enhanced SSL approach to address these challenges by effectively utilizing unlabeled data through a combination of pseudo-labeling and contrastive learning. The key contribution of our method is the introduction of a Dynamic Sample Reweighting strategy to select reliable unlabeled samples, thereby improving the model’s utilization of unlabeled data. Additionally, we incorporate multiple data augmentation strategies based on the Mean Teacher (MT) model to ensure consistent outputs across different perturbations. To better capture and integrate multi-scale features, we propose a novel feature fusion network, the Medical Multi-scale Feature Fusion Network (MedFuseNet), which enhances the model’s ability to classify complex medical images. Finally, we introduce a pseudo-label guided contrastive learning (PGC) loss function that improves intra-class compactness and inter-class separability of the model’s feature representations. Extensive experiments on three public medical image datasets demonstrate that our method outperforms existing SSL approaches, achieving 93.16% accuracy on the ISIC2018 dataset using only 20% labeled data, highlighting the potential of our approach to advance medical image classification under limited supervision.

Novel Method for Evaluating Wetland Ecological Environment Quality Based on Coupled Remote Sensing Ecological Index and Landscape Pattern Indices: Case Study of Dianchi Lake Wetlands, China

Wetlands serve as crucial ecological buffers, significantly influencing temperature reduction, carbon storage, regional climate regulation, and urban wastewater treatment. To elucidate the relationship between wetland landscape patterns and ecological environment, and to accurately assess lake ecosystems, this study proposes a semi-supervised classification method based on RSEI and K-Means. By integrating landscape pattern indices, the Remote Sensing Ecological Index (RSEI), and disturbance proximity, a comprehensive evaluation of the ecological quality of the Dianchi wetlands was conducted. The results indicate that the RSEI-K-Means method, with K set to 50, achieved overall accuracies (OAs) and Kappa values of 0.91 and 0.88, surpassing the SVM’s 0.85 and 0.80. This method effectively combines ecological and landscape indices without relying on extensive training samples, enhancing accuracy and speed in wetland information extraction and addressing the challenges of spatial heterogeneity. This study reveals that from 2007 to 2009, and 2013 to 2015, landscape patterns were significantly influenced by the rapid expansion of Kunming city, exacerbating wetland fragmentation. Notably, significant ecological quality changes were observed in 2009 and 2013, with gradual recovery post-2013 due to strengthened environmental protection policies. The RSEI disturbance proximity analysis indicated that the affected areas were primarily concentrated in regions of high human activity, confirming the method’s high sensitivity and effectiveness. This study can help in wetland ecosystem research and management.

Selective-Hop Graph Neural Networks for semi-supervised node classification

Message-passing Graph Neural Networks (GNNs) are remarkably successful in graph representation learning. Most of them propagate and aggregate information within fixed-size receptive-fields (RFs) for all nodes. In this paper, our empirical studies discover that different-sized RFs are often desired for distinct graphs and target nodes. To this end, we propose a dynamic selection mechanism by allowing each node to adaptively adjust the size of its RFs according to the graph properties. In particular, we design an architectural unit called Selective-Hop (SH) block, in which information from neighborhoods within different hops are merged using dual attention mechanisms: (i) node-wise attention on each network branch to capture mutual information between the subgraphs and the entire graph; and (ii) channel-wise attention on different network branches to detect the significance of different hops at the node-level granularity. As such, the SH block determines neighbors flexibly without the constraints of hops. By stacking several SH blocks in GNN style, we instantiate two new GNN variants, SHGCN and SHSGC. Empirical results on eight benchmark datasets for semi-supervised node classification reveal that our new models achieve a considerable improvement over the state-of-the-art methods under low-homophily or heterophily and maintain competitive performance under homophily.

Multi-scale structure-guided graph generation for multi-view semi-supervised classification

Graph convolutional network has emerged as a focal point in machine learning because of its robust graph processing capability. Most existing graph convolutional network-based approaches are designed for single-view data, yet in many practical scenarios, data is represented through multiple views. Moreover, due to the complexity of multiple views, normal graph generation methods cannot mitigate redundancy to generate a high quality graph. Although the ability of graph convolutional network is undeniable, the quality of graph directly affects its performance. To tackle the aforementioned challenges, this paper proposes a multi-scale graph generation deep learning framework, called multi-scale semi-supervised graph generation based multi-view classification, consisting of two modules: edge sampling and path sampling. The former aims to generate an adjacency graph by selecting edges based on the maximum likelihood among graphs from different views. Meanwhile, the latter seeks to construct an adjacency graph according to the characteristics of paths within the graphs. Finally, the statistical technique is employed to extract commonality and generate a fused graph. Extensive experimental results robustly demonstrate the superior performance of our proposed framework, compared to other state-of-the-art multi-view semi-supervised approaches.

Positive Unlabeled Learning Selected Not At Random (PULSNAR): class proportion estimation without the selected completely at random assumption.

Positive and unlabeled (PU) learning is a type of semi-supervised binary classification where the machine learning algorithm differentiates between a set of positive instances (labeled) and a set of both positive and negative instances (unlabeled). PU learning has broad applications in settings where confirmed negatives are unavailable or difficult to obtain, and there is value in discovering positives among the unlabeled (e.g., viable drugs among untested compounds). Most PU learning algorithms make the selected completely at random (SCAR) assumption, namely that positives are selected independently of their features. However, in many real-world applications, such as healthcare, positives are not SCAR (e.g., severe cases are more likely to be diagnosed), leading to a poor estimate of the proportion, α, of positives among unlabeled examples and poor model calibration, resulting in an uncertain decision threshold for selecting positives. PU learning algorithms vary; some estimate only the proportion, α, of positives in the unlabeled set, while others calculate the probability that each specific unlabeled instance is positive, and some can do both. We propose two PU learning algorithms to estimate α, calculate calibrated probabilities for PU instances, and improve classification metrics: i) PULSCAR (positive unlabeled learning selected completely at random), and ii) PULSNAR (positive unlabeled learning selected not at random). PULSNAR employs a divide-and-conquer approach to cluster SNAR positives into subtypes and estimates α for each subtype by applying PULSCAR to positives from each cluster and all unlabeled. In our experiments, PULSNAR outperformed state-of-the-art approaches on both synthetic and real-world benchmark datasets.

Semi-supervised spectral classification of DESI white dwarfs by dimensionality reduction

Abstract As a new generation of large-sky spectroscopic surveys comes online, the enormous data volume poses unprecedented challenges in classifying spectra. Modern unsupervised techniques have the power to group spectra based on their dominant features, circumventing the complete reliance on training data suffered by supervised methods. We outline the use of dimensionality reduction to generate a 2D map of the structure of an intermediate-resolution spectroscopic dataset. This technique efficiently separates white dwarfs of different spectral classes in the Dark Energy Spectroscopic Instrument’s Early Data Release (DESI EDR), identifying spectral features that had been missed even by visual classification. By focusing the method on particular spectral regions, we identify white dwarfs with helium features at 90percnt recall, and cataclysmic variables at 100percnt recall, illustrating rapid selection of low-contamination samples from spectroscopic surveys. We also demonstrate the use of dimensionality reduction in a supervised manner, outlining a procedure to classify any white dwarf spectrum in comparison with those in the DESI EDR. With upcoming surveys promising tens of millions of spectra, our work highlights the potential for semi-supervised techniques as an efficient means of classification and dataset visualisation.