Semi-supervised Classification Method Research Articles

Semi-supervised learning with graph convolutional extreme learning machines

Extreme Learning Machine (ELM) has been widely used for various classification problems. However, the traditional ELMs are typically based on the regular Euclidean data, thus ignoring the intrinsic structured information among data and resulting in poor robustness. In this paper, we present a novel semi-supervised learning framework, termed Graph Convolutional Extreme Learning Machines (GCELM), based on extending the traditional ELM to the non-Euclidean domain. Technically, we recast ELM layers into a randomized graph convolutional embedding layer followed by a graph convolutional regression layer, which endows ELM with the capability to handle graphs in the non-Euclidean domain. Benefiting from the diversity of the randomized graph convolution, we further propose an enhanced GCELM, i.e., Voting-based GCELM (V-GCELM), by using a simple voting ensemble strategy. The proposed methods preserve the advantages of ELMs, thus being more efficient than the gradient-based graph convolutional networks but without loss of graph learning ability. Extensive experiments on 36 benchmark datasets demonstrate that the proposed methods significantly outperform many previous semi-supervised classification methods.

A local semi-supervised ensemble learning strategy for the data‐driven soft sensor of the power prediction in wind power generation

Soft sensors have been widely employed to estimate the power prediction in wind power generation that are difficult to measure in real-time. However, strongly nonlinear, dynamic time-varying, and multi-rate data characteristics are major causes of poor soft sensor performance. To address this problem, a strategy of building a soft sensor model based on local semi-supervised ensemble learning of least squares support vector regression (LS-EL-LSSVR) is proposed in this paper. Firstly, to reduce the influence of nonlinearity, a localization method based on the combination of time and space criteria is proposed, which can overcome the drawback of the clustering method based on a spatial criterion. Subsequently, the self-training semi-supervised classification method is used to achieve semi-supervised clustering of unlabeled samples to improve the generalization performance of the soft sensor model. Afterward, based on the local semi-supervised clustering sample dataset, the soft sensor model is established according to the least squares support vector machine method. Moreover, a selective ensemble learning method based on the prediction accuracy of local models and an adaptive combination weight calculation method of sub-models is proposed to realize selective ensemble learning and improve the prediction accuracy. The parameters of the proposed method are finalized automatically by the particle swarm optimization technique. The proposed method is simulated by using the actual spatial dynamic wind power prediction dataset, the results demonstrate the effectiveness of the proposed method in dealing with nonlinear, dynamic time-varying, and multi-rate data regression problems in wind power generation process.

Dual-feature-embeddings-based semi-supervised learning for cognitive engagement classification in online course discussions

Online course discussions contain abundant cognitive information from learners. Previous models required a large amount of labeled data to classify cognitive engagement from the perspective of semantic features alone. However, these models only contain semantic features but cannot fully represent textual information and have poor performance in cases of scarce labeled data. Moreover, cognitive psychological features imply important information that cannot be captured by semantic features. Therefore, this paper proposes a dual feature embedding-based semi-supervised cognitive classification method that exploits the additional inductive biases caused by implicit cognitive features to supplement generic semantic features. Additional inductive biases facilitate the propagation of labeled and unlabeled data and improve the consistency between unlabeled and augmented data. Unsupervised data augmentation (UDA) is used to obtain augmented data by inserting advanced noise into unlabeled data in semi-supervised learning. Furthermore, bidirectional encoder representations from transformers (BERT) are used to extract generic semantics, and linguistic inquiry and word count (LIWC) are adopted to fetch implicit cognitive features from discussion texts. Therefore, we refer to the proposed method as B-LIWC-UDA, sequentially fusing the dual features in the explicit and hidden levels to obtain dual feature embeddings. The cognitive engagement classification model was trained using supervised and consistent training methods. We conducted experiments using datasets obtained from two real-world online course discussions. The experimental results demonstrate that, in terms of major evaluation metrics, the proposed B-LIWC-UDA method performs better than state-of-the-art text classification methods used for identifying cognitive engagement.

NaNG-ST: A natural neighborhood graph-based self-training method for semi-supervised classification

The self-training method has been favored by scholars in semi-supervised classification. One of the greatest challenges in self-training methods is finding high-confidence unlabeled samples at each iteration. While multiple variations of self-training methods are developed, the strategy of finding high-confidence unlabeled samples heavlies on parameters and only utilizes labeled data alone, which makes the self-training method limited by the distribution of labeled data. To solve the above issues, a novel natural neighborhood graph-based self-training method (NaNG-ST) is proposed. In NaNG-ST, a parameter-free natural neighborhood graph (NaNG) is first constructed. The NaNG roughly reveals the real data distribution by exploiting unlabeled and labeled data. Based on NaNG, homogeneous and heterogeneous edges are defined to divide unlabeled samples into three cases. After that, homogeneous and heterogeneous edges can use the revealed distribution rather than labeled data alone to help NaNG-ST fast and effectively find confident unlabeled samples without any parameters. Besides, they also help NaNG-ST not to be limited by the distribution of initial labeled data. When a few initial labeled data can not roughly represent the distribution of the entire data, the NaNG helps NaNG-ST restore the real data distribution better. Intensive experiments on real-world data sets prove that NaNG-ST outperforms 7 popular semi-supervised self-taught approaches in terms of classification accuracy, mean F-measure and required running time.

An effective knowledge transfer method based on semi-supervised learning for evolutionary optimization

Effective knowledge transfer has been proven to achieve superior performance in evolutionary optimization. Evolutionary multitasking optimization (EMT), which can solve several optimization tasks simultaneously using evolutionary algorithms, continues to be a young research field but is growing rapidly. Due to the parallelism of population-based search, the performance of component tasks can be improved through effective knowledge transfer between different tasks. The main challenge in the EMT field is addressing the negative transfer. Aiming to overcome this challenge, this paper proposes an EMT algorithm that transfers effective knowledge through semi-supervised learning. In addition, a semi-supervised classification method is designed based on the cluster assumption, which is part of the geometric basis of semi-supervised learning. By using both labeled and unlabeled samples generated in the optimization process, the proposed method can identify individuals that contain valuable knowledge and select them to transfer the knowledge between tasks. In this way, the performance of the EMT algorithm can be significantly improved. The effectiveness of the proposed method is verified by empirical tests and comparison with two benchmarks. Further, a case study is conducted. The results indicate that the proposed algorithm can achieve highly competitive performance compared with the state-of-the-art EMT algorithms.

MMatch: Semi-Supervised Discriminative Representation Learning for Multi-View Classification

Semi-supervised multi-view learning has been an important research topic due to its capability to exploit complementary information from unlabeled multi-view data. This work proposes MMatch, a new semi-supervised discriminative representation learning method for multi-view classification. Unlike existing multi-view representation learning methods that seldom consider the negative impact caused by particular views with unclear classification structures (weak discriminative views). MMatch jointly learns view-specific representations and class probabilities of training data. The representations concatenated to integrate multiple views’ information to form a global representation. Moreover, MMatch performs the smoothness constraint on the class probabilities of the global representation to improve pseudo labels, whereas the pseudo labels regularize the structure of view-specific representations. A discriminative global representation is mined with the training process, and the negative impact of weak discriminative views is overcome. Besides, MMatch learns consistent classification while preserving diverse information from multiple views. Experiments on several multi-view datasets demonstrate the effectiveness of MMatch.

Semi-supervised Time Series Classification Model with Self-supervised Learning

Semi-supervised learning is a powerful machine learning method. It can be used for model training when only part of the data are labeled. Unlike discrete data, time series data generally have some temporal relation, which can be considered as a supervised signal in semi-supervised learning to supervise the learning of unlabeled time series data. However, the currently known semi-supervised time series classification (TSC) methods always ignore or under-explore the temporal relation structure and fail to fully use the unlabeled time series data. Therefore, we propose a Semi-supervised Time Series Classification Model with Self-supervised Learning (SSTSC). It takes self-supervised learning as the auxiliary task and jointly optimizes it with the main TSC task. Specifically, it performs the TSC task on the labeled time series data; For the unlabeled time series data, it splits the “past-anchor-future” segments and constructs the positive/negative temporal relation samples with different combinations to accurately predict the temporal relations and capture the higher-quality semantic context in self-supervised learning as a supervised signal for TSC task. Experimental results demonstrate that SSTSC has better effects than the baselines from different perspectives.

A Novel Classification Performance Approach for Remotely Sensed Multispectral Image Data by Using Data Mining Techniques

In order to combine multimedia imagery and multispectral remote sensing data to analyze information, preprocessing becomes a necessary part of it. As one of the most important branches in the field of data analysis, it is widely used in many fields such as classification, regression, missing value filling, and machine learning. As a lazy algorithm, this method requires no prior statistical knowledge and no additional data to train description rules and is easy to implement. This study compares classification algorithm performances of data mining clustering algorithms for remotely sensed multispectral image data using WEKA data mining software. Clustering algorithm selection is very important for data mining classification method-based clustering. The class attribute for remotely sensed multispectral image data is obtained from six different clustering algorithms for classification. Classification algorithm performances computed depending on the data labeling of six different clustering algorithms in terms of correctly classified instances and kappa statistics for seven different classification algorithms. A strategy is developed for selecting the best unsupervised clustering algorithm, among different clustering algorithms, giving the highest supervised classification accuracy in terms of correctly classified instances and kappa statistics for semi supervised classification of remotely-sensed multispectral image data. The performances of seven semi-supervised classification methods assessed depending on six different unsupervised clustering algorithms for supervised classification of remotely sensed multispectral image data. This study determines data free clustering algorithms for classification.

RESPONSE OF TAXONOMIC AND FUNCTIONAL BENTHIC MACROFAUNA RICHNESS TO LAND USE AND LAND COVER AT BRAZILIAN RESERVOIRS DURING EXTREME DROUGHT

Introduction: Reservoirs are artificial aquatic ecosystems created by damming rivers, and are influenced by different sorts of impacts. Anthropic land use and occupation are one of the main causes of the loss of environmental quality and biological diversity in aquatic ecosystems. The objective of this work was to evaluate the effects of land use and occupation on the taxonomic and functional richness of benthic macroinvertebrates in reservoirs in the Brazilian semiarid region during a period of extreme drought. Material and methods: The study was conducted in six reservoirs located in the semiarid region from Brazilian Northeast (Paraíba and Rio Grande do Norte states): Sabugí, Passagem das Traíras, Cruzeta, Cordeiro, Sumé and Poções. The collections of benthic macrofauna, physical and chemical variables of water and the mapping of land use and occupation were performed for the years 2014 and 2019, in June and September. The sampling period was characterized as the most severe drought in the last 50 years in this study area. The land use and occupation were verified through the semi-supervised classification method of images captured by the Landsat 8 satellite, within 100 meters of the reservoir margin. The diversity of benthic macrofauna was analyzed through the taxonomic richness and functional richness, according to five functional characteristics: feeding trophic groups, breathing mode, voltinism, body size and body protection. To check for significant differences in physical and chemical water variables and taxonomic and functional richness among land use and land cover categories we performed PERMANOVA test. Results and Discussion: Seven land use and land cover categories were found in the reservoirs studied in the years 2014 and 2019, these are: water, agriculture, pasture, exposed soil, human occupation, open vegetation (native and exotic species of small/medium size with continuous spacing) and tree/shrub vegetation (native and exotic species of medium/large size with associated herbaceous stratum and predominance of semi-continuous dorsel). In all reservoirs the prevalent-use was pasture. Physical and chemical variables of water varied significantly among land use and land cover types, they were soluble reactive phosphate (p=0.02), total phosphorus (p=0.002) and water volume (p=0.01). The highest concentrations of total phosphorus were associated with predominant human occupation and soluble reactive phosphate with agriculture. The highest percentage of water volume was associated with a predominant use of tree/shrub vegetation. In reservoirs of semiarid regions with low connectivity due to intermittent rivers and extreme drought periods, adjacent anthropic activities such as agriculture, pasture and human occupation are the main external sources of nutrients. The taxonomic and functional richness of benthic macrofauna was higher in areas of open vegetation, however, it did not vary significantly between land use and land cover categories. Conclusions: Our results show that the effects of anthropic use and occupation in reservoirs associated with water scarcity lead to the homogenization of the macrobenthic community, which can generate negative consequences in processes sustained by the community.

Ptr4BERT: Automatic Semisupervised Chinese Government Message Text Classification Method Based on Transformer-Based Pointer Generator Network

With the development of Internet technology, government affairs can be handled online. More and more citizens are using online platforms to report to government departments, which is generating a lot of textual data. Among them, the basic but important problem is to automatically classify the different categories of messages, so that staff from different departments can process relevant information quickly. However, government messages have problems such as fast update rate, a large amount of information, long texts, and difficulty in capturing key points, which make supervised learning methods unsuitable for processing such texts. To address these problems, we propose a semisupervised text classification method based on a transformer-based pointer generator network named Ptr4BERT, which uses the pointer generator network with BERT(bidirectional encoder representation from transformers) embedding as a preprocessor for feature extraction. In this method, text classification can achieve very good results with a small set of labeled data, by extracting features exclusively from the message text. In order to verify the effect of our proposed model, we performed some experiments. Besides, we designed a crawler program and obtained two datasets from different websites, which are named HNMes and QDMes. Experimental results have shown that the proposed method outperforms the state-of-the-art methods significantly.

SECANT: a biology-guided semi-supervised method for clustering, classification, and annotation of single-cell multi-omics.

The recent advance of single cell sequencing (scRNA-seq) technology such as Cellular Indexing of Transcriptomes and Epitopes by Sequencing (CITE-seq) allows researchers to quantify cell surface protein abundance and RNA expression simultaneously at single cell resolution. Although CITE-seq and other similar technologies have gained enormous popularity, novel methods for analyzing this type of single cell multi-omics data are in urgent need. A limited number of available tools utilize data-driven approach, which may undermine the biological importance of surface protein data. In this study, we developed SECANT, a biology-guided SEmi-supervised method for Clustering, classification, and ANnoTation of single-cell multi-omics. SECANT is used to analyze CITE-seq data, or jointly analyze CITE-seq and scRNA-seq data. The novelties of SECANT include (1) using confident cell type label identified from surface protein data as guidance for cell clustering, (2) providing general annotation of confident cell types for each cell cluster, (3) utilizing cells with uncertain or missing cell type label to increase performance, and (4) accurate prediction of confident cell types for scRNA-seq data. Besides, as a model-based approach, SECANT can quantify the uncertainty of the results through easily interpretable posterior probability, and our framework can be potentially extended to handle other types of multi-omics data. We successfully demonstrated the validity and advantages of SECANT via simulation studies and analysis of public and in-house datasets from multiple tissues. We believe this new method will be complementary to existing tools for characterizing novel cell types and make new biological discoveries using single-cell multi-omics data.

Semi-supervised classification of fundus images combined with CNN and GCN.

Diabetic retinopathy (DR) is one of the most serious complications of diabetes, which is a kind of fundus lesion with specific changes. Early diagnosis of DR can effectively reduce the visual damage caused by DR. Due to the variety and different morphology of DR lesions, automatic classification of fundus images in mass screening can greatly save clinicians' diagnosis time. To alleviate these problems, in this paper, we propose a novel framework-graph attentional convolutional neural network (GACNN). The network consists of convolutional neural network (CNN) and graph convolutional network (GCN). The global and spatial features of fundus images are extracted by using CNN and GCN, and attention mechanism is introduced to enhance the adaptability of GCN to topology map. We adopt semi-supervised method for classification, which greatly improves the generalization ability of the network. In order to verify the effectiveness of the network, we conducted comparative experiments and ablation experiments. We use confusion matrix, precision, recall, kappa score, and accuracy as evaluation indexes. With the increase of the labeling rates, the classification accuracy is higher. Particularly, when the labeling rate is set to 100%, the classification accuracy of GACNN reaches 93.35%. Compared with DenseNet121, the accuracy rate is improved by 6.24%. Semi-supervised classification based on attention mechanism can effectively improve the classification performance of the model, and attain preferable results in classification indexes such as accuracy and recall. GACNN provides a feasible classification scheme for fundus images, which effectively reduces the screening human resources.

A plot-based analysis of the vegetation of the Northern Territory, Australia: a first assessment within the International Vegetation Classification framework

Aims: To develop an interim classification of the vegetation of the Northern Territory at the International Vegetation Classification (IVC) division (level 4) and macrogroup (level 5) levels. These types are produced to assist in the development of an integrated nationwide plot and floristically based classification of Australia allowing integration within a global perspective. Study Area: The Northern Territory of Australia covers an area of 1.42 million square kilometres, almost 20% of Australia’s land mass. It comprises three distinct climatic zones including tropical, subtropical and arid vegetation types. Methods: We used collated vegetation data held by two organisations: the Northern Territory Government, Department of Environment, Parks and Water Security and the Terrestrial Ecosystem Research Network (a total of 45,710 plots used). We applied semi-supervised quantitative classification methods to define vegetation types at the IVC division and macrogroup levels. Analyses used kR-CLUSTER methods on presence/absence data. Macrogroups were characterised by taxa with the highest frequency of occurrence across plots. Additional analyses were conducted (cluster) to elucidate interrelationships between macrogroups and to assist in the assessment of division level typology. Results: We propose 21 macrogroups and place these within higher thematic levels of the IVC. Conclusions: We found that the IVC hierarchy and associated standard procedures and protocols provide a useful classification tool for Australian ecosystems. The divisions and macrogroups provide a valid framework for subsequent analysis of Northern Territory vegetation types at the detailed levels of the IVC. A consistent typology for the Northern Territory (and hopefully in future, for all of Australia) has numerous benefits, in that they can be used for various applications using a well-structured, systematic and authoritative description and classification that is placed in a continental and global context, readily enabling the one system to be used in studies from the local to global level. Taxonomic reference: Northern Territory Herbarium (2022). Abbreviations: DVT = Definitive Vegetation Type; IVC = International Vegetation Classification; nMDS = non-metric multidimensional scaling; NT = Northern Territory; NTVSD = Northern Territory Vegetation Site Database; NVIS = National Vegetation Information System; WA = Western Australia.

Multigraph Random Walk for Joint Learning of Multiview Clustering and Semisupervised Classification

Recent researches on multiview learning have received widespread attention due to the increasing generalization of multiview data. As an effective probabilistic model, random walk has also shown encouraging performance in various fields. To further exploit the potential of utilizing random walk schemes to address multiview learning problems, this article proposes a simple yet efficient multigraph random walk scheme for both multiview clustering and semisupervised classification tasks. The proposed model integrates random walk with multiview learning, and recursively learns a globally stable probability distribution matrix from multiple views, on the basis of which the label indicator is obtained in the scene of clustering or semisupervised classification. Furthermore, an adaptive weight vector is learned to incorporate the diversity and complementarity of multiview data. Besides, the relationships between the proposed scheme and spectral clustering, neighborhood embedding and manifold embedding are analyzed theoretically. Finally, comprehensive comparative experiments are conducted with several state-of-the-art multiview clustering and semisupervised classification methods on eight real-world datasets. The experimental results demonstrate the superiority of the proposed method in terms of both clustering and classification performance.

Self-boosted with dynamic semi-supervised clustering method for imbalanced big data classification

Self-boosted with dynamic semi-supervised clustering method for imbalanced big data classification

Self-Boosted With Dynamic Semi-Supervised Clustering Method for Imbalanced Big Data Classification

Big data plays a major role in the learning, manipulation, and forecasting of information intelligence. Due to the imbalance of data delivery, the learning and retrieval of information from such large datasets can result in limited classification outcomes and wrong decisions. Traditional machine learning classifiers successfully handling the imbalanced datasets still there is inadequacy in overfitting problems, training cost, and sample hardness in classification. In order to forecast a better classification, the research work proposed the novel “Self-Boosted with Dynamic Semi-Supervised Clustering Method”. The method is initially preprocessed by constructing sample blocks using Hybrid Associated Nearest Neighbor heuristic over-sampling to replicate the minority samples and merge each copy with every sub-set of majority samples to remove the overfitting issue thus slightly reduce noise with the imbalanced data. After preprocessing the data, massive data classification requires big data space which leads to large training costs.

Semi-supervised Mode Classification of Inter-city Trips from Cellular Network Data

Good knowledge of travel patterns is essential in transportation planning. Cellular network data as a large-scale passive data source provides billions of daily location updates allowing us to observe human mobility with all travel modes. However, many transport planning applications require an understanding of travel patterns separated by travel mode, requiring the classification of trips by travel mode. Most previous studies have used rule-based or geometric classification, which often fails when the routes for different modes are similar or supervised classification, requiring labelled training trips. Sufficient amounts of labelled training trips are unfortunately often unavailable in practice. We propose semi-supervised classification as a novel approach of classifying large sets of trips extracted from cellular network data in inter-city origin–destination pairs as either using road or rail. Our methods require no labelled trips which is an important advantage as labeled data is often not available in practice. We propose three methods which first label a small share of trips using geometric classification. We then use structures in a large set of unlabelled trips using a supervised classification method (geometric-labelling), iterative semi-supervised training (self-labelling) and by transferring information between origin–destination pairs (continuity-labelling). We apply the semi-supervised classification methods on a dataset of 9545 unlabelled trips in two inter-city origin–destination pairs. We find that the methods can identify structures in the cells used during trips in the unlabelled data corresponding to the available route alternatives. We validate the classification methods using a dataset of 255 manually labelled trips in the two origin–destination pairs. While geometric classification misclassifies 4.2% and 5.6% of the trips in the two origin–destination pairs, all trips can be classified correctly using semi-supervised classification.

Fine-grained interactive attention learning for semi-supervised white blood cell classification

White blood cell (WBC) is an essential part of the human immune system. To diagnose blood diseases, hematologists have to think about the WBC information. For instance, the number of each type of WBCs often implies the health condition of the human body. Thus, the classification of white blood cell images plays a significant role in the medical diagnosis process. However, manual WBC inspection is time-consuming and labor-intensive for experts, which means automated classification methods are needed for WBC recognition. Another problem is that the traditional automatic recognition system needs a large amount of annotated medical images for training, which is highly costly. In this respect, the semi-supervised learning framework has recently been widely used for medical diagnosis due to its specificity, which can explore relevant information from massive unlabeled data. In this study, a novel semi-supervised white blood cell classification method is proposed, named by Fine-grained Interactive Attention Learning (FIAL). It consists of a Semi-Supervised Teacher-Student (SSTS) module and a Fine-Grained Interactive Attention (FGIA) mechanism. In detail, SSTS employs limited labeled WBC images and generates predicted probability vectors for a large amount of unlabeled WBC samples, like a human. After top-k selection in predicted probabilities, the efficient data can be exploited from unlabeled WBC images for training. With a very small amount of annotated WBC images, FIAL achieves an average accuracy of 93.2% on BCCD dataset when giving 75 labeled images for each category, which sufficiently elaborates our excellent capability on semi-supervised white blood cell image classification task.

A semi-supervised learning method for surface defect classification of magnetic tiles

A semi-supervised learning method for surface defect classification of magnetic tiles

A Semi-supervised Deep Learning Method for Cervical Cell Classification.

Currently, the Thinprep cytologic test (TCT) is the most popular cervical cancer cytology test technique. It can detect precancerous conditions and microbial infections. However, this technique entirely relies on manual operation and doctors' naked eye observation, resulting in a heavy workload and low accuracy rate. Recently, automatic pathological diagnosis has been developed to solve this problem. Cervical cell classification is a key technology in the intelligent cervical cancer diagnosis system. Training a deep neural network-based classification model requires a large amount of data. However, cervical cell labeling requires specialized physicians and the cost of labeling is high, resulting in a lack of sufficient labeling data in this field. To address this problem, we propose a method to ensure high accuracy in cervical cell classification with a small amount of labeled data by introducing manual features and a voting mechanism to achieve data expansion in semi-supervised learning. The method consists of three main steps, using a clarity function to filter out high-quality cervical cell images, annotating a small amount of them, and balancing the training data using a voting mechanism. With a small amount of labeled data, the accuracy of the proposed method in this paper can reach to 91.94%.