Semi-supervised Learning Algorithm Research Articles

Supervised and semi-supervised probabilistic learning with deep neural networks for concurrent process-quality monitoring

Concurrent process-quality monitoring helps discover quality-relevant process anomalies and quality-irrelevant process anomalies. It especially works well in chemical plants with faults that cause quality problems. Traditional monitoring strategies are limitedly applied in chemical plants because quality targets in training data are insufficient. It is hard for inflexible models to fully capture the strongly nonlinear process-quality correlations. Also, deterministic models are mapped from process variables to qualities without any consideration of uncertainties. Simultaneously, a slow sampling rate for quality variables is ubiquitous in chemical plants since a product quality test is often time-consuming and expensive. Motivated by these limitations, this paper proposes a new concurrent process-quality monitoring scheme based on a probabilistic generative deep learning model developed from variational autoencoder. The supervised model is firstly developed and then the semi-supervised version is extended to solve the issue of missing targets. Especially, the semi-supervised learning algorithm is accomplished with an optimal parameter estimation in the light of maximum likelihood principle and no any hyperparameters are introduced. Two case studies validate that the proposed method effectively outperforms the other comparative methods in concurrent process-quality monitoring.

Semi-supervised active learning algorithm for SVMs based on QBC and tri-training

For the problem that large-scale labeled samples are not easy to acquire in the course of Support Vector Machines (SVMs) training, a Semi-Supervised Active Learning Algorithm for SVMs (QTB-ASVM) is proposed in the paper, which efficiently combines the semi–supervised learning based on Tri-Training and active learning based on Query By Committee (QBC) with SVMs. With this method, QBC active learning is used to select the samples which are the most valuable to current SVM classifier, and Tri-Training is used to exploit useful information that remains in the unlabeled samples. The experimental results show that the proposed approach can considerably reduce the labeled samples and costs compared to the SVMs which is either not applied with semi-supervised learning or active learning or applied with only one of them, and at the same time it can ensure that the accurate classification performance is kept as the passive SVM, while improving generalization performance and also expediting the SVM training.

Semi-Supervised Learning in Medical Images Through Graph-Embedded Random Forest.

One major challenge in medical imaging analysis is the lack of label and annotation which usually requires medical knowledge and training. This issue is particularly serious in the brain image analysis such as the analysis of retinal vasculature, which directly reflects the vascular condition of Central Nervous System (CNS). In this paper, we present a novel semi-supervised learning algorithm to boost the performance of random forest under limited labeled data by exploiting the local structure of unlabeled data. We identify the key bottleneck of random forest to be the information gain calculation and replace it with a graph-embedded entropy which is more reliable for insufficient labeled data scenario. By properly modifying the training process of standard random forest, our algorithm significantly improves the performance while preserving the virtue of random forest such as low computational burden and robustness over over-fitting. Our method has shown a superior performance on both medical imaging analysis and machine learning benchmarks.

Semi-Supervised Learning to Enhance Speech Signal for Mobile Communication

Machine learning algorithm to enhance the complex speech signal for mobile communication is one of the research problems in signal processing. The objective of this research paper is to develop a learning algorithm that improves the quality and intelligibility of voice signals that gets are corrupted by real world noise while they are transmitted through the channel. In this paper, we consider a semi-supervised machine learning algorithm for mobile phones that comes with system software to improve SNR of speech signal which is corrupted by manmade disturbance. Most of the disturbances are non-stationary where the effect of noise is non-uniform for all spectral components. In the projected algorithm training, the system is completed with a set of speech and noise data base. System parameters are derived during training process; these parameters are updated as per the disturbance present in the signal. These parameters are used to remove the noise present in speech signal. The obtained results show a substantial progress in SNR by 5–8% as compared to traditional methods.

A semi-supervised learning algorithm via adaptive Laplacian graph

Many semi-supervised learning methods have been developed in recent years, especially graph-based approaches, which have achieved satisfactory performance in the practical applications. There are two points that need to be noticed. Firstly, the quality of the graph directly affects the final classification accuracy. However, graph-based algorithms mostly use k-Nearest Neighbor to construct the graph. And the directly constructed graph is inaccurate due to outliers and erroneous features in the data. Secondly, the amount of labeled data is a small part of all data. It cannot be guaranteed that all categories of data are included in the labeled data and the labels of data are not totally correct in practice. To address the aforementioned problems, we propose a new graph-based semi-supervised method named ALGSSL via adaptive Laplacian graph. In the algorithm, we adaptively update the graph to reduce the sensitiveness of the construction of initial graph. Meanwhile, we use the regularization parameters to set confidence on existing labels, which can reduce the impact of the error labels on the result and discover the new category. Experiments on three toy datasets and nine benchmark datasets demonstrate the proposed method can achieve good performance.

Semisupervised Learning for Seismic Monitoring Applications

Abstract The impressive performance that deep neural networks demonstrate on a range of seismic monitoring tasks depends largely on the availability of event catalogs that have been manually curated over many years or decades. However, the quality, duration, and availability of seismic event catalogs vary significantly across the range of monitoring operations, regions, and objectives. Semisupervised learning (SSL) enables learning from both labeled and unlabeled data and provides a framework to leverage the abundance of unreviewed seismic data for training deep neural networks on a variety of target tasks. We apply two SSL algorithms (mean-teacher and virtual adversarial training) as well as a novel hybrid technique (exponential average adversarial training) to seismic event classification to examine how unlabeled data with SSL can enhance model performance. In general, we find that SSL can perform as well as supervised learning with fewer labels. We also observe in some scenarios that almost half of the benefits of SSL are the result of the meaningful regularization enforced through SSL techniques and may not be attributable to unlabeled data directly. Lastly, the benefits from unlabeled data scale with the difficulty of the predictive task when we evaluate the use of unlabeled data to characterize sources in new geographic regions. In geographic areas where supervised model performance is low, SSL significantly increases the accuracy of source-type classification using unlabeled data.

Neurocognitive and functional heterogeneity in depressed youth.

Depression is a common psychiatric illness that often begins in youth, and is sometimes associated with cognitive deficits. However, there is significant variability in cognitive dysfunction, likely reflecting biological heterogeneity. We sought to identify neurocognitive subtypes and their neurofunctional signatures in a large cross-sectional sample of depressed youth. Participants were drawn from the Philadelphia Neurodevelopmental Cohort, including 712 youth with a lifetime history of a major depressive episode and 712 typically developing (TD) youth matched on age and sex. A subset (MDD n = 368, TD n = 200) also completed neuroimaging. Cognition was assessed with the Penn Computerized Neurocognitive Battery. A recently developed semi-supervised machine learning algorithm was used to delineate neurocognitive subtypes. Subtypes were evaluated for differences in both clinical psychopathology and brain activation during an n-back working memory fMRI task. We identified three neurocognitive subtypes in the depressed group. Subtype 1 was high-performing (high accuracy, moderate speed), Subtype 2 was cognitively impaired (low accuracy, slow speed), and Subtype 3 was impulsive (low accuracy, fast speed). While subtypes did not differ in clinical psychopathology, they diverged in their activation profiles in regions critical for executive function, which mirrored differences in cognition. Taken together, these data suggest disparate mechanisms of cognitive vulnerability and resilience in depressed youth, which may inform the identification of biomarkers for prognosis and treatment response.

Multiple-feature-driven co-training method for crop mapping based on remote sensing time series imagery

ABSTRACT Remote sensing time series imagery (RSTSI) provides a useful tool for crop mapping, as it provides crucial spectral, temporal, and spatial (STS) features. However, its high dimensionality coupled with the limited number of training samples leads to an ill-posed classification problem and the Hughes phenomenon. To solve this problem, this study presents a multiple-feature-driven co-training method (MFDC) for accurately mapping crop types based on RSTSI with a limited number of training samples. In MFDC, four complementary pre-defined views, which represent STS features, are generated for the utilization of multiple features. Then, to enhance the classifier’s generalization ability, a novel labelled sample augmentation method that combines the Breaking Tiles algorithm and co-training is proposed. Third, to ensure the effectiveness of ensemble learning in co-training as well as to further speed up the learning process, a multi-view semi-supervised feature learning algorithm that expands the single view semi-supervised learning algorithm to multiple views is proposed and embedded in co-training. Finally, a weighted majority vote method is utilized to obtain the classification results. The experimental results for study areas in the United States indicate that the proposed method can accurately map crop types with a limited number of labelled training samples without a significant computational cost.

Research on state evaluation and risk assessment for relay protection system based on machine learning algorithm

The relay protection system plays an important role in ensuring the stable operation of power systems. Combined with operation data collected from a region in China, this study is aimed at providing a reliable quantitative basis for relay protection systems’ operating maintenance by the aid of a semi-supervised Mahalanobis distance machine learning algorithm. The evaluation result is first applied as a training set on the basis of the analytic hierarchy process fuzzy synthetic evaluation. Then, contrastive analysis is conducted in terms of accuracy, processing time, and feasibility. It includes comparative cases with a supervised multiple regression analysis algorithm and unsupervised K-means algorithm. The comparison result reveals that the algorithm can effectively and accurately predict the running state of the equipment and offer a quantitative reference for relative maintenance strategy.

Semi-Supervised Machine Learning Aided Anomaly Detection Method in Cellular Networks

The ever-increasing amount of data in cellular networks poses challenges for network operators to monitor the quality of experience (QoE). Traditional key quality indicators (KQIs)-based hard decision methods are difficult to undertake the task of QoE anomaly detection in the case of big data. To solve this problem, in this paper, we propose a KQIs-based QoE anomaly detection framework using semi-supervised machine learning algorithm, i.e., iterative positive sample aided one-class support vector machine (IPS-OCSVM). There are four steps for realizing the proposed method while the key step is combining machine learning with the network operator's expert knowledge using OCSVM. Our proposed IPS-OCSVM framework realizes QoE anomaly detection through soft decision and can easily fine-tune the anomaly detection ability on demand. Moreover, we prove that the fluctuation of KQIs thresholds based on expert knowledge has a limited impact on the result of anomaly detection. Finally, experiment results are given to confirm the proposed IPS-OCSVM framework for QoE anomaly detection in cellular networks.

Semi-Supervised Concept Learning by Concept-Cognitive Learning and Concept Space

In human concept learning, people can naturally combine a handful of labeled data with abundant unlabeled data when they make classification decisions, which is also known as semi-supervised learning (SSL) in machine learning. Especially, human concept learning not only is a static process in human cognition but also can vary gradually with dynamic environments. Nevertheless, the classical SSL algorithms must be redesigned to accommodate newly input data. In this sense, concept-cognitive learning may be a good choice, as it can implement dynamic processes by imitating human cognitive processes. Meanwhile, numerous SSL methods were designed based on the feature vector information of instances, while ignoring concept structural information that is a very important process in human knowledge organization. Based on this idea, a novel SSL method, named semi-supervised concept learning method (S2CL), is proposed for dynamic SSL by employing concept spaces, in which knowledge is represented by hierarchical concept structures. Moreover, to make full use of the global and local conceptual information, we further propose an extended version of S2CL (namely, <inline-formula><tex-math notation="LaTeX">$\text{S2CL}^{\alpha }$</tex-math></inline-formula> ) for concept learning. More specifically, to effectively exploit the unlabeled data, this paper first shows some new related theories for S2CL (or <inline-formula><tex-math notation="LaTeX">$\text{S2CL}^{\alpha }$</tex-math></inline-formula> ) based on a regular formal decision context; then a novel SSL framework is designed, and its corresponding algorithm is developed. Finally, we conduct some experiments on various datasets to demonstrate the effectiveness of our methods, which include concept classification and incremental learning under a large quantity of unlabeled data.

Node influence-based label propagation algorithm for semi-supervised learning

Graph-based semi-supervised learning (GSSL) has received more and more attention due to its efficiency and accuracy. Label propagation is a critical step in GSSL that propagates label information to unlabeled data through the structure of graph. However, the traditional label propagation algorithms treat all unlabeled samples as equivalent and blindly propagate label information to all neighbors without considering their reliabilities. In this case, some unreliable samples may mislead the process of label propagation, thus greatly reducing the accuracy of classification. In order to solve this problem, this paper proposes a novel label propagation algorithm called node influence-based label propagation (NILP). Based on the structure of graph, the NILP algorithm measures the influences of nodes by calculating their degrees and local densities. In the process of label propagation, the label information is preferentially transmitted to the influential neighbors to control the propagation sequence and prevent wrong propagation. Moreover, our algorithm improves the transition matrix by integrating label information and feature information. The experimental results on both synthetic and real-world benchmark datasets show that the proposed method is superior to some existing label propagation algorithms. Especially when the number of labeled samples is very small, the advantage of NILP algorithm is more obvious.

The Turnout Abnormality Diagnosis Based on Semi-Supervised Learning Method

In China, the turnout abnormality very easily causes traffic accident or affects the efficiency due to the operating environment of railway transportation. The existing monitoring means are relatively backward, and mature automatic diagnosis method is lacking. In this study, a method based on semi-supervised learning algorithm for abnormal state diagnosis of turnout action curve is proposed, which is used to analyze and extract the electrical characteristics of the turnout by using the turnout action curve and the static and dynamic properties collected by the railway centralized monitoring system. The support vector machine model is used to construct the initial classifier with a small number of labeled samples, and the labeled samples are expanded from a large number of unlabeled samples. The switch curve is analyzed and diagnosed by using unlabeled data with a small amount of labeled data. The experimental results show that the method can automatically diagnose turnout electrical characteristics with high accuracy. While the cost of this method is relatively low compared with supervised learning, it can achieve higher accuracy and improve the practicability of fault diagnosis of turnout.

SSKM_Succ: A Novel Succinylation Sites Prediction Method Incorporating K-Means Clustering With a New Semi-Supervised Learning Algorithm.

Protein succinylation is a type of post-translational modification (PTM) that occurs on lysine sites and plays a key role in protein conformation regulation and cellular function control. When training in computational method, it is difficult to designate negative samples because of the uncertainty of non-succinylation lysine sites, and if not handled properly, it may affect the performance of computational models dramatically. Therefore, we propose a new semi-supervised learning method to identify reliable non-succinylation lysine sites as negative samples. This method, named SSKM_Succ, also employs K-means clustering to divide data into 5 clusters. Besides, information of proximal PTMs and three kinds of sequence features (grey pseudo amino acid composition, K-space and position-special amino acid propensity) are utilized to formulate protein. Then, we perform a two-step feature selection to remove redundant features and construct the optimization model for each cluster. Finally, support vector machine is applied to construct a prediction model for each cluster. Promising results are obtained by this method with an accuracy of 80.18 percent for succinylation sites on the independent testing dataset. Meanwhile, we compare the result with other existing tools, and it shows that our method is promising for predicting succinylation sites. Through analysis, we further verify that succinylated protein has potential effects on amino acid degradation and fatty acid metabolism, and speculate that protein succinylation may be closely related to neurodegenerative diseases. The code of SSKM_Succ is available on the web https://github.com/yangyq505/SSKM_Succ.git.

Semi-supervised Learning Algorithm Based on Linear Lie Group for Imbalanced Multi-class Classification

In practical application, the data are imbalanced, it is difficult to find the balanced, rather skewed data is the common occurrence. This poses a severe challenge to the classification algorithm. At present, imbalanced data classification methods are mainly for binary classes designed, and it is difficult to extend them to multiple classes. In this study, we introduced Lie group machine learning and proposed a semi-supervised learning algorithm based on the linear Lie group. First, the sample set is represented by a matrix, the isomorphism(or homomorphism)-GL(n) linear Lie group of the corresponding learning system is found, and the labeled data are used to represent the object to be learned by linear Lie group. Then, according to the algebraic structure of the linear Lie group, it is marked by the group method. We performed experiments on 18 benchmark multi-class imbalanced datasets to demonstrate the performance of our proposed method and measured the performance of multi-class imbalanced data using four state-of-the-art learning algorithms (mean of accuracy, mean of f-measure, and mean of area under the curve). The experimental results demonstrate that the proposed method is effective and improves the performance.

Multi-stage domain-specific pretraining for improved detection and localization of Barrett's neoplasia: A comprehensive clinically validated study.

Patients suffering from Barrett's Esophagus (BE) are at an increased risk of developing esophageal adenocarcinoma and early detection is crucial for a good prognosis. To aid the endoscopists with the early detection for this preliminary stage of esophageal cancer, this work concentrates on the development and extensive evaluation of a state-of-the-art computer-aided classification and localization algorithm for dysplastic lesions in BE. To this end, we have employed a large-scale endoscopic data set, consisting of 494,355 images, in combination with a novel semi-supervised learning algorithm to pretrain several instances of the proposed neural network architecture. Next, several Barrett-specific data sets that are increasingly closer to the target domain with significantly more data compared to other related work, were used in a multi-stage transfer learning strategy. Additionally, the algorithm was evaluated on two prospectively gathered external test sets and compared against 53 medical professionals. Finally, the model was also evaluated in a live setting without interfering with the current biopsy protocol. Results from the performed experiments show that the proposed model improves on the state-of-the-art on all measured metrics. More specifically, compared to the best performing state-of-the-art model, the specificity is improved by more than 20% points while simultaneously preserving high sensitivity and reducing the false positive rate substantially. Our algorithm yields similar scores on the localization metrics, where the intersection of all experts is correctly indicated in approximately 92% of the cases. Furthermore, the live pilot study shows great performance in a clinical setting with a patient level accuracy, sensitivity, and specificity of 90%. Finally, the proposed algorithm outperforms each individual medical expert by at least 5% and the average assessor by more than 10% over all assessor groups with respect to accuracy.

A Sample Dependent Decision Fusion Algorithm for Graph-based Semi-supervised Learning

On many occasions, the evaluation of a phenomenon based on a single feature could not solely be resulted in comprehensive and accurate results. Moreover, even if we have several features, we don’t know in advance, which feature offers a better description of the phenomenon. Thus, selecting the best features and especially their combination could lead to better results. An affinity graph is a tool that can describe the relationship between the samples. In this paper, we proposed a graph-based sample-based ranking method that sorts the graphs based on six proposed parameters. The sorting is performed such that the graphs at the top of the list have better performance compared to the graphs at the bottom. Furthermore, we propose a fusion method to merge the information of various features and improve the accuracy of label propagation. Moreover, a method is proposed for parameter optimizations and the ultimate decision fusion. The experimental results indicate that the proposed scheme, apart from correctly ranking the graphs according to their accuracy, in the fusion step, increases the accuracy compared to the use of a single feature.

Robust Maximum Mixture Correntropy Criterion-Based Semi-Supervised ELM With Variable Center

The recent correntropy criterion based semi-supervised random neural network extreme learning machine (RC-SSELM) achieved outstanding performance in dealing with datasets with large outliers and non-Gaussian noises. To further improve the effectiveness and flexibility of the algorithm in combating large and complex outliers, we explore a more effective semi-supervised ELM data learning algorithm with the robust maximum mixture correntropy criterion (MMCC) based optimization scheme in this brief. Meanwhile, the generalized correntropy criterion kernel function with the variable kernel center is applied to MMCC, and the resultant novel semi-supervised learning algorithm is abbreviated as MC-SSELM <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">vc</sup> . The fixed-point iteration learning algorithm is adopted for the output weight optimization of MC-SSELM <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">vc</sup> . Experiment conducted on many benchmark datasets are given to show the effectiveness of MC-SSELM <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">vc</sup> and comparisons to several state-of-the-art semi-supervised learning algorithms are provided for the superiority demonstration.

Deviation based clustering for unsupervised person re-identification

Recently, unsupervised person re-identification (re-ID) has gained a lot of attention, since it does not depend on intensive manual annotation and is more practical to deploy in the real world directly. An inspiring method, the Bottom-up Clustering (BUC), achieves the state-of-the-art among unsupervised re-ID methods and outperforms most semi-supervised and transfer learning algorithms. The BUC utilizes the minimum-distance between samples in different clusters as the merging criterion, and the number of samples as the penalization term. However, the minimum-distance criterion only considers one pair of samples between two clusters, and cannot exploit the information of all samples in clusters. The penalization, intuitively, is unsuitable for the dataset with irregular sample quantity. To relieve this problem, we propose a deviation based clustering re-ID approach, which takes the inter- and intra-cluster deviation into consideration. The inter-cluster deviation denotes the increase of deviation after merging two clusters, considering all samples between the two clusters to merge. The intra-cluster deviation, working as penalization, denotes the distance between samples and the center in each cluster, and thus it can help to mitigate the side-effect of irregular datasets. The two criterions can reflect the inter- and intra- dispersion precisely. Based on these criterions, we group similar samples into clusters and utilize the cluster identities as a pseudo annotation to train our model. To evaluate our proposed approach, we implement abundant experiments on two popular re-ID datasets, where one has irregular sample quality (i.e., Market-1501) and the other has regular sample quality (i.e., DukeMTMC-reID). Evaluations show that our method outperforms BUC by 1.8% on Rank-1 (i.e., 68.0% accuracy) and 2.1% on mAP (i.e., 40.4% accuracy) for the Market-1501 dataset, and well maintains the benefit of BUC on the DukeMIMC-reID dataset with the accuracy of 47.8% in Rank-1 and 27.0% in mAP.

Distributed Information-Theoretic Semisupervised Learning for Multilabel Classification.

Multilabel classification (MLC) has received much attention recently. The existing MLC algorithms usually learn multiple classifiers simultaneously by exploiting the correlations among different labels. However, it is difficult and/or expensive to collect a large amount of multilabeled data in practice. The lack of labeled data significantly deteriorates the performance of classification. Moreover, the existing algorithms belong to centralized learning, that is, all the data with their labels must be transmitted to a fusion center for processing. But in many real applications, data may be dispersedly collected/stored in distributed nodes of networks. Due to the concerns of communication cost, processing ability, and data privacy, it is impossible to transmit and/or process the data centrally at one node. Considering this, the problem of distributed MLC over networks is studied, and two distributed information-theoretic semisupervised multilabel learning (dITS2ML2) algorithms are proposed, which are, respectively, used for solving linear and nonlinear MLC problems. In the proposed algorithms, a cost-sensitive objective function is designed, in which a new label correlation term defined on some anchor data is suggested. Besides, to decentralize the global objective function, a distributed matrix completion algorithm is developed to distributively complete the label matrix of the anchor data. Then, by exchanging and combining a few intermediate quantities instead of the original data for both linear and nonlinear cases, the model parameters can be adaptively estimated. The convergence of the proposed dITS2ML2 algorithms is analyzed, and their effectiveness in MLC is verified by simulations on various real datasets.