Semi-supervised Learning Problem Research Articles

A Semi-Supervised Learning Method for MiRNA-Disease Association Prediction Based on Variational Autoencoder.

MicroRNAs (miRNAs) are a class of non-coding RNAs that play critical role in many biological processes, such as cell growth, development, differentiation and aging. Increasing studies have revealed that miRNAs are closely involved in many human diseases. Therefore, the prediction of miRNA-disease associations is of great significance to the study of the pathogenesis, diagnosis and intervention of human disease. However, biological experimentally methods are usually expensive in time and money, while computational methods can provide an efficient way to infer the underlying disease-related miRNAs. In this study, we propose a novel method to predict potential miRNA-disease associations, called SVAEMDA. Our method mainly consider the miRNA-disease association prediction as semi-supervised learning problem. SVAEMDA integrates disease semantic similarity, miRNA functional similarity and respective Gaussian interaction profile (GIP) similarities. The integrated similarities are used to learn the representations of diseases and miRNAs. SVAEMDA trains a variational autoencoder based predictor by using known miRNA-disease associations, with the form of concatenated dense vectors. Reconstruction probability of the predictor is used to measure the correlation of the miRNA-disease pairs. Experimental results show that SVAEMDA outperforms other stat-of-the-art methods. AUC values of SVAEMDA of global leave-one-out cross validation (LOOCV) and 5-fold cross validation (5-fold CV) are 0.9464 and 0.9428 respectively. In addition, case studies of three common human diseases indicate that SVAEMDA obtains 100 percent of the top 50 predicted candidates in the benchmark databases. Therefore, SVAEMDA can efficiently and accurately predict the potential associations between diseases and miRNAs.

Deep graph cut network for weakly-supervised semantic segmentation

The scarcity of fully-annotated data becomes the biggest obstacle that prevents many deep learning approaches from widely applied. Weakly-supervised visual learning which can utilize inexact annotations is developed rapidly to remedy such a situation. In this paper, we study the weakly-supervised task achieving pixel-level semantic segmentation only with image-level labels as supervision. Different from other methods, our approach tries to transform the weakly-supervised visual learning problem into a semi-supervised visual learning problem and then utilizes semi-supervised learning methods to solve it. Utilizing this transformation, we can adopt effective semi-supervised methods to perform transductive learning with context information. In the semi-supervised learning module, we propose to use the graph cut algorithm to label more supervision from the activation seeds generated from a classification network. The generated labels can provide the segmentation model with effective supervision information; moreover, the graph cut module can benefit from features extracted by the segmentation model. Then, each of them updates and optimizes the other iteratively until convergence. Experiment results on PASCAL VOC and COCO benchmarks demonstrate the effectiveness of the proposed deep graph cut algorithm for weakly-supervised semantic segmentation.

Regularization graph convolutional networks with data augmentation

Recently, there is a lot of progress in designing effective convolutional schemes on irregular graph structures. These graph convolutional networks mostly solve the semi-supervised learning problem on the graph where the number of labeled nodes typically is very small and the number of unlabeled nodes is very large. In the standard GCN, only labeled nodes will enter the loss function; unlabeled data will not get into the loss function. The model will be unstable due to the unconstrained nature of unlabeled nodes. This will lead to overfitting and negatively affect the generalization ability of the model. To solve this problem, we propose to regularize the graph convolutional networks using data augmentation. We provide the details of the regularization GCN and SGC model and design rich data augmentation schemes including individual features perturbation, individual edge addition, and combination of them. To verify the performance of our method, we conduct experiments on standard benchmarks and explore the intensity effect of different data augmentation strategies. The experimental results on the node classification tasks demonstrate that our method can outperform baseline methods.

Graph regularized low-rank representation for semi-supervised learning

Low-rank representation (LRR) has attracted wide attention of researchers in recent years due to its excellent performance in the exploration of high-dimensional subspace structures. However, in the existing semi-supervised learning problem based on the LRR method, graph construction and semi-supervised learning are two separate steps. Therefore, the existing label information in the data set is not well used to guide the construction of the affinity graph. Therefore, these methods do not guarantee that the final result is a global optimal solution. This paper proposes a graph regularized low-rank representation for semi-supervised learning, called GLR2S2. This method combines the construction of affinity graph with semi supervised learning and unifies them into an optimization framework. By solving the joint optimization problem, the global optimal solution can be obtained. Experimental results on several standard data sets show that the GLR2S2 method proposed in this paper is effective.

An Uncertainty-Driven GCN Refinement Strategy for Organ Segmentation

Organ segmentation in CT volumes is an important pre-processing step in many computer assisted intervention and diagnosis methods. In recent years, convolutional neural networks have dominated the state of the art in this task. However, since this problem presents a challenging environment due to high variability in the organ’s shape and similarity between tissues, the generation of false negative and false positive regions in the output segmentation is a common issue. Recent works have shown that the uncertainty analysis of the model can provide us with useful information about potential errors in the segmentation. In this context, we proposed a segmentation refinement method based on uncertainty analysis and graph convolutional networks. We employ the uncertainty levels of the convolutional network in a particular input volume to formulate a semi-supervised graph learning problem that is solved by training a graph convolutional network. To test our method we refine the initial output of a 2D U-Net. We validate our framework with the NIH pancreas dataset and the spleen dataset of the medical segmentation decathlon. We show that our method outperforms the state-of-the-art CRF refinement method by improving the dice score by 1% for the pancreas and 2% for spleen, with respect to the original U-Net’s prediction. Finally, we perform a sensitivity analysis on the parameters of our proposal and discuss the applicability to other CNN architectures, the results, and current limitations of the model for future work in this research direction. For reproducibility purposes, we make our code publicly available at https://github.com/rodsom22/gcn_refinement

MetaRisk: Semi-supervised few-shot operational risk classification in banking industry

We study the operational risk classification problem, a critical yet challenging problem in the banking industry. In practice, banks build supervised multi-label classification models to identify the pre-defined risks using financial news sources. However, the models are often suboptimal due to the lack of labeled data and diverse combinations of risk types. To address these practical issues, we re-frame multi-label supervised operational risk classification as a semi-supervised few-shot learning problem, named MetaRisk, which can then be effectively learned using the prototypical network. We also propose a weighted scheme to help obtain accurately prototype vectors of multi-risk classes. We evaluate the proposed approach MetaRisk using a real-world operational risk classification dataset, and the results demonstrate that it outperforms a set of standard baselines. Especially, MetaRisk is capable of predicting risk types that are new to the system. We expect our work provides a direct and relevant toolkit that may assist risk officers to predict and intervene risks in the banking industry.

Feedback-driven semi-supervised synthesis of program transformations

While editing code, it is common for developers to make multiple related repeated edits that are all instances of a more general program transformation. Since this process can be tedious and error-prone, we study the problem of automatically learning program transformations from past edits, which can then be used to predict future edits. We take a novel view of the problem as a semi-supervised learning problem: apart from the concrete edits that are instances of the general transformation, the learning procedure also exploits access to additional inputs (program subtrees) that are marked as positive or negative depending on whether the transformation applies on those inputs. We present a procedure to solve the semi-supervised transformation learning problem using anti-unification and programming-by-example synthesis technology. To eliminate reliance on access to marked additional inputs, we generalize the semi-supervised learning procedure to a feedback-driven procedure that also generates the marked additional inputs in an iterative loop. We apply these ideas to build and evaluate three applications that use different mechanisms for generating feedback. Compared to existing tools that learn program transformations from edits, our feedback-driven semi-supervised approach is vastly more effective in successfully predicting edits with significantly lesser amounts of past edit data.

X-ModalNet: A semi-supervised deep cross-modal network for classification of remote sensing data

This paper addresses the problem of semi-supervised transfer learning with limited cross-modality data in remote sensing. A large amount of multi-modal earth observation images, such as multispectral imagery (MSI) or synthetic aperture radar (SAR) data, are openly available on a global scale, enabling parsing global urban scenes through remote sensing imagery. However, their ability in identifying materials (pixel-wise classification) remains limited, due to the noisy collection environment and poor discriminative information as well as limited number of well-annotated training images. To this end, we propose a novel cross-modal deep-learning framework, called X-ModalNet, with three well-designed modules: self-adversarial module, interactive learning module, and label propagation module, by learning to transfer more discriminative information from a small-scale hyperspectral image (HSI) into the classification task using a large-scale MSI or SAR data. Significantly, X-ModalNet generalizes well, owing to propagating labels on an updatable graph constructed by high-level features on the top of the network, yielding semi-supervised cross-modality learning. We evaluate X-ModalNet on two multi-modal remote sensing datasets (HSI-MSI and HSI-SAR) and achieve a significant improvement in comparison with several state-of-the-art methods.

Deep detector classifier (DeepDC) for moving objects segmentation and classification in video surveillance

In this study, the authors present a new approach to segment and classify moving objects in video sequences by combining an unsupervised anomaly discovery framework called DeepSphere and generative adversarial networks. The proposed deep detector classifier employs and validates DeepSphere, which aims mainly to identify the anomalous cases in the spatial and temporal context in order to perform foreground objects segmentation. For post-processing, some morphological operations are considered to better segment and extract the desired objects. Finally, they take advantage of the power of generative models, which recognise the problem of semi-supervised learning as a specific missing data imputation task in order to classify the segmented objects. They evaluate the method with multiple datasets and the results confirm the effectiveness of the proposed approach, which achieves superior performance over the state-of-the-art methods having the capabilities of segmenting and classifying moving objects from videos surveillance.

Semi-Supervised Learning under Class Distribution Mismatch

Semi-supervised learning (SSL) aims to avoid the need for collecting prohibitively expensive labelled training data. Whilst demonstrating impressive performance boost, existing SSL methods artificially assume that small labelled data and large unlabelled data are drawn from the same class distribution. In a more realistic scenario with class distribution mismatch between the two sets, they often suffer severe performance degradation due to error propagation introduced by irrelevant unlabelled samples. Our work addresses this under-studied and realistic SSL problem by a novel algorithm named Uncertainty-Aware Self-Distillation (UASD). Specifically, UASD produces soft targets that avoid catastrophic error propagation, and empower learning effectively from unconstrained unlabelled data with out-of-distribution (OOD) samples. This is based on joint Self-Distillation and OOD filtering in a unified formulation. Without bells and whistles, UASD significantly outperforms six state-of-the-art methods in more realistic SSL under class distribution mismatch on three popular image classification datasets: CIFAR10, CIFAR100, and TinyImageNet.

Laplacian Support Vector Machine for Vibration-Based Robotic Terrain Classification

The achievement of robot autonomy has environmental perception as a prerequisite. The hazards rendered from uneven, soft and slippery terrains, which are generally named non-geometric hazards, are another potential threat reducing the traversing efficient, and therefore receiving more and more attention from the robotics community. In the paper, the vibration-based terrain classification (VTC) is investigated by taking a very practical issue, i.e., lack of labels, into consideration. According to the intrinsic temporal correlation existing in the sampled terrain sequence, a modified Laplacian SVM is proposed to utilise the unlabelled data to improve the classification performance. To the best of our knowledge, this is the first paper studying semi-supervised learning problem in robotic terrain classification. The experiment demonstrates that: (1) supervised learning (SVM) achieves a relatively low classification accuracy if given insufficient labels; (2) feature-space homogeneity based semi-supervised learning (traditional Laplacian SVM) cannot improve supervised learning’s accuracy, and even makes it worse; (3) feature- and temporal-space based semi-supervised learning (modified Laplacian SVM), which is proposed in the paper, could increase the classification accuracy very significantly.

Model-based clustering with envelopes

Clustering analysis is an important unsupervised learning technique in multivariate statistics and machine learning. In this paper, we propose a set of new mixture models called CLEMM (in short for Clustering with Envelope Mixture Models) that is based on the widely used Gaussian mixture model assumptions and the nascent research area of envelope methodology. Formulated mostly for regression models, envelope methodology aims for simultaneous dimension reduction and efficient parameter estimation, and includes a very recent formulation of envelope discriminant subspace for classification and discriminant analysis. Motivated by the envelope discriminant subspace pursuit in classification, we consider parsimonious probabilistic mixture models where the cluster analysis can be improved by projecting the data onto a latent lower-dimensional subspace. The proposed CLEMM framework and the associated envelope-EM algorithms thus provide foundations for envelope methods in unsupervised and semi-supervised learning problems. Numerical studies on simulated data and two benchmark data sets show significant improvement of our propose methods over the classical methods such as Gaussian mixture models, K-means and hierarchical clustering algorithms. An R package is available at https://github.com/kusakehan/CLEMM.

Graph Regularized Variational Ladder Networks for Semi-Supervised Learning

To tackle the problem of semi-supervised learning (SSL), we propose a new autoencoder-based deep model. Ladder networks (LN) is an autoencoder-based method for representation learning which has been successfully applied on unsupervised learning and semi-supervised learning. However, It ignores the manifold information of high-dimensional data and usually achieves unmeaning features which are very difficult to use in the subsequent tasks, such as prediction and recognition. To these issues, we proposed Graph Regularized Variational Ladder Networks (GRVLN), which explicitly and implicitly employs the manifold structure of data. Our contributions can be summarized as two folds: (1) Graph regularization is used to build all decoder layers, which explicitly promotes the manifold learning via graph laplacian matrixs; (2) Variational autoencoder is used as the backbone instead of traditional autoencoder in the encoder layers for implicitly learning the manifold structure of data distribution. Compared with ladder networks and other autoencoder-based methods, GRVLN achieves superior performance in semi-supervised classification tasks. Experimental results show that our method also has a comparable performance with state-of-the-art methods on several benchmark data sets.

Application of Massive Parallel Deep Learning Algorithm in the Prediction of Colorectal Carcinogenesis of Familial Polyposis

Based on the massively parallel deep learning algorithm, this paper studies familial polyposis colorectal carcinogenesis, and proposes a semi-supervised multi-task survival analysis method based on deep learning, which transforms the survival analysis problem into multi-timepoint survival probability prediction. The multi-task learning model is composed of semi-supervised learning problems. We use semi-supervised loss and sorting loss to deal with data of censorship and the non-increasing probability of survival probability. It established a prognostic risk prediction model for familial polyposis colorectal cancer based on a semi-supervised logistic regression method and learns from supervised learning from five aspects of discriminating ability, interpretability, and clinical practicality. The method comparison expands the current understanding of the generalization capabilities of different models and provides a reference for the establishment of clinical prediction models. The effectiveness of this method was verified by external data and provided technical support for constructing a prognostic model with application value for multi-center real clinical data. This model has demonstrated better prediction performance than common models in the prognostic task of familial polyposis colorectal cancer, and successfully described the mechanism of action of predictors.

Semi-Supervised Learning Based on GAN With Mean and Variance Feature Matching

The improved generative adversarial network (improved GAN) is a successful method using a generative adversarial model to solve the problem of semi-supervised learning (SSL). The improved GAN learns a generator with the technique of mean feature matching which penalizes the discrepancy of the first-order moment of the latent features. To better describe common attributes of a distribution, this paper proposes a novel SSL method which incorporates the first-order and the second-order moments of the features in an intermediate layer of the discriminator, called mean and variance feature matching GAN (MVFM-GAN). To capture more precisely the data manifold, not only the mean but also the variance is used in the latent feature learning. Compared with improved GAN and other traditional methods, MVFM-GAN achieves superior performance in semi-supervised classification tasks and a better stability of GAN training, particularly in the cases when the number of labeled samples is low. It shows a comparable performance with the state-of-the-art methods on several benchmark data sets. As a byproduct of the novel approach, MVFM-GAN generates realistic images of good visual quality.

Loss-Sensitive Generative Adversarial Networks on Lipschitz Densities

In this paper, we present the Lipschitz regularization theory and algorithms for a novel Loss-Sensitive Generative Adversarial Network (LS-GAN). Specifically, it trains a loss function to distinguish between real and fake samples by designated margins, while learning a generator alternately to produce realistic samples by minimizing their losses. The LS-GAN further regularizes its loss function with a Lipschitz regularity condition on the density of real data, yielding a regularized model that can better generalize to produce new data from a reasonable number of training examples than the classic GAN. We will further present a Generalized LS-GAN (GLS-GAN) and show it contains a large family of regularized GAN models, including both LS-GAN and Wasserstein GAN, as its special cases. Compared with the other GAN models, we will conduct experiments to show both LS-GAN and GLS-GAN exhibit competitive ability in generating new images in terms of the Minimum Reconstruction Error (MRE) assessed on a separate test set. We further extend the LS-GAN to a conditional form for supervised and semi-supervised learning problems, and demonstrate its outstanding performance on image classification tasks.

Partial Bayesian Co-training for Virtual Metrology

Building accurate regression models using limited data is a challenging problem in manufacturing data analysis. In this paper, we study a particular semisupervised learning problem where labeled data are limited, while unlabeled data are plentiful. In these conditions, conventional single-view learning methods are prone to overfitting. To tackle this problem, we develop a novel co-training technique, namely partial Bayesian co-training (PBCT). PBCT scales down the original set of features to create a partial view, and then exploit side information from the partial view to enhance the complete model. The PBCT model also allows integrating domain knowledge to enhance model accuracy. The proposed method is validated with experiments on industrial manufacturing data. The experimental results show that under a reduction of labeled data by up to 50%, a robust estimation is still attainable. This suggests that the PBCT model is a promising solution to a broad spectrum of applications.

Semi-Supervised Graph Based Embedding With Non-Convex Sparse Coding Techniques

We consider the problem of semi-supervised graph-based learning upon multimodal and mixmodal data. Since in semi-supervised settings, the labeled information is very limited, we first propose a non-convex sparse-coding based label propagation ( $\alpha$ α -SLP) method to estimate the soft labels, and thereby to enrich the supervised information. By considering the structural properties of multimodal and mixmodal data, we present a semi-supervised graph-based embedding (SGE) approach that incorporates the soft label information with the hierarchical local geometric information of within-class, between-class, and overall-class data. Based on this, subspaces characterizing the multimodal and mixmodal data structure can be derived by maximizing the weighted between-class separability and minimizing the locality-preserved within-class as well as overall-class distances of the training samples. We further extend SGE into semi-supervised sparse subspace learning scenarios and present an $\alpha$ α -structural-regularization-induced SGE ( $\alpha$ α -SSGE) model, which can give better results in extracting discriminative groups of features by utilizing the non-convex structural regularization techniques. Experiments for multimodal and mixmodal digit as well as face recognition verify the validity and effectiveness of the proposed models.

GrAMME: Semisupervised Learning Using Multilayered Graph Attention Models.

Modern data analysis pipelines are becoming increasingly complex due to the presence of multiview information sources. While graphs are effective in modeling complex relationships, in many scenarios, a single graph is rarely sufficient to succinctly represent all interactions, and hence, multilayered graphs have become popular. Though this leads to richer representations, extending solutions from the single-graph case is not straightforward. Consequently, there is a strong need for novel solutions to solve classical problems, such as node classification, in the multilayered case. In this article, we consider the problem of semisupervised learning with multilayered graphs. Though deep network embeddings, e.g., DeepWalk, are widely adopted for community discovery, we argue that feature learning with random node attributes, using graph neural networks, can be more effective. To this end, we propose to use attention models for effective feature learning and develop two novel architectures, GrAMME-SG and GrAMME-Fusion, that exploit the interlayer dependences for building multilayered graph embeddings. Using empirical studies on several benchmark data sets, we evaluate the proposed approaches and demonstrate significant performance improvements in comparison with the state-of-the-art network embedding strategies. The results also show that using simple random features is an effective choice, even in cases where explicit node attributes are not available.

HesGCN: Hessian graph convolutional networks for semi-supervised classification

Manifold or local geometry of samples have been recognized as a powerful tool in machine learning areas, especially in the graph-based semi-supervised learning (GSSL) problems. Over recent decades, plenty of manifold assumption-based SSL algorithms (MSSL) have been proposed including graph embedding and graph regularization models, where the objective is to utilize the local geometry of data distributions. One of most representative MSSL approaches is graph convolutional networks (GCN), which effectively generalizes the convolutional neural networks to deal with the graphs with the arbitrary structures by constructing and fusing the Laplacian-based structure information. However, the null space of the Laplacian remains unchanged along the underlying manifold, it causes the poor extrapolating ability of the model. In this paper, we introduce a variant of GCN, i.e. Hessian graph convolutional networks (HesGCN). In particularly, we get a more efficient convolution layer rule by optimizing the one-order spectral graph Hessian convolutions. In addition, the spectral graph Hessian convolutions is a combination of the Hessian matrix and the spectral graph convolutions. Hessian gets a richer null space by the existence of its two-order derivatives, which can describe the intrinsic local geometry structure of data accurately. Thus, HesGCN can learn more efficient data features by fusing the original feature information with its structure information based on Hessian. We conduct abundant experiments on four public datasets. Extensive experiment results validate the superiority of our proposed HesGCN compared with many state-of-the-art methods.