Semi-supervised Learning Setting Research Articles

Predicting pathological complete response based on weakly and semi-supervised joint learning in breast cancer multi-parametric MRI

Neoadjuvant chemotherapy (NAC) is the primary treatment used to reduce the tumor size in early breast cancer. Patients who achieve a pathological complete response (pCR) after NAC treatment have a significantly higher five-year survival rate. However, accurately predicting whether patients could achieve pCR remains challenging due to the limited availability of manually annotated MRI data. This study develops a weakly and semi-supervised joint learning model that integrates multi-parametric MR images to predict pCR to NAC in breast cancer patients. First, the attention-based multi-instance learning model is designed to characterize the representation of multi-parametric MR images in a weakly supervised learning setting. The Mean-Teacher learning framework is then developed to locate tumor regions for extracting radiochemical parameters in a semi-supervised learning setting. Finally, all extracted MR imaging features are fused to predict pCR to NAC. Our experiments were conducted on a cohort of 442 patients with multi-parametric MR images and NAC outcomes. The results demonstrate that our proposed model, which leverages multi-parametric MRI data, provides the AUC value of over 0.85 in predicting pCR to NAC, outperforming other comparative methods.

Learning to Predict Gradients for Semi-Supervised Continual Learning.

A key challenge for machine intelligence is to learn new visual concepts without forgetting the previously acquired knowledge. Continual learning (CL) is aimed toward addressing this challenge. However, there still exists a gap between CL and human learning. In particular, humans are able to continually learn from the samples associated with known or unknown labels in their daily lives, whereas existing CL and semi-supervised CL (SSCL) methods assume that the training samples are associated with known labels. Specifically, we are interested in two questions: 1) how to utilize unrelated unlabeled data for the SSCL task and 2) how unlabeled data affect learning and catastrophic forgetting in the CL task. To explore these issues, we formulate a new SSCL method, which can be generically applied to existing CL models. Furthermore, we propose a novel gradient learner to learn from labeled data to predict gradients on unlabeled data. In this way, the unlabeled data can fit into the supervised CL framework. We extensively evaluate the proposed method on mainstream CL methods, adversarial CL (ACL), and semi-supervised learning (SSL) tasks. The proposed method achieves state-of-the-art performance on classification accuracy and backward transfer (BWT) in the CL setting while achieving the desired performance on classification accuracy in the SSL setting. This implies that the unlabeled images can enhance the generalizability of CL models on the predictive ability of unseen data and significantly alleviate catastrophic forgetting. The code is available at https://github.com/luoyan407/grad_prediction.git.

Self-Supervised 3D Action Representation Learning With Skeleton Cloud Colorization.

3D Skeleton-based human action recognition has attracted increasing attention in recent years. Most of the existing work focuses on supervised learning which requires a large number of labeled action sequences that are often expensive and time-consuming to annotate. In this paper, we address self-supervised 3D action representation learning for skeleton-based action recognition. We investigate self-supervised representation learning and design a novel skeleton cloud colorization technique that is capable of learning spatial and temporal skeleton representations from unlabeled skeleton sequence data. We represent a skeleton action sequence as a 3D skeleton cloud and colorize each point in the cloud according to its temporal and spatial orders in the original (unannotated) skeleton sequence. Leveraging the colorized skeleton point cloud, we design an auto-encoder framework that can learn spatial-temporal features from the artificial color labels of skeleton joints effectively. Specifically, we design a two-steam pretraining network that leverages fine-grained and coarse-grained colorization to learn multi-scale spatial-temporal features. In addition, we design a Masked Skeleton Cloud Repainting task that can pretrain the designed auto-encoder framework to learn informative representations. We evaluate our skeleton cloud colorization approach with linear classifiers trained under different configurations, including unsupervised, semi-supervised, fully-supervised, and transfer learning settings. Extensive experiments on NTU RGB+D, NTU RGB+D 120, PKU-MMD, NW-UCLA, and UWA3D datasets show that the proposed method outperforms existing unsupervised and semi-supervised 3D action recognition methods by large margins and achieves competitive performance in supervised 3D action recognition as well.

Self-Supervised Contrastive Representation Learning for Semi-Supervised Time-Series Classification.

Learning time-series representations when only unlabeled data or few labeled samples are available can be a challenging task. Recently, contrastive self-supervised learning has shown great improvement in extracting useful representations from unlabeled data via contrasting different augmented views of data. In this work, we propose a novel Time-Series representation learning framework via Temporal and Contextual Contrasting (TS-TCC) that learns representations from unlabeled data with contrastive learning. Specifically, we propose time-series-specific weak and strong augmentations and use their views to learn robust temporal relations in the proposed temporal contrasting module, besides learning discriminative representations by our proposed contextual contrasting module. Additionally, we conduct a systematic study of time-series data augmentation selection, which is a key part of contrastive learning. We also extend TS-TCC to the semi-supervised learning settings and propose a Class-Aware TS-TCC (CA-TCC) that benefits from the available few labeled data to further improve representations learned by TS-TCC. Specifically, we leverage the robust pseudo labels produced by TS-TCC to realize a class-aware contrastive loss. Extensive experiments show that the linear evaluation of the features learned by our proposed framework performs comparably with the fully supervised training. Additionally, our framework shows high efficiency in few labeled data and transfer learning scenarios. The code is publicly available at https://github.com/emadeldeen24/CA-TCC.

On the use of matrix profiles and optimal transport theory for multivariate time series anomaly detection within structural health monitoring

In order for a practical application of structural health monitoring to be considered successful, not only is the detection of anomalies important but so is the tracking of various other behaviors, which may be induced by changing operational conditions or environmental variations. Rather than being problems of anomaly detection, these are known as problems of changing contexts or changing semantics. In this paper, we propose a novel, and efficient method which is able to perform both tasks: anomaly detection, and semantic analysis, simultaneously, and which is also able to scale to a multi-sensor network array. The proposed methodology works to combine a recent development from the field of time series data mining known as matrix profiling, as well as ideas from a popular area of mathematics known as optimal transport theory. In particular, matrix profiles will be used to generate curves of semantic segmentation, over multivariate time series data. From here, it will be demonstrated that by finding the unbalanced optimal transport barycenter one can successfully pool together information from a sensor array network in such a way that the presence of false positive transition points is reduced significantly when compared to the baseline mean. Moreover, this will be demonstrated to occur in a hierarchical fashion by employing a variational mode decomposition across each sensor in the network array. Overall, the proposed methodology: (i) Can work in the small data context without requiring a large amount of data for model training and validation, (ii) Applies readily to unsupervised and semi-supervised learning settings, (iii) Can extend to the online learning setting, (iv) Is adaptable and robust for sensor array network problems, (v) Does not require specifying any threshold, and (vi) Is highly interpretable. Finally, the proposed methodology will be validated on three different levels of investigation: (i) A numerical dataset of an aerospace model based on a two-dimensional plate model, (ii) An experimental dataset of a building from Los Alamos National Laboratories, and finally, (iii) A field investigation on a cable-stayed bridge located in Sydney, Australia.

Semisupervised Federated Learning for Temporal News Hyperpatism Detection

The proliferation of false and erroneous information on the Internet has posed a challenge to the accurate exchange of information. To address this issue, a semisupervised system based on self-embedding has been proposed. This system verifies information before it is shared, allowing only reliable and accurate content to be disseminated and protecting individuals from the negative effects of false information. In this article, we present a news article retrieval model based on active learning (AL) in a semisupervised learning setting. This model has the advantages of limited communication requirements, strong scalability, increased data privacy, and a time-dependent retrieval model. We use lexicon expansion, content segmentation, and temporal events to generate a bidirectional encoder representations from transformer (BERT) attention embedding query for the temporal understanding of sequential news articles. To generate pseudo-labels, we combine the partially trained model with the original tagged data. An attention network is used to update pseudo-labels of data samples when the label of a sample is correctly or incorrectly predicted. Finally, the modified classifiers are combined to make predictions. Experimental results indicate that the proposed model has 81% performance, showing that co-training and semisupervised learning can improve the performance of temporal expansion and profiling algorithms.

Learning Graph Representations With Maximal Cliques.

Non-Euclidean property of graph structures has faced interesting challenges when deep learning methods are applied. Graph convolutional networks (GCNs) can be regarded as one of the successful approaches to classification tasks on graph data, although the structure of this approach limits its performance. In this work, a novel representation learning approach is introduced based on spectral convolutions on graph-structured data in a semisupervised learning setting. Our proposed method, COnvOlving cLiques (COOL), is constructed as a neighborhood aggregation approach for learning node representations using established GCN architectures. This approach relies on aggregating local information by finding maximal cliques. Unlike the existing graph neural networks which follow a traditional neighborhood averaging scheme, COOL allows for aggregation of densely connected neighboring nodes of potentially differing locality. This leads to substantial improvements on multiple transductive node classification tasks.

Leveraging Symbolic Knowledge Bases for Commonsense Natural Language Inference Using Pattern Theory.

The commonsense natural language inference (CNLI) tasks aim to select the most likely follow-up statement to a contextual description of ordinary, everyday events and facts. Current approaches to transfer learning of CNLI models across tasks require many labeled data from the new task. This paper presents a way to reduce this need for additional annotated training data from the new task by leveraging symbolic knowledge bases, such as ConceptNet. We formulate a teacher-student framework for mixed symbolic-neural reasoning, with the large-scale symbolic knowledge base serving as the teacher and a trained CNLI model as the student. This hybrid distillation process involves two steps. The first step is a symbolic reasoning process. Given a collection of unlabeled data, we use an abductive reasoning framework based on Grenander's pattern theory to create weakly labeled data. Pattern theory is an energy-based graphical probabilistic framework for reasoning among random variables with varying dependency structures. In the second step, the weakly labeled data, along with a fraction of the labeled data, is used to transfer-learn the CNLI model into the new task. The goal is to reduce the fraction of labeled data required. We demonstrate the efficacy of our approach by using three publicly available datasets (OpenBookQA, SWAG, and HellaSWAG) and evaluating three CNLI models (BERT, LSTM, and ESIM) that represent different tasks. We show that, on average, we achieve 63% of the top performance of a fully supervised BERT model with no labeled data. With only 1,000 labeled samples, we can improve this performance to 72%. Interestingly, without training, the teacher mechanism itself has significant inference power. The pattern theory framework achieves 32.7% accuracy on OpenBookQA, outperforming transformer-based models such as GPT (26.6%), GPT-2 (30.2%), and BERT (27.1%) by a significant margin. We demonstrate that the framework can be generalized to successfully train neural CNLI models using knowledge distillation under unsupervised and semi-supervised learning settings. Our results show that it outperforms all unsupervised and weakly supervised baselines and some early supervised approaches, while offering competitive performance with fully supervised baselines. Additionally, we show that the abductive learning framework can be adapted for other downstream tasks, such as unsupervised semantic textual similarity, unsupervised sentiment classification, and zero-shot text classification, without significant modification to the framework. Finally, user studies show that the generated interpretations enhance its explainability by providing key insights into its reasoning mechanism.

Influential Attributed Communities via Graph Convolutional Network (InfACom-GCN)

Community search is a basic problem in graph analysis. In many applications, network nodes have certain properties that are important for the community to make sense of the application; hence, attributes are associated with nodes to capture their properties. Community influence is an important community property that can be used to rank communities in a network based on the relevance/importance of a particular attribute. Unfortunately, most of the community search algorithms introduced previously in attributed networks research work ignored the community influence. When searching for influential communities, two potential data sources can be used: network attributes and nodes. Dealing with structure-related attributes is a challenge. Recently, the graph neural network (GNN) has completely changed the field of graph representation learning by effectively learning node embedding and has achieved the most advanced results in tasks such as node classification and connection prediction. In this paper, we investigate the problem of searching for the influential communities in attributed networks. We propose an efficient algorithm for retrieving the influential communities in a large attributed network. The proposed approach contains two main steps: (1) Community detection using a graph convolutional network in a semi-supervised learning setting considering the correlation between attributes and the overall graph information, and (2) constructing the influential communities resulting from step 1. The proposed approach is evaluated on various real datasets. The experimental results show the efficiency and effectiveness of the proposed implementations.

Self-supervised learning for macromolecular structure classification based on cryo-electron tomograms.

Macromolecular structure classification from cryo-electron tomography (cryo-ET) data is important for understanding macro-molecular dynamics. It has a wide range of applications and is essential in enhancing our knowledge of the sub-cellular environment. However, a major limitation has been insufficient labelled cryo-ET data. In this work, we use Contrastive Self-supervised Learning (CSSL) to improve the previous approaches for macromolecular structure classification from cryo-ET data with limited labels. We first pretrain an encoder with unlabelled data using CSSL and then fine-tune the pretrained weights on the downstream classification task. To this end, we design a cryo-ET domain-specific data-augmentation pipeline. The benefit of augmenting cryo-ET datasets is most prominent when the original dataset is limited in size. Overall, extensive experiments performed on real and simulated cryo-ET data in the semi-supervised learning setting demonstrate the effectiveness of our approach in macromolecular labeling and classification.

Perceiving Stroke-Semantic Context: Hierarchical Contrastive Learning for Robust Scene Text Recognition

We introduce Perceiving Stroke-Semantic Context (PerSec), a new approach to self-supervised representation learning tailored for Scene Text Recognition (STR) task. Considering scene text images carry both visual and semantic properties, we equip our PerSec with dual context perceivers which can contrast and learn latent representations from low-level stroke and high-level semantic contextual spaces simultaneously via hierarchical contrastive learning on unlabeled text image data. Experiments in un- and semi-supervised learning settings on STR benchmarks demonstrate our proposed framework can yield a more robust representation for both CTC-based and attention-based decoders than other contrastive learning methods. To fully investigate the potential of our method, we also collect a dataset of 100 million unlabeled text images, named UTI-100M, covering 5 scenes and 4 languages. By leveraging hundred-million-level unlabeled data, our PerSec shows significant performance improvement when fine-tuning the learned representation on the labeled data. Furthermore, we observe that the representation learned by PerSec presents great generalization, especially under few labeled data scenes.

Fuzziness based semi-supervised multimodal learning for patient’s activity recognition using RGBDT videos

Automatic recognition of bedridden patients’ physical activity has important applications in the clinical process. Such recognition tasks are usually accomplished on visual data captured by RGB, depth, and/or thermal cameras by utilizing supervised machine learning. However, supervised machine learning requires a large amount of labeled data and the accuracy depends on extracting appropriate features based on the domain knowledge. A plausible solution to these issues is using semi-supervised learning that focuses less on labeled data and domain knowledge. In this paper, a novel fuzziness-based semi-supervised multimodal learning algorithm, called (FSSL-PAR) is proposed for bedridden patient activity recognition. We use a synergistic interaction on RGB, Depth, and Thermal videos to assess the effect of visual multimodality for the first time in this semi-supervised learning setting. Experiments are conducted on a dataset collected by mimicking the patients with Acute Brain Injury (ABI) from a neurorehabilitation center. The results exhibit the superiority of the proposed algorithm over the existing supervised learning algorithms. We also see a positive correlation between the performance and the size of the labeled data in the proposed FSSL-PAR.

Robust and efficient semi-supervised estimation of average treatment effects with application to electronic health records data.

We consider the problem of estimating the average treatment effect (ATE) in a semi-supervised learning setting, where a very small proportion of the entire set of observations are labeled with the true outcome but features predictive of the outcome are available among all observations. This problem arises, for example, when estimating treatment effects in electronic health records (EHR) data because gold-standard outcomes are often not directly observable from the records but are observed for a limited number of patients through small-scale manual chart review. We develop an imputation-based approach for estimating the ATE that is robust to misspecification of the imputation model. This effectively allows information from the predictive features to be safely leveraged to improve efficiency in estimating the ATE. The estimator is additionally doubly-robust in that it is consistent under correct specification of either an initial propensity score model or a baseline outcome model. It is also locally semiparametric efficient under an ideal semi-supervised model where the distribution of the unlabeled data is known. Simulations exhibit the efficiency and robustness of the proposed method compared to existing approaches in finite samples. We illustrate the method by comparing rates of treatment response to two biologic agents for treatment inflammatory bowel disease using EHR data from Partners'Healthcare.

Supervised and semi-supervised twin parametric-margin regularized extreme learning machine

Twin extreme learning machine (TELM) has attracted considerable attention and achieved great success in the machine learning field. However, its performance will be severely affected when outliers exist in the dataset since TELM does not consider heteroscedasticity in practical applications. To improve the performance of TELM, a novel learning framework called twin parametric-margin extreme learning machine (TPMELM) was proposed. Further, to enhance the classification performance of our TPMELM in a semi-supervised learning setting, a Laplacian TPMELM (Lap-TPMELM) was developed by introducing manifold regularization into TPMELM. Using the geometric information of the marginal distribution embedded in unlabeled samples, Lap-TPMELM can effectively construct a more reasonable classifier. The TPMELM and Lap-TPMELM are suitable for many situations, especially when the data has heteroscedastic error structure. Moreover, the TPMELM and Lap-TPMELM are helpful in clarifying theoretical interpretation of parameters which control the bounds on proportions of support vectors and boundary errors. An efficient technique (successive over-relaxation, SOR) is applied in TPMELM and Lap-TPMELM, respectively. Experimental results show the effectiveness and reliability of the proposed methods.

Fuzzy weighted sparse reconstruction error-steered semi-supervised learning for face recognition

Since the number of labeled data is limited in the semi-supervised learning settings, we propose a fuzzy weighted sparse reconstruction error-steered semi-supervised learning method for face recognition. The fuzzy membership functions are introduced to the reconstruction error calculation for the unlabeled data. A weight function is utilized to capture the locality property of data when learning the sparse coefficients. The fuzzy weighted sparse reconstruction error-steered semi-supervised learning not only inherits the advantages of sparse representation classification techniques and neighborhood methods, but also steers the reconstruction errors of unlabeled data. Experimental studies on well-known face image datasets demonstrate that the proposed method outperforms the comparative approaches.

Multi-task Self-Supervised Learning for Human Activity Detection

Deep learning methods are successfully used in applications pertaining to ubiquitous computing, pervasive intelligence, health, and well-being. Specifically, the area of human activity recognition (HAR) is primarily transformed by the convolutional and recurrent neural networks, thanks to their ability to learn semantic representations directly from raw input. However, in order to extract generalizable features massive amounts of well-curated data are required, which is a notoriously challenging task; hindered by privacy issues and annotation costs. Therefore, unsupervised representation learning (i.e., learning without manually labeling the instances) is of prime importance to leverage the vast amount of unlabeled data produced by smart devices. In this work, we propose a novel self-supervised technique for feature learning from sensory data that does not require access to any form of semantic labels, i.e., activity classes. We learn a multi-task temporal convolutional network to recognize transformations applied on an input signal. By exploiting these transformations, we demonstrate that simple auxiliary tasks of the binary classification result in a strong supervisory signal for extracting useful features for the down-stream task. We extensively evaluate the proposed approach on several publicly available datasets for smartphone-based HAR in unsupervised, semi-supervised and transfer learning settings. Our method achieves performance levels superior to or comparable with fully-supervised networks trained directly with activity labels, and it performs significantly better than unsupervised learning through autoencoders. Notably, for the semi-supervised case, the self-supervised features substantially boost the detection rate by attaining a kappa score between 0.7 - 0.8 with only 10 labeled examples per class. We get similar impressive performance even if the features are transferred from a different data source. Self-supervision drastically reduces the requirement of labeled activity data, effectively narrowing the gap between supervised and unsupervised techniques for learning meaningful representations. While this paper focuses on HAR as the application domain, the proposed approach is general and could be applied to a wide variety of problems in other areas.

Kernel conditional clustering and kernel conditional semi-supervised learning

The results of clustering are often affected by covariates that are independent of the clusters one would like to discover. Traditionally, alternative clustering algorithms can be used to solve such clustering problems. However, these suffer from at least one of the following problems: (1) Continuous covariates or nonlinearly separable clusters cannot be handled; (2) assumptions are made about the distribution of the data; (3) one or more hyper-parameters need to be set. The presence of covariates also has an effect in a different type of problem such as semi-supervised learning. To the best of our knowledge, there is no existing method addressing the semi-supervised learning setting in the presence of covariates. Here we propose two novel algorithms, named kernel conditional clustering (KCC) and kernel conditional semi-supervised learning (KCSSL), whose objectives are derived from a kernel-based conditional dependence measure. KCC is parameter-light and makes no assumptions about the cluster structure, the covariates, or the distribution of the data, while KCSSL is fully parameter-free. On both simulated and real-world datasets, the proposed KCC and KCSSL algorithms perform better than state-of-the-art methods. The former detects the ground truth cluster structures more accurately, and the latter makes more accurate predictions.

Semi-Supervised Deep Neural Network for Joint Intensity Estimation of Multiple Facial Action Units

Facial action units (AUs) are defined to depict movements of facial muscles, which are basic elements to encode facial expressions. Automatic AU intensity estimation is an important task in affective computing. Previous works leverage the representation power of deep neural networks (DNNs) to improve the performance of intensity estimation. However, a large number of intensity annotations are required to train DNNs that contain millions of parameters. But it is expensive and difficult to build a large-scale database with AU intensity annotation since AU annotation requires annotators have strong domain expertise. We propose a novel semi-supervised deep convolutional network that leverages extremely limited AU annotations for AU intensity estimation. It requires only intensity annotations of keyframes of training sequences. Domain knowledge on AUs is leveraged to provide weak supervisory information, including relative appearance similarity, temporal intensity ordering, facial symmetry, and contrastive appearance difference. We also propose a strategy to train a model for joint intensity estimation of multiple AUs under the setting of semi-supervised learning, which greatly improves the efficiency during inference. We perform empirical experiments on two public benchmark expression databases and make comparisons with state-of-the-art methods to demonstrate the effectiveness of the proposed method.

Learning a discriminant graph-based embedding with feature selection for image categorization

Graph-based embedding methods are very useful for reducing the dimension of high-dimensional data and for extracting their relevant features. In this paper, we introduce a novel nonlinear method called Flexible Discriminant graph-based Embedding with feature selection (FDEFS). The proposed algorithm aims to classify image sample data in supervised learning and semi-supervised learning settings. Specifically, our method incorporates the Manifold Smoothness, Margin Discriminant Embedding and the Sparse Regression for feature selection. The weights add ℓ2,1-norm regularization for local linear approximation. The sparse regression implicitly performs feature selection on the original features of data matrix and of the linear transform. We also provide an effective solution method to optimize the objective function. We apply the algorithm on six public image datasets including scene, face and object datasets. These experiments demonstrate the effectiveness of the proposed embedding method. They also show that proposed the method compares favorably with many competing embedding methods.

Graph-based sparse linear discriminant analysis for high-dimensional classification

Linear discriminant analysis (LDA) is a well-known classification technique that enjoyed great success in practical applications. Despite its effectiveness for traditional low-dimensional problems, extensions of LDA are necessary in order to classify high-dimensional data. Many variants of LDA have been proposed in the literature. However, most of these methods do not fully incorporate the structure information among predictors when such information is available. In this paper, we introduce a new high-dimensional LDA technique, namely graph-based sparse LDA (GSLDA), that utilizes the graph structure among the features. In particular, we use the regularized regression formulation for penalized LDA techniques, and propose to impose a structure-based sparse penalty on the discriminant vector β. The graph structure can be either given or estimated from the training data. Moreover, we explore the relationship between the within-class feature structure and the overall feature structure. Based on this relationship, we further propose a variant of our proposed GSLDA to utilize effectively unlabeled data, which can be abundant in the semi-supervised learning setting. With the new regularization, we can obtain a sparse estimate of β and more accurate and interpretable classifiers than many existing methods. Both the selection consistency of β estimation and the convergence rate of the classifier are established, and the resulting classifier has an asymptotic Bayes error rate. Finally, we demonstrate the competitive performance of the proposed GSLDA on both simulated and real data studies.