Multi-Party Federated Recommendation Based on Semi-Supervised Learning

Deep learning promotes the fields of image processing, machine translation and natural language processing etc. It also can be used in network anomaly detection. In practice, it is not hard to obtain normal instances. However, it is always difficult to label anomalous instances. Semi-supervised learning can be utilized to resolve this problem. In this paper, we make a comprehensive study of semi-supervised deep learning techniques for network anomaly detection. Three kinds of deep learning techniques including GAN (Generative Adversarial networks), Auto-encoder and LSTM (Long Short-Term Memory) are studied on the latest network traffic dataset of CICIDS2017. Five deep architectures based on semi-supervised learning are designed, including BiGAN, regular GAN, WGAN, Auto-encoder and LSTM. Seven schemes of semi-supervised deep learning for anomaly detection are proposed according to different functions of anomaly score. Grid search is utilized to find the threshold of anomaly detection. Two traditional schemes of machine learning are also adopted to compare performance. There are altogether nine schemes of anomaly detection for CICIDS2017. From results of the experiment for network anomaly detection, it can be found that Auto-encoder outperforms LSTM and the three kinds of GAN. BiGAN and LSTM are both better than WGAN and regular GAN. All the seven schemes of semi-supervised deep learning for anomaly detection outperform the two traditional schemes. The work and results in this paper are meaningful on the application of semi-supervised deep learning for network anomaly detection.

Recently, the problem of indoor localization based on WLAN signals is attracting increasing attention due to the development of mobile devices and the widespread construction of networks. However, no definitive solution for achieving a low-cost and accurate positioning system has been found. In most traditional approaches, solving the indoor localization problem requires the availability of a large number of labeled training samples, the collection of which requires considerable manual effort. Previous research has not provided a means of simultaneously reducing human calibration effort and improving location accuracy. This paper introduces fusion semi-supervised extreme learning machine (FSELM), a novel semi-supervised learning algorithm based on the fusion of information from Wi-Fi and Bluetooth Low Energy (BLE) signals. Unlike previous semi-supervised methods, which consider multiple signals individually, FSELM fuses multiple signals into a unified model. When applied to sparsely calibrated localization problems, our proposed method is advantageous in three respects. First, it can dramatically reduce the human calibration effort required when using a semi-supervised learning framework. Second, it utilizes fused Wi-Fi and BLE fingerprints to markedly improve the location accuracy. Third, it inherits the beneficial properties of ELMs with regard to training and testing speeds because the input weights and biases of hidden nodes can be generated randomly. As demonstrated by experimental results obtained on practical indoor localization datasets, FSELM possesses a better semi-supervised manifold learning ability and achieves higher location accuracy than several previous batch supervised learning approaches (ELM, BP and SVM) and semi-supervised learning approaches (SELM, S-RVFL and FS-RVFL). Moreover, FSELM needs less training and testing time, making it easier to apply in practice. We conclude through experiments that FSELM yields good results when applied to a multi-signal-based semi-supervised learning problem. The contributions of this paper can be summarized as follows: First, the findings indicate that effective multi-data fusion can be achieved not only through data-layer fusion, feature-layer fusion and decision-layer fusion but also through the fusion of constraints within a model. Second, for semi-supervised learning problems, it is necessary to combine the advantages of different types of data by optimizing the model’s parameters.

The need for manual and detailed annotations limits the applicability of supervised deep learning algorithms in medical image analyses, specifically in the field of pathology. Semi-supervised learning (SSL) provides an effective way for leveraging unlabeled data to relieve the heavy reliance on the amount of labeled samples when training a model. Although SSL has shown good performance, the performance of recent state-of-the-art SSL methods on pathology images is still under study. The problem for selecting the most optimal data to label for SSL is not fully explored. To tackle this challenge, we propose a semi-supervised active learning framework with a region-based selection criterion. This framework iteratively selects regions for annotation query to quickly expand the diversity and volume of the labeled set. We evaluate our framework on a grey-matter/white-matter segmentation problem using gigapixel pathology images from autopsied human brain tissues. With only 0.1% regions labeled, our proposed algorithm can reach a competitive IoU score compared to fully-supervised learning and outperform the current state-of-the-art SSL by more than 10% of IoU score and DICE coefficient.

In human concept learning, people can naturally combine a handful of labeled data with abundant unlabeled data when they make classification decisions, which is also known as semi-supervised learning (SSL) in machine learning. Especially, human concept learning not only is a static process in human cognition but also can vary gradually with dynamic environments. Nevertheless, the classical SSL algorithms must be redesigned to accommodate newly input data. In this sense, concept-cognitive learning may be a good choice, as it can implement dynamic processes by imitating human cognitive processes. Meanwhile, numerous SSL methods were designed based on the feature vector information of instances, while ignoring concept structural information that is a very important process in human knowledge organization. Based on this idea, a novel SSL method, named semi-supervised concept learning method (S2CL), is proposed for dynamic SSL by employing concept spaces, in which knowledge is represented by hierarchical concept structures. Moreover, to make full use of the global and local conceptual information, we further propose an extended version of S2CL (namely, <inline-formula><tex-math notation="LaTeX">$\text{S2CL}^{\alpha }$</tex-math></inline-formula> ) for concept learning. More specifically, to effectively exploit the unlabeled data, this paper first shows some new related theories for S2CL (or <inline-formula><tex-math notation="LaTeX">$\text{S2CL}^{\alpha }$</tex-math></inline-formula> ) based on a regular formal decision context; then a novel SSL framework is designed, and its corresponding algorithm is developed. Finally, we conduct some experiments on various datasets to demonstrate the effectiveness of our methods, which include concept classification and incremental learning under a large quantity of unlabeled data.

ALG: Adaptive low-rank graph regularization for scalable semi-supervised and unsupervised learning

Semi-supervised transfer learning with hierarchical self-regularization

Semi-supervised learning acts as an effective way to leverage massive unlabeled data. In this paper, we propose a novel training strategy, termed as Semi-supervised Contrastive Learning (SsCL), which combines the well-known contrastive loss in self-supervised learning with the cross entropy loss in semi-supervised learning, and jointly optimizes the two objectives in an end-to-end way. The highlight is that different from self-training based semi-supervised learning that conducts prediction and retraining over the same model weights, SsCL interchanges the predictions over the unlabeled data between the two branches, and thus formulates a co-calibration procedure, which we find is beneficial for better prediction and avoid being trapped in local minimum. Towards this goal, the contrastive loss branch models pairwise similarities among samples, using the pseudo labels generated from the cross entropy branch, and in turn calibrates the prediction distribution of the cross entropy branch with the contrastive similarity. We show that SsCL produces more discriminative representation and is beneficial to semi-supervised learning. Notably, on ImageNet with ResNet50 as the backbone, SsCL achieves 60.2% and 72.1% top-1 accuracy with 1% and 10% labeled samples, respectively, which significantly outperforms the baseline, and is better than previous semi-supervised and self-supervised methods.

Semi-supervised extreme learning machine with manifold and pairwise constraints regularization

Extreme learning machine (ELM) not only is an effective classifier in supervised learning, but also can be applied on unsupervised learning and semi-supervised learning. The model structure of semi-supervised machine learning machine (SS-ELM) is same as ELM, the difference between them is the cost function. In this paper, we introduce kernel function to SS-ELM and proposed semi-supervised extreme learning machine with kernel (SS-KELM). Wavelet analysis has the characteristics of multivariate interpolation and sparse change, and the wavelet kernel is a kind of multidimensional wavelet function that can approximate arbitrary nonlinear functions. Therefore, we propose semi-supervised extreme learning machine with wavelet kernel (SS-WKELM) based on the wavelet kernel function and SS-ELM. The experimental results show the feasibility and validity of SS-WKELM in classification.

Deep learning approaches achieve state-of-the-art performance for classifying radiology images, but rely on large labelled datasets that require resource-intensive annotation by specialists. Both semi-supervised learning and active learning can be utilised to mitigate this annotation burden. However, there is limited work on combining the advantages of semi-supervised and active learning approaches for multi-label medical image classification. Here, we introduce a novel Consistency-based Semi-supervised Evidential Active Learning framework (CSEAL). Specifically, we leverage predictive uncertainty based on theories of evidence and subjective logic to develop an end-to-end integrated approach that combines consistency-based semi-supervised learning with uncertainty-based active learning. We apply our approach to enhance four leading consistency-based semi-supervised learning methods: Pseudo-labelling, Virtual Adversarial Training, Mean Teacher and NoTeacher. Extensive evaluations on multi-label Chest X-Ray classification tasks demonstrate that CSEAL achieves substantive performance improvements over two leading semi-supervised active learning baselines. Further, a class-wise breakdown of results shows that our approach can substantially improve accuracy on rarer abnormalities with fewer labelled samples.KeywordsSemi-supervised learningActive learningTheory of evidenceSubjective logicMulti-label classification

A semi-supervised deep learning approach for cropped image detection

A unified deep semi-supervised graph learning scheme based on nodes re-weighting and manifold regularization

One major challenge in the current brain–computer interface research is the accurate classification of time-varying electroencephalographic (EEG) signals. The labeled EEG samples are usually scarce, while the unlabeled samples are available in large quantities and easy to collect in real applications. Semi-supervised learning (SSL) methods can utilize both labeled and unlabeled data to improve performance over supervised approaches. However, it has been reported that the unlabeled data may undermine the performance of SSL in some cases. To improve the safety of SSL, we proposed a new safety-control mechanism by analyzing the differences between unlabeled data analysis in supervised and semi-supervised learning. We then develop and implement a safe classification method based on the semi-supervised extreme learning machine (SS-ELM). Following this approach, the Wasserstein distance is used to measure the similarities between the predictions obtained from ELM and SS-ELM algorithms, and a different risk degree is thereby calculated for each unlabeled data instance. A risk-based regularization term is then constructed and embedded into the objective function of the SS-ELM. Extensive experiments were conducted using benchmark and EEG datasets to evaluate the effectiveness of the proposed method. Experimental results show that the performance of the new algorithm is comparable to SS-ELM and superior to ELM on average. It is thereby shown that the proposed method is safe and efficient for the classification of EEG signals.

Predicting the energy consumption of buildings plays a critical role in supporting utility providers, users, and facility managers in minimizing energy waste and optimizing operational efficiency. However, this prediction becomes difficult because of the limited availability of supervised labeled data to train Artificial Intelligence (AI) models. This data availability becomes either expensive or difficult due to privacy protection. To overcome the scarcity of balanced labeled data, semi-supervised learning utilizes extensive unlabeled data. Motivated by this, we propose semi-supervised learning to train AI model. For the AI model, we employ the Belief Rule-Based Expert System (BRBES) because of its domain knowledge-based prediction and uncertainty handling mechanism. For improved accuracy of the BRBES, we utilize initial labeled data to optimize BRBES’ parameters and structure through evolutionary learning until its accuracy reaches the confidence threshold. As semi-supervised learning, we employ a self-training model to assign pseudo-labels, predicted by the BRBES, to unlabeled data generated through weak and strong augmentation. We reoptimize the BRBES with labeled and pseudo-labeled data, resulting in a semi-supervised BRBES. Finally, this semi-supervised BRBES explains its prediction to the end-user in nontechnical human language, resulting in a trust relationship. To validate our proposed semi-supervised explainable BRBES framework, a case study based on Skellefteå, Sweden, is used to predict and explain energy consumption of buildings. Experimental results show 20 ± 0.71% higher accuracy of the semi-supervised BRBES than state-of-the-art semi-supervised machine learning models. Moreover, the semi-supervised BRBES framework turns out to be 29 ± 0.67% more explainable than these semi-supervised machine learning models.

With the continuous advancement of industrial automation, intelligent defect detection has become a crucial means of ensuring product quality. However, the cost and labor required to obtain high-quality labeled data limit the widespread practical application of traditional supervised learn-ing methods. Therefore, semi-supervised learning (SSL) and unsupervised learning (UL) methods have received extensive attention from researchers due to their superior performance in low-labeled or unlabeled scenarios. This paper provides a systematic survey of typical semi-supervised and unsupervised learning methods in the field of industrial defect detection, analyzes the core concepts and key technologies of unsupervised learning and semi-supervised learning, as well as compara-tive analysis on relevant datasets, and finally proposes future development direc-tions.