Semi-supervised Method Research Articles

Role of Machine and Deep Learning Algorithms in Secure Intrusion Detection Systems (IDS) for IOT & Smart Cities

In this study the authors have examines various machine learning algorithms that could be used in IDS for making secure IoT and Smart Cities. The study examines various deep learning architectures of supervised, unsupervised, and semi-supervised learning methods to improve security and resource usage. Federated learning, edge computing, explainable AI, adversarial machine learning defense, and transfer learning are also explored for smart farming and IoT challenges. Machine learning has the potential to improve security and agricultural sustainability, but it must be researched and developed.

A Semi-supervised crowd counting method based on patch crowds statistics

A Semi-supervised crowd counting method based on patch crowds statistics

Almost exact recovery in noisy semi-supervised learning

Abstract Graph-based semi-supervised learning methods combine the graph structure and labeled data to classify unlabeled data. In this work, we study the effect of a noisy oracle on classification. In particular, we derive the maximum a posteriori (MAP) estimator for clustering a degree corrected stochastic block model when a noisy oracle reveals a fraction of the labels. We then propose an algorithm derived from a continuous relaxation of the MAP, and we establish its consistency. Numerical experiments show that our approach achieves promising performance on synthetic and real data sets, even in the case of very noisy labeled data.

SSML: Semi-supervised metric learning with hard samples for hyperspectral image classification

Deep learning is widely used in hyperspectral image (HSI) classification due to its powerful learning capabilities. However, its excellent performance typically requires a large number of samples, which can be time-consuming and labor-intensive to produce. The limitation of available samples greatly constrains the model's generalization performance. To alleviate the sample pressure, we propose a semi-supervised metric learning method for HSI classification that focuses on hard samples and can obtain reliable pseudo-labels through multiscale prediction. Additionally, we employ a hard sample learning strategy for network training, which concentrates on enhancing network discrimination by adaptively optimizing intra-class and inter-class distances. Performance tests on three public datasets indicate that the proposed method surpasses other state-of-the-art methods across multiple validation metrics.

Unsupervised PolSAR Image Classification Based on Superpixel Pseudo-Labels and a Similarity-Matching Network

Supervised polarimetric synthetic aperture radar (PolSAR) image classification demands a large amount of precisely labeled data. However, such data are difficult to obtain. Therefore, many unsupervised methods have been proposed for unsupervised PolSAR image classification. The classification maps of unsupervised methods contain many high-confidence samples. These samples, which are often ignored, can be used as supervisory information to improve classification performance on PolSAR images. This study proposes a new unsupervised PolSAR image classification framework. The framework combines high-confidence superpixel pseudo-labeled samples and semi-supervised classification methods. The experiments indicated that this framework could achieve higher-level effectiveness in unsupervised PolSAR image classification. First, superpixel segmentation was performed on PolSAR images, and the geometric centers of the superpixels were generated. Second, the classification maps of rotation-domain deep mutual information (RDDMI), an unsupervised PolSAR image classification method, were used as the pseudo-labels of the central points of the superpixels. Finally, the unlabeled samples and the high-confidence pseudo-labeled samples were used to train an excellent semi-supervised method, similarity matching (SimMatch). Experiments on three real PolSAR datasets illustrated that, compared with the excellent RDDMI, the accuracy of the proposed method was increased by 1.70%, 0.99%, and 0.8%. The proposed framework provides significant performance improvements and is an efficient method for improving unsupervised PolSAR image classification.

NiReject: toward automated bad channel detection in functional near-infrared spectroscopy.

The increasing sample sizes and channel densities in functional near-infrared spectroscopy (fNIRS) necessitate precise and scalable identification of signals that do not permit reliable analysis to exclude them. Despite the relevance of detecting these "bad channels," little is known about the behavior of fNIRS detection methods, and the potential of unsupervised and semi-supervised machine learning remains unexplored. We developed three novel machine learning-based detectors, unsupervised, semi-supervised, and hybrid NiReject, and compared them with existing approaches. We conducted a systematic literature search and demonstrated the influence of bad channel detection. Based on 29,924 signals from two independently rated datasets and a simulated scenario space of diverse phenomena, we evaluated the NiReject models, six of the most established detection methods in fNIRS, and 11 prominent methods from other domains. Although the results indicated that a lack of proper detection can strongly bias findings, detection methods were reported in only 32% of the included studies. Semi-supervised models, specifically semi-supervised NiReject, outperformed both established thresholding-based and unsupervised detectors. Hybrid NiReject, utilizing a human feedback loop, addressed the practical challenges of semi-supervised methods while maintaining precise detection and low rating effort. This work contributes toward more automated and reliable fNIRS signal quality control by comprehensively evaluating existing and introducing novel machine learning-based techniques and outlining practical considerations for bad channel detection.

A Novel Landmark-based Semi-supervised Deep Learning Method for Cerebral Aneurysm Detection Using TOF-MRA

Background: Time-of-flight (TOF) magnetic resonance angiography (MRA) is widely used to identify aneurysm in human brain. Various deep learning models have been developed to help TOF-MRA reading in the field. The performance of those TOF-MRA analysis tools, however, faces several limitations in cerebral aneurysm detection. These challenges primarily come from the fact that cerebral aneurysms occupy less than 0.1% of the total TOF-MRA voxel size. This study aims to improve the efficiency of cerebral aneurysm detection by developing a landmark-based semi-supervised deep learning method, a technology that automatically generates landmark boxes in areas with a high probability of cerebral aneurysm occurrence.Methods: We used data from a total of 500 aneurysm-positive and 50 aneurysm-negative subjects. The aneurysm detection model was developed using clustering and a dilated residual network.Results: When the number of landmarks was ten and their size was 36 mm<sup>3</sup>, the best performance was achieved in our experiment. Although landmark occupies a small portion of the entire image, up to 98.2% of landmarks were cerebral aneurysms. The sensitivity of the model for cerebral aneurysm detection was 83.0%, with a false positive rate of 3.4%.Conclusions: This study developed a deep learning model using TOF-MRA image. This model generates the most suitable landmarks for each individual, excluding unnecessary areas for cerebral aneurysm detection, which makes it possible to focus on areas with a high probability of occurrence. This model is expected to enhance the efficiency and accuracy of cerebral aneurysm detection in the field.

Character region extraction of wheel water meter based on object detection

Currently, research on automatic meter reading mainly focuses on meter reading recognition, while neglecting the fundamental role of counter detection in the entire automatic meter reading system. In fact, only by accurately locating the counter area can the influence of dial factors be completely eliminated, thus ensuring the accuracy and reliability of subsequent water meter reading recognition. In view of this phenomenon, the focus of this study is on the counter detection stage. Firstly, a target detection-based image skew correction method is proposed to solve the problem of image skew caused by shooting angle and other reasons. This method ensures the accuracy of subsequent counter area positioning and the neatness of cutting effect. Secondly, a semi-supervised target detection training method is proposed to solve the problem of time and manpower costs required in large-scale data situations. In addition, we have made publicly available a dataset containing 1070 water meter images for non-commercial purposes, which can be obtained from the Github11https://github.com/QuanhuiZhao/water-datasets.. Finally, we evaluated our model on three completely different datasets and compared it with the best positioning results of other models. The experimental results show that compared with other models, the proposed model in this paper has improved the positioning accuracy by 5.82%, 5.96%, and 9.20% on three datasets respectively. Furthermore, in the final visualization comparison, the model accurately identifies the counter region even when faced with complex real-world environments.

Applications of cluster-based transfer learning in image and localization tasks

Transfer learning can address the issue of insufficient labels in machine learning. Using knowledge in a labeled domain (source domain) can assist in acquiring and learning knowledge in a domain (target domain) that lacks some or all labels. In this paper, we propose a new cluster-based semi-supervised transfer learning (CBSSTL) under a new assumption that samples in the target domain are unlabeled but contain cluster information. Furthermore, we propose a new transfer learning framework and a method for fine-tuning parameters. We tested and compared the proposed method with other unsupervised and semi-supervised transfer learning methods on well-known image datasets. The experimental results demonstrate the effectiveness of the proposed method. Additionally, we created a localization dataset for transfer learning. Finally, we tested and analyzed the proposed method on this dataset. Its particularly challenging nature makes it difficult for our method to work effectively.

Semi-supervised multi-view feature selection with adaptive similarity fusion and learning

Existing multi-view semi-supervised feature selection methods typically need to calculate the inversion of high-order dense matrices, rendering them impractical for large-scale applications. Meanwhile, traditional works construct similarity graphs on different views and directly fuse these graphs from the view level, ignoring the differences among samples in various views and the interplay between graph learning and feature selection. Consequently, both the reliability of graphs and the discrimination of selected features are compromised. To address these issues, we propose a novel multi-view semi-supervised feature selection with Adaptive Similarity Fusion and Learning (ASFL) for large-scale tasks. Specifically, ASFL constructs bipartite graphs for each view and then leverages the relationships between samples and anchors to align anchors and graphs across different views, preserving the complementarity and consistency among views. Moreover, an effective view-to-sample fusion manner is designed to coalesce the aligned graphs while simultaneously exploiting the neighbor structures in projection subspaces to construct the joint graph compatible across views, reducing the adverse effects of noisy features and outliers. By incorporating bipartite graph fusion and learning, label propagation, and l2,0-norm multi-view feature selection into a unified framework, ASFL not only avoids the expensive computation in the solution procedures but also enhances the quality of selected features. An effective optimization strategy with fast convergence is developed to solve the objective function, and experimental results validate its efficiency and effectiveness over state-of-the-art methods.

Salt dome identification using incremental semi-supervised learning and unsupervised learning-based label generation

Salt domes represent distinctive geological anomalies in seismic data, crucial for pinpointing hydrocarbon reservoirs and strategizing drilling paths. Conventional seismic attributes or computer vision methods usually fail to capture the intricate details of salt domes, resulting in interpretation results marred by noise. While deep learning presents a promising approach for intelligent 3D salt dome interpretation, its effectiveness is heavily dependent on the availability of labeled samples. To facilitate accurate interpretation, we propose an innovative workflow that integrates an unsupervised label generation component with an incremental semi-supervised learning framework utilizing the U-Net architecture. To generate salt dome labels, we prioritize both the root mean square (RMS) amplitude attribute and variance attribute (VA) as foundational data. Utilizing convolutional autoencoders (CAE), we establish a relationship between the input RMS attribute and the output reconstructed attribute. The intermediate features extracted by CAE are transformed into the salt boundary feature via principal component analysis and K-Means clustering. Concurrently, we employ K-Means clustering on VA to ascertain the salt internal feature. We further propose a feature aggregation method to consolidate the salt boundary feature and the salt internal feature for label generation of the salt dome. For 3D salt dome interpretation, we begin by predicting adjacent test datasets using labels generated by the unsupervised salt dome label generation module. The prediction results of these test datasets are then integrated into the training datasets to enhance the interpretation performance of U-Net, steering it towards an incremental semi-supervised learning method for salt dome interpretation. Additionally, we extend this research by applying transfer learning techniques for identifying mound-shoals using the same semi-supervised model parameters initially developed for interpreting salt domes. This method is validated using datasets from the Netherlands F3 block for salt domes and the North China block for mound-shoals. The results demonstrate that this innovative process only requires a minimal number of labels from unsupervised methods to precisely interpret salt domes across 3D seismic data. Furthermore, the low-level features of salt domes learned from neural network can be seamlessly transferred to mound-shoal identification. This automated approach significantly streamlines the interpretation process, reducing the time and resources traditionally necessary for reservoir analysis.

PE-MCAT: Leveraging Image Sensor Fusion and Adaptive Thresholds for Semi-Supervised 3D Object Detection.

Existing 3D object detection frameworks in sensor-based applications heavily rely on large-scale annotated data to achieve optimal performance. However, obtaining such annotations from sensor data-like LiDAR or image sensors-is both time-consuming and costly. Semi-supervised learning offers an efficient solution to this challenge and holds significant potential for sensor-driven artificial intelligence (AI) applications. While it reduces the need for labeled data, semi-supervised learning still depends on a small amount of labeled samples for training. In the initial stages, relying on such limited samples can adversely affect the effective training of student-teacher networks. In this paper, we propose PE-MCAT, a semi-supervised 3D object detection method that generates high-precision pseudo-labels. First, to address the challenges of insufficient local feature capture and poor robustness in point cloud data, we introduce a point enrichment module. This module incorporates information from image sensors and combines multiple feature fusion methods of local and self-features to directly enhance the quality of point clouds and pseudo-labels, compensating for the limitations posed by using only a few labeled samples. Second, we explore the relationship between the teacher network and the pseudo-labels it generates. We propose a multi-class adaptive threshold strategy to initially filter and create a high-quality pseudo-label set. Furthermore, a joint variable threshold strategy is introduced to refine this set further, enhancing the selection of superior pseudo-labels.Extensive experiments demonstrate that PE-MCAT consistently outperforms recent state-of-the-art methods across different datasets. Specifically, on the KITTI dataset and using only 2% of labeled samples, our method improved the mean Average Precision (mAP) by 0.7% for cars, 3.7% for pedestrians, and 3.0% for cyclists.

A Pseudo-Labeling Multi-Screening-Based Semi-Supervised Learning Method for Few-Shot Fault Diagnosis.

In few-shot fault diagnosis tasks in which the effective label samples are scarce, the existing semi-supervised learning (SSL)-based methods have obtained impressive results. However, in industry, some low-quality label samples are hidden in the collected dataset, which can cause a serious shift in model training and lead to the performance of SSL-based method degradation. To address this issue, the latest prototypical network-based SSL techniques are studied. However, most prototypical network-based scenarios consider that each sample has the same contribution to the class prototype, which ignores the impact of individual differences. This article proposes a new SSL method based on pseudo-labeling multi-screening for few-shot bearing fault diagnosis. In the proposed work, a pseudo-labeling multi-screening strategy is explored to accurately screen the pseudo-labeling for improving the generalization ability of the prototypical network. In addition, the AdaBoost adaptation-based weighted technique is employed to obtain accurate class prototypes by clustering multiple samples, improving the performance that deteriorated by low-quality samples. Specifically, the squeeze and excitation block technique is used to enhance the useful feature information and suppress non-useful feature information for extracting accuracy features. Finally, three well-known bearing datasets are selected to verify the effectiveness of the proposed method. The experiments illustrated that our method can receive better performance than that of the state-of-the-art methods.

Consistency Self-Training Semi-Supervised Method for Road Extraction from Remote Sensing Images

Road extraction techniques based on remote sensing image have significantly advanced. Currently, fully supervised road segmentation neural networks based on remote sensing images require a significant number of densely labeled road samples, limiting their applicability in large-scale scenarios. Consequently, semi-supervised methods that utilize fewer labeled data have gained increasing attention. However, the imbalance between a small quantity of labeled data and a large volume of unlabeled data leads to local detail errors and overall cognitive mistakes in semi-supervised road extraction. To address this challenge, this paper proposes a novel consistency self-training semi-supervised method (CSSnet), which effectively learns from a limited number of labeled data samples and a large amount of unlabeled data. This method integrates self-training semi-supervised segmentation with semi-supervised classification. The semi-supervised segmentation component relies on an enhanced generative adversarial network for semantic segmentation, which significantly reduces local detail errors. The semi-supervised classification component relies on an upgraded mean-teacher network to handle overall cognitive errors. Our method exhibits excellent performance with a modest amount of labeled data. This study was validated on three separate road datasets comprising high-resolution remote sensing satellite images and UAV photographs. Experimental findings showed that our method consistently outperformed state-of-the-art semi-supervised methods and several classic fully supervised methods.

Semi-supervised comparative learning compensation method for chemical gas sensor drift.

The gradual and unpredictable variation in chemo-sensory signal responses when exposed to the same analyte under identical conditions, commonly referred to as sensor drift, has long been recognized as one of the most serious challenges faced by chemical sensors. The traditional drift compensation method is both labor-intensive and expensive, as it requires frequent collection and labeling of gas samples for recalibration. Introducing a small number of meaningful drift calibration samples can be an attractive strategy to reduce the computational load and improve the performance of the updated classifier. However, under the influence of drift, new challenges arise due to the difference in the distribution of source and target domain data. This paper proposes a novel algorithm framework called semi-supervised contrastive learning drift compensation (SSCLDC). The framework automatically extracts high-level abstract features based on a multilayer perceptron to better represent the structure of the source data. In addition, to address the issue of data distribution differences caused by drift between the source and target domains. We add a small number of reference sample pairs into the training for semi-supervised learning. Combining a contrastive loss function that can represent the matching degree of paired samples effectively overcomes the problem of sensor drift. The Kennard-Stone sequential algorithm is used to select the representative reference sample from the set of candidate reference samples. Experiments conducted on a widely used long-term chemical gas sensor drift dataset demonstrate that the proposed method outperforms several classic drift compensation techniques, highlighting its effectiveness and practical applicability.

A comprehensive review of deep learning for medical image segmentation

Medical image segmentation provides detailed mappings of regions of interest, facilitating precise identification of critical areas and greatly aiding in the diagnosis, treatment, and understanding of diverse medical conditions. However, conventional techniques frequently rely on hand-crafted feature-based approaches, posing challenges when dealing with complex medical images, leading to issues such as low accuracy and sensitivity to noise. Recent years have seen substantial research focused on the effectiveness of deep learning models for segmenting medical images. In this study, we present a comprehensive review of the various deep learning-based approaches for medical image segmentation and provide a detailed analysis of their contributions to the domain. These methods can be broadly categorized into five groups: CNN-based methods, Transformer-based methods, Mamba-based methods, semi-supervised learning methods, and weakly supervised learning methods. Convolutional Neural Networks (CNNs), with their efficient feature self-learning, have driven major advances in medical image segmentation. Subsequently, Transformers, leveraging self-attention mechanisms, have achieved performance on par with or surpassing Convolutional Neural Networks. Mamba-based methods, as a novel selective state-space model, are emerging as a promising direction. Furthermore, due to the limited availability of annotated medical images, research in weakly supervised and semi-supervised learning continues to evolve. This review covers common evaluation methods, datasets, and deep learning applications in diagnosing and treating skin lesions, hippocampus, tumors, and polyps. Finally, we identify key challenges such as limited data, diverse modalities, noise, and clinical applicability, and propose future research in zero-shot segmentation, transfer learning, and multi-modal techniques to advance the development of medical image segmentation technology.

Research on elastic parameter inversion method based on seismic facies-controlled deep learning network

Deep learning has been widely applied in the field of geophysics, and existing research literature has demonstrated that intelligent geophysical inversion methods have high vertical resolution but low horizontal resolution. The reason lies in the fact that existing horizontal constraint methods mainly adopt convolutional models, without fully considering other prior information of seismic data. Within the same sedimentary unit, seismic response characteristics vary gradually due to similar lithology and geological characteristics. Therefore, the seismic facies information extracted from seismic data is integrated into deep learning network to enhance the horizontal prediction stability of the network. Firstly, according to the spatial and temporal characteristics of seismic data, a fusion network of three-dimensional convolutional neural network (3D-CNN), gated recurrent unit (GRU) and attention mechanism is established to improve the vertical resolution of inversion results. Then, seismic facies classification of the target layer is achieved by applying the K-means clustering method. Subsequently, to improve the horizontal resolution of the inversion results, seismic facies classification is transformed into temporal encoding data using the position coding theory in natural language processing, to form a seismic facies-controlled deep learning network. Finally, the deep learning network is trained and tested in the thin interlayer model and practical application adopting a semi-supervised learning method. The results indicate that incorporating seismic facies-controlled technology in the deep learning network can improve the horizontal resolution of the inversion results.

Digital Grading Company: Deep Neural Networks for Predictive Analytics for Alternative Asset Portfolio Management

In this study, we investigate the application of deep neural networks for predictive analytics in the realm of alternative asset portfolio management, focusing on Trading Card Games (TCGs) like Pokemon. Our primary objective is to understand the optimal trading cards for investment and to determine the appropriate timing for divestment and retention of these assets. Drawing on the successful implementation of machine learning predictive analytics in traditional financial markets, we aim to extend these techniques to alternative investments. To achieve this, we employ a two-pronged approach using a comprehensive dataset of historical sales values for Pokemon TCGs spanning five years, comprising over 200 million unlabeled data points. The first approach utilizes a semi-supervised sequence learning method, where a Long Short-Term Memory (LSTM) model is initially trained as a sequence autoencoder. The pre-trained model's weights are then used in a supervised phase to fine-tune the predictions for future trading card values. The second approach involves heuristic pretraining, which leverages market-derived indicators from existing literature to bridge the gap between traditional feature engineering and deep learning methods. Our technology stack is designed to handle end-to-end data collection, analytics generation, and execution of recommended trading strategies. Simulated backtesting results demonstrate that our Digital Grading Company (DGC) portfolio model outperforms traditional indices, achieving a 357% return compared to the 61% return of the S&P 500 Index over the same period. Real-time analysis further indicates that our model consistently identifies profitable trading opportunities in the TCG market using hourly data and proprietary algorithmic trade logic, executing trades 24/7. The DGC Predictive Portfolio, based on ungraded Pokemon TCG trading cards, also shows superior performance when compared to the Nasdaq 100 Index (Invesco QQQ ETF). Our analytics framework not only predicts the direction of price movement but also provides insights into the magnitude of changes, enabling a more dynamic and responsive trading strategy. The findings suggest that the integration of deep learning models with heuristic indicators has the potential to revolutionize portfolio management in alternative investments, paving the way for more accurate and profitable decision-making in niche markets. This research highlights the promising role of machine learning in enhancing alternative asset management strategies, suggesting future applications in other asset classes. We conclude that deep neural networks, when combined with data-driven heuristics, can significantly enhance trading outcomes, making them valuable tools for investors in the evolving landscape of alternative investments.

Semi-supervised segmentation of cardiac chambers from LGE-CMR using feature consistency awareness

BackgroundLate gadolinium enhancement cardiac magnetic resonance imaging (LGE-CMR) is a valuable cardiovascular imaging technique. Segmentation of cardiac chambers from LGE-CMR is a fundamental step in electrophysiological modeling and cardiovascular disease diagnosis. Deep learning methods have demonstrated extremely promising performance. However, excellent performance often depended on a large amount of finely annotated data. The purpose of this manuscript was to develop a semi-supervised segmentation method to use unlabeled data to improve model performance.MethodsThis manuscript proposed a semi-supervised network that integrates triple-consistency constraints (data-level, task-level, and feature-level) for cardiac chambers segmentation from LGE-CMR. Specifically, we designed a network that integrated segmentation and edge prediction tasks based on the mean teacher architecture. This addressed the problem of ignoring some challenging regions because of excluding low-confidence regions of previous research. We also applied a voxel-level contrastive learning strategy to achieve feature-level consistency, helping the model pay attention to the consistency between features overlooked in previous research.ResultsIn terms of the Dice, Jaccard, Average Surface Distance (ASD), and 95% Hausdorff Distance (95HD) metrics, for the atrial segmentation dataset, the proposed method achieved scores of 88.34%, 79.30%, 7.92, and 2.02 when trained with 10% labeled data, and 90.70%, 83.09%, 6.41, and 1.72 when trained with 20% labeled data. For the ventricular segmentation task, the results were 87.22%, 77.95%, 2.27, and 0.61 with 10% labeled data, and 88.99%, 80.45%, 1.87, and 0.51 with 20% labeled data, respectively.ConclusionExperiments demonstrated that our method outperforms previous semi-supervised methods, showing the potential of the proposed network for semi-supervised segmentation problems.

Tri-training algorithm based nuclear power systems semi-supervised fault diagnosis under multiple restricted data conditions

An improved tri-training semi-supervised classification method was proposed as a solution to the restricted data conditions of supervised classification models relying on labeled data, semi-supervised models not considering imbalanced proportion of faults data and the presence of faults data with similar feature or varying severity in existing data-driven nuclear power system fault diagnosis studies. Based on the structure of three identical sub-classifiers in the original tri-training algorithm, data update paths have been added, the updated data selection has been strengthened, and the initial information and permissions of sub-models have been differentiated. The novel method only requires the dataset to meet the Smoothness Assumption, and through multiple strict data update paths, it improves the utilization of unlabeled training data while reducing Pseudo-labeling process errors. The difference in permissions and initial information between sub-classifiers is utilized to improve the training rigor for fault types with smaller sample sizes, in order to alleviate the problem of imbalanced data. Based on a marine nuclear power system secondary circuit model and a widely recognized nuclear power plant dataset, multiple nuclear power system common restricted fault diagnosis datasets faced in ships and power plants were obtained to verify the novel method's advantages and generalization. The conclusion is that under the same conditions, compared with the original tri-training method and other semi-supervised learning methods, the Accuracy, AUC, Precision Rate and Recall Rate of the novel method have improved by about 21 %, 10 %, 20 %, and 21 %, respectively, and can reduce about 25 % of misjudgments between similar feature faults.