Semi-supervised Training Method Research Articles

An Ergonomic Risk Assessment System Based on 3D Human Pose Estimation and Collaborative Robot

Human pose estimation focuses on methods that allow us to assess ergonomic risk in the workplace and aims to prevent work-related musculoskeletal disorders (WMSDs). The recent increase in the use of Industry 4.0 technologies has allowed advances to be made in machine learning (ML) techniques for image processing to enable automated ergonomic risk assessment. In this context, this study aimed to develop a method of calculating joint angles from digital snapshots or videos using computer vision and ML techniques to achieve a more accurate evaluation of ergonomic risk. Starting with an ergonomic analysis, this study explored the use of a semi-supervised training method to detect the skeletons of workers and to estimate the positions and angles of their joints. A criticality index, based on RULA scores and fuzzy rules, is then calculated to evaluate possible corrective actions aimed at reducing WMSDs and improving production capacity using a collaborative robot that supports workers in carrying out critical operations. This method is tested in a real industrial case in which the manual assembly of electrical components is conducted, achieving a reduction in overall ergonomic stress of 13% and an increase in production capacity of 33% during a work shift. The proposed approach can overcome the limitations of recent developments based on computer vision or wearable sensors by performing an assessment with an objective and flexible approach to postural analysis development.

Wheat Teacher: A One-Stage Anchor-Based Semi-Supervised Wheat Head Detector Utilizing Pseudo-Labeling and Consistency Regularization Methods

Wheat breeding heavily relies on the observation of various traits during the wheat growth process. Among all traits, wheat head density stands out as a particularly crucial characteristic. Despite the realization of high-throughput phenotypic data collection for wheat, the development of efficient and robust models for extracting traits from raw data remains a significant challenge. Numerous fully supervised target detection algorithms have been employed to address the wheat head detection problem. However, constrained by the exorbitant cost of dataset creation, especially the manual annotation cost, fully supervised target detection algorithms struggle to unleash their full potential. Semi-supervised training methods can leverage unlabeled data to enhance model performance, addressing the issue of insufficient labeled data. This paper introduces a one-stage anchor-based semi-supervised wheat head detector, named “Wheat Teacher”, which combines two semi-supervised methods, pseudo-labeling, and consistency regularization. Furthermore, two novel dynamic threshold components, Pseudo-label Dynamic Allocator and Loss Dynamic Threshold, are designed specifically for wheat head detection scenarios to allocate pseudo-labels and filter losses. We conducted detailed experiments on the largest wheat head public dataset, GWHD2021. Compared with various types of detectors, Wheat Teacher achieved a mAP0.5 of 92.8% with only 20% labeled data. This result surpassed the test outcomes of two fully supervised object detection models trained with 100% labeled data, and the difference with the other two fully supervised models trained with 100% labeled data was within 1%. Moreover, Wheat Teacher exhibits improvements of 2.1%, 3.6%, 5.1%, 37.7%, and 25.8% in mAP0.5 under different labeled data usage ratios of 20%, 10%, 5%, 2%, and 1%, respectively, validating the effectiveness of our semi-supervised approach. These experiments demonstrate the significant potential of Wheat Teacher in wheat head detection.

An intelligent identification method based on self-adaptive mechanism regulated neural network for chemical process

BackgroundThe complex characteristics of chemical process, such as multivariable, nonlinear, time-varying and strong coupling often lead to the poor effect of traditional identification theory in practical application. The development of deep learning in recent years has brought a breakthrough for nonlinear system identification, but more progress is still needed. MethodsThis paper proposes a nonlinear identification method based on self-adaptive mechanism regulated Long Short-Term Memory (LSTM) network for chemical process dynamic simulation. First, to increase the reliability of the application of neural network (NN) for chemical process identification and improve the generalization ability, the known differential equation describing the mechanism is taken as a regularization term to constrain the output of the NN. Then, a specific training method is proposed, which introduces trainable self-adaptive weights to force the neural network to focus on the regions with large training error. In addition, a semi-supervised network training method is proposed for the case that some parameters in the mechanism equation are unknown. Finally, a dynamic virtual device (VD) model is established, which can simulate the dynamic response of controlled objects. Significant findingsTo evaluate the efficiency of the developed identification method, various comparative experiments are conducted on pH neutralization and continuous stirred tank reactor (CSTR) processes. The experimental results show that the proposed identification method can obtain a nonlinear dynamic model with robustness, high accuracy and strong generalization ability.

Breaking the barriers of data scarcity in drug-target affinity prediction.

Accurate prediction of drug-target affinity (DTA) is of vital importance in early-stage drug discovery, facilitating the identification of drugs that can effectively interact with specific targets and regulate their activities. While wet experiments remain the most reliable method, they are time-consuming and resource-intensive, resulting in limited data availability that poses challenges for deep learning approaches. Existing methods have primarily focused on developing techniques based on the available DTA data, without adequately addressing the data scarcity issue. To overcome this challenge, we present the Semi-Supervised Multi-task training (SSM) framework for DTA prediction, which incorporates three simple yet highly effective strategies: (1) A multi-task training approach that combines DTA prediction with masked language modeling using paired drug-target data. (2) A semi-supervised training method that leverages large-scale unpaired molecules and proteins to enhance drug and target representations. This approach differs from previous methods that only employed molecules or proteins in pre-training. (3) The integration of a lightweight cross-attention module to improve the interaction between drugs and targets, further enhancing prediction accuracy. Through extensive experiments on benchmark datasets such as BindingDB, DAVIS and KIBA, we demonstrate the superior performance of our framework. Additionally, we conduct case studies on specific drug-target binding activities, virtual screening experiments, drug feature visualizations and real-world applications, all of which showcase the significant potential of our work. In conclusion, our proposed SSM-DTA framework addresses the data limitation challenge in DTA prediction and yields promising results, paving the way for more efficient and accurate drug discovery processes.

An MRI image automatic diagnosis model for lumbar disc herniation using semi-supervised learning

Lumbar disc herniation is a common disease that causes low back pain. Due to the high cost of medical diagnosis, as well as a shortage and uneven distribution of medical resources, a system that can automatically analyze and diagnose lumbar spine Magnetic Resonance Imaging (MRI) is becoming an urgent need. This study uses deep learning methods to establish a classifier to diagnose lumbar disc herniation. An MRI classification dataset of lumbar disc herniation consisting of public MRI images is presented and is used to train the proposed classifier. Because a common difficulty in applying computer vision technology to medical images is labeling training data, we use a semi-supervised model training method, while multilayer transverse axial MRI images are used as the model input. In this method, we first use unlabelled MRI images for random self-supervised pre-training and the pre-trained model as a feature extractor for MRI images. Then, all marked cross-sections of each intervertebral disc are used to calculate the feature vector through the feature extractor. The information of all feature vectors is integrated, while a multilayer perceptron is used for classification training. After training, the model achieved 87.11% accuracy, 87.50% sensitivity, 86.72% specificity and 0.9487 AUC (Area Under the ROC Curve) index on the test set. To analyze the rationality of the diagnostic results more quickly, we output the severity of degenerative changes in each region using a heatmap.

Semi-supervised interactive fusion network for MR image segmentation.

Medical image segmentation is an important task in the diagnosis and treatment of cancers. The low contrast and highly flexible anatomical structure make it challenging to accurately segment the organs or lesions. To improve the segmentation accuracy of the organs or lesions in magnetic resonance (MR) images, which can be useful in clinical diagnosis and treatment ofcancers. First, a selective feature interaction (SFI) module is designed to selectively extract the similar features of the sequence images based on the similarity interaction. Second, a multi-scale guided feature reconstruction (MGFR) module is designed to reconstruct low-level semantic features and focus on small targets and the edges of the pancreas. Third, to reduce manual annotation of large amounts of data, a semi-supervised training method is also proposed. Uncertainty estimation is used to further improve the segmentation accuracy. Three hundred ninety-five 3D MR images from 395 patients with pancreatic cancer, 259 3D MR images from 259 patients with brain tumors, and four-fold cross-validation strategy are used to evaluate the proposed method. Compared to state-of-the-art deep learning segmentation networks, the proposed method can achieve better segmentation of pancreas or tumors in MR images. SFI-Net can fuse dual sequence MR images for abnormal pancreas or tumor segmentation. The proposed semi-supervised strategy can further improve the performance ofSFI-Net.

Artificial neural networks based multimodal device for autism spectrum disorder.

The neuro developmental condition known as Autism Spectrum Disorder (ASD) affects people on a lifetime basis and exhibits itself in a wide range of ways. In this research work a brand-new semi-supervised training method for the recognition of discrete multi-modal autism spectrum disorder is proposed. At the coarse-grained level, we consider that various methodologies are anticipated to explore equivalent information about child autism. To build DC AlexNet, this combines two small network branches and a large network (trunk network). The network trunk is programmed just to become familiar with the distinguishing characteristics shared by face images at different resolutions. It is built using recently suggested residential components. To project images to a place where their ranges are as little as possible, two branch networks are programmed to learn coupled-mappings (CMs) that are particular to a given resolution. The suggested technique is properly assessed utilizing the databases for the OMEGE and DIAEMO datasets by evaluating it to state-of-the-art techniques in terms of many parameters. Deep Coupled AlexNet is developed to obtain 98.13 % of accuracy, 95.1 % of precision, 94.3 % of recall and 95.4 of F1-score for OMEGE dataset. Moreover, 98.6 % of accuracy, 97.2 % of precision, 98.5 of recall and 97.5 % of F1-score for DIAEMO dataset (Tab. 8, Fig. 10, Ref. 16). Keywords: autism spectrum disorder, artificial neural networks, emotion recognition, interaction design, multimodal factors.

Improvement of generalization capability of 2D salt segmentation via iterative semisupervised learning

Accurate interpretations of subsurface salts are vital to oil and gas exploration. However, manually interpreting them from seismic depth images is labor-intensive. Consequently, the use of deep-learning tools, such as a convolutional neural network, for automatic salt interpretation recently became popular. Because of poor generalization capabilities, interpreting salt boundaries using these tools is difficult when labeled data are available from one geologic region, and we like to make predictions for other nearby regions with varied geologic features. At the same time, due to vast amount of the data involved and the associated computational complexities needed for training, such generalization is necessary for solving practical salt interpretation problems. We have adopted a semisupervised training method, which allows the predicted model to iteratively improve as more and more information is distilled from the unlabeled data into the model. In addition, by performing a mixup between labeled and unlabeled data during training, we encouraged the predicted models to linearly behave across training samples, thereby improving the generalization capability of the method. For each iteration, we used the model obtained from the previous iteration to generate pseudolabels for the unlabeled data. This automated consecutive data distillation allowed our model prediction to improve with iteration, without any need for human intervention. To demonstrate the effectiveness and efficiency, we applied the method on 2D images extracted from a real 3D seismic data volume. By comparing our predictions and fully supervised baseline predictions with those that were manually interpreted and that we consider as “ground truth,” we found that the prediction quality our new method surpassed the baseline prediction. Therefore, we concluded that our new method is a viable tool for automated salt delineation from seismic depth images.

FMWDCT: Foreground Mixup Into Weighted Dual-Network Cross Training for Semisupervised Remote Sensing Road Extraction

With the development of deep learning, the application of automatic road extraction has achieved great success. However, the main challenge is how to make full use of a large number of unlabeled images to improve segmentation models and how alleviate sample imbalance in road extraction tasks. In this paper, we propose a novel semi-supervised remote sensing road extraction approach is refined as Foreground Mixup into Weighted Dual-network Cross Training (FMWDCT), which combines labeled images with unlabeled images to extract road from remote sensing images. FMWDCT is composed of Dual-network Cross Training (DCT) and Foreground Pasting (FP). DCT is a new semi-supervised training method and FP is an effective data perturbation method for road extraction. We firstly paste the foreground pixels obtained from labeled images into unlabeled images to produce mixed input images. The mixed pseudo labels are then generated by a combination of high-confidence predictions from the augmented network and labeled masks. Finally, the mixed pseudo labels are used to guide another adversarial basic network for cross training, and this basic network is used to smoothly update the augmented network that corresponds to it. The proposed FMWDCT effectively solve the overfitting problem and imbalance problem of positive and negative sample in the case of a few labeled training samples. We demonstrate the effectiveness of our method on three road extraction datasets, and achieve better performance with few labeled data. Extensive experiments show that the proposed semi-supervised method can learn latent information from the unlabeled data to improve performance.

Uncertainty-Aware Semi-Supervised Method Using Large Unlabeled and Limited Labeled COVID-19 Data

The new coronavirus has caused more than one million deaths and continues to spread rapidly. This virus targets the lungs, causing respiratory distress which can be mild or severe. The X-ray or computed tomography ( CT ) images of lungs can reveal whether the patient is infected with COVID-19 or not. Many researchers are trying to improve COVID-19 detection using artificial intelligence. Our motivation is to develop an automatic method that can cope with scenarios in which preparing labeled data is time consuming or expensive. In this article, we propose a Semi-supervised Classification using Limited Labeled Data ( SCLLD ) relying on Sobel edge detection and Generative Adversarial Networks ( GANs ) to automate the COVID-19 diagnosis. The GAN discriminator output is a probabilistic value which is used for classification in this work. The proposed system is trained using 10,000 CT scans collected from Omid Hospital, whereas a public dataset is also used for validating our system. The proposed method is compared with other state-of-the-art supervised methods such as Gaussian processes. To the best of our knowledge, this is the first time a semi-supervised method for COVID-19 detection is presented. Our system is capable of learning from a mixture of limited labeled and unlabeled data where supervised learners fail due to a lack of sufficient amount of labeled data. Thus, our semi-supervised training method significantly outperforms the supervised training of Convolutional Neural Network ( CNN ) when labeled training data is scarce. The 95% confidence intervals for our method in terms of accuracy, sensitivity, and specificity are 99.56 ± 0.20%, 99.88 ± 0.24%, and 99.40 ± 0.18%, respectively, whereas intervals for the CNN (trained supervised) are 68.34 ± 4.11%, 91.2 ± 6.15%, and 46.40 ± 5.21%.

Semi-Supervised Spatiotemporal Deep Learning for Intrusions Detection in IoT Networks

The rapid growth of the Internet of Things (IoT) technologies has generated a huge amount of traffic that can be exploited for detecting intrusions through IoT networks. Despite the great effort made in annotating IoT traffic records, the number of labeled records is still very small, increasing the difficulty in recognizing attacks and intrusions. This study introduces a semi-supervised deep learning approach for intrusion detection (SS-Deep-ID), in which we propose a multiscale residual temporal convolutional (MS-Res) module to finetune the network capability in learning spatiotemporal representations. An improved traffic attention (TA) mechanism is introduced to estimate the importance score that helps the model to concentrate on important information during learning. Furthermore, a hierarchical semi-supervised training method is introduced which takes into account the sequential characteristics of the IoT traffic data during training. The proposed SS-Deep-ID is easily integrated into a fog-enabled IoT network to offer efficient real-time intrusion detection. Finally, empirical evaluations on two recent data sets (CIC-IDS2017 and CIC-IDS2018) demonstrate that SS-Deep-ID improves the efficiency of intrusion detection and increases the robustness of performance while maintaining computational efficiency.

Semi-Supervised Dim and Small Infrared Ship Detection Network Based on Haar Wavelet

Traditional deep learning detection network has poor effect on the detection of infrared dim and small targets on the sea in the case of interference or bad weather. In this paper, an improved dim and small infrared ship detection network based on Haar wavelet is proposed. The HaarConv module is designed based on the high-frequency features obtained by Haar wavelet decomposition, which further increases the feature extraction ability of the backbone network for small targets. Meanwhile, the HaarUp-HaarDown module is designed by using Haar forward and inverse transform, replacing the up-sampling layer of the feature pyramid network and the down-sampling layer of backbone network to retain smaller target features. Furthermore, the pseudo-label-based method enables the network to conduct semi-supervised learning, which reduces the labeling cost and improves detection accuracy while expanding the amount of training data. The above method is applied to the YOLOv5-s lightweight network and 11278 infrared images (3352 labeled) of dim and small ships are collected as a dataset. The results show that the introduction of semi-supervised training method effectively expands the training dataset, and the mAP@.5:.95 increases by 23.5%. The proposed Haar wavelet improvement method can effectively improve the detection accuracy of dim and small infrared ship targets by more than 2%, and the number of parameters increases by only about 0.02M. Compared with existing methods, the proposed method reaches the state-of-the-art result and has good generalization performance.

Evaluation of semi-supervised learning using sparse labeling to segment cell nuclei

Abstract The analysis of microscopic images from cell cultures plays an important role in the development of drugs. The segmentation of such images is a basic step to extract the viable information on which further evaluation steps are build. Classical image processing pipelines often fail under heterogeneous conditions. In the recent years deep neuronal networks gained attention due to their great potentials in image segmentation. One main pitfall of deep learning is often seen in the amount of labeled data required for training such models. Especially for 3D images the process to generate such data is tedious and time consuming and thus seen as a possible reason for the lack of establishment of deep learning models for 3D data. Efforts have been made to minimize the time needed to create labeled training data or to reduce the amount of labels needed for training. In this paper we present a new semisupervised training method for image segmentation of microscopic cell recordings based on an iterative approach utilizing unlabeled data during training. This method helps to further reduce the amount of labels required to effectively train deep learning models for image segmentation. By labeling less than one percent of the training data, a performance of 90% compared to a full annotation with 342 nuclei can be achieved.

A Hybrid Deep Learning System for Real-World Mobile User Authentication Using Motion Sensors.

With the popularity of smartphones and the development of hardware, mobile devices are widely used by people. To ensure availability and security, how to protect private data in mobile devices without disturbing users has become a key issue. Mobile user authentication methods based on motion sensors have been proposed by many works, but the existing methods have a series of problems such as poor de-noising ability, insufficient availability, and low coverage of feature extraction. Based on the shortcomings of existing methods, this paper proposes a hybrid deep learning system for complex real-world mobile authentication. The system includes: (1) a variational mode decomposition (VMD) based de-noising method to enhance the singular value of sensors, such as discontinuities and mutations, and increase the extraction range of the feature; (2) semi-supervised collaborative training (Tri-Training) methods to effectively deal with mislabeling problems in complex real-world situations; and (3) a combined convolutional neural network (CNN) and support vector machine (SVM) model for effective hybrid feature extraction and training. The training results under large-scale, real-world data show that the proposed system can achieve 95.01% authentication accuracy, and the effect is better than the existing frontier methods.

Semi-Supervised Speech Recognition Acoustic Model Training Using Policy Gradient

In this paper, we propose a policy gradient-based semi-supervised speech recognition acoustic model training. In practice, self-training and teacher/student learning are one of the widely used semi-supervised training methods due to their scalability and effectiveness. These methods are based on generating pseudo labels for unlabeled samples using a pre-trained model and selecting reliable samples using confidence measure. However, there are some considerations in this approach. The generated pseudo labels can be biased depending on which pre-trained model is used, and the training process can be complicated because the confidence measure is usually carried out in post-processing using external knowledge. Therefore, to address these issues, we propose a policy gradient method-based approach. Policy gradient is a reinforcement learning algorithm to find an optimal behavior strategy for an agent to obtain optimal rewards. The policy gradient-based approach provides a framework for exploring unlabeled data as well as exploiting labeled data, and it also provides a way to incorporate external knowledge in the same training cycle. The proposed approach was evaluated on an in-house non-native Korean recognition domain. The experimental results show that the method is effective in semi-supervised acoustic model training.

Deep Contextualized Self-training for Low Resource Dependency Parsing

Neural dependency parsing has proven very effective, achieving state-of-the-art results on numerous domains and languages. Unfortunately, it requires large amounts of labeled data, which is costly and laborious to create. In this paper we propose a self-training algorithm that alleviates this annotation bottleneck by training a parser on its own output. Our Deep Contextualized Self-training (DCST) algorithm utilizes representation models trained on sequence labeling tasks that are derived from the parser’s output when applied to unlabeled data, and integrates these models with the base parser through a gating mechanism. We conduct experiments across multiple languages, both in low resource in-domain and in cross-domain setups, and demonstrate that DCST substantially outperforms traditional self-training as well as recent semi-supervised training methods. 1

DIOD: Fast and Efficient Weakly Semi-Supervised Deep Complex ISAR Object Detection.

Inverse synthetic aperture radar (ISAR) object detection is one of the most important and challenging problems in computer vision tasks. To provide a convenient and high-quality ISAR object detection method, a fast and efficient weakly semi-supervised method, called deep ISAR object detection (DIOD), is proposed, based on advanced region proposal networks (ARPNs) and weakly semi-supervised deep joint sparse learning: 1) to generate high-level region proposals and localize potential ISAR objects robustly and accurately in minimal time, ARPN is proposed based on a multiscale fully convolutional region proposal network and a region proposal classification and ranking strategy. ARPN shares common convolutional layers with the Inception-ResNet-based system and offers almost cost-free proposal computation with excellent performance; 2) to solve the difficult problem of the lack of sufficient annotated training data, especially in the ISAR field, a convenient and efficient weakly semi-supervised training method is proposed with the weakly annotated and unannotated ISAR images. Particularly, a pairwise-ranking loss handles the weakly annotated images, while a triplet-ranking loss is employed to harness the unannotated images; and 3) to further improve the accuracy and speed of the whole system, a novel sharable-individual mechanism and a relational-regularized joint sparse learning strategy are introduced to achieve more discriminative and comprehensive representations while learning the shared- and individual-features and their correlations. Extensive experiments are performed on two real-world ISAR datasets, showing that DIOD outperforms existing state-of-the-art methods and achieves higher accuracy with shorter execution time.

DeepHeart: Semi-Supervised Sequence Learning for Cardiovascular Risk Prediction

We train and validate a semi-supervised, multi-task LSTM on 57,675 person-weeks of data from off-the-shelf wearable heart rate sensors, showing high accuracy at detecting multiple medical conditions, including diabetes (0.8451), high cholesterol (0.7441), high blood pressure (0.8086), and sleep apnea (0.8298). We compare two semi-supervised training methods, semi-supervised sequence learning and heuristic pretraining, and show they outperform hand-engineered biomarkers from the medical literature. We believe our work suggests a new approach to patient risk stratification based on cardiovascular risk scores derived from popular wearables such as Fitbit, Apple Watch, or Android Wear.

Bird Song Scene Analysis Using a Spatial-Cue-Based Probabilistic Model

[abstFig src='/00290001/22.jpg' width='300' text='Spatial-cue-based probabilistic model' ] This paper addresses bird song scene analysis based on semi-automatic annotation. Research in animal behavior, especially in birds, would be aided by automated or semi-automated systems that can localize sounds, measure their timing, and identify their sources. This is difficult to achieve in real environments, in which several birds at different locations may be singing at the same time. Analysis of recordings from the wild has usually required manual annotation. These annotations may be inaccurate or inconsistent, as they may vary within and between observers. Here we suggest a system that uses automated methods from robot audition, including sound source detection, localization, separation and identification. In robot audition, these technologies are assessed separately, but combining them has often led to poor performance in natural setting. We propose a new Spatial-Cue-Based Probabilistic Model (SCBPM) for their integration focusing on spatial information. A second problem has been that supervised machine learning methods usually require a pre-trained model, which may need a large training set of annotated labels. We have employed a semi-automatic annotation approach, in which a semi-supervised training method is deduced for a new model. This method requires much less pre-annotation. Preliminary experiments with recordings of bird songs from the wild revealed that our system outperformed the identification accuracy of a method based on conventional robot audition.**This paper is an extension of a proceeding of IROS2015.

Lithuanian Broadcast Speech Transcription Using Semi-supervised Acoustic Model Training

This paper reports on an experimental work to build a speech transcription system for Lithuanian broadcast data, relying on unsupervised and semi-supervised training methods as well as on other low-knowledge methods to compensate for missing resources. Unsupervised acoustic model training is investigated using 360hours of untranscribed speech data. A graphemic pronunciation approach is used to simplify the pronunciation model generation and there-fore ease the language model adaptation for the system users. Discriminative training on top of semi-supervised training is also investigated, as well as various types of acoustic features and their combinations. Experimental results are provided for each of our development steps as well as contrastive results comparing various options. Using the best system configuration a word error rate of 18.3% is obtained on a set of development data from the Quaero program.