Semi-supervised Object Research Articles

Consistency-based semi-supervised learning for oriented object detection

Semi-Supervised Object Detection (SSOD) has emerged as a potent framework that leverages unlabeled data to reduce annotation costs while enhancing model performance. Nevertheless, existing SSOD methods primarily concentrate on the conventional localization of horizontal objects, overlooking the common arbitrary-oriented objects in aerial scenes. Extending semi-supervised learning to oriented object detection still encounters two major challenges: (1) There is an inconsistency between classification and localization in oriented object detectors, which means that pseudo-labels selected solely based on classification scores cannot properly represent the localization quality. (2) The static and monotonic pseudo-label selection process fails to dynamically filter pseudo-labels for different tasks and categories throughout the iteration process, inevitably introducing accumulated noise and bias into the supervisory signals for the student model. To mitigate this problems, we proposed a novel consistency-based semi-supervised oriented object detection framework, named CSLO-Det. Specifically, Task Alignment Learning (TAL) is proposed to improve the consistency in terms of both features and learning objectives, thereby facilitating a more robust selection of pseudo-labels. Noise-Resistant Pseudo-label Mining (NR-PM) is introduced to separately and dynamically exploits positives for the classification and localization task. Moreover, diverging from the conventional semi-supervised learning frameworks that utilize pseudo-boxes, our method applies pixel-level dense supervision during unsupervised training, which improves detection performance. Comprehensive experiment results reveal that the proposed CSLO-Det attains state-of-the-art performance under multiple semi-supervised datasets.

Unsupervised Object Cosegmentation Method Devoted to Image Classification

Rich heterogeneous data provided by social networks can be very big, which imposes considerable challenges for object extraction and image classification. Therefore, the objective of this work is to propose an unsupervised object cosegmentation method that could be notably efficient to improve image classification performance. The main goal of cosegmentation is to extract the salient common objects within each image. To this end, we propose to minimize an energy function based on the Markov Random Field using the saliency detection, while considering linear dependence of generated foreground histograms of the input image collection. In fact, the saliency detection is processed in two steps. In each image, we detect salient objects, by considering appearance similarity and spatial distributions of image pixels. Then, fuzzy quantification is used to correct the belonging of pixels to the foreground objects. Finally, an iterative optimization permits to enhance the final segmentation results. The proposed method has been validated as a preprocessing step for image classification. Indeed, to enhance cosegmentation-based classification performance, we have applied a semi-supervised object classification based on ensemble projection. Qualitative and quantitative evaluations of the proposed cosegmentation and classification techniques on the iCoseg, CDS and Oxford Flowers 17 datasets demonstrate the effectiveness of the proposed framework.

Image-level supervision and self-training for transformer-based cross-modality tumor segmentation

Deep neural networks are commonly used for automated medical image segmentation, but models will frequently struggle to generalize well across different imaging modalities. This issue is particularly problematic due to the limited availability of annotated data, both in the target as well as the source modality, making it difficult to deploy these models on a larger scale. To overcome these challenges, we propose a new semi-supervised training strategy called MoDATTS. Our approach is designed for accurate cross-modality 3D tumor segmentation on unpaired bi-modal datasets. An image-to-image translation strategy between modalities is used to produce synthetic but annotated images and labels in the desired modality and improve generalization to the unannotated target modality. We also use powerful vision transformer architectures for both image translation (TransUNet) and segmentation (Medformer) tasks and introduce an iterative self-training procedure in the later task to further close the domain gap between modalities, thus also training on unlabeled images in the target modality. MoDATTS additionally allows the possibility to exploit image-level labels with a semi-supervised objective that encourages the model to disentangle tumors from the background. This semi-supervised methodology helps in particular to maintain downstream segmentation performance when pixel-level label scarcity is also present in the source modality dataset, or when the source dataset contains healthy controls. The proposed model achieves superior performance compared to other methods from participating teams in the CrossMoDA 2022 vestibular schwannoma (VS) segmentation challenge, as evidenced by its reported top Dice score of 0.87±0.04 for the VS segmentation. MoDATTS also yields consistent improvements in Dice scores over baselines on a cross-modality adult brain gliomas segmentation task composed of four different contrasts from the BraTS 2020 challenge dataset, where 95% of a target supervised model performance is reached when no target modality annotations are available. We report that 99% and 100% of this maximum performance can be attained if 20% and 50% of the target data is additionally annotated, which further demonstrates that MoDATTS can be leveraged to reduce the annotation burden.

Adaptive Adversarial Self-Training for Semi-Supervised Object Detection in Complex Maritime Scenes

Adaptive Adversarial Self-Training for Semi-Supervised Object Detection in Complex Maritime Scenes

Semi-Supervised One-Stage Object Detection for Maize Leaf Disease

Maize is one of the most important crops globally, and accurate diagnosis of leaf diseases is crucial for ensuring increased yields. Despite the continuous progress in computer vision technology, detecting maize leaf diseases based on deep learning still relies on a large amount of manually labeled data, and the labeling process is time-consuming and labor-intensive. Moreover, the detectors currently used for identifying maize leaf diseases have relatively low accuracy in complex experimental fields. Therefore, the proposed Agronomic Teacher, an object detection algorithm that utilizes limited labeled and abundant unlabeled data, is applied to maize leaf disease recognition. In this work, a semi-supervised object detection framework is built based on a single-stage detector, integrating the Weighted Average Pseudo-labeling Assignment (WAP) strategy and AgroYOLO detector combining Agro-Backbone network with Agro-Neck network. The WAP strategy uses weight adjustments to set objectness and classification scores as evaluation criteria for pseudo-labels reliability assignment. Agro-Backbone network accurately extracts features of maize leaf diseases and obtains richer semantic information. Agro-Neck network enhances feature fusion by utilizing multi-layer features for collaborative combinations. The effectiveness of the proposed method is validated on the MaizeData and PascalVOC datasets at different annotation ratios. Compared to the baseline model, Agronomic Teacher leverages abundant unlabeled data to achieve a 6.5% increase in mAP (0.5) on the 30% labeled MaizeData. On the 30% labeled PascalVOC dataset, the mAP (0.5) improved by 8.2%, demonstrating the method’s potential for generalization.

IML-SSOD: Interconnected and multi-layer threshold learning for semi-supervised detection

While semi-supervised anchored detector of the R-CNN series has achieved remarkable success, semi-supervised anchor-free detector lacks the ability to generate high-quality flexible pseudo labels, resulting in serious inconsistencies in SSOD. In order to make the network learn more reliable and consistent label data to solve the problem of information bias, we propose an interconnected and multi-layer threshold learning for semi-supervised object detection (IML-SSOD). The Joint Guided Estimation (JGE) module uses the Core Zone refinement module to improve the position accuracy score of low semantic information, and combines the classification and the centerness score as evaluation criteria to predict stable labels. The multi-layer threshold filtering method selects more potential label samples for the student network ensuring the information used in training. Extensive experiments on MS COCO and PASCAL VOC datasets demonstrated the effectiveness of IML-SSOD. Compared with existing methods, our method on VOC achieved 81.9% AP50 and 57.89% AP50:95, which is highly competitive.

A Semi-Supervised Object Detection Method for Close Range Detection of Spacecraft and Space Debris

A Semi-Supervised Object Detection Method for Close Range Detection of Spacecraft and Space Debris

Point based weakly semi-supervised biomarker detection with cross-scale and label assignment in retinal OCT images

Background and ObjectiveOptical coherence tomography (OCT) is currently one of the most advanced retinal imaging methods. Retinal biomarkers in OCT images are of clinical significance and can assist ophthalmologists in diagnosing lesions. Compared with fundus images, OCT can provide higher resolution segmentation. However, image annotation at the bounding box level needs to be performed by ophthalmologists carefully and is difficult to obtain. In addition, the large variation in shape of different retinal markers and the inconspicuous appearance of biomarkers make it difficult for existing deep learning-based methods to effectively detect them. To overcome the above challenges, we propose a novel network for the detection of retinal biomarkers in OCT images. MethodsWe first address the issue of labeling cost using a novel weakly semi-supervised object detection method with point annotations which can reduce bounding box-level annotation efforts. To extend the method to the detection of biomarkers in OCT images, we propose multiple consistent regularizations for point-to-box regression network to deal with the shortage of supervision, which aims to learn more accurate regression mappings. Furthermore, in the subsequent fully supervised detection, we propose a cross-scale feature enhancement module to alleviate the detection problems caused by the large-scale variation of biomarkers. We also propose a dynamic label assignment strategy to distinguish samples of different importance more flexibly, thereby reducing detection errors due to the indistinguishable appearance of the biomarkers. ResultsWhen using our detection network, our regressor also achieves an AP value of 20.83 s when utilizing a 5 % fully labeled dataset partition, surpassing the performance of other comparative methods at 5 % and 10 %. Even coming close to the 20.87 % result achieved by Point DETR under 20 % full labeling conditions. When using Group R-CNN as the point-to-box regressor, our detector achieves 27.21 % AP in the 50 % fully labeled dataset experiment. 7.42 % AP improvement is achieved compared to our detection network baseline Faster R-CNN. ConclusionsThe experimental findings not only demonstrate the effectiveness of our approach with minimal bounding box annotations but also highlight the enhanced biomarker detection performance of the proposed module. We have included a detailed algorithmic flow in the supplementary material.

Semi-Supervised Object Detection with Multi-Scale Regularization and Bounding Box Re-Prediction

Semi-supervised object detection has become a hot topic in recent years, but there are still some challenges regarding false detection, duplicate detection, and inaccurate localization. This paper presents a semi-supervised object detection method with multi-scale regularization and bounding box re-prediction. Specifically, to improve the generalization of the two-stage object detector and to make consistent predictions related to the image and its down-sampled counterpart, a novel multi-scale regularization loss is proposed for the region proposal network and the region-of-interest head. Then, in addition to using the classification probabilities of the pseudo-labels to exploit the unlabeled data, this paper proposes a novel bounding box re-prediction strategy to re-predict the bounding boxes of the pseudo-labels in the unlabeled images and select the pseudo-labels with reliable bounding boxes (location coordinates) to improve the model’s localization accuracy based on its unsupervised localization loss. Experiments on the public MS COCO and Pascal VOC show that our proposed method achieves a competitive detection performance compared to other state-of-the-art methods. Furthermore, our method offers a multi-scale regularization strategy and a reliably located pseudo-label screening strategy, both of which facilitate the development of semi-supervised object detection techniques and boost the object detection performance in autonomous driving, industrial inspection, and agriculture automation.

Elaborate Teacher: Improved Semi-Supervised Object Detection with Rich Image Exploiting

Elaborate Teacher: Improved Semi-Supervised Object Detection with Rich Image Exploiting

A Novel Semi-Supervised Object Detection Approach via Scale ReBalancing and Global Proposal Contrast Consistency

A Novel Semi-Supervised Object Detection Approach via Scale ReBalancing and Global Proposal Contrast Consistency

USD: Uncertainty-Based One-Phase Learning to Enhance Pseudo-Label Reliability for Semi-Supervised Object Detection

USD: Uncertainty-Based One-Phase Learning to Enhance Pseudo-Label Reliability for Semi-Supervised Object Detection

Boosting Semi-Supervised Object Detection in Remote Sensing Images With Active Teaching

Boosting Semi-Supervised Object Detection in Remote Sensing Images With Active Teaching

Uncertain region mining semi-supervised object detection

Uncertain region mining semi-supervised object detection

Semi-supervised structural damage assessment via autoregressive models and evolutionary optimization

Strategies based on Structural Health Monitoring (SHM) allow identifying the occurrence of damage in sensitive structures, such as bridges, tall buildings, dams, stadiums, among others. However, few strategies allow determining the exact damage locations and intensities through the structure’s dynamic responses, i.e., accelerations, displacements, strains, etc. Many structures around the world are reaching their service lives and yet are still being used, as their simple replacement is economically impractical. Therefore, having an advanced SHM technique may present itself as a favorable solution for evaluating the structure’s condition under changes in their vibration properties. In this paper, the four main autoregressive models (AR, ARMA, and their exogenous input counterparts ARX and ARMAX) are combined with Differential Evolution (DE) algorithm and employed to deal with multiple time series patterns of vibration signals. The procedure of model order optimization is performed both with a proposed semi-supervised objective function and compared with the well-known Akaike criterion (AIC). The semi-supervised design consists in exploring the behavior of progressive deterioration to localize the focus of it. Damage levels are calculated for each sensor based on the Mahalanobis distance (MD) of the resultant autoregressive coefficients. This enhanced framework is applied to a three-story frame and a real-world bridge (Z24 bridge). The suggested strategy reaches similar or better damage localization in all cases, while significantly outperforming the compared literature in terms of computational cost and time, making it more promising for real-time SHM.

Semi-supervised object detection based on single-stage detector for thighbone fracture localization

Semi-supervised object detection based on single-stage detector for thighbone fracture localization

A two-stage semi-supervised object detection method for SAR images with missing labels based on meta pseudo-labels

Recently, deep learning has been applied to analyze satellite images. However, applying fully supervised object detection (FSOD) is impractical because it is challenging to detect and annotate objects that are relatively small in high-resolution satellite images. In addition, most data owned by public institutions, such as military reconnaissance videos, do not contain sufficient object class label information. Therefore, the application of semi-supervised objection detection (SSOD) is more practical. The SSOD performance is often determined by generated pseudo-labels. Therefore, this study proposes a 2-stage SSOD model to generate accurate pseudo-labels. In the first stage, annotations, including the location information of unlabeled data, are detected using a modified faster R-CNN model. In the second stage, pseudo-labels of objects are additionally suggested through the meta pseudo-label method. Finally, reliable pseudo-labels are generated by comparing and combining the pseudo-labels generated in each stage, improving the SSOD model performance. Extensive experiments were conducted using the xView3 dataset provided by the U.S. Defense Innovation Unit (DIU). The proposed method performed approximately 11% better than the FSOD model, which does not learn pseudo-labels for datasets with missing labels according to the evaluation metrics proposed by the DIU covering both object detection and classification performance.

Semi-Supervised Building Detection from High-Resolution Remote Sensing Imagery

Urban building information reflects the status and trends of a region’s development and is essential for urban sustainability. Detection of buildings from high-resolution (HR) remote sensing images (RSIs) provides a practical approach for quickly acquiring building information. Mainstream building detection methods are based on fully supervised deep learning networks, which require a large number of labeled RSIs. In practice, manually labeling building instances in RSIs is labor-intensive and time-consuming. This study introduces semi-supervised deep learning techniques for building detection and proposes a semi-supervised building detection framework to alleviate this problem. Specifically, the framework is based on teacher–student mutual learning and consists of two key modules: the color and Gaussian augmentation (CGA) module and the consistency learning (CL) module. The CGA module is designed to enrich the diversity of building features and the quantity of labeled images for better training of an object detector. The CL module derives a novel consistency loss by imposing consistency of predictions from augmented unlabeled images to enhance the detection ability on the unlabeled RSIs. The experimental results on three challenging datasets show that the proposed framework outperforms state-of-the-art building detection methods and semi-supervised object detection methods. This study develops a new approach for optimizing the building detection task and a methodological reference for the various object detection tasks on RSIs.

Salvage of Supervision in Weakly Supervised Object Detection and Segmentation

Weakly supervised vision tasks, including detection and segmentation, have attracted much attention in the vision community recently. However, the lack of detailed and precise annotations in the weakly supervised case leads to a large accuracy gap between weakly- and fully-supervised methods. In this paper, we propose a new framework, Salvage of Supervision(SoS), with the key idea being to effectively harness every potentially useful supervisory signal in weakly supervised vision tasks. Starting with weakly supervised object detection(WSOD), we propose SoS-WSOD to shrink the technology gap between WSOD and FSOD, which utilizes the weak image-level labels, the pseudo-labels, and the power of semi-supervised object detection for WSOD. Moreover, SoS-WSOD removes restrictions in traditional WSOD methods, including the reliance on ImageNet pretraining and inability to use modern backbones. The SoS framework also extends to weakly supervised semantic segmentation and instance segmentation. On several weakly supervised vision benchmarks, SoS achieves significant performance boost and generalization ability.

Robust Teacher: Self-correcting pseudo-label-guided semi-supervised learning for object detection

Semi-supervised object detection methods have become increasingly popular in computer vision owing to their success in reducing data labeling costs. However, low-quality pseudo-labels exhibit difficulties in correction and severely limit the performance of existing state-of-the-art models. To address these problems, based on the contemporary teacher–student dual framework, we develop a novel self-correcting pseudo-label module to generate more reliable predictions for unlabeled data. Simultaneously, to mitigate the inherent class bias in pseudo-labels, we integrate multi-label classification results for measuring the re-balanced focal loss to enable class-agnostic, robust semi-supervised learning. Furthermore, pseudo-label-guided copy-paste is proposed as a simple yet efficient data augmentation technique to enhance instance representation learning within diverse complex scenes. The above key designs constitute our proposed approach, which we call Robust Teacher. Extensive experiments on PASCAL VOC and MS COCO demonstrate that Robust Teacher achieves competitive results and outperforms state-of-the-art models by a large margin. Our comprehensive ablation studies further verify the effectiveness of the key components. The code will be publicly available at https://github.com/Complicateddd/RobustT.