Semi-supervised Object Detection Research Articles

Consistency-based semi-supervised learning for oriented object detection

Semi-Supervised Object Detection (SSOD) has emerged as a potent framework that leverages unlabeled data to reduce annotation costs while enhancing model performance. Nevertheless, existing SSOD methods primarily concentrate on the conventional localization of horizontal objects, overlooking the common arbitrary-oriented objects in aerial scenes. Extending semi-supervised learning to oriented object detection still encounters two major challenges: (1) There is an inconsistency between classification and localization in oriented object detectors, which means that pseudo-labels selected solely based on classification scores cannot properly represent the localization quality. (2) The static and monotonic pseudo-label selection process fails to dynamically filter pseudo-labels for different tasks and categories throughout the iteration process, inevitably introducing accumulated noise and bias into the supervisory signals for the student model. To mitigate this problems, we proposed a novel consistency-based semi-supervised oriented object detection framework, named CSLO-Det. Specifically, Task Alignment Learning (TAL) is proposed to improve the consistency in terms of both features and learning objectives, thereby facilitating a more robust selection of pseudo-labels. Noise-Resistant Pseudo-label Mining (NR-PM) is introduced to separately and dynamically exploits positives for the classification and localization task. Moreover, diverging from the conventional semi-supervised learning frameworks that utilize pseudo-boxes, our method applies pixel-level dense supervision during unsupervised training, which improves detection performance. Comprehensive experiment results reveal that the proposed CSLO-Det attains state-of-the-art performance under multiple semi-supervised datasets.

Adaptive Adversarial Self-Training for Semi-Supervised Object Detection in Complex Maritime Scenes

Adaptive Adversarial Self-Training for Semi-Supervised Object Detection in Complex Maritime Scenes

Semi-Supervised One-Stage Object Detection for Maize Leaf Disease

Maize is one of the most important crops globally, and accurate diagnosis of leaf diseases is crucial for ensuring increased yields. Despite the continuous progress in computer vision technology, detecting maize leaf diseases based on deep learning still relies on a large amount of manually labeled data, and the labeling process is time-consuming and labor-intensive. Moreover, the detectors currently used for identifying maize leaf diseases have relatively low accuracy in complex experimental fields. Therefore, the proposed Agronomic Teacher, an object detection algorithm that utilizes limited labeled and abundant unlabeled data, is applied to maize leaf disease recognition. In this work, a semi-supervised object detection framework is built based on a single-stage detector, integrating the Weighted Average Pseudo-labeling Assignment (WAP) strategy and AgroYOLO detector combining Agro-Backbone network with Agro-Neck network. The WAP strategy uses weight adjustments to set objectness and classification scores as evaluation criteria for pseudo-labels reliability assignment. Agro-Backbone network accurately extracts features of maize leaf diseases and obtains richer semantic information. Agro-Neck network enhances feature fusion by utilizing multi-layer features for collaborative combinations. The effectiveness of the proposed method is validated on the MaizeData and PascalVOC datasets at different annotation ratios. Compared to the baseline model, Agronomic Teacher leverages abundant unlabeled data to achieve a 6.5% increase in mAP (0.5) on the 30% labeled MaizeData. On the 30% labeled PascalVOC dataset, the mAP (0.5) improved by 8.2%, demonstrating the method’s potential for generalization.

IML-SSOD: Interconnected and multi-layer threshold learning for semi-supervised detection

While semi-supervised anchored detector of the R-CNN series has achieved remarkable success, semi-supervised anchor-free detector lacks the ability to generate high-quality flexible pseudo labels, resulting in serious inconsistencies in SSOD. In order to make the network learn more reliable and consistent label data to solve the problem of information bias, we propose an interconnected and multi-layer threshold learning for semi-supervised object detection (IML-SSOD). The Joint Guided Estimation (JGE) module uses the Core Zone refinement module to improve the position accuracy score of low semantic information, and combines the classification and the centerness score as evaluation criteria to predict stable labels. The multi-layer threshold filtering method selects more potential label samples for the student network ensuring the information used in training. Extensive experiments on MS COCO and PASCAL VOC datasets demonstrated the effectiveness of IML-SSOD. Compared with existing methods, our method on VOC achieved 81.9% AP50 and 57.89% AP50:95, which is highly competitive.

A Semi-Supervised Object Detection Method for Close Range Detection of Spacecraft and Space Debris

A Semi-Supervised Object Detection Method for Close Range Detection of Spacecraft and Space Debris

Point based weakly semi-supervised biomarker detection with cross-scale and label assignment in retinal OCT images

Background and ObjectiveOptical coherence tomography (OCT) is currently one of the most advanced retinal imaging methods. Retinal biomarkers in OCT images are of clinical significance and can assist ophthalmologists in diagnosing lesions. Compared with fundus images, OCT can provide higher resolution segmentation. However, image annotation at the bounding box level needs to be performed by ophthalmologists carefully and is difficult to obtain. In addition, the large variation in shape of different retinal markers and the inconspicuous appearance of biomarkers make it difficult for existing deep learning-based methods to effectively detect them. To overcome the above challenges, we propose a novel network for the detection of retinal biomarkers in OCT images. MethodsWe first address the issue of labeling cost using a novel weakly semi-supervised object detection method with point annotations which can reduce bounding box-level annotation efforts. To extend the method to the detection of biomarkers in OCT images, we propose multiple consistent regularizations for point-to-box regression network to deal with the shortage of supervision, which aims to learn more accurate regression mappings. Furthermore, in the subsequent fully supervised detection, we propose a cross-scale feature enhancement module to alleviate the detection problems caused by the large-scale variation of biomarkers. We also propose a dynamic label assignment strategy to distinguish samples of different importance more flexibly, thereby reducing detection errors due to the indistinguishable appearance of the biomarkers. ResultsWhen using our detection network, our regressor also achieves an AP value of 20.83 s when utilizing a 5 % fully labeled dataset partition, surpassing the performance of other comparative methods at 5 % and 10 %. Even coming close to the 20.87 % result achieved by Point DETR under 20 % full labeling conditions. When using Group R-CNN as the point-to-box regressor, our detector achieves 27.21 % AP in the 50 % fully labeled dataset experiment. 7.42 % AP improvement is achieved compared to our detection network baseline Faster R-CNN. ConclusionsThe experimental findings not only demonstrate the effectiveness of our approach with minimal bounding box annotations but also highlight the enhanced biomarker detection performance of the proposed module. We have included a detailed algorithmic flow in the supplementary material.

Semi-Supervised Object Detection with Multi-Scale Regularization and Bounding Box Re-Prediction

Semi-supervised object detection has become a hot topic in recent years, but there are still some challenges regarding false detection, duplicate detection, and inaccurate localization. This paper presents a semi-supervised object detection method with multi-scale regularization and bounding box re-prediction. Specifically, to improve the generalization of the two-stage object detector and to make consistent predictions related to the image and its down-sampled counterpart, a novel multi-scale regularization loss is proposed for the region proposal network and the region-of-interest head. Then, in addition to using the classification probabilities of the pseudo-labels to exploit the unlabeled data, this paper proposes a novel bounding box re-prediction strategy to re-predict the bounding boxes of the pseudo-labels in the unlabeled images and select the pseudo-labels with reliable bounding boxes (location coordinates) to improve the model’s localization accuracy based on its unsupervised localization loss. Experiments on the public MS COCO and Pascal VOC show that our proposed method achieves a competitive detection performance compared to other state-of-the-art methods. Furthermore, our method offers a multi-scale regularization strategy and a reliably located pseudo-label screening strategy, both of which facilitate the development of semi-supervised object detection techniques and boost the object detection performance in autonomous driving, industrial inspection, and agriculture automation.

Elaborate Teacher: Improved Semi-Supervised Object Detection with Rich Image Exploiting

Elaborate Teacher: Improved Semi-Supervised Object Detection with Rich Image Exploiting

A Novel Semi-Supervised Object Detection Approach via Scale ReBalancing and Global Proposal Contrast Consistency

A Novel Semi-Supervised Object Detection Approach via Scale ReBalancing and Global Proposal Contrast Consistency

USD: Uncertainty-Based One-Phase Learning to Enhance Pseudo-Label Reliability for Semi-Supervised Object Detection

USD: Uncertainty-Based One-Phase Learning to Enhance Pseudo-Label Reliability for Semi-Supervised Object Detection

Boosting Semi-Supervised Object Detection in Remote Sensing Images With Active Teaching

Boosting Semi-Supervised Object Detection in Remote Sensing Images With Active Teaching

Uncertain region mining semi-supervised object detection

Uncertain region mining semi-supervised object detection

Semi-supervised object detection based on single-stage detector for thighbone fracture localization

Semi-supervised object detection based on single-stage detector for thighbone fracture localization

A two-stage semi-supervised object detection method for SAR images with missing labels based on meta pseudo-labels

Recently, deep learning has been applied to analyze satellite images. However, applying fully supervised object detection (FSOD) is impractical because it is challenging to detect and annotate objects that are relatively small in high-resolution satellite images. In addition, most data owned by public institutions, such as military reconnaissance videos, do not contain sufficient object class label information. Therefore, the application of semi-supervised objection detection (SSOD) is more practical. The SSOD performance is often determined by generated pseudo-labels. Therefore, this study proposes a 2-stage SSOD model to generate accurate pseudo-labels. In the first stage, annotations, including the location information of unlabeled data, are detected using a modified faster R-CNN model. In the second stage, pseudo-labels of objects are additionally suggested through the meta pseudo-label method. Finally, reliable pseudo-labels are generated by comparing and combining the pseudo-labels generated in each stage, improving the SSOD model performance. Extensive experiments were conducted using the xView3 dataset provided by the U.S. Defense Innovation Unit (DIU). The proposed method performed approximately 11% better than the FSOD model, which does not learn pseudo-labels for datasets with missing labels according to the evaluation metrics proposed by the DIU covering both object detection and classification performance.

Semi-Supervised Building Detection from High-Resolution Remote Sensing Imagery

Urban building information reflects the status and trends of a region’s development and is essential for urban sustainability. Detection of buildings from high-resolution (HR) remote sensing images (RSIs) provides a practical approach for quickly acquiring building information. Mainstream building detection methods are based on fully supervised deep learning networks, which require a large number of labeled RSIs. In practice, manually labeling building instances in RSIs is labor-intensive and time-consuming. This study introduces semi-supervised deep learning techniques for building detection and proposes a semi-supervised building detection framework to alleviate this problem. Specifically, the framework is based on teacher–student mutual learning and consists of two key modules: the color and Gaussian augmentation (CGA) module and the consistency learning (CL) module. The CGA module is designed to enrich the diversity of building features and the quantity of labeled images for better training of an object detector. The CL module derives a novel consistency loss by imposing consistency of predictions from augmented unlabeled images to enhance the detection ability on the unlabeled RSIs. The experimental results on three challenging datasets show that the proposed framework outperforms state-of-the-art building detection methods and semi-supervised object detection methods. This study develops a new approach for optimizing the building detection task and a methodological reference for the various object detection tasks on RSIs.

Salvage of Supervision in Weakly Supervised Object Detection and Segmentation

Weakly supervised vision tasks, including detection and segmentation, have attracted much attention in the vision community recently. However, the lack of detailed and precise annotations in the weakly supervised case leads to a large accuracy gap between weakly- and fully-supervised methods. In this paper, we propose a new framework, Salvage of Supervision(SoS), with the key idea being to effectively harness every potentially useful supervisory signal in weakly supervised vision tasks. Starting with weakly supervised object detection(WSOD), we propose SoS-WSOD to shrink the technology gap between WSOD and FSOD, which utilizes the weak image-level labels, the pseudo-labels, and the power of semi-supervised object detection for WSOD. Moreover, SoS-WSOD removes restrictions in traditional WSOD methods, including the reliance on ImageNet pretraining and inability to use modern backbones. The SoS framework also extends to weakly supervised semantic segmentation and instance segmentation. On several weakly supervised vision benchmarks, SoS achieves significant performance boost and generalization ability.

Robust Teacher: Self-correcting pseudo-label-guided semi-supervised learning for object detection

Semi-supervised object detection methods have become increasingly popular in computer vision owing to their success in reducing data labeling costs. However, low-quality pseudo-labels exhibit difficulties in correction and severely limit the performance of existing state-of-the-art models. To address these problems, based on the contemporary teacher–student dual framework, we develop a novel self-correcting pseudo-label module to generate more reliable predictions for unlabeled data. Simultaneously, to mitigate the inherent class bias in pseudo-labels, we integrate multi-label classification results for measuring the re-balanced focal loss to enable class-agnostic, robust semi-supervised learning. Furthermore, pseudo-label-guided copy-paste is proposed as a simple yet efficient data augmentation technique to enhance instance representation learning within diverse complex scenes. The above key designs constitute our proposed approach, which we call Robust Teacher. Extensive experiments on PASCAL VOC and MS COCO demonstrate that Robust Teacher achieves competitive results and outperforms state-of-the-art models by a large margin. Our comprehensive ablation studies further verify the effectiveness of the key components. The code will be publicly available at https://github.com/Complicateddd/RobustT.

Penalizing proposals using classifiers for semi-supervised object detection

Obtaining gold standard annotated data for object detection is often costly, involving human-level effort. Semi-supervised object detection algorithms solve the problem with a small amount of gold-standard labels and a large unlabeled dataset used to generate silver-standard labels. But training on the silver standard labels does not produce good results, because they are machine-generated annotations. In this work, we design a modified loss function to train on large silver standard annotated sets generated by a weak annotator. We include a confidence metric associated with the annotation as an additional term in the loss function, signifying the quality of the annotation. We test the effectiveness of our approach on various test sets and use numerous variations to compare the results with some of the current approaches to object detection. In comparison with the baseline where no confidence metric is used, we achieved a 4% gain in mAP with 25% labeled data and 10% gain in mAP with 50% labeled data by using the proposed confidence metric.

Mind the Gap: Polishing Pseudo Labels for Accurate Semi-supervised Object Detection

Exploiting pseudo labels (e.g., categories and bounding boxes) of unannotated objects produced by a teacher detector have underpinned much of recent progress in semi-supervised object detection (SSOD). However, due to the limited generalization capacity of the teacher detector caused by the scarce annotations, the produced pseudo labels often deviate from ground truth, especially those with relatively low classification confidences, thus limiting the generalization performance of SSOD. To mitigate this problem, we propose a dual pseudo-label polishing framework for SSOD. Instead of directly exploiting the pseudo labels produced by the teacher detector, we take the first attempt at reducing their deviation from ground truth using dual polishing learning, where two differently structured polishing networks are elaborately developed and trained using synthesized paired pseudo labels and the corresponding ground truth for categories and bounding boxes on the given annotated objects, respectively. By doing this, both polishing networks can infer more accurate pseudo labels for unannotated objects through sufficiently exploiting their context knowledge based on the initially produced pseudo labels, and thus improve the generalization performance of SSOD. Moreover, such a scheme can be seamlessly plugged into the existing SSOD framework for joint end-to-end learning. In addition, we propose to disentangle the polished pseudo categories and bounding boxes of unannotated objects for separate category classification and bounding box regression in SSOD, which enables introducing more unannotated objects during model training and thus further improves the performance. Experiments on both PASCAL VOC and MS-COCO benchmarks demonstrate the superiority of the proposed method over existing state-of-the-art baselines. The code can be found at https://github.com/snowdusky/DualPolishLearning.

Adapting Object Size Variance and Class Imbalance for Semi-supervised Object Detection

Semi-supervised object detection (SSOD) attracts extensive research interest due to its great significance in reducing the data annotation effort. Collecting high-quality and category-balanced pseudo labels for unlabeled images is critical to addressing the SSOD problem. However, most of the existing pseudo-labeling-based methods depend on a large and fixed threshold to select high-quality pseudo labels from the predictions of a teacher model. Considering different object classes usually have different detection difficulty levels due to scale variance and data distribution imbalance, conventional pseudo-labeling-based methods are arduous to explore the value of unlabeled data sufficiently. To address these issues, we propose an adaptive pseudo labeling strategy, which can assign thresholds to classes with respect to their “hardness”. This is beneficial for ensuring the high quality of easier classes and increasing the quantity of harder classes simultaneously. Besides, label refinement modules are set up based on box jittering for guaranteeing the localization quality of pseudo labels. To further improve the algorithm’s robustness against scale variance and make the most of pseudo labels, we devise a joint feature-level and prediction-level consistency learning pipeline for transferring the information of the teacher model to the student model. Extensive experiments on COCO and VOC datasets indicate that our method achieves state-of-the-art performance. Especially, it brings mean average precision gains of 2.08 and 1.28 on MS-COCO dataset with 5% and 10% labeled images, respectively.