Pseudo Annotations Research Articles

Learning Transferable Conceptual Prototypes for Interpretable Unsupervised Domain Adaptation.

Despite the great progress of unsupervised domain adaptation (UDA) with the deep neural networks, current UDA models are opaque and cannot provide promising explanations, limiting their applications in the scenarios that require safe and controllable model decisions. At present, a surge of work focuses on designing deep interpretable methods with adequate data annotations and only a few methods consider the distributional shift problem. Most existing interpretable UDA methods are post-hoc ones, which cannot facilitate the model learning process for performance enhancement. In this paper, we propose an inherently interpretable method, named Transferable Conceptual Prototype Learning (TCPL), which could simultaneously interpret and improve the processes of knowledge transfer and decision-making in UDA. To achieve this goal, we design a hierarchically prototypical module that transfers categorical basic concepts from the source domain to the target domain and learns domain-shared prototypes for explaining the underlying reasoning process. With the learned transferable prototypes, a self-predictive consistent pseudo-label strategy that fuses confidence, predictions, and prototype information, is designed for selecting suitable target samples for pseudo annotations and gradually narrowing down the domain gap. Comprehensive experiments show that the proposed method can not only provide effective and intuitive explanations but also outperform previous state-of-the-arts. Code is available at https://drive.google.com/file/d/1b1EHFghiF1ExD-Cn1HYg75VutfkXWp60/view?usp=sharing.

Branches Mutual Promotion for End-to-End Weakly Supervised Semantic Segmentation.

End-to-end weakly supervised semantic segmentation (E2E-WSSS) aims at optimizing a segmentation model in a single-stage training process based on only image annotations. Existing methods adopt an online-trained classification branch to provide pseudo annotations for supervising the segmentation branch. However, this strategy makes the classification branch dominate the whole concurrent training process, hindering these two branches from assisting each other. In our work, we treat these two branches equally by viewing them as diverse ways to generate the segmentation map, and add interactions on both their supervision and operation to achieve mutual promotion. For this purpose, a bidirectional supervision mechanism is elaborated to force the consistency between the outputs of these two branches. Thus, the segmentation branch can also give feedback to the classification branch to enhance the quality of localization seeds. Moreover, our method also designs interaction operations between these two branches to exchange their knowledge to assist each other. Experiments indicate our work outperforms existing end-to-end weakly supervised segmentation methods. Codes are available at https://github.com/zh460045050/BMP-WSSS.

Domain-knowledge Inspired Pseudo Supervision (DIPS) for unsupervised image-to-image translation models to support cross-domain classification

The ability to classify images is dependent on having access to large labeled datasets and testing on data from the same domain of which the model was trained on. Classification becomes more challenging when dealing with new data from a different domain, where gathering and especially labeling a larger image dataset for retraining a classification model requires a labor-intensive human effort. Cross-domain classification frameworks were developed to handle this data domain shift problem by utilizing unsupervised image-to-image translation models to translate an input image from the unlabeled domain to the labeled domain. The problem with these unsupervised models lies in their unsupervised nature. For lack of annotations, it is not possible to use the traditional supervised metrics to evaluate these translation models to pick the best-saved checkpoint model. This paper introduces a new method called Domain-knowledge Inspired Pseudo Supervision (DIPS) which utilizes Gaussian Mixture Models and domain knowledge to generate pseudo annotations to enable the use of traditional supervised metrics. This method was designed specifically to support cross-domain classification applications contrary to other typically used metrics such as the Fréchet Inception Distance (FID) which were designed to evaluate the model in terms of the quality of the generated image from a human-eye perspective. DIPS outperforms state-of-the-art GAN evaluation metrics when selecting the optimal saved checkpoint. Furthermore, DIPS showcases its robustness and interpretability by demonstrating a strong correlation with truly supervised metrics, highlighting its superiority over existing state-of-the-art alternatives The boiling crisis problem has been approached as a case study. The code and data to replicate the results can be found on the official GitHub-repository11https://github.com/Hindawi91/DIPS..

Unsupervised domain alignment of fingerprint denoising models using pseudo annotations

Unsupervised domain alignment of fingerprint denoising models using pseudo annotations

Semisupervised Boundary Detection for Aluminum Grains Combined With Transfer Learning and Region Growing.

In the manufacturing process of aluminum alloy, the size, distribution, and shape of microscopic grains indicate the mechanical characteristics and product quality. However, for metallographic images that can reveal microstructures, the cost of expert labeling at pixel level is high. To solve the problem, we propose a semisupervised learning strategy for grain boundary detection with a few labeled images and abundant unlabeled samples. To expand the helpful information, transfer learning and rule-based region growing are considered. Specifically, a deep network used for extracting multiscale features is designed. With constant training, through a few labeled metallographic images and abundant transferred natural images, pseudo annotations are generated gradually for unlabeled metallographic images iteratively by feature similarity and boundary region growing. The increased unlabeled samples with their pseudo annotations would be involved in the following training process in semisupervised self-training mode to improve the generalization ability of model, together with the domain adaptation block. In experiments, the proposed two methods named semiricher convolutional features-generative adversarial networks (SemiRCF-GAN) and semiricher convolutional features-maximum mean discrepancy (SemiRCF-MMD) can effectively detect grain boundaries with only one labeled metallographic image, and achieve F1 scores of 0.73 and 0.72, respectively, which surpass typical methods.

Towards a category-extended object detector with limited data

Object detectors are typically learned on fully-annotated training data with fixed predefined categories. However, categories are often required to be increased progressively. Usually, only the original training set annotated with old classes and some new training data labeled with new classes are available in such scenarios. Based on the limited datasets, a unified detector that can handle all categories is strongly needed. We propose a practical scheme to achieve it in this work. A conflict-free loss is designed to avoid label ambiguity, leading to an acceptable detector in one training round. To further improve performance, we propose a retraining phase in which Monte Carlo Dropout is employed to calculate the localization confidence to mine more accurate bounding boxes, and an overlap-weighted method is proposed for making better use of pseudo annotations during retraining. Extensive experiments demonstrate the effectiveness of our method.

Static-Dynamic Co-teaching for Class-Incremental 3D Object Detection

Deep learning-based approaches have shown remarkable performance in the 3D object detection task. However, they suffer from a catastrophic performance drop on the originally trained classes when incrementally learning new classes without revisiting the old data. This "catastrophic forgetting" phenomenon impedes the deployment of 3D object detection approaches in real-world scenarios, where continuous learning systems are needed. In this paper, we study the unexplored yet important class-incremental 3D object detection problem and present the first solution - SDCoT, a novel static-dynamic co-teaching method. Our SDCoT alleviates the catastrophic forgetting of old classes via a static teacher, which provides pseudo annotations for old classes in the new samples and regularizes the current model by extracting previous knowledge with a distillation loss. At the same time, SDCoT consistently learns the underlying knowledge from new data via a dynamic teacher. We conduct extensive experiments on two benchmark datasets and demonstrate the superior performance of our SDCoT over baseline approaches in several incremental learning scenarios. Our code is available at https://github.com/Na-Z/SDCoT.

Uncertainty-Aware Deep Learning With Cross-Task Supervision for PHE Segmentation on CT Images.

Perihematomal edema (PHE) volume, surrounding spontaneous intracerebral hemorrhage (SICH), is an important biomarker for the presence of SICH-associated diseases. However, due to irregular shapes and extremely low contrast of PHE on CT images, manually annotating PHE in pixel-wise is time-consuming and labour intensive even for experienced experts, which makes it almost infeasible to deploy current supervised deep learning approaches for automated PHE segmentation. How to develop annotation-efficient deep learning to achieve accurate PHE segmentation is an open problem. In this paper, we, for the first time, propose a cross-task supervised framework by introducing slice-level PHE labels and pixel-wise SICH annotations, which are more accessible in clinical scenarios compared to pixel-wise PHE annotations. Specifically, we first train a multi-level classifier based on slice-level PHE labels to produce high-quality class activation maps (CAMs) as pseudo PHE annotations. Then, we train a deep learning model to produce accurate PHE segmentation by iteratively refining the pseudo annotations via an uncertainty-aware corrective training strategy for noise removal and a distance-aware loss for background compression. Experimental results demonstrate that, the proposed framework achieves a comparative performance with the fully supervised methods on PHE segmentation, and largely improves the baseline performance where only pseudo PHE labels are used for training. We believe the findings from this study of using cross-task supervision for annotation-efficient deep learning can be applied to other medical imaging applications.

Learning scene-adaptive pseudo annotations for pedestrian detection in semi-supervised scenarios

Sufficient labeled training data may not be available for pedestrian detection in many real-world scenes. Semi-supervised settings naturally apply for the case where an adequate number of images are collected in a target scene but only a small proportion of them can be manually annotated. A common strategy is to adopt a detector trained on a well-established dataset (source data) or the limited annotated data to pseudo-annotate unannotated images. However, the domain gap and the lack of supervision in the target scene may lead to low-quality pseudo annotations. In this paper, we propose a Scene-adaptive Pseudo Annotation (SaPA) approach, which aims at exploiting two types of training data: source data providing sufficient supervision and unannotated target data offering domain-specific information. To utilize the source data, an Annotation Network (AnnNet) competes with a domain discriminator to learn domain-invariant features. To exploit the unannotated data, we temporally aggregate the parameters of AnnNet to build a more robust network, which is able to provide training goals for AnnNet. This new approach improves the generalization performance of AnnNet, which eventually leads to high-quality pseudo annotations to the unannotated data. Both manual and pseudo annotations are leveraged to train a more precise and scene-specific detector. We perform extensive experiments on multiple benchmarks to verify the effectiveness and superiority of SaPA.

Boundarymix: Generating pseudo-training images for improving segmentation with scribble annotations

Weakly-supervised semantic segmentation, as a promising solution to alleviate the burden of collecting per-pixel annotations, aims to train a segmentation model from partial weak annotations. Scribble on the object is one of the commonly used weak annotations and has shown to be sufficient for learning a decent segmentation model. Despite being effective, scribble-based weakly-supervised learning methods often lead to imprecise segmentation on object boundaries. This is mainly because the scribble annotations usually locate inside the objects and the dataset lacks annotations close to the semantic boundaries. To alleviate this issue, this paper proposes a simple-but-effective solution, i.e., BoundaryMix, which generates pseudo training image-annotation pairs from the original images to supplement the missing semantic boundaries. Specifically, given a prediction of segmentation, we cut off the regions around the estimated boundaries, which are error-prone and replace them with the contents from another image, which in effect creates new samples with less ambiguity around semantic boundaries. With training on scribbles and the on-the-fly generated pseudo annotations, the network acquires better prediction capability around the boundary region and thus improves the overall segmentation performance. By conducting experiments on PASCAL VOC 2012 dataset and POTSDAM dataset with only scribble annotations, we demonstrate the excellent performance of the proposed method and the almost closed gap between scribble-supervised and fully-supervised image segmentation.

Efficient Weakly-Supervised Object Detection With Pseudo Annotations

Weakly-supervised object detection (WSOD) has attracted lots of attention in recent years. However, there is still a big gap between WSOD and generic object detection. The main barriers to the efficiency of WSOD are the ineffective data augmentations and inaccurate bounding box predictions. Given only image-level annotations, it is hard for WSOD to effectively utilize variant data augmentations and accurately regress the bounding boxes. Although a fully-supervised object detector can be trained using annotations generated from the weakly-supervised object detector, the performance is still severely limited due to the low quality of mined pseudo annotations. This paper proposes an efficient WSOD method with pseudo annotations (EWPA) to make better use of imperfect annotations. With the assistance of pseudo annotations, EWPA can effectively regress more accurate bounding boxes while the traditional WSOD can only locate the salient parts of an object. Furthermore, pseudo annotations can help design more complex data augmentations, driving the network to learn more discriminative feature representations. Extensive experiments are conducted on PASCAL VOC 2007 and 2012 datasets and validate the effectiveness of EWPA.

3D Image Segmentation With Sparse Annotation by Self-Training and Internal Registration.

Anatomical image segmentation is one of the foundations for medical planning. Recently, convolutional neural networks (CNN) have achieved much success in segmenting volumetric (3D) images when a large number of fully annotated 3D samples are available. However, rarely a volumetric medical image dataset containing a sufficient number of segmented 3D images is accessible since providing manual segmentation masks is monotonous and time-consuming. Thus, to alleviate the burden of manual annotation, we attempt to effectively train a 3D CNN using a sparse annotation where ground truth on just one 2D slice of the axial axis of each training 3D image is available. To tackle this problem, we propose a self-training framework that alternates between two steps consisting of assigning pseudo annotations to unlabeled voxels and updating the 3D segmentation network by employing both the labeled and pseudo labeled voxels. To produce pseudo labels more accurately, we benefit from both propagation of labels (or pseudo-labels) between adjacent slices and 3D processing of voxels. More precisely, a 2D registration-based method is proposed to gradually propagate labels between consecutive 2D slices and a 3D U-Net is employed to utilize volumetric information. Ablation studies on benchmarks show that cooperation between the 2D registration and the 3D segmentation provides accurate pseudo-labels that enable the segmentation network to be trained effectively when for each training sample only even one segmented slice by an expert is available. Our method is assessed on the CHAOS and Visceral datasets to segment abdominal organs. Results demonstrate that despite utilizing just one segmented slice for each 3D image (that is weaker supervision in comparison with the compared weakly supervised methods) can result in higher performance and also achieve closer results to the fully supervised manner.

Self-Enhanced R-CNNs for Human Detection With Semi-Supervised Assumptions

Vision-based human detection is a fundamental task in visual content analysis. It has a wide range of applications, especially for person search and retrieval. To reduce the reliance of detection models on large amount of labeled data, we modify Faster R-CNN to facilitate semi-supervised human detection. Specifically, a Reliability Analysis (RA) module is included as an add-on into our Self-Enhanced R-CNN (SE-RCNN) model. The unlabeled images can be pseudo-annotated reliably under the help of this module. As a result, both labeled and unlabeled data are fed simultaneously for model optimization. The additional supervision, in turn, guides the training of a detection module in our model. The two aspects, extracting precise proposals and generating reliable pseudo annotations, can be mutually reinforced. Unlike previous related works, it is the first attempt to build a single-stage semi-supervised human detection model. In our experiment, we observe that the RA module plays an important role in exploiting unlabeled data and leads to state-of-the-art results of SE-RCNN on multiple benchmarks.

Semi-Supervised Human Detection via Region Proposal Networks Aided by Verification.

In this paper, we explore how to leverage readily available unlabeled data to improve semi-supervised human detection performance. For this purpose, we specifically modify the region proposal network (RPN) for learning on a partially labeled dataset. Based on commonly observed false positive types, a verification module is developed to assess foreground human objects in the candidate regions to provide an important cue for filtering the RPN's proposals. The remaining proposals with high confidence scores are then used as pseudo annotations for re-training our detection model. To reduce the risk of error propagation in the training process, we adopt a self-paced training strategy to progressively include more pseudo annotations generated by the previous model over multiple training rounds. The resulting detector re-trained on the augmented data can be expected to have better detection performance. The effectiveness of the main components of this framework is verified through extensive experiments, and the proposed approach achieves state-of-the-art detection results on multiple scene-specific human detection benchmarks in the semi-supervised setting.