Box Annotations Research Articles

A Fixed-Point Approach to Unified Prompt-Based Counting

Existing class-agnostic counting models typically rely on a single type of prompt, e.g., box annotations. This paper aims to establish a comprehensive prompt-based counting framework capable of generating density maps for concerned objects indicated by various prompt types, such as box, point, and text. To achieve this goal, we begin by converting prompts from different modalities into prompt masks without requiring training. These masks are then integrated into a class-agnostic counting methodology for predicting density maps. Furthermore, we introduce a fixed-point inference along with an associated loss function to improve counting accuracy, all without introducing new parameters. The effectiveness of this method is substantiated both theoretically and experimentally. Additionally, a contrastive training scheme is implemented to mitigate dataset bias inherent in current class-agnostic counting datasets, a strategy whose effectiveness is confirmed by our ablation study. Our model excels in prominent class-agnostic datasets and exhibits superior performance in cross-dataset adaptation tasks.

OBBInst: Remote sensing instance segmentation with oriented bounding box supervision

Remote sensing (RS) instance segmentation is an important but challenging task due to multi-oriented, densely arranged objects and lack of mask annotation. Compared with redundant horizontal bounding-box (HBB) and expensive pixel-level annotation, oriented bounding box (OBB) annotations can provide compact object depicts with lower annotation costs. Therefore, we propose the first weakly supervised remote sensing instance segmentation method with OBB supervision (namely OBBInst) to reduce the annotation burden and make full use of existing abundant OBB annotations. Based on BoxInst (a high-performance instance segmentation method with box annotations), OBBInst has customized a framework for OBB annotation to unify the incompatibility between existing HBB-based and OBB-based methods. In addition, we propose an oriented projection method with a corresponding loss function to achieve more precise target depicts of OBB annotation. Moreover, we propose an edge similarity loss to incorporate Canny edge prior into deep learning framework for more precise edge identification of densely arranged objects. We have conducted extensive experiments on iSAID and HRSC datasets, and the experimental results demonstrate that OBBInst can achieve the state-of-the-art performance as compared to existing box-supervised methods. In addition, OBBInst dramatically narrows the performance gap between weakly and fully supervised instance segmentation (23.9% vs. 35.6% in iSAID dataset and 79.5% vs. 84.9% in HRSC dataset).

Towards High Quality Multi-Object Tracking and Segmentation without Mask Supervision.

Recently studies have shown the potential of weakly supervised multi-object tracking and segmentation, but the drawbacks of coarse pseudo mask label and limited utilization of temporal information remain to be unresolved. To address these issues, we present a framework that directly uses box label to supervise the segmentation network without resorting to pseudo mask label. In addition, we propose to fully exploit the temporal information from two perspectives. Firstly, we integrate optical flow-based pairwise consistency to ensure mask consistency across frames, thereby improving mask quality for segmentation. Secondly, we propose a temporally adjacent pair-based sampling strategy to adapt instance embedding learning for data association in tracking. We combine these techniques into an end-to-end deep model, named BoxMOTS, which requires only box annotation without mask supervision. Extensive experiments demonstrate that our model surpasses current state-of-the-art by a large margin, and produces promising results on KITTI MOTS and BDD100K MOTS. The source code is available at https://github.com/Spritea/BoxMOTS.

Weakly supervised real-time instance segmentation for ultrasound images of median nerves

Medical ultrasound technology has garnered significant attention in recent years, with Ultrasound-guided regional anesthesia (UGRA) and carpal tunnel diagnosis (CTS) being two notable examples. Instance segmentation, based on deep learning approaches, is a promising choice to support the analysis of ultrasound data. However, many instance segmentation models cannot achieve the requirement of ultrasound technology e.g. real-time. Moreover, fully supervised instance segmentation models require large numbers of images and corresponding mask annotations for training, which can be time-consuming and labor-intensive in the case of medical ultrasound data. This paper proposes a novel weakly supervised framework, CoarseInst, to achieve real-time instance segmentation of ultrasound images with only box annotations. CoarseInst not only improves the network structure, but also proposes a two-stage “coarse-to-fine” training strategy. Specifically, median nerves are used as the target application for UGRA and CTS. CoarseInst consists of two stages, with pseudo mask labels generated in the coarse mask generation stage for self-training. An object enhancement block is incorporated to mitigate the performance loss caused by parameter reduction in this stage. Additionally, we introduce a pair of loss functions, the amplification loss, and the deflation loss, that work together to generate the masks. A center area mask searching algorithm is also proposed to generate labels for the deflation loss. In the self-training stage, a novel self-feature similarity loss is designed to generate more precise masks. Experimental results on a practical ultrasound dataset demonstrate that CoarseInst could achieve better performance than some state-of-the-art fully supervised works.

WSRC: Weakly Supervised Faster RCNN Toward Accurate Traffic Object Detection

Weakly supervised object detection has attracted increasing interest in recent years as driven by data annotation bottleneck in deep network training. On the other hand, the performance of weakly supervised models still lags far behind due to the missing of object location information during training. This paper presents a Weakly Supervised Faster RCNN (WSRC) that aims to train accurate traffic object detectors without using any box annotations, addressing the phenomenon that the traffic object detector trained by traditional method cannot be applied in the real traffic scene due to the lack of data annotation. The WSRC designs an innovative proposal evaluation network (PEN) that is trained by using a large amount of proposals that are scored according to their Intersection-over-Union (IoU) overlap with pseudo ground-truth boxes. Novel loss functions are designed to teach the PEN and WSRC to identify good proposals and regress to accurate object boxes under the guidance of loss dynamics and proposal scores. The proposed WSRC achieves superior detection performance over the datasets Pascal VOC2007, MS COCO and Cityscape, demonstrating its effectiveness while short of annotation boxes.

Rethinking mask heads for partially supervised instance segmentation

We focus on partially supervised instance segmentation where only a subset of categories are mask-annotated (seen) and the model is expected to generalize to unseen categories for which only box annotations are provided to eliminate laborious mask annotations. Many recent studies train a class-agnostic segmentation network to distinguish foreground areas in each proposal. However, class-agnostic models behave poorly in complex contexts when the foreground object overlaps with other irreverent objects. Identifying specific object categories is simpler than distinguishing foreground from background since the definition of the foreground is ambiguous even for a human. However, training class-specific model is unfeasible under the partially supervised setting since the mask annotations of unseen categories are absent during training. To overcome this issue, we put forward a teacher-student architecture where the teacher learns general yet comprehensive knowledge and the students, guided by the teacher, delve deeper into specific categories. Concretely, the teacher learns to segment foreground from proposals and the student is devoted to segmenting objects of specific categories. Extensive experiments on the challenging COCO dataset demonstrate our method consistently improve the performance of several recent state-of-the-art methods for the partially setting. Especially, for overlapped objects, our method significantly outperforms the competitors with a clear margin, demonstrating the superiority of our method.