COCO Dataset Research Articles

Region-based online selective examination for weakly supervised semantic segmentation

Current weakly supervised semantic segmentation methods usually generate noisy pseudo-labels. Training segmentation models with these labels tends to overfit the noise, leading to poor performance. Existing approaches often rely on iterative updates of pseudo-labels at pixel or image-level, ignoring the importance of region-level characteristics. The recently introduced Segment Anything Model (SAM) advances multiple approaches by fusing such region-level masks with noisy pseudo-labels. However, the fusion of noisy pseudo-labels using SAM is still challenging due to the lack of semantic information. To address these challenges, we propose a Region-based Online Selective Examination (ROSE). To be specific, we first consolidate SAM masks in a bottom-up manner to form a unified region prior. Then, leveraging these priors, region-level visual information is aggregated through the proposed region voting strategy. Furthermore, a cross-view selective examination method effectively explores semantic consistency between different image views and performs an examination to correct noisy pseudo-labels. The experimental results show that our ROSE achieves a new state-of-the-art on the Pascal VOC and COCO datasets. Moreover, the training time of our ROSE is over 10 times faster than previous methods.

RVAIC: Refined visual attention for improved image captioning

Visual attention has emerged as a prominent approach for improving the effectiveness of image captioning, as it enables the decoder network to focus selectively on the most salient regions in the image content, thereby facilitating the generation of precise and informative captions. Although visual attention achieves the improvement, the small numerical values of its input have a negative impact on its softmax, decreasing its effectiveness. To address this limitation, we propose a refined visual attention (RVA) framework that internally reweights visual attention by leveraging the language context of previously generated words. We first feed the language context into a fully connected layer to obtain appropriate dimensions for the visual features. Then, we use a sigmoid function to obtain a probability distribution to reweight the softmax’s input by applying the multiplication process. Experiments conducted on the MS COCO dataset demonstrate that RVA outperforms traditional visual attention and other existing image captioning methods, highlighting its effectiveness in enhancing the accuracy and informativeness of image captions.

Сегментация перекрывающихся изображений деревьев на цифровых снимках лесных массивов

The use of decision support systems based on computer vision and artificial intelligence significantly improves the working conditions for the operators of technological machines in the timber sector, whose work implies high intensity and psycho-emotional overload. By means of computer vision and artificial intelligence the operator can quickly and easily obtain the data on the state of the cutting area and adopt the optimal solution for holding the working operation. This facilitates his work and reduces the time spent searching and analyzing the data on the cutting area. Meanwhile, one of the key elements of such a system is a subsystem for automatic segmentation of objects in the photograph. We have explored the possibility of segmenting overlapping objects in the photographs of forest areas using a convolutional neural network based on the Mask R-CNN architecture. Unlike in most works on similar topics, the objects of this study are color photographs taken by an RGB camera rather than a lidar. This creates the prospect for reducing the cost of hardware and software systems used to support decision-making by the operators of logging machines. The images of the stems and crowns of coniferous and deciduous trees overlapping each other are the segmented objects under consideration. Using the GIMP graphic editor, we have manually marked the color photographs depicting a total of 134 trees of 4 different species: spruce, aspen, birch and pine. Utilizing the developed database, we have carried out an experiment to further train the Mask R-CNN convolutional neural network for segmentation of overlapping parts of the trees in the digital photographs of forest areas. The neural network has been pre-trained using the Microsoft COCO dataset containing more than 200,000 images of 80 different classes of objects such as people, cars, animals and various items. While training the neural network, the images supplied to its input were subjected to a series of various linear and nonlinear geometric transformations, which made it possible to increase the volume of training data by 11 times. As a result, the accuracy of segmentation of the images of the stems and crowns of coniferous and deciduous trees overlapping each other has reached 79 %, which allows the use of neural networks based on a similar architecture in decision support systems for logging machine operators.

A Lightweight Context-Aware Feature Transformer Network for Human Pose Estimation

We propose a Context-aware Feature Transformer Network (CaFTNet), a novel network for human pose estimation. To address the issue of limited modeling of global dependencies in convolutional neural networks, we design the Transformerneck to strengthen the expressive power of features. Transformerneck directly substitutes 3×3 convolution in the bottleneck of HRNet with a Contextual Transformer (CoT) block while reducing the complexity of the network. Specifically, the CoT first produces keys with static contextual information through 3×3 convolution. Then, relying on query and contextualization keys, dynamic contexts are generated through two concatenated 1×1 convolutions. Static and dynamic contexts are eventually fused as an output. Additionally, for multi-scale networks, in order to further refine the features of the fusion output, we propose an Attention Feature Aggregation Module (AFAM). Technically, given an intermediate input, the AFAM successively deduces attention maps along the channel and spatial dimensions. Then, an adaptive refinement module (ARM) is exploited to activate the obtained attention maps. Finally, the input undergoes adaptive feature refinement through multiplication with the activated attention maps. Through the above procedures, our lightweight network provides powerful clues for the detection of keypoints. Experiments are performed on the COCO and MPII datasets. The model achieves a 76.2 AP on the COCO val2017 dataset. Compared to other methods with a CNN as the backbone, CaFTNet has a 72.9% reduced number of parameters. On the MPII dataset, our method uses only 60.7% of the number of parameters, acquiring similar results to other methods with a CNN as the backbone.

Weakly supervised instance segmentation via peak mining and filtering

AbstractLearning the full extent of pixel‐level instance response in a weakly supervised manner remains unsatisfactory. Peak response maps (PRMs) localizes the discriminative object regions but cannot provide complete instance information, suffering from incomplete segmentation and unreliable mask prediction by noisy proposal retrieval. This work tackles this challenging problem by mining diverse class peak responses that include more discriminative and complete object regions and retrieving more reliable proposals from noisy segment proposal galleries. First, the existing method is enhanced with two more classification branches, thus contributing to more diverse and abundant instance regions from peak response maps. The mined class peak responses from two of the branches are then merged to generate more complete peak response maps by a clustering approach in their deep feature space. Then, instance segmentation masks are retrieved from a noisy object segment proposal gallery with class confidence, which is calculated by a normal classifier to obtain cleaner mask prediction. Finally, the pseudo‐supervision can be used to train an instance segmentation network in a fully supervised manner. Experiments on the PASCAL VOC 2012 dataset and COCO dataset show that the approach works effectively and outperforms other counterparts by a margin of more than 6 %, 4%, and 3% with the mean average precision (mAP) at IoU threshold of 0.25, 0.5 and 0.75, respectively.

YOLO-Rlepose: Improved YOLO Based on Swin Transformer and Rle-Oks Loss for Multi-Person Pose Estimation

In recent years, there has been significant progress in human pose estimation, fueled by the widespread adoption of deep convolutional neural networks. However, despite these advancements, multi-person 2D pose estimation still remains highly challenging due to factors such as occlusion, noise, and non-rigid body movements. Currently, most multi-person pose estimation approaches handle joint localization and association separately. This study proposes a direct regression-based method to estimate the 2D human pose from a single image. The authors name this network YOLO-Rlepose. Compared to traditional methods, YOLO-Rlepose leverages Transformer models to better capture global dependencies between image feature blocks and preserves sufficient spatial information for keypoint detection through a multi-head self-attention mechanism. To further improve the accuracy of the YOLO-Rlepose model, this paper proposes the following enhancements. Firstly, this study introduces the C3 Module with Swin Transformer (C3STR). This module builds upon the C3 module in You Only Look Once (YOLO) by incorporating a Swin Transformer branch, enhancing the YOLO-Rlepose model’s ability to capture global information and rich contextual information. Next, a novel loss function named Rle-Oks loss is proposed. The loss function facilitates the training process by learning the distributional changes through Residual Log-likelihood Estimation. To assign different weights based on the importance of different keypoints in the human body, this study introduces a weight coefficient into the loss function. The experiments proved the efficiency of the proposed YOLO-Rlepose model. On the COCO dataset, the model outperforms the previous SOTA method by 2.11% in AP.

A Robust Approach to Object Detection in Images using Improved T_CenterNet Method

Aims: A Robust Approach to Object Detection in Images using Improved T_CenterNet Method Background: Currently, two-stage image object detectors are the most common method of detecting objects. However, due to the slow processing of two-stage detectors, it is not always possible to apply them in real-time. Objective: To develop a more robust and effective method for detecting objects in images by building upon and improving the T_CenterNet algorithm Method: In this research paper, we propose CenterNet, a method based on a one-stage detector that takes into consideration the heat map branch of CenterNet, which is the most accurate anchorfree detector. To improve the outcomes of the upcoming image as well as detection efficiency, we disseminate the previously accurate long-term detection using a heatmap and a cascade guiding framework. Result: On the COCO dataset, our technique obtains detection outcome AP50 -65.5%, AP! (Small Objects)-33.8, AP! (Medium Objects)-47.0 at 38 frames per second. The results of the study show the superiority of the suggested approach over the conventional one. Conclusion: In this research, we have introduced an enhanced approach to image object detection using the improved T_CenterNet method. While the original CenterNet algorithm demonstrated superior precision compared to other single-stage algorithms, it suffered from slow detection speed. Our proposed approach, the Hourglass-208 model, built upon CenterNet, significantly improved detection performance. The proposed T_CenterNet algorithm improves upon the original CenterNet by incorporating the Hourglass backbone and feature map fusion techniques, enabling enhanced object detections. Additionally, we applied the Smooth L1 loss function to object size estimation within T_CenterNet, greatly enhancing overall detection performance by improving object size regression.

Image-free single-pixel keypoint detection for privacy preserving human pose estimation.

Computer vision technology has been applied in various fields such as identification, surveillance, and robot vision. However, computer vision algorithms used for human-related tasks operate on human images, which raises data security and privacy concerns. In this Letter, we propose an image-free human keypoint detection technique using a few coded illuminations and a single-pixel detector. Our proposed method can complete the keypoint detection task at an ultralow sampling rate on a measured one-dimensional sequence without image reconstruction, thus protecting privacy from the data collection stage and preventing the acquisition of detailed visual information from the source. The network is designed to optimize both the illumination patterns and the human keypoint predictor with an encoder-decoder framework. For model training and validation, we used 2000 images from Leeds Sport Dataset and COCO Dataset. By incorporating EfficientNet backbone, the inference time is reduced from 4 s to 0.10 s. In the simulation, the proposed network achieves 91.7% average precision. Our experimental results show an average precision of 88.4% at a remarkably low sampling rate of 0.015. In summary, our proposed method has the advantages of privacy protection and resource efficiency, which can be applied to many monitoring and healthcare tasks, such as clinical monitoring, construction site monitoring, and home service robots.

Disentangling the intrinsic feature from the related feature in image classification using knowledge distillation and object replacement

In our previous work, the features extracted by deep learning models are conceptually divided into the intrinsic features and the related features, where the object-oriented features are the intrinsic features and the background-oriented features are the related features. And the intrinsic feature-based classification results are regarded more reliable. In this work, we have improved our previously used metric and propose a new metric called Reliable Classification Accuracy (RCA) to more effectively quantify the dependency of classification results on the intrinsic features. Compared to our previous method that heightens the intrinsic feature exaction, we propose a novel method to further disentangle the intrinsic features from the related features. The proposed method comprises specially designed knowledge distillation and unique data augmentation. Unlike traditional knowledge distillation, the teacher model shares an identical network structure with the student model but has a special training process: the teacher model is pre-trained on a carefully designed dataset for pure intrinsic feature extraction and then guides the student model to focus on the category-related objects while ignoring the backgrounds in general datasets. And in the unique data augmentation, the category-related objects and the backgrounds from different images are re-matched in a graph-based method to further disentangle the intrinsic features from the related features in the guided feature extraction by specially designed knowledge distillation. The experiment results prove that the proposed method significantly improves both RCA and classification accuracy on the filtered datasets from MS COCO, Pascal VOC and Open Image Dataset.

SDSDet: A real-time object detector for small, dense, multi-scale remote sensing objects

Object detection in remote sensing images (RSIs) plays a crucial role in aerial and satellite image analysis. Existing methods lack the capability to effectively detect small and multi-scale objects in RSIs. Consequently, achieving an optimal trade-off between speed and accuracy remains unattainable. Extensive investigation reveals that state-of-the-art detectors have largely overlooked two critical aspects: Spatial artifacts from convolution operations and gradient confusion caused by neighboring levels in the Feature Pyramid Network. To address the first problem, we propose adopting a non-reorganized patch-embedding layer in the downsampling stage and a dual-path learning network (DPLNet) as the backbone, which can effectively mitigate the adverse effects of the edge pixel feature bias in feature maps. Additionally, using DPLNet as the backbone network can minimize costs while learning the intrinsic feature information of objects in RSI. For the second aspect, we propose a neighbor-erasing module with only one gradient flow (OGF-NEM). This module utilizes deep features to erase large objects to highlight small objects in shallow features and changes the backpropagation path to prevent the backflow of unreasonable gradients and the erosion of information from neighbor scales. Thus, a novel detector, called SDSDet, is proposed, which achieves excellent performance for small, dense, and multi-scale objects in RSIs. We have conducted exhaustive experiments on DOTA and MS COCO datasets. Specifically, the SDSDet achieves 42.8% AP on DOTA and 33.3% AP on MS COCO, together with nearly 4.87 M model size and 95 FPS.

Enhancing fine-detail image synthesis from text descriptions by text aggregation and connection fusion module

Synthesizing images with fine details from text descriptions is a challenge. The existing single-stage generative adversarial networks (GANs) fuse sentence features into the image generation process through affine transformation, which alleviate the problems of missing details and large computation from stacked networks. However, existing single-stage networks ignore the word features in the text description, resulting in a lack of detail in the generated image. To address this issue, we proposed a text aggregation module (TAM) to fuse sentence features and word features in a text by a simple spatial attention mechanism. Then we built a text connection fusion (TCF) block consisting mainly of gated recurrent unit (GRU) and up-sampled block. It can connect text features used in the up-sampled blocks to improve text utilization. Besides, to further improve the semantic consistency between text and the generated images, we introduce the deep attentional multimodal similarity model (DAMSM) loss, which monitors the similarity between text and improves semantic consistency. Experimental results prove that our method is superior to the state-of-the-art models on the CUB and COCO datasets, regarding both image fidelity and semantic consistency with the text.

ResDNet: Efficient Dense Multi-Scale Representations With Residual Learning for High-Level Vision Tasks.

Deep feature fusion plays a significant role in the strong learning ability of convolutional neural networks (CNNs) for computer vision tasks. Recently, works continually demonstrate the advantages of efficient aggregation strategy and some of them refer to multiscale representations. In this article, we describe a novel network architecture for high-level computer vision tasks where densely connected feature fusion provides multiscale representations for the residual network. We term our method the ResDNet which is a simple and efficient backbone made up of sequential ResDNet modules containing the variants of dense blocks named sliding dense blocks (SDBs). Compared with DenseNet, ResDNet enhances the feature fusion and reduces the redundancy by shallower densely connected architectures. Experimental results on three classification benchmarks including CIFAR-10, CIFAR-100, and ImageNet demonstrate the effectiveness of ResDNet. ResDNet always outperforms DenseNet using much less computation on CIFAR-100. On ImageNet, ResDNet-B-129 achieves 1.94% and 0.89% top-1 accuracy improvement over ResNet-50 and DenseNet-201 with similar complexity. Besides, ResDNet with more than 1000 layers achieves remarkable accuracy on CIFAR compared with other state-of-the-art results. Based on MMdetection implementation of RetinaNet, ResDNet-B-129 improves mAP from 36.3 to 39.5 compared with ResNet-50 on COCO dataset.

Reference Twice: A Simple and Unified Baseline for Few-Shot Instance Segmentation.

Few-Shot Instance Segmentation (FSIS) requires detecting and segmenting novel classes with limited support examples. Existing methods based on Region Proposal Networks (RPNs) face two issues: 1) Overfitting suppresses novel class objects; 2) Dual-branch models require complex spatial correlation strategies to prevent spatial information loss when generating class prototypes. We introduce a unified framework, Reference Twice (RefT), to exploit the relationship between support and query features for FSIS and related tasks. Our three main contributions are: 1) A novel transformer-based baseline that avoids overfitting, offering a new direction for FSIS; 2) Demonstrating that support object queries encode key factors after base training, allowing query features to be enhanced twice at both feature and query levels using simple cross-attention, thus avoiding complex spatial correlation interaction; 3) Introducing a class-enhanced base knowledge distillation loss to address the issue of DETR-like models struggling with incremental settings due to the input projection layer, enabling easy extension to incremental FSIS. Extensive experimental evaluations on the COCO dataset under three FSIS settings demonstrate that our method performs favorably against existing approaches across different shots, e.g., +8.2/ + 9.4 performance gain over state-of-the-art methods with 10/30-shots. Source code and models will be available at this github site.

ISTR: Mask-Embedding-Based Instance Segmentation Transformer.

Transformer-based instance-level recognition has attracted increasing research attention recently due to the superior performance. However, although attempts have been made to encode masks as embeddings into Transformer-based frameworks, how to combine mask embeddings and spatial information for a transformer-based approach is still not fully explored. In this paper, we revisit the design of mask-embedding-based pipelines and propose an Instance Segmentation TRansformer (ISTR) with Mask Meta-Embeddings (MME), leveraging the strengths of transformer models in encoding embedding information and incorporating spatial information from mask embeddings. ISTR incorporates a recurrent refining head that consists of a Dynamic Box Predictor (DBP), a Mask Information Generator (MIG), and a Mask Meta-Decoder (MMD). To improve the quality of mask embeddings, MME interprets the mask encoding-decoding processes as a mutual information maximization problem, which unifies the objective functions of different decoding schemes such as Principal Component Analysis (PCA) and Discrete Cosine Transform (DCT) with a meta-formulation. Under the meta-formulation, a learnable Spatial Mask Tuner (SMT) is further proposed, which fuses the spatial and embedding information produced from MIG and can significantly boost the segmentation performance. The resulting varieties, i.e., ISTR-PCA, ISTR-DCT, and ISTR-SMT, demonstrate the effectiveness and efficiency of incorporating mask embeddings with the query-based instance segmentation pipelines. On the COCO dataset, ISTR surpasses all predominant mask-embedding-based models by a large margin, and achieves competitive performance compared to concurrent state-of-the-art models. On the Cityscapes dataset, ISTR also outperforms several strong baselines. Our code has been made available at: https://github.com/hujiecpp/ISTR.

Spatial Structure Constraints for Weakly Supervised Semantic Segmentation.

The image-level label has prevailed in weakly supervised semantic segmentation tasks due to its easy availability. Since image-level labels can only indicate the existence or absence of specific categories of objects, visualization-based techniques have been widely adopted to provide object location clues. Considering class activation maps (CAMs) can only locate the most discriminative part of objects, recent approaches usually adopt an expansion strategy to enlarge the activation area for more integral object localization. However, without proper constraints, the expanded activation will easily intrude into the background region. In this paper, we propose spatial structure constraints (SSC) for weakly supervised semantic segmentation to alleviate the unwanted object over-activation of attention expansion. Specifically, we propose a CAM-driven reconstruction module to directly reconstruct the input image from deep CAM features, which constrains the diffusion of last-layer object attention by preserving the coarse spatial structure of the image content. Moreover, we propose an activation self-modulation module to refine CAMs with finer spatial structure details by enhancing regional consistency. Without external saliency models to provide background clues, our approach achieves 72.7% and 47.0% mIoU on the PASCAL VOC 2012 and COCO datasets, respectively, demonstrating the superiority of our proposed approach. The source codes and models have been made available at https://github.com/NUST-Machine-Intelligence-Laboratory/SSC.

Joint-Guided Distillation Binary Neural Network via Dynamic Channel-Wise Diversity Enhancement for Object Detection

Through truncating the weights and activations of a deep neural network, conventional binary quantization imposes limitations on the representation capability of the network parameters, which hence deteriorates the detection performance of the network. In this paper, a joint-guided distillation binary neural network via dynamic channel-wise diversity enhancement for object detection (JDBNet) is proposed to mitigate the gap of quantization errors. Our JDBNet includes a dynamic channel-wise diversity scheme and real-valued joint-guided teacher assistance to enhance the representation capability of the binary neural network in the object detection tasks. In the dynamic diversity scheme, the learning channel-wise bias (LCB) layer supports adjusting the magnitude of the parameters in which the sensitivity of the model parameters to the arbitrary quantization method is reduced, thereby improving the diversity expression ability of the feature parameters. In the joint-guided strategy, the single-precision implicit knowledge from the guiding teacher in the multilevel layer is utilized to supervise and penalize the quantitative model, enhancing the fitting performance of parameters in the binary quantized model. Extensive experiments on the PASCAL VOC, MS COCO, and VisDrone-DET datasets demonstrate that our JDBNet outperforms the state-of-the-art binary object detection networks in terms of mean Average Precision.

Recurrent Affine Transformation for Text-to-Image Synthesis

Text-to-image synthesis aims to generate realistic images conditioned on text descriptions. To fuse text information into synthesized images, conditional affine transformations (CATs), such as conditional batch normalization (CBN) and conditional instance normalization (CIN), are usually used to predict batch statistics of different layers. However, ordinary CAT blocks control the batch statistics independently disregarding the consistency among neighboring layers. To address the above issue, we propose a new fusion approach names recurrent affine transformation (RAT) for synthesizing images conditioned on text information. RAT connects all the CAT blocks with recurrent connections for explicitly fitting the temporal consistency between CAT blocks. To verify the effectiveness of RAT, we propose a novel visualization method to show how generative adversarial network (GAN) fuses conditional information. Our microscopic and macroscopic visualizations not only demonstrate the effectiveness of RAT but also turn out to be a useful perspective to analyze how GAN fuses conditional information. In addition, we propose a more stable spatial attention mechanism for the discriminator, which helps the text description to supervise the generator to synthesize more relevant image contents. Extensive experiments on the CUB, Oxford-102, and COCO datasets demonstrate the proposed model's superiority in comparison to state-of-the-art models. Our code is available on <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/senmaoy/RAT-GAN</uri> .

Small target detection algorithm based on multi-branch stacking and new sampling transition module.

Aiming at the problem that the SSD algorithm does not fully extract the feature information contained in each feature layer, as well as the feature information is easily lost during the sampling process, which makes the feature expression ineffective and leads to insufficient performance in small target detection. In this paper, AMT-SSD is proposed, a small target detection algorithm that incorporates the multi-branch stacking and new sampling transition module of the attention mechanism. In this algorithm, the composite attention mechanism is utilized to improve the correlation of features of the samples to be detected in terms of spatial and channels, and the efficiency of the algorithm; secondly, multi-branch stacking module is used to extract multi-size features for each feature layer, and different sizes of convolution kernels are utilized in parallel to fully extract their features and improve the expression of features; meanwhile, during the sampling process, the problem of missing features is solved by applying inverse subpixel convolution in the new sampling transition module. Experimentally, the AMT-SSD algorithm achieves 84.6% and 53.4% mAP metrics on the PASCAL VOC dataset and MS COCO dataset, respectively. This indicates that the AMT-SSD algorithm can effectively extract feature information that is beneficial to detection samples, and also performs well in reducing feature loss, which is effective for the algorithm to improve the algorithm on small targets.

Relation Knowledge Distillation by Auxiliary Learning for Object Detection.

Balancing the trade-off between accuracy and speed for obtaining higher performance without sacrificing the inference time is a challenging topic for object detection task. Knowledge distillation, which serves as a kind of model compression techniques, provides a potential and feasible way to handle above efficiency and effectiveness issue through transferring the dark knowledge from the sophisticated teacher detector to the simple student one. Despite demonstrating promising solutions to make harmonies between accuracy and speed, current knowledge distillation for object detection methods still suffer from two limitations. Firstly, most of the methods are inherited or refereed from the frameworks in image classification task, and deploy an implicit manner by imitating or constraining the features from the intermediate layers or the output predictions between the teacher and student models. While little consideration has been raised to the intrinsic relevance of the classification and localization predictions in object detection task. Besides, these methods fail to investigate the relationship between detection and distillation tasks in knowledge distillation pipeline, and they train the whole network by simply integrating losses from these two different tasks through hand-crafted designation parameters. For addressing the aforementioned issues, we propose a novel Relation Knowledge Distillation by Auxiliary Learning for Object Detection (ReAL) method in this paper. Specifically, we first design a prediction relation distillation module which makes the student model directly mimic the output predictions from the teacher one, and conduct self and mutual relation distillation losses to excavate the relation information between teacher and student models. Moreover, for better devolving into the relationship between different tasks in distillation pipeline, we introduce the auxiliary learning into knowledge distillation for object detection and develop a dynamic weight adaptation strategy. Through regarding detection task as primary task and treating distillation task as auxiliary task in auxiliary learning framework, we dynamically adjust and regularize the corresponding weights of the losses for these tasks during the training process. Experiments on MS COCO dataset are conducted using various detector combinations of teacher and student models and the results show that our proposed ReAL can achieve obvious improvement on different distillation model configurations, while performing favorably against state-of-the-arts.

ECEA: Extensible Co-Existing Attention for Few-Shot Object Detection.

Few-shot object detection (FSOD) identifies objects from extremely few annotated samples. Most existing FSOD methods, recently, apply the two-stage learning paradigm, which transfers the knowledge learned from abundant base classes to assist the few-shot detectors by learning the global features. However, such existing FSOD approaches seldom consider the localization of objects from local to global. Limited by the scarce training data in FSOD, the training samples of novel classes typically capture part of objects, resulting in such FSOD methods being unable to detect the completely unseen object during testing. To tackle this problem, we propose an Extensible Co-Existing Attention (ECEA) module to enable the model to infer the global object according to the local parts. Specifically, we first devise an extensible attention mechanism that starts with a local region and extends attention to co-existing regions that are similar and adjacent to the given local region. We then implement the extensible attention mechanism in different feature scales to progressively discover the full object in various receptive fields. In the training process, the model learns the extensible ability on the base stage with abundant samples and transfers it to the novel stage of continuous extensible learning, which can assist the few-shot model to quickly adapt in extending local regions to co-existing regions. Extensive experiments on the PASCAL VOC and COCO datasets show that our ECEA module can assist the few-shot detector to completely predict the object despite some regions failing to appear in the training samples and achieve the new state-of-the-art compared with existing FSOD methods. Code is released at https://github.com/zhimengXin/ECEA.