PASCAL VOC Research Articles

Dynamic YOLO for small underwater object detection

The practical application of object detection inevitably encounters challenges posed by small objects. In underwater object detection, a crucial method for marine exploration, the presence of small objects in underwater environments significantly hampers the performance of detection. In this paper, a dynamic YOLO detector is proposed as a solution to alleviate this problem. Specifically, a light-weight backbone network is first constructed based on deformable convolution v3, with some specialized designs for small object detection. Secondly, a unified feature fusion framework based on channel-wise, scale-wise, and spatial-aware attention is proposed to fuse feature maps from different scales. This is particularly critical for detecting small objects since it allows us to fully exploit the enhanced capabilities offered by our proposed backbone network. Finally, a simple but effective detection head is designed to handle the conflict between classification and localization by disentangling and aligning the two tasks. Extensive experiments are conducted on benchmark datasets to demonstrate the effectiveness of the proposed model. Without bells and whistles, dynamic YOLO outperforms the recent state-of-the-art methods by a large margin of +0.8\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$+\\,0.8$$\\end{document} AP and +1.8\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$+\\,1.8$$\\end{document} APS\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\ ext {AP}_{S}$$\\end{document} on the DUO dataset. Experimental results on Pascal VOC and MS COCO datasets also demonstrate the superiority of the proposed method. At last, ablation studies are conducted on DUO dataset to validate the effectiveness and efficiency of each design in dynamic YOLO. Source code will be available at https://github.com/chenjie04/Dynamic-YOLO.

Knowledge Distillation Based on Narrow-Deep Networks

Deep neural networks perform better than shallow neural networks, but the former tends to be deeper or wider, introducing large numbers of parameters and computations. We know that networks that are too wide have a high risk of overfitting and networks that are too deep require a large amount of computation. This paper proposed a narrow-deep ResNet, increasing the depth of the network while avoiding other issues caused by making the network too wide, and used the strategy of knowledge distillation, where we set up a trained teacher model to train an unmodified, wide, and narrow-deep ResNet that allows students to learn the teacher’s output. To validate the effectiveness of this method, it is tested on Cifar-100 and Pascal VOC datasets. The method proposed in this paper allows a small model to have about the same accuracy rate as a large model, while dramatically shrinking the response time and computational effort.

SpanEffiDet: Span-Scale and Span-Path Feature Fusion for Object Detection

Lower versions of EfficientDet (such as D0, D1) have smaller network structures and parameter sizes, but lower detection accuracy. Higher versions exhibit higher accuracy, but the increase in network complexity poses challenges for real-time processing and hardware requirements. To meet the higher accuracy requirements under limited computational resources, this paper introduces SpanEffiDet based on the channel adaptive frequency filter (CAFF) and the Span-Path Bidirectional Feature Pyramid structure. Firstly, the CAFF module proposed in this paper realizes the frequency domain transformation of channel information through Fourier transform and effectively extracts the key features through semantic adaptive frequency filtering, thus, eliminating channel redundant information of EfficientNet. Simultaneously, the module has the ability to compute the weights across the channels and at fine granularity, and capture the detailed information of element features. Secondly, a two-way characteristic pyramid network multi-level cross-BIFPN, which can achieve multi-layer and multi-nodes, is proposed to build cross-level information transmission to incorporate both semantic and positional information of the target. This design enables the network to more effectively detect objects with significant size differences in complex environments. Finally, by introducing generalized focal Loss V2, reliable localization quality estimation scores are predicted from the distribution statistics of bounding boxes, thereby improving localization accuracy. The experimental results indicate that on the MS COCO dataset, SpanEffiDet-D0 achieved an AP improvement of 3.3% compared to the original EfficientDet series algorithms. Similarly, on the PASCAL VOC2007 and 2012 datasets, the mAP of SpanEffiDet-D0 is respectively 1.66 and 2.65% higher than that of EfficientDet-D0.

One stage multi-scale efficient network for underwater target detection.

Due to the complexity of the underwater environment, existing methods for underwater target detection present low precision on small or dense targets. To address these issues, a novel method is proposed for underwater target detection based on YOLOv5s (You Only Look Once version 5 small), which aims to improve the precision and robustness. In this study, an efficient feature extraction network is introduced to extract significant features, and a novel attention mechanism with deformable convolution is designed to improve the feature representation. Subsequently, an adaptive spatial fusion operation is introduced at the neck of YOLOv5s to facilitate feature fusion from various layers. By integrating low-level features with high-level features, the adaptive fusion feature pyramid network effectively integrates global semantic information and decreases the semantic gap between features from various layers, contributing to the high detection precision. Comprehensive experiments demonstrate that the proposed method achieves an mAP50 of 86.97% on the Underwater Robot Professional Contest of China 2020 dataset, 3.07% higher than YOLOv5s. Furthermore, the proposed method achieves a detection precision of 76.0% on the PASCAL VOC2007 dataset, surpassing several outstanding methods.

Proposal-Free Fully Convolutional Network: Object Detection Based on a Box Map.

Region proposal-based detectors, such as Region-Convolutional Neural Networks (R-CNNs), Fast R-CNNs, Faster R-CNNs, and Region-Based Fully Convolutional Networks (R-FCNs), employ a two-stage process involving region proposal generation followed by classification. This approach is effective but computationally intensive and typically slower than proposal-free methods. Therefore, region proposal-free detectors are becoming popular to balance accuracy and speed. This paper proposes a proposal-free, fully convolutional network (PF-FCN) that outperforms other state-of-the-art, proposal-free methods. Unlike traditional region proposal-free methods, PF-FCN can generate a "box map" based on regression training techniques. This box map comprises a set of vectors, each designed to produce bounding boxes corresponding to the positions of objects in the input image. The channel and spatial contextualized sub-network are further designed to learn a "box map". In comparison to renowned proposal-free detectors such as CornerNet, CenterNet, and You Look Only Once (YOLO), PF-FCN utilizes a fully convolutional, single-pass method. By reducing the need for fully connected layers and filtering center points, the method considerably reduces the number of trained parameters and optimizes the scalability across varying input sizes. Evaluations of benchmark datasets suggest the effectiveness of PF-FCN: the proposed model achieved an mAP of 89.6% on PASCAL VOC 2012 and 71.7% on MS COCO, which are higher than those of the baseline Fully Convolutional One-Stage Detector (FCOS) and other classical proposal-free detectors. The results prove the significance of proposal-free detectors in both practical applications and future research.

Towards Stabilized Few-Shot Object Detection with Less Forgetting via Sample Normalization.

Few-shot object detection is a challenging task aimed at recognizing novel classes and localizing with limited labeled data. Although substantial achievements have been obtained, existing methods mostly struggle with forgetting and lack stability across various few-shot training samples. In this paper, we reveal two gaps affecting meta-knowledge transfer, leading to unstable performance and forgetting in meta-learning-based frameworks. To this end, we propose sample normalization, a simple yet effective method that enhances performance stability and decreases forgetting. Additionally, we apply Z-score normalization to mitigate the hubness problem in high-dimensional feature space. Experimental results on the PASCAL VOC data set demonstrate that our approach outperforms existing methods in both accuracy and stability, achieving up to +4.4 mAP@0.5 and +5.3 mAR in a single run, with +4.8 mAP@0.5 and +5.1 mAR over 10 random experiments on average. Furthermore, our method alleviates the drop in performance of base classes. The code will be released to facilitate future research.

Small object intelligent Detection method based on Adaptive Cascading Context

With the technology advances, deep learning-based object detection has made unprecedented progress. However, the small spatial ratio of object pixels affects the effective extraction of deep details features, resulting in poor detection results in small object detection. To improve the accuracy of small object detection, an adaptive Cascading Context small (ACC) object detection method is proposed based on YOLOv5. Firstly, a separate shallow layer feature was proposed to obtain more detailed information beneficial to small object detection. Secondly, an adaptive cascade method is proposed to fuse the output features of the three layers of the pyramid to adaptively filter negative semantic information, while fusing with shallow features to solve the problem of low classification accuracy caused by insufficient semantic information of shallow features. Finally, an adaptive context model is proposed to use a deformable convolution to obtain spatial context features of shallow small objects, associating the targets with the background, thereby improving the accuracy of small object detection. The experimental results show that the detection accuracy of the proposed method has been improved by 6.12%, 3.35%, 3.33%, and 5.2%, respectively, compared with the source code on the PASCAL VOC, NWPU VHR-10, KITTI, and RSOD datasets, which fully demonstrate the effectiveness of our method in small object detection.

Employing feature mixture for active learning of object detection

Active learning aims to select the most informative samples for annotation from a large amount of unlabeled data, in order to reduce time-consuming and labor-intensive manual labeling efforts. Although active learning for object detection has made substantial progress in recent years, developing an accurate and efficient active learning algorithm for object detection remains a challenge. In this paper, we propose a novel unsupervised active learning method for deep object detection. This is based on our hypotheses that an object is more likely to be wrongly predicted by the model, if the prediction changes when its feature representations are slightly mixed by another feature representations at a very small ratio. Such unlabeled samples can be regarded as informative samples that can be selected by active learning. Our method employs base representations of all categories generated from the object detection network to examine the robustness of every detected object. We design a scoring function to calculate the informative score of each unlabeled image. We conduct extensive experiments on two public datasets, i.e., PASCAL VOC and MS-COCO. Experiment results show that our approach consistently outperforms state-of-the-art single-model based methods with significant margins. Our approach also performs on par with multi-model based methods, at much lesser computational cost.

DP-DDCL: A discriminative prototype with dual decoupled contrast learning method for few-shot object detection

Few-shot object detection (FSOD) can effectively improve object detection performance with limited training data, attracting increasing interest from researchers. Due to the limited sample size, there is often a large variance in the learned features, resulting in deviating from the class center and being confused with other classes. People are often able to accurately identify few-shot objects based on discriminative information. Motivated by it, we propose a discriminative prototype with dual decoupled contrast learning (DP-DDCL) method to address the issue of limited training data in FSOD. By introducing domain knowledge comprising the CLIP and attribute knowledge graph to obtain explicit and implicit semantic and visual information, we construct a novel discriminative prototype that enhances the representation of different classes (especially few-shot samples). Simultaneously, dual decoupled contrast learning consists of the decoupled of positive and negative samples, and the decoupled of discriminative prototype and instance contrast learning is proposed. It makes samples of the same class cluster around their respective class prototype while maintaining a clear semantic boundary between different classes. Extensive experiments demonstrate the efficacy of our method, surpassing the current state-of-the-art in any-shot and all-data splits, with maximum performance gains of up to +9.15% on the standard PASCAL VOC benchmark and +2.5% on the challenging COCO benchmark.

CRTED: Few-Shot Object Detection via Correlation-RPN and Transformer Encoder–Decoder

Few-shot object detection (FSOD) aims to address the challenge of requiring a substantial number of annotations for training in conventional object detection, which is very labor-intensive. However, the existing few-shot methods achieve high precision with the sacrifice of time for exhaustive fine-tuning or have poor performance in novel-class adaptation. We presume the major reason is that the valuable correlation feature among different categories is insufficiently exploited, hindering the generalization of knowledge from base to novel categories for object detection. In this paper, we propose few-shot object detection via Correlation-RPN and transformer encoder–decoder (CRTED), a novel training network to learn object-relevant features of inter-class correlation and intra-class compactness while suppressing object-agnostic features in the background with limited annotated samples. And we also introduce a four-way tuple-contrast training strategy to positively activate the training progress of our object detector. Experiments over two few-shot benchmarks (Pascal VOC, MS-COCO) demonstrate that our proposed CRTED without further fine-tuning can achieve comparable performance with current state-of-the-art fine-tuned works. The codes and pre-trained models will be released.

PCSformer: Pair-wise Cross-scale Sub-prototypes mining with CNN-transformers for weakly supervised semantic segmentation

Generating initial seeds is an important step in weakly supervised semantic segmentation (WSSS). Our approach concentrates on generating and refining initial seeds. The convolutional neural networks (CNNs)–based initial seeds focus only on the most discriminative regions and lack global information about the target. The Vision Transformer (ViT)–based approach can capture long-range feature dependencies due to the unique advantage of the self-attention mechanism. Still, we find that it suffers from distractor object leakage and background leakage problems. Based on these observations, we propose PCSformer, which improves the model’s ability to extract features through a Pair-wise Cross-scale (PC) strategy and solves the problem of distractor object leakage by further extracting potential target features through Sub-Prototypes (SP) mining. In addition, the proposed Conflict Self-Elimination (CSE) module further alleviates the background leakage problem. We validate our approach on the widely adopted Pascal VOC 2012 and MS COCO 2014, and extensive experiments demonstrate our superior performance. Furthermore, our method proves to be competitive for WSSS in medical images and challenging scenarios involving deformable and cluttered scenes. Additionally, we extend the PCSformer to weakly supervised object localization tasks, further highlighting its scalability and versatility. The code is available at https://github.com/ChunmengLiu1/PCSformer.

Value-Guided Adaptive Data Augmentation for Imbalanced Small Object Detection

Data augmentation is considered a promising technique to resolve the imbalance of large and small objects. Unfortunately, most existing methods augment all small objects indiscriminately, regardless of their learnability and proportion. This tends to result in wasteful enlargement for many weak, low-information objects but under-augmentation for rare and learnable objects. To this end, we propose a value-guided adaptive data augmentation for scale- and proportion-imbalanced small object detection (ValCopy-Paste). Specifically, we first develop a non-learning object value criteria to determine whether one object should be expanded. Both scale-based learnability and quantity-based necessity are involved in this criteria. Then, the value distribution of objects in the dataset can be further constructed on the basis of the relevant object values. This helps to ensure that those uncommon, learnable objects that deserve enhancement are more likely to be enhanced. Additionally, we propose to enhance the data by pasting the sampled objects into relatively smooth portions of fresh background images, rather than arbitrary areas of any background images. This helps to boost data diversity while reducing the interference from complicated backgrounds. Evidently, our method does not require sophisticated training and just depends on the size and distribution of the objects in the dataset. Extensive experiments on MS COCO 2017 and PASCAL VOC 2012 demonstrate that our method achieves better performance than state-of-the-art methods.

TA-YOLO: a lightweight small object detection model based on multi-dimensional trans-attention module for remote sensing images

Object detection plays a vital role in remote sensing applications. Although object detection has achieved proud results in natural images, these methods are difficult to be directly applied to remote sensing images. Remote sensing images often have complex backgrounds and small objects, which results in a highly unbalanced distribution of foreground and complex background information. In order to solve the above problems, this paper proposes a multi-head channel and spatial trans-attention (MCSTA) module, which performs remote pixel interaction from the channel and spatial dimensions respectively to complete the attention feature capture function. It is a plug-and-play module that can be easily embedded in any other natural image object detection convolutional neural network, making it quickly applicable to remote sensing images. First, in order to reduce computational complexity and improve feature richness, we use a special linear convolution to obtain three projection features instead of the simple matrix multiplication transformation in Transformer. Second, we obtain trans-attention maps in different dimensions in a manner similar to the self-attention mechanism to capture the interrelationships of features in channels and spaces. In this process, we use a multi-head mechanism to perform parallel operations to improve speed. Furthermore, in order to avoid large-scale matrix operations, we specially designed an attention blocking mode to reduce computer memory usage and increase operation speed. Finally, we embedded the trans-attention module into YOLOv8, added a new detection head and optimized the feature fusion method, thus designing a lightweight small object detection model named TA-YOLO for remote sensing images. It has fewer parameters than the benchmark model YOLOv8, and its mAP on the PASCAL VOC and VisDrone data sets increased by 1.3% and 6.2% respectively. The experimental results prove the powerful function of the trans-attention module and the excellent performance of TA-YOLO.

Bridging knowledge distillation gap for few-sample unsupervised semantic segmentation

Due to privacy, security, and costly labeling of images, unsupervised semantic segmentation with very few samples has become a promising direction, but still remains unexplored. This inspires us to introduce the few-sample unsupervised semantic segmentation task, which is very challenging because generalizing the segmentation model from only a few unlabeled images is far from sufficient. We address this problem in the knowledge distillation perspective, by proposing a medium-sized auxiliary network as the bridge, which narrows down the semantic knowledge gap between teacher network (large) and student network (small). To this end, we develop the Knowledge Distillation Bridge (KDB) framework for few-sample unsupervised semantic segmentation. In particular, it consists of the teacher-auxiliary-student architecture, which adopts the block-wise distillation that encourages the auxiliary to imitate the teacher and the student to imitate the auxiliary. In this way, the knowledge gap between the source feature distribution and the target one is reduced, allowing the student with the smaller network to be readily deployed in highly-demanding environment. Meanwhile, each channel characterizes different semantics in feature map, which motivates us to distill the features of decoder in a channel-wise manner. Extensive experiments on two benchmarks including Pascal VOC2012 and Cityscapes demonstrate the promising performance of the proposed method, which strikes a good balance between precision and speed, e.g., it achieves the inference speed of 230 fps for a 512×512 image.

Graph relationship-driven label coded mapping and compensation for multi-label textile fiber recognition

Recently, neural network-based methods for multi-label textile fiber recognition have achieved considerable success. However, a significant limitation of most current approaches is their disregard for the valuable dependencies that exist among different fiber categories. The universal multi-label image recognition methods that consider label relationships often fall short when applied to the challenge posed by the mixture of fibers in textile images. And these relationship modeling manners have not yet been used in the field of multi-label textile fiber recognition. In this work, a graph relationship-driven method is proposed for the recognition of multi-label textile fibers. Based on the graph attention network, the proposed method introduces global relation graph, label coded mapping, and label compensation to equip the feature learning backbone with the relationship modeling ability. First, feature learning backbone extracts the semantic features from images, and global relation graph mines shallow representations of global label relationships from label embeddings. Then, label coded mapping combines these semantic features and global features to model joint relationships. The obtained joint representations constitute the label code, which is used to map the fiber class indices. Finally, label compensation further extracts the deep representations of global label relationships and utilizes them to compensate the fiber class indices to obtain final prediction scores. By employing experiments on a textile fiber dataset and a rearranged dataset, the effectiveness and superiority of the proposed method are validated. Further experiments on PASCAL VOC 2007 showcase the potential applications of the proposed method beyond textile fiber recognition.

MSO‐DETR: Metric space optimization for few‐shot object detection

AbstractIn the metric‐based meta‐learning detection model, the distribution of training samples in the metric space has great influence on the detection performance, and this influence is usually ignored by traditional meta‐detectors. In addition, the design of metric space might be interfered with by the background noise of training samples. To tackle these issues, we propose a metric space optimisation method based on hyperbolic geometry attention and class‐agnostic activation maps. First, the geometric properties of hyperbolic spaces to establish a structured metric space are used. A variety of feature samples of different classes are embedded into the hyperbolic space with extremely low distortion. This metric space is more suitable for representing tree‐like structures between categories for image scene analysis. Meanwhile, a novel similarity measure function based on Poincaré distance is proposed to evaluate the distance of various types of objects in the feature space. In addition, the class‐agnostic activation maps (CCAMs) are employed to re‐calibrate the weight of foreground feature information and suppress background information. Finally, the decoder processes the high‐level feature information as the decoding of the query object and detects objects by predicting their locations and corresponding task encodings. Experimental evaluation is conducted on Pascal VOC and MS COCO datasets. The experiment results show that the effectiveness of the authors’ method surpasses the performance baseline of the excellent few‐shot detection models.

Distilling Structural Knowledge for Platform-Aware Semantic Segmentation

Knowledge Distillation (KD) aims to distill the dark knowledge of a high-powered teacher network into a student network, which can improve the capacity of student network and has been successfully applied to semantic segmentation. However, the standard knowledge distillation approaches merely represent the supervisory signal of teacher network as the dark knowledge, while ignoring the impact of network architecture during distillation. In this paper, we found that the student network with a more similar architecture against the teacher network obtains more performance gain from distillation. Therefore, a more generalized paradigm for knowledge distillation is to distill both the soft-label and the structure of the teacher network. We propose a novel Structural Distillation (SD) method which introduces the structural similarity constraints into vanilla knowledge distillation. We leverage Neural Architecture Search technique to search optimal student structure for semantic segmentation from a well-designed search space, which mimics the given teacher both in terms of soft-label and network structure. Experiment results make clear that our proposed method outperforms both the NAS with conventional Knowledge Distillation and human-designed methods, and achieves sota performance on the Cityscapes dataset under various platform-aware latency constraints. Furthermore, the best architecture discovered on Cityscapes also transfers well to the PASCAL VOC2012 dataset.

Weakly supervised semantic segmentation based on superpixel affinity

Weakly supervised semantic segmentation (WSSS) usually employs the method of modifying and extending class activation map (CAM) seeds to achieve semantic segmentation. However, the down-sampling of the network weakens the edge awareness of CAMs, leading to under- or over-activation problems, which affects the segmentation quality. Considering the excellent contour attachment property of superpixels and the high semantic similarity between pixels within the same superpixel, we propose a superpixel affinity-based method that uses multi-scale features to aggregate superpixels with the same semantics, providing complete localization supervision for the generation of CAMs. In order to improve the accuracy of semantic labels for superpixels, we utilize a method of deep feature reorganization to improve the quality of network-generated CAM seeds. The experimental results indicate that the proposed method has achieved satisfactory performance on PASCAL VOC 2012 and MS COCO 2014 datasets.

Positional normalization-based mixed-image data augmentation and ensemble self-distillation algorithm

The prior approach to data augmentation using positional normalization necessitated manual determination of weight parameters for blending two images. However, theoretically, these parameters cannot cater to all samples. We propose a novel data augmentation approach called pnMix (Positional normalization-based data mixing) based on data blending and positional normalization to address these issues. Moreover, we have improved the teacher model of an existing network structure based on self-distillation technology using an ensembling approach called ESKD. Our method enhances performance across multiple datasets and tasks. We outperform previous methods in complex real-world datasets such as imagenet-a and imagenet-o, which we believe are highly meaningful for practical applications. Specifically, we obtain good results on the CIFAR100 and Imagenet image classification datasets, the Pascal VOC and COCO object detection datasets, and the COCO and Cityscapes image segmentation datasets. Furthermore, we analyze the pnMix method's advantages compared to other data augmentation methods through visualization techniques, demonstrating its effectiveness and interpretability. Our source code and pre-trained models are available at https://github.com/sydney72380/pnMix.

Research on Image Classification And Semantic Segmentation Model Based on Convolutional Neural Network

This paper investigates convolutional neural network (CNN)-based approaches for image classification and semantic segmentation, with a focus on addressing spatial detail loss and multi-scale feature integration issues prevalent in semantic segmentation. The introduced EDNET model tackles these challenges through the incorporation of spatial information branches and the design of efficient feature fusion mechanisms. It further enhances performance via the use of global pooling and boundary refinement modules. Evaluations on the PASCAL VOC 2012 dataset reveal an 11.67% increase in mean intersection-over-union (IoU) compared to standard fully convolutional networks, demonstrating substantial improvement over comparable techniques. These results confirm the efficacy and practicality of the EDNET framework.