VOC Dataset Research Articles

TS-CAM: Token Semantic Coupled Attention Map for Weakly Supervised Object Localization.

Weakly supervised object localization (WSOL), which trains object localization models using solely image category annotations, remains a challenging problem. Existing approaches based on convolutional neural networks (CNNs) tend to miss full object extent while activating discriminative object parts. Based on our analysis, this is caused by CNN's intrinsic characteristics, which experiences difficulty to capture object semantics at long distances. In this article, we introduce the vision transformer to WSOL, with the aim to capture long-range semantic dependency of features by leveraging transformer's cascaded self-attention mechanism. We propose the token semantic coupled attention map (TS-CAM) method, which first decomposes class-aware semantics and then couples the semantics with attention maps for semantic-aware activation. To capture object semantics at long distances and avoid partial activation, TS-CAM performs spatial embedding by partitioning an image to a set of patch tokens. To incorporate object category information to patch tokens, TS-CAM reallocates category-related semantics to each patch token. The patch tokens are finally coupled with attention maps which are semantic-agnostic to perform semantic-aware object localization. By introducing semantic tokens to produce semantic-aware attention maps, we further explore the capability of TS-CAM for multicategory object localization. Experiments show that TS-CAM outperforms its CNN-CAM counterpart by 11.6% and 28.9% on ILSVRC and CUB-200-2011 datasets, respectively, improving the state-of-the-art with large margins. TS-CAM also demonstrates superiority for multicategory object localization on the Pascal VOC dataset. The code is available at github.com/yuanyao366/ts-cam-extension.

Research and Application of the Median Filtering Method in Enhancing the Imperceptibility of Perturbations in Adversarial Examples

In the field of object detection, the adversarial attack method based on generative adversarial network efficiently generates adversarial examples, thereby significantly reducing time costs. However, this approach overlooks the imperceptibility of perturbations in adversarial examples, resulting in poor visual performance and insufficient invisibility of the generated adversarial examples. To further enhance the imperceptibility of perturbations in adversarial examples, a method utilizing median filtering is proposed to address these generated perturbations. Experimental evaluations were conducted on the Pascal VOC dataset. The results demonstrate that, compared to the original image, there is an increase of at least 17.2% in the structural similarity index (SSIM) for generated adversarial examples. Additionally, the peak signal-to-noise ratio (PSNR) increases by at least 27.5%, while learned perceptual image patch similarity (LPIPS) decreases by at least 84.6%. These findings indicate that the perturbations in generated adversarial examples are more difficult to detect, with significantly improved imperceptibility and closer resemblance to the original image without compromising their high aggressiveness.

DFFNet: a lightweight approach for efficient feature-optimized fusion in steel strip surface defect detection

Steel surface defect detection is crucial in manufacturing, but achieving high accuracy and real-time performance with limited computing resources is challenging. To address this issue, this paper proposes DFFNet, a lightweight fusion network, for fast and accurate steel surface defect detection. Firstly, a lightweight backbone network called LDD is introduced, utilizing partial convolution to reduce computational complexity and extract spatial features efficiently. Then, PANet is enhanced using the Efficient Feature-Optimized Converged Network and a Feature Enhancement Aggregation Module (FEAM) to improve feature fusion. FEAM combines the Efficient Layer Aggregation Network and reparameterization techniques to extend the receptive field for defect perception, and reduce information loss for small defects. Finally, a WIOU loss function with a dynamic non-monotonic mechanism is designed to improve defect localization in complex scenes. Evaluation results on the NEU-DET dataset demonstrate that the proposed DFFNet achieves competitive accuracy with lower computational complexity, with a detection speed of 101 FPS, meeting real-time performance requirements in industrial settings. Furthermore, experimental results on the PASCAL VOC and MS COCO datasets demonstrate the strong generalization capability of DFFNet for object detection in diverse scenarios.

Dynamic YOLO for small underwater object detection

The practical application of object detection inevitably encounters challenges posed by small objects. In underwater object detection, a crucial method for marine exploration, the presence of small objects in underwater environments significantly hampers the performance of detection. In this paper, a dynamic YOLO detector is proposed as a solution to alleviate this problem. Specifically, a light-weight backbone network is first constructed based on deformable convolution v3, with some specialized designs for small object detection. Secondly, a unified feature fusion framework based on channel-wise, scale-wise, and spatial-aware attention is proposed to fuse feature maps from different scales. This is particularly critical for detecting small objects since it allows us to fully exploit the enhanced capabilities offered by our proposed backbone network. Finally, a simple but effective detection head is designed to handle the conflict between classification and localization by disentangling and aligning the two tasks. Extensive experiments are conducted on benchmark datasets to demonstrate the effectiveness of the proposed model. Without bells and whistles, dynamic YOLO outperforms the recent state-of-the-art methods by a large margin of +0.8\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$+\\,0.8$$\\end{document} AP and +1.8\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$+\\,1.8$$\\end{document} APS\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\ ext {AP}_{S}$$\\end{document} on the DUO dataset. Experimental results on Pascal VOC and MS COCO datasets also demonstrate the superiority of the proposed method. At last, ablation studies are conducted on DUO dataset to validate the effectiveness and efficiency of each design in dynamic YOLO. Source code will be available at https://github.com/chenjie04/Dynamic-YOLO.

Knowledge Distillation Based on Narrow-Deep Networks

Deep neural networks perform better than shallow neural networks, but the former tends to be deeper or wider, introducing large numbers of parameters and computations. We know that networks that are too wide have a high risk of overfitting and networks that are too deep require a large amount of computation. This paper proposed a narrow-deep ResNet, increasing the depth of the network while avoiding other issues caused by making the network too wide, and used the strategy of knowledge distillation, where we set up a trained teacher model to train an unmodified, wide, and narrow-deep ResNet that allows students to learn the teacher’s output. To validate the effectiveness of this method, it is tested on Cifar-100 and Pascal VOC datasets. The method proposed in this paper allows a small model to have about the same accuracy rate as a large model, while dramatically shrinking the response time and computational effort.

One stage multi-scale efficient network for underwater target detection.

Due to the complexity of the underwater environment, existing methods for underwater target detection present low precision on small or dense targets. To address these issues, a novel method is proposed for underwater target detection based on YOLOv5s (You Only Look Once version 5 small), which aims to improve the precision and robustness. In this study, an efficient feature extraction network is introduced to extract significant features, and a novel attention mechanism with deformable convolution is designed to improve the feature representation. Subsequently, an adaptive spatial fusion operation is introduced at the neck of YOLOv5s to facilitate feature fusion from various layers. By integrating low-level features with high-level features, the adaptive fusion feature pyramid network effectively integrates global semantic information and decreases the semantic gap between features from various layers, contributing to the high detection precision. Comprehensive experiments demonstrate that the proposed method achieves an mAP50 of 86.97% on the Underwater Robot Professional Contest of China 2020 dataset, 3.07% higher than YOLOv5s. Furthermore, the proposed method achieves a detection precision of 76.0% on the PASCAL VOC2007 dataset, surpassing several outstanding methods.

Towards Stabilized Few-Shot Object Detection with Less Forgetting via Sample Normalization.

Few-shot object detection is a challenging task aimed at recognizing novel classes and localizing with limited labeled data. Although substantial achievements have been obtained, existing methods mostly struggle with forgetting and lack stability across various few-shot training samples. In this paper, we reveal two gaps affecting meta-knowledge transfer, leading to unstable performance and forgetting in meta-learning-based frameworks. To this end, we propose sample normalization, a simple yet effective method that enhances performance stability and decreases forgetting. Additionally, we apply Z-score normalization to mitigate the hubness problem in high-dimensional feature space. Experimental results on the PASCAL VOC data set demonstrate that our approach outperforms existing methods in both accuracy and stability, achieving up to +4.4 mAP@0.5 and +5.3 mAR in a single run, with +4.8 mAP@0.5 and +5.1 mAR over 10 random experiments on average. Furthermore, our method alleviates the drop in performance of base classes. The code will be released to facilitate future research.

Object detection on low-resolution images with two-stage enhancement

Although deep learning-based object detection methods have achieved superior performance on conventional benchmark datasets, it is still difficult to detect objects from low-resolution (LR) images under diverse degradation conditions. To this end, a two-stage enhancement method for the LR image object detection (TELOD) framework is proposed. In the first stage, an extremely lightweight task disentanglement enhancement network (TDEN) is developed as a super-resolution (SR) sub-network before the detector. In the TDEN, the SR images can be obtained by applying the recurrent connection manner between an image restoration branch (IRB) and a resolution enhancement branch (REB) to enhance the input LR images. Specifically, the TDEN reduces the difficulty of image reconstruction by dividing the total image enhancement task into two sub-tasks, which are accomplished by the IRB and REB, respectively. Furthermore, a shared feature extractor is applied across two sub-tasks to explore common and accurate feature representations. In the second stage, an auxiliary feature enhancement head (AFEH) driven by high-resolution (HR) image priors is designed to improve the task-specific features produced by the detection Neck without any extra inference costs. In particular, the feature interaction module is built into the AFEH to integrate the features from the enhancement and detection phases to learn comprehensive information for detection. Extensive experiments show that the proposed TELOD significantly outperforms other methods. Specifically, the TELOD achieves mAP improvements of 1.8% and 3.3% over the second best method AERIS on degraded VOC and COCO datasets, respectively.

Small object intelligent Detection method based on Adaptive Cascading Context

With the technology advances, deep learning-based object detection has made unprecedented progress. However, the small spatial ratio of object pixels affects the effective extraction of deep details features, resulting in poor detection results in small object detection. To improve the accuracy of small object detection, an adaptive Cascading Context small (ACC) object detection method is proposed based on YOLOv5. Firstly, a separate shallow layer feature was proposed to obtain more detailed information beneficial to small object detection. Secondly, an adaptive cascade method is proposed to fuse the output features of the three layers of the pyramid to adaptively filter negative semantic information, while fusing with shallow features to solve the problem of low classification accuracy caused by insufficient semantic information of shallow features. Finally, an adaptive context model is proposed to use a deformable convolution to obtain spatial context features of shallow small objects, associating the targets with the background, thereby improving the accuracy of small object detection. The experimental results show that the detection accuracy of the proposed method has been improved by 6.12%, 3.35%, 3.33%, and 5.2%, respectively, compared with the source code on the PASCAL VOC, NWPU VHR-10, KITTI, and RSOD datasets, which fully demonstrate the effectiveness of our method in small object detection.

A Few-Shot Object Detection Method for Endangered Species

Endangered species detection plays an important role in biodiversity conservation and is significant in maintaining ecological balance. Existing deep learning-based object detection methods are overly dependent on a large number of supervised samples, and building such endangered species datasets is usually costly. Aiming at the problems faced by endangered species detection, such as low accuracy and easy loss of location information, an efficient endangered species detection method with fewer samples is proposed to extend the few-shot object detection technique to the field of endangered species detection, which requires only a small number of training samples to obtain excellent detection results. First, SE-Res2Net is proposed to optimize the feature extraction capability. Secondly, an RPN network with multiple attention mechanism is proposed. Finally, for the classification confusion problem, a weighted prototype-based comparison branch is introduced to construct weighted category prototype vectors, which effectively improves the performance of the original classifier. Under the setting of 30 samples in the endangered species dataset, the average detection accuracy value of the method, mAP50, reaches 76.54%, which is 7.98% higher than that of the pre-improved FSCE method. This paper also compares the algorithm on the PASCOL VOC dataset, which is optimal and has good generalization ability compared to the other five algorithms.

Weakly supervised instance segmentation via class double-activation maps and boundary localization

Weakly supervised instance segmentation based on image-level class labels has recently gained much attention, in which the primary key step is to generate the pseudo labels based on class activation maps (CAMs). Most methods adopt binary cross-entropy (BCE) loss to train the classification model. However, since BCE loss is not class mutually exclusive, activations among classes occur independently. Thus, not only do foreground classes are wrongly activated as background, but also incorrect activations among confusing classes are occurred in the foreground. To solve this problem, we propose the Class Double-Activation Map, called Double-CAM. Firstly, the vanilla CAM is extracted from the multi-label classifier and then fused with the output feature map of backbone. The enhanced feature map of each class is fed into the single-label classification branch with softmax cross-entropy (SCE) loss and entropy minimization module, from which the more accurate Double-CAM is extracted. It refines the vanilla CAM to improve the quality of pseudo labels. Secondly, to mine object edge cues from Double-CAM, we propose the Boundary Localization (BL) module to synthesize boundary annotations, so as to provide constraints for label propagation more explicitly without adding additional supervision. The quality of pseudo masks is also improved substantially with the addition of BL module. Finally, the generated pseudo labels are used to train fully supervised instance segmentation networks. The evaluations on VOC and COCO datasets show that our method achieves excellent performance, outperforming mainstream weakly supervised segmentation methods at the same supervisory level, even those that depend on stronger supervision.

TA-YOLO: a lightweight small object detection model based on multi-dimensional trans-attention module for remote sensing images

Object detection plays a vital role in remote sensing applications. Although object detection has achieved proud results in natural images, these methods are difficult to be directly applied to remote sensing images. Remote sensing images often have complex backgrounds and small objects, which results in a highly unbalanced distribution of foreground and complex background information. In order to solve the above problems, this paper proposes a multi-head channel and spatial trans-attention (MCSTA) module, which performs remote pixel interaction from the channel and spatial dimensions respectively to complete the attention feature capture function. It is a plug-and-play module that can be easily embedded in any other natural image object detection convolutional neural network, making it quickly applicable to remote sensing images. First, in order to reduce computational complexity and improve feature richness, we use a special linear convolution to obtain three projection features instead of the simple matrix multiplication transformation in Transformer. Second, we obtain trans-attention maps in different dimensions in a manner similar to the self-attention mechanism to capture the interrelationships of features in channels and spaces. In this process, we use a multi-head mechanism to perform parallel operations to improve speed. Furthermore, in order to avoid large-scale matrix operations, we specially designed an attention blocking mode to reduce computer memory usage and increase operation speed. Finally, we embedded the trans-attention module into YOLOv8, added a new detection head and optimized the feature fusion method, thus designing a lightweight small object detection model named TA-YOLO for remote sensing images. It has fewer parameters than the benchmark model YOLOv8, and its mAP on the PASCAL VOC and VisDrone data sets increased by 1.3% and 6.2% respectively. The experimental results prove the powerful function of the trans-attention module and the excellent performance of TA-YOLO.

MSO‐DETR: Metric space optimization for few‐shot object detection

AbstractIn the metric‐based meta‐learning detection model, the distribution of training samples in the metric space has great influence on the detection performance, and this influence is usually ignored by traditional meta‐detectors. In addition, the design of metric space might be interfered with by the background noise of training samples. To tackle these issues, we propose a metric space optimisation method based on hyperbolic geometry attention and class‐agnostic activation maps. First, the geometric properties of hyperbolic spaces to establish a structured metric space are used. A variety of feature samples of different classes are embedded into the hyperbolic space with extremely low distortion. This metric space is more suitable for representing tree‐like structures between categories for image scene analysis. Meanwhile, a novel similarity measure function based on Poincaré distance is proposed to evaluate the distance of various types of objects in the feature space. In addition, the class‐agnostic activation maps (CCAMs) are employed to re‐calibrate the weight of foreground feature information and suppress background information. Finally, the decoder processes the high‐level feature information as the decoding of the query object and detects objects by predicting their locations and corresponding task encodings. Experimental evaluation is conducted on Pascal VOC and MS COCO datasets. The experiment results show that the effectiveness of the authors’ method surpasses the performance baseline of the excellent few‐shot detection models.

Distilling Structural Knowledge for Platform-Aware Semantic Segmentation

Knowledge Distillation (KD) aims to distill the dark knowledge of a high-powered teacher network into a student network, which can improve the capacity of student network and has been successfully applied to semantic segmentation. However, the standard knowledge distillation approaches merely represent the supervisory signal of teacher network as the dark knowledge, while ignoring the impact of network architecture during distillation. In this paper, we found that the student network with a more similar architecture against the teacher network obtains more performance gain from distillation. Therefore, a more generalized paradigm for knowledge distillation is to distill both the soft-label and the structure of the teacher network. We propose a novel Structural Distillation (SD) method which introduces the structural similarity constraints into vanilla knowledge distillation. We leverage Neural Architecture Search technique to search optimal student structure for semantic segmentation from a well-designed search space, which mimics the given teacher both in terms of soft-label and network structure. Experiment results make clear that our proposed method outperforms both the NAS with conventional Knowledge Distillation and human-designed methods, and achieves sota performance on the Cityscapes dataset under various platform-aware latency constraints. Furthermore, the best architecture discovered on Cityscapes also transfers well to the PASCAL VOC2012 dataset.

Positional normalization-based mixed-image data augmentation and ensemble self-distillation algorithm

The prior approach to data augmentation using positional normalization necessitated manual determination of weight parameters for blending two images. However, theoretically, these parameters cannot cater to all samples. We propose a novel data augmentation approach called pnMix (Positional normalization-based data mixing) based on data blending and positional normalization to address these issues. Moreover, we have improved the teacher model of an existing network structure based on self-distillation technology using an ensembling approach called ESKD. Our method enhances performance across multiple datasets and tasks. We outperform previous methods in complex real-world datasets such as imagenet-a and imagenet-o, which we believe are highly meaningful for practical applications. Specifically, we obtain good results on the CIFAR100 and Imagenet image classification datasets, the Pascal VOC and COCO object detection datasets, and the COCO and Cityscapes image segmentation datasets. Furthermore, we analyze the pnMix method's advantages compared to other data augmentation methods through visualization techniques, demonstrating its effectiveness and interpretability. Our source code and pre-trained models are available at https://github.com/sydney72380/pnMix.

Fish detection based on Gather-and-Distribute mechanism Multi-scale feature fusion network and Structural Re-parameterization method

ABSTRACT To solve the problems of localization and identification of fish in the complex fishway environment, improving the accuracy of fish detection, this paper proposes an object detection algorithm YOLORG, and a fishway fish detection dataset (FFDD). The FFDD contains 4,591 images from the web and lab shots and labeled with the LabelIMG tool, covering fish in a wide range of complex scenarios. The YOLORG algorithm, based on YOLOv8, improves the traditional FPN–PAN network into a C2f Multi-scale feature fusion network with a Gather-and-Distribute mechanism, which solves the problem of information loss accompanied by the network in the fusion of feature maps of different sizes. Also, we propose a C2D Structural Re-parameterization module with a rich gradient flow and good performance to further improve the detection accuracy of the algorithm. The experimental results show that the YOLORG algorithm improves the mAP50 and mAP50-95 by 1.2 and 1.8% compared to the original network under the joint VOC dataset, and also performs very well in terms of accuracy compared to other state-of-the-art object detection algorithms, and is able to detect fish in very turbid environments after training on the FFDD.

Adjustable patch and feature prior token-based transformer for weakly supervised semantic segmentation

Weakly supervised semantic segmentation is a challenging task, utilizing only low-cost weak supervision to produce pixel-level predictions. Existing transformer-based methods for weakly supervised semantic segmentation have some limitations: (1) Fixing patch size might destroy the structured semantics, which is unfriendly to objects of different scales, and (2) Ignoring the prior features when using multi-head attention mechanisms might lead to inaccurate segmentation localization. To tackle these issues, we proposed an effective transformer framework coupled with the adjustable patch and prior feature tokens, termed as APFPformer, in which an adjustable patch module is developed to split the image according to the area of the salient object for preserving the structured semantics in patches. A prior feature token module is devised to exploit the edge and texture information as prior tokens, ensuring the gain of discriminative representation. Additionally, a single-stage scheme is applied to reduce the computation and rapid segmentation process. Our experiments demonstrate the superiority of our approach over early methods, gaining competitive mean Intersection-over-Union scores of 67.9% on the PASCAL VOC2012 dataset and 40.2% on the MS COCO dataset.

Weakly-supervised Incremental learning for Semantic segmentation with Class Hierarchy

Although current semantic segmentation approaches have achieved impressive performance, their ability to incrementally learn new classes is limited. Moreover, pixel-by-pixel annotations are costly and time-consuming. Therefore, a new field called Weakly-supervised Incremental Learning for Semantic Segmentation (WILSS) has emerged, which learns new classes using image-level labels. However, image-level labels do not provide sufficient detail, and we discover that the state-of-the-art of WILSS suffers from confusion between old knowledge and new knowledge. To address this issue, we propose Weakly-supervised Incremental learning for Semantic segmentation with Class Hierarchy (WISH), a method that considers the hierarchical structure of each class when determining which knowledge to trust in cases of confusion between old and new knowledge. Our method has achieved new state-of-the-art performances in all settings compared to the previous methods on the Pascal VOC and MS COCO datasets.

PIAENet: Pyramid integration and attention enhanced network for object detection

Object detection is a challenging task that requires a trade-off between accuracy and efficiency. Previous approaches have focused mainly on optimizing one aspect at the expense of the other, making them unsuitable for resource-constrained devices. To address this issue, we propose a new object detection network architecture, the Pyramid Integration and Attention Enhanced Network (PIAENet). PIAENet is a lightweight architecture that can achieve high accuracy and efficiency. We utilize a lightweight EfficientNet-B2 backbone for feature extraction to maintain accuracy while reducing computational overhead. The core components of PIAENet, the Pyramid Integration Module (PIM) and the Attention Enhanced Module (AEM), work together to improve the performance of object detection. PIM fuses multi-scale features using multiple branches to enhance the receptive field of the model, while AEM strengthens the fusion of features using two attention mechanisms to suppress the influence of irrelevant information. Our proposed method has been evaluated on the PASCAL VOC and KITTI datasets. The results have shown our method outperforms most of the existing state-of-the-art methods.

MBAN: multi-branch attention network for small object detection.

Recent years small object detection has seen remarkable advancement. However, small objects are difficult to accurately detect in complex scenes due to their low resolution. The downsampling operation inevitably leads to the loss of information for small objects. In order to solve these issues, this article proposes a novel Multi-branch Attention Network (MBAN) to improve the detection performance of small objects. Firstly, an innovative Multi-branch Attention Module (MBAM) is proposed, which consists of two parts, i.e. Multi-branch structure consisting of convolution and maxpooling, and the parameter-free SimAM attention mechanism. By combining these two parts, the number of network parameters is reduced, the information loss of small objects is reduced, and the representation of small object features is enhanced. Furthermore, to systematically solve the problem of small object localization, a pre-processing method called Adaptive Clustering Relocation (ACR) is proposed. To validate our network, we conducted extensive experiments on two benchmark datasets, i.e. NWPU VHR-10 and PASCAL VOC. The findings from the experiment demonstrates the significant performance gains of MBAN over most existing algorithms, the mAP of MBAN achieved 96.55% and 84.96% on NWPU VHR-10 and PASCAL VOC datasets, respectively, which proves that MBAN has significant performance in small object detection.