PASCAL VOC Research Articles

Non-target feature filtering for weakly supervised semantic segmentation

Weakly supervised semantic segmentation (WSSS) utilizes weak labels to learn semantic segmentation models, significantly reducing reliance on pixel-level annotations. WSSS typically employs a multi-label classification network to extract image features for constructing localization maps. The quality of the localization map critically influences the performance of WSSS. However, non-target semantic noise within the features impedes the improvement of localization map quality. To address this issue, we propose a non-target feature filtering class activation mapping (NFF-CAM) for WSSS, which can reduce non-target semantic signals and generate higher-quality localization maps. Specifically, the class-constrained dual cosine clustering (CDCC) and channel identification (CI) modules are introduced in NFF-CAM. CDCC effectively addresses the issue of unsuitability in the clustering group relationships of the original features under specified class conditions. CI can efficiently identify channel features containing non-target semantic information. We conduct extensive evaluations of NFF-CAM on popular datasets, including PASCAL VOC 2012 and MS COCO 2014. Experimental results show that NFF-CAM can effectively improve the segmentation performance of off-the-shelf methods.

Multi-level similarity transfer and adaptive fusion data augmentation for few-shot object detection

Few-shot object detection method aims to learn novel classes through a small number of annotated novel class samples without having a catastrophic impact on previously learned knowledge, thereby expanding the trained model’s ability to detect novel classes. For existing few-shot object detection methods, there is a prominent false positive issue for the novel class samples due to the similarity in appearance features and feature distribution between the novel classes and the base classes. That is, the following two issues need to be solved: (1) How to detect these false positive samples in large-scale dataset, and (2) How to utilize the correlations between these false positive samples and other samples to improve the accuracy of the detection model. To address the first issue, an adaptive fusion data augmentation strategy is utilized to enhance the diversity of novel class samples and further alleviate the issue of false positive novel class samples. To address the second issue, a similarity transfer strategy is here proposed to effectively utilize the correlations between different categories. Experimental results demonstrate that the proposed method performs well in various settings of PASCAL VOC and MSCOCO datasets, achieving 48.7 and 11.3 on PASCAL VOC and MSCOCO under few-shot settings (shot = 1) in terms of nAP50 respectively.

MCTformer+: Multi-Class Token Transformer for Weakly Supervised Semantic Segmentation.

This paper proposes a novel transformer-based framework to generate accurate class-specific object localization maps for weakly supervised semantic segmentation (WSSS). Leveraging the insight that the attended regions of the one-class token in the standard vision transformer can generate class-agnostic localization maps, we investigate the transformer's capacity to capture class-specific attention for class-discriminative object localization by learning multiple class tokens. We present the Multi-Class Token transformer, which incorporates multiple class tokens to enable class-aware interactions with patch tokens. This is facilitated by a class-aware training strategy that establishes a one-to-one correspondence between output class tokens and ground-truth class labels. We also introduce a Contrastive-Class-Token (CCT) module to enhance the learning of discriminative class tokens, enabling the model to better capture the unique characteristics of each class. Consequently, the proposed framework effectively generates class-discriminative object localization maps from the class-to-patch attentions associated with different class tokens. To refine these localization maps, we propose the utilization of patch-level pairwise affinity derived from the patch-to-patch transformer attention. Furthermore, the proposed framework seamlessly complements the Class Activation Mapping (CAM) method, yielding significant improvements in WSSS performance on PASCAL VOC 2012 and MS COCO 2014. These results underline the importance of the class token for WSSS.

Orthogonal Progressive Network for Few-shot Object Detection

Few-Shot Object Detection (FSOD) is a significant application of few-shot learning in object detection tasks. Its primary objective is to enable the model to quickly acquire the ability to detect novel categories through a limited number of annotated samples. However, such a small number of training samples is insufficient for the model to be fully trained, which leads to difficulties in completely decoupling the regression and classification tasks, and confusion among categories. To address this issue, this paper proposes to remap features into location and class spaces. By imposing task orthogonality and class orthogonality constraints on the fine-tuning process, task decoupling and class decoupling are achieved. Furthermore, we design and implement a progressive fine-tuning strategy that iteratively optimizes in a lagged and progressive manner through the combination of training samples and historical features, mitigating the risks of catastrophic forgetting and overfitting. Experimental results demonstrate that our method effectively improves the model’s performance in FSOD tasks, achieving state-of-the-art metrics on the PASCAL VOC and MS COCO benchmarks.

Cross-domain fine grained strip steel defect detection method based on semi-supervised learning and Multi-head Self Attention coordination

The identification of steel strip defects plays a pivotal role in assessing steel quality and advancing production technology. However, the majority of intelligent defect recognition algorithms for steel strips, based on deep learning, primarily focus on supervised learning. These methods depend on a multitude of training samples, incurring additional manual labelling costs, and exhibit low recognition efficiency. In contrast to supervised learning, we integrate the fine-grained characteristics of strip defects. We propose a cross-domain, fine-grained strip defect detection method based on semi-supervised learning and Multi-head self-attention coordination, along with an improvement strategy, resulting in a novel network structure: Multi-head Self Attention and Semi-supervised collaborative detection network (MSD Net). This method initiates the cross-domain migration of defect samples through Cycle Generative Adversarial Networks (Cycle GAN), creating new semi-supervised training samples from source domain and target domain data to enhance data distribution diversity. The detection model is then constructed leveraging the advantages of Multi-head Self Attention (MSA) in augmenting the global receptive field of feature extraction. The proposed semi-supervised learning method employs a pseudo-label allocation strategy to guide the model in fully utilizing the distribution fitting of unlabeled samples. This allows the deep neural network to learn a more comprehensive multivariate data distribution within the training set, thereby enhancing the generalization ability of the semi-supervised model. Experimental results on the benchmark dataset for steel strip defect detection demonstrate that the cross-domain semi-supervised method achieves a test accuracy of 96.1 % on mAP@0.5, surpassing the supervised baseline model by 4.8 %. Our method also outperforms the baseline supervised model in the accuracy of small target recognition on PASCAL VOC 2007 datasets. Additionally, we have implemented a strip defect detection system based on edge computing for real-time deployment of the proposed algorithm. Testing in an actual industrial setting further validates the efficacy of our proposed method in practical applications. Our work encourages further exploration, the task of public datasets can be obtained at https://github.com/songzhiweiknight/NEU-DET-Datasets.git.

DMTN-Net: Semantic Segmentation Architecture for Surface Unmanned Vessels

Aiming at the problems of insufficient navigation area recognition accuracy, fuzzy boundary of obstacle segmentation, and high consumption of computational resources in the autonomous navigation of water navigation sensors, such as USVs, this paper proposes a DMTN-Net network architecture based on DeeplabV3+ to improve the accuracy and efficiency of environment sensing. Firstly, DMTN-Net adopts the lightweight MobileNetV2 as the backbone, which reduces the amount of computation. Secondly, the innovative N-Decoder structure integrates cSE and Triplet Attention, which enhances the feature representation and improves the segmentation performance. Finally, various experiments were conducted on the MassMind dataset, Pascal VOC2007 dataset, and related sea areas. The experimental results show that DMTN-Net performs well on MassMind and Pascal VOC2007 datasets, and compared with other mainstream networks, the indexes of mIoU, mPA, and mPrecision are significantly improved, and the computational cost is greatly reduced. In addition, the offshore navigation experiments further validate its performance advantages and provide solid support for the practicalization of USV waterborne sensors.

Text-Guided Unknown Pseudo-Labeling for Open-World Object Detection

Open-world object detection (OWOD) focuses on training models with partially known class labels, enabling the detection of objects from known classes while concurrently identifying objects from unknown classes. Current models often perform suboptimally in generating pseudo-labels for unknown objects based on objectness scores due to inherent biases towards known classes. To address this issue, we propose a cross-modal learning model named Text-Guided Unknown Pseudo-Labeling for Open-world Object Detection(TGOOD) building on the Featurized Query R-CNN (FQR-CNN) framework. Specifically, we introduce a module called Similarity-Random-Similarity (SRS) to guide the model in detecting unknown objects during training. Additionally, we replace the one-to-one label assignment strategy in FQR-CNN with a one-to-many (OTM) label assignment strategy to provide more supervisory information during training. Moreover, we propose the ROI features Refinement Module (RRM) to enhance the discriminability of all objects. Experimental evaluations on the PASCAL VOC, MS-COCO, and COCO-O benchmarks demonstrate TGOOD’s superior open-world detection capability.

Lightweight enhanced YOLOv8n underwater object detection network for low light environments

In response to the challenges of target misidentification, missed detection, and other issues arising from severe light attenuation, low visibility, and complex environments in current underwater target detection, we propose a lightweight low-light underwater target detection network, named PDSC-YOLOv8n. Firstly, we enhance the input images using the improved Pro MSRCR algorithm for data augmentation. Secondly, we replace the traditional convolutions in the backbone and neck networks of YOLOv8n with Ghost and GSConv modules respectively to achieve lightweight network modeling. Additionally, we integrate the improved DCNv3 module into the C2f module of the backbone network to enhance the capability of target feature extraction. Furthermore, we introduce the GAM into the SPPF and incorporate the CBAM attention mechanism into the last layer of the backbone network to enhance the model’s perceptual and generalization capabilities. Finally, we optimize the training process of the model using WIoUv3 as the loss function. The model is successfully deployed on an embedded platform, achieving real-time detection of underwater targets on the embedded platform. We first conduct experiments on the RUOD underwater dataset. Meanwhile, we also utilized the Pascal VOC2012 dataset to evaluate our approach. The mAP@0.5 and mAP@0.5:0.95 of the original YOLOv8n algorithm on RUOD dataset were 79.6% and 58.2%, respectively, and the PDSC -YOLOv8n algorithm mAP@0.5 and mAP@0.5:0.95 can reach 86.1% and 60.8%. The number of parameters of the model is reduced by about 15.5%, the detection accuracy is improved by 6.5%. The original YOLOv8n algorithm was 73% and 53.2% mAP@0.5 and mAP@0.5:0.95 on the Pascal VOC dataset, respectively. The PDSC-YOLOv8n algorithm mAP@0.5 and mAP@0.5:0.95 were 78.5% and 57%, respectively. The superior performance of PDSC-YOLOv8n indicates its effectiveness in the field of underwater target detection.

Semantic Segmentation Based on Geometric Calibration Using AI and AR in Health Care

In the medical field, medical imaging is essential for precise diagnosis, treatment planning, and condition monitoring. The goal of this work is to improve the field of healthcare imaging by investigating the complementary applications of augmented reality (AR) and artificial intelligence (AI) in semantic segmentation. More precise identification and delineation of anatomical structures and abnormalities in medical images is made possible by AI-driven semantic segmentation, opening the door to more precise diagnosis and treatment plans. Modern AI algorithms are employed in the suggested methodology to perform semantic segmentation in a variety of medical imaging modalities, including ultrasound, CT, and MRI scans. The detection of anomalies such as tumours, irregularities in organs, and neurological disorders is made easier by these algorithms. The foundation for integrating augmented reality technologies into the healthcare ecosystem is provided by the segmented medical images. AR enhances the interaction and visualization of segmented medical data in the context of healthcare. Real-time augmented reality overlays help surgeons by improving surgical navigation and precision. Additionally, AR apps help with medical education by offering professionals and students alike an immersive learning environment. AR provides interactive visualizations to help patients understand their medical conditions when they are going for therapy and rehabilitation. The difficulties in integrating these technologies in the healthcare industry are discussed in this paper, with a focus on the significance of data privacy, regulatory compliance, and easy integration with current healthcare systems. The successful deployment of AI technologies necessitates interdisciplinary collaboration among AI developers, healthcare professionals, and AR specialists to ensure compliance with ethical, legal, and clinical standards mandated in the healthcare domain. The proposed Segmentation mask is redefined with geometric calibration and used with U-Net for image segmentation. The segmentation is experimented on Cityscapes and PASCAL VOC 2012 datasets. The experimental results show that proposed semantic segmentation based on geometric calibration yields more accuracy than its counterparts.

Attention Pyramid Networks for Object Detection With Semantic Information Fusion

Pyramid context information fusion has limitations in distinguishing background and important semantic information. The attention mechanism solves the limitations, but the detection effect for small target detection and occluded object detection is not good. In order to solve the above problems, a target detection method based on attention pyramid information fusion (APIFNet) is proposed. It consists of Dynamic Pyramid Attention Fusion Module (DPAF) and Attention Semantic Contextual Information Module (ACM). First, the DPAF module fuses information at different scales and simultaneously guides the learning of low-level features by high-level features to enhance semantic information and spatial details. Then, in order to detect small targets and occluded objects, the ACM module effectively enhances the detection ability of small targets by emphasizing the importance of foreground contextual semantic information and suppressing unimportant semantic information. Ultimately, APIFNet outperforms other methods in terms of evaluation performance on the COCO and PASCAL VOC data sets.

Refined feature enhancement network for object detection

Convolutional neural networks-based object detection techniques have achieved positive performances. However, due to the limitations of local receptive field, some existing object detection methods cannot effectively capture global information in feature extraction phases, and thus lead to unsatisfactory detection performance. Moreover, the feature information extracted by the backbone network may be redundant. To alleviate these problems, in this paper we propose a refined feature enhancement network (RFENet) for object detection. Specifically, we first propose a feature enhancement module (FEM) to capture more global and local information from feature maps with certain long-range dependencies. We further propose a multi-branch dilated attention mechanism (MDAM) to refine the extracted features in a weighted form, which can select more important spatial and channel information and broaden the receptive field of the network. Finally, we validate RFENet on MS-COCO2017, PASCAL VOC2012, and PASCAL VOC07+12 datasets, respectively. Compared to the baseline network, our RFENet improves by 2.4 AP on MS-COCO2017 dataset, 3.4 mAP on PASCAL VOC2012 dataset, and 2.7 mAP on PASCAL VOC07+12 dataset. Extensive experiments show that our RFENet can perform competitively on different datasets. The code is available at https://github.com/object9detection/RFENet.

YOLO vs CNN : A Comparative Study on Object Detection

Abstract—With the availability of large amount of data and increasing demand of digitizing the visual systems, lot of research on object detection techniques has been the foremost requirement since last ten years. As the technological advancements are pacing at such a higher rate with increasing computational power and new architectures for CNN has already been proposed in a successive manner to enhance the detection techniques. For Real time Object identification, CNN has proposed several architectures but YOLO proves itself better than others based on the speed of identification. This paper presents the brief overview and comparative analysis of various object detection techniques like SSD, RCNN, YOLO and their subsequent versions for different applications. On comparing, it is found that single stage detectors outperform two stage detectors. Keywords—Pascal VOC, MS COCO, YOLO, CNN, Object detection, Object Recognition, SSD.

Infrared target detection algorithm based on multipath coordinate attention mechanism

Abstract The current generation of infrared target detection algorithms frequently exhibits a high degree of dependency on parameter configurations within complex operational environments. This often results in a reduction in detection accuracy, an increase in the number of model parameters, and a slowing of the detection process. To address these limitations, a new algorithm, CGhostNet-Attention-YOLO (CAY), is proposed in this paper. Firstly, we designed a lightweight backbone network, CGhostNet, with the objective of improving feature extraction efficiency, thereby enabling accurate and real-time feature extraction. Furthermore, we proposed a multipath coordinate attention mechanism, which incorporates both channel and positional information, thereby facilitating enhanced context awareness and the comprehension of relationships between different positions. This effectively enhances the model’s ability to comprehend the overall meaning and addresses the issue of missed detections in infrared targets, significantly improving detection accuracy. Moreover, we employed the Inner-SIoU loss function to accelerate model convergence, reduce loss, and enhance the robustness of the model. Finally, comparative experiments were conducted on our dataset (IFD) as well as publicly available datasets, including FLIR, Pascal VOC, and NEU-DET. The results demonstrate that the CAY algorithm achieved a mean Average Precision (mAP@0.5) of 81.3% on the IFD dataset, 86.1% on the FLIR dataset, 79.2% on the Pascal VOC dataset, and 79.9% on the NEU-DET dataset, with a 27% reduction in the number of parameters. These findings validate the feasibility of the proposed algorithm.

YOLO with feature enhancement and its application in intelligent assembly

Object detection is the most important part in intelligent assembly tasks, accurate and fast detection for different targets can complete positioning and assembly tasks more automatically and efficiently. In this paper, a feature enhancement object detection model based on YOLO is proposed. Firstly, the expression ability of feature layer is enhanced through RFP (Recursive Feature Pyramid) structure. The ARSPP (Atrous Residual Spatial Pyramid Pooling) is proposed to have a further enhancement for the feature layers output by the backbone network, it improves the recognition performance for multi-scale targets of model by using different size of dilated convolution and residual connection. Finally, the contiguous pyramid features are fused and enhanced through the attention mechanism, the results are used for the input of next recursive or predictive output. The model proposed in this paper effectively improves the detection accuracy of YOLO, it has 3% MAP improvement in PASCAL VOC dataset. The validity and accuracy of the model are verified in the robot intelligent assembly recognition task.

Illuminating Salient Contributions in Neuron Activation with Attribution Equilibrium.

With the remarkable success of deep neural networks, there is a growing interest in research aimed at providing clear interpretations of their decision-making processes. In this paper, we introduce Attribution Equilibrium, a novel method to decompose output predictions into fine-grained attributions, balancing positive and negative relevance for clearer visualization of the evidence behind a network decision. We carefully analyze conventional approaches to decision explanation and present a different perspective on the conservation of evidence. We define the evidence as a gap between positive and negative influences among gradient-derived initial contribution maps. Then, we incorporate antagonistic elements and a user-defined criterion for the degree of positive attribution during propagation. Additionally, we consider the role of inactivated neurons in the propagation rule, thereby enhancing the discernment of less relevant elements such as the background. We conduct various assessments in a verified experimental environment with PASCAL VOC 2007, MS COCO 2014, and ImageNet datasets. The results demonstrate that our method outperforms existing attribution methods both qualitatively and quantitatively in identifying the key input features that influence model decisions.

Survivor detection approach for post earthquake search and rescue missions based on deep learning inspired algorithms

Rapid and reliable detection of human survivors trapped under debris is crucial for effective post-earthquake search and rescue (SAR) operations. This paper presents a novel approach to survivor detection using a snake robot equipped with deep learning (DL) based object identification algorithms. We evaluated the performance of three main algorithms: Faster R-CNN, Single Shot MultiBox Detector (SSD), and You Only Look Once (YOLO). While these algorithms are initially trained on the PASCAL VOC 2012 dataset for human identification, we address the lack of a dedicated dataset for trapped individuals by compiling a new dataset of 200 images that specifically depicts this scenario, featuring cluttered environment and occlusion. Our evaluation takes into account detection accuracy, confidence interval, and running time. The results demonstrate that the YOLOv10 algorithm achieves the 98.4 mAP@0.5, accuracy of 98.5% for inference time of 15 ms. We validate the performance of these algorithms using images of human survivors trapped under debris and subjected to various occlusions.

An One-step Triple Enhanced weakly supervised semantic segmentation using image-level labels.

Weakly supervised semantic segmentation, based on image-level labels, abandons the pixel-level labels relied upon by traditional semantic segmentation algorithms. It only utilizes images as supervision information, thereby reducing the time cost and human resources required for marking pixel data. The prevailing approach in weakly supervised segmentation involves two-step method, introducing an additional network and numerous parameters, thereby complicating the model structure. Furthermore, image-level labels typically furnishes only category information for the entire image, lacking specific location details and accurate target boundaries during model training. We propose an innovative One-Step Triple Enhanced weakly supervised semantic segmentation network(OSTE). OSTE streamlines the model structure, which can accomplish both pseudo-labels generation and semantic segmentation tasks in just one step. Furthermore, we augment the weakly supervised semantic segmentation network in three key aspects based on the class activation map construction method, thereby enhancing segmentation accuracy: Firstly, by integrating local information from the activation map with the image, we can enhance the network's localization and expansion capabilities to obtain more accurate and rich location information. Then, we refine the seed areas of the class activation map by exploiting the correlation between multi-level feature. Finally, we incorporate conditional random field theory to generate pseudo-labels with higher confidence and richer boundary information. In comparison to the prevailing two-step weakly supervised semantic segmentation schemes, the segmentation network proposed in this paper achieves a more competitive mean Intersection over Union (mIoU) score of 58.47% on Pascal VOC. Additionally, it enhances the mIoU score by at least 5.03% when compared to existing end-to-end schemes.

Region-aware image-based human action retrieval with transformers

Human action understanding is a fundamental and challenging task in computer vision. Although there exists tremendous research on this area, most works focus on action recognition, while action retrieval has received less attention. In this paper, we focus on the neglected but important task of image-based action retrieval which aims to find images that depict the same action as a query image. We establish benchmarks for this task and set up important baseline methods for fair comparison. We present a Transformer-based model that learns rich action representations from three aspects: the anchored person, contextual regions, and the global image. A fusion transformer is designed to model the relationships among different features and effectively fuse them into an action representation. Experiments on both the Stanford-40 and PASCAL VOC 2012 Action datasets show that the proposed method significantly outperforms previous approaches for image-based action retrieval.

Pixel shuffling is all you need: spatially aware convmixer for dense prediction tasks

ConvMixer is an extremely simple model that could perform better than the state-of-the-art convolutional-based and vision transformer-based methods thanks to mixing the input image patches using a standard convolution. The global mixing process of the patches is only valid for the classification tasks, but it cannot be used for dense prediction tasks as the spatial information of the image is lost in the mixing process. We propose a more efficient technique for image patching, known as pixel shuffling, as it can preserve spatial information. We downsample the input image using the pixel shuffle downsampling in the same form of image patches so that the ConvMixer can be extended for the dense prediction tasks. This paper proves that pixel shuffle downsampling is more efficient than the standard image patching as it outperforms the original ConvMixer architecture in the CIFAR10 and ImageNet-1k classification tasks. We also suggest spatially-aware ConvMixer architectures based on efficient pixel shuffle downsampling and upsampling operations for semantic segmentation and monocular depth estimation. We performed extensive experiments to test the proposed architectures on several datasets; Pascal VOC2012, Cityscapes, and ADE20k for semantic segmentation, NYU-depthV2, and Cityscapes for depth estimation. We show that SA-ConvMixer is efficient enough to get relatively high accuracy at many tasks in a few training epochs (150∼400). The proposed SA-ConvMixer could achieve an ImageNet-1K Top-1 classification accuracy of 87.02%, mean intersection over union (mIOU) of 87.1% in the PASCAL VOC2012 semantic segmentation task, and absolute relative error of 0.096 in the NYU depthv2 depth estimation task. The implementation code of the proposed method is available at: https://github.com/HatemHosam/SA-ConvMixer/.

Mitigating Adversarial Attacks in Object Detection through Conditional Diffusion Models

The field of object detection has witnessed significant advancements in recent years, thanks to the remarkable progress in artificial intelligence and deep learning. These breakthroughs have significantly enhanced the accuracy and efficiency of detecting and categorizing objects in digital images. Nonetheless, contemporary object detection technologies have certain limitations, such as their inability to counter white-box attacks, insufficient denoising, suboptimal reconstruction, and gradient confusion. To overcome these hurdles, this study proposes an innovative approach that uses conditional diffusion models to perturb adversarial examples. The process begins with the application of a random chessboard mask to the adversarial example, followed by the addition of a slight noise to fill the masked area during the forward process. The adversarial image is then restored to its original form through a reverse generative process that only considers the masked pixels, not the entire image. Next, we use the complement of the initial mask as the mask for the second stage to reconstruct the image once more. This two-stage masking process allows for the complete removal of global disturbances and aids in image reconstruction. In particular, we employ a conditional diffusion model based on a class-conditional U-Net architecture, with the source image further conditioned through concatenation. Our method outperforms the recently introduced HARP method by 5% and 6.5% in mAP on the COCO2017 and PASCAL VOC datasets, respectively, under non-APT PGD attacks. Comprehensive experimental results confirm that our method can effectively restore adversarial examples, demonstrating its practical utility.