Object Detection Tasks Research Articles

Three-Dimensional Point Cloud Object Detection Based on Feature Fusion and Enhancement

With the continuous emergence and development of 3D sensors in recent years, it has become increasingly convenient to collect point cloud data for 3D object detection tasks, such as the field of autonomous driving. But when using these existing methods, there are two problems that cannot be ignored: (1) The bird’s eye view (BEV) is a widely used method in 3D objective detection; however, the BEV usually compresses dimensions by combined height, dimension, and channels, which makes the process of feature extraction in feature fusion more difficult. (2) Light detection and ranging (LiDAR) has a much larger effective scanning depth, which causes the sector to become sparse in deep space and the uneven distribution of point cloud data. This results in few features in the distribution of neighboring points around the key points of interest. The following is the solution proposed in this paper: (1) This paper proposes multi-scale feature fusion composed of feature maps at different levels made of Deep Layer Aggregation (DLA) and a feature fusion module for the BEV. (2) A point completion network is used to improve the prediction results by completing the feature points inside the candidate boxes in the second stage, thereby strengthening their position features. Supervised contrastive learning is applied to enhance the segmentation results, improving the discrimination capability between the foreground and background. Experiments show these new additions can achieve improvements of 2.7%, 2.4%, and 2.5%, respectively, on KITTI easy, moderate, and hard tasks. Further ablation experiments show that each addition has promising improvement over the baseline.

Salient object detection in egocentric videos

AbstractIn the realm of video salient object detection (VSOD), the majority of research has traditionally been centered on third‐person perspective videos. However, this focus overlooks the unique requirements of certain first‐person tasks, such as autonomous driving or robot vision. To bridge this gap, a novel dataset and a camera‐based VSOD model, CaMSD, specifically designed for egocentric videos, is introduced. First, the SalEgo dataset, comprising 17,400 fully annotated frames for video salient object detection, is presented. Second, a computational model that incorporates a camera movement module is proposed, designed to emulate the patterns observed when humans view videos. Additionally, to achieve precise segmentation of a single salient object during switches between salient objects, as opposed to simultaneously segmenting two objects, a saliency enhancement module based on the Squeeze and Excitation Block is incorporated. Experimental results show that the approach outperforms other state‐of‐the‐art methods in egocentric video salient object detection tasks. Dataset and codes can be found at https://github.com/hzhang1999/SalEgo.

Learning Point Processes and Convolutional Neural Networks for Object Detection in Satellite Images

Convolutional neural networks (CNN) have shown great results for object-detection tasks by learning texture and pattern-extraction filters. However, object-level interactions are harder to grasp without increasing the complexity of the architectures. On the other hand, Point Process models propose to solve the detection of the configuration of objects as a whole, allowing the factoring in of the image data and the objects’ prior interactions. In this paper, we propose combining the information extracted by a CNN with priors on objects within a Markov Marked Point Process framework. We also propose a method to learn the parameters of this Energy-Based Model. We apply this model to the detection of small vehicles in optical satellite imagery, where the image information needs to be complemented with object interaction priors because of noise and small object sizes.

A Region-Adaptive Local Perturbation-Based Method for Generating Adversarial Examples in Synthetic Aperture Radar Object Detection

In synthetic aperture radar (SAR) imaging, intelligent object detection methods are facing significant challenges in terms of model robustness and application security, which are posed by adversarial examples. The existing adversarial example generation methods for SAR object detection can be divided into two main types: global perturbation attacks and local perturbation attacks. Due to the dynamic changes and irregular spatial distribution of SAR coherent speckle backgrounds, the attack effectiveness of global perturbation attacks is significantly reduced by coherent speckle. In contrast, by focusing on the image objects, local perturbation attacks achieve targeted and effective advantages over global perturbations by minimizing interference from the SAR coherent speckle background. However, the adaptability of conventional local perturbations is limited because they employ a fixed size without considering the diverse sizes and shapes of SAR objects under various conditions. This paper presents a framework for region-adaptive local perturbations (RaLP) specifically designed for SAR object detection tasks. The framework consists of two modules. To address the issue of coherent speckle noise interference in SAR imagery, we develop a local perturbation generator (LPG) module. By filtering the original image, this module reduces the speckle features introduced during perturbation generation. It then superimposes adversarial perturbations in the form of local perturbations on areas of the object with weaker speckles, thereby reducing the mutual interference between coherent speckles and adversarial perturbation. To address the issue of insufficient adaptability in terms of the size variation in local adversarial perturbations, we propose an adaptive perturbation optimizer (APO) module. This optimizer adapts the size of the adversarial perturbations based on the size and shape of the object, effectively solving the problem of adaptive perturbation size and enhancing the universality of the attack. The experimental results show that RaLP reduces the detection accuracy of the YOLOv3 detector by 29.0%, 29.9%, and 32.3% on the SSDD, SAR-Ship, and AIR-SARShip datasets, respectively, and the model-to-model and dataset-to-dataset transferability of RaLP attacks are verified.

Vision Transformer-based overlay processor for Edge Computing

Accelerating Visual Neural Networks in Edge Computing environments is crucial for processing image and video data. Visual Neural Networks, including Convolutional Neural Networks and Vision Transformers, are central to image recognition, video analysis, and object detection tasks. Deploying these networks on edge devices and accelerating them can significantly enhance data processing speed and efficiency. The large number of parameters, complex computational flows, and various structural variants of Transformer models present both opportunities and challenges. We propose Vis-TOP (Vision Transformer Overlay Processor), an overlay processor designed for all types of Vision Transformer models. Vis-TOP, distinct from coarse-grained general-purpose accelerators like GPUs and fine-grained custom designs, encapsulates Vision Transformer characteristics into a three-layer, two-level mapping structure, enabling flexible model switching without hardware architecture modifications. Concurrently, we designed a corresponding instruction bundle and hardware architecture within this mapping structure. We implemented the overlay processor design on the ZCU102 after quantizing the Swin Transformer model to 8-bit fixed points (fix_8). Experimentally, our throughput surpasses GPU implementation by 1.5 times. Our throughput per DSP is 2.2 to 11.7 times higher than that of existing Transformer-like accelerators. Overall, our approach satisfies real-time AI requirements in resource consumption and inference speed. Vis-TOP offers a cost-effective image processing solution for Edge Computing on reconfigurable devices, enhancing computational resource utilization, saving data transfer time and costs, and reducing latency.

Target Detection and Segmentation Technology for Zero-shot Learning

Zero-shot learning (ZSL) in the field of computer vision refers to enabling the model to recognize and understand categories that have not been encountered during the training phase. It is particularly critical for object detection and segmentation tasks, because these tasks require the model to have good generalization capabilities to unknown categories. Object detection requires the model to determine the location of the object, while segmentation further requires the precise demarcation of the object's boundaries. In ZSL research, knowledge representation and transfer are core issues. Researchers have tried to use semantic attributes as a knowledge bridge to connect categories seen during the training phase and categories not seen during the testing phase. These attributes may be color, shape, etc., but this method requires accurate attribute annotation, which is often not easy to achieve in practice. Therefore, researchers have begun to explore the use of non-visual information such as knowledge maps and text descriptions to enrich the recognition capabilities of models, but this also introduces the challenge of information integration and alignment. At present, ZSL has made certain progress in target detection and segmentation tasks, but there is still a significant gap compared with traditional supervised learning. This is mainly due to the limited ability of ZSL models to generalize to new categories. To this end, researchers have begun to explore combining ZSL with other technologies, such as generative adversarial networks (GANs) and reinforcement learning, to enhance the model's detection and segmentation capabilities for new categories. Future research needs to focus on several aspects. The first is how to design a more effective knowledge representation and transfer mechanism so that the model can better utilize existing knowledge. The second step is to develop new algorithms to improve the performance of ZSL in complex environments. In addition, research should focus on how to reduce the dependence on computing resources so that the ZSL method can run effectively in resource-limited environments. In summary, the research on target detection and segmentation technology of zero-shot learning is a cutting-edge topic in the field of computer vision. Despite the challenges, with the deepening of research, we expect these technologies to contribute to improving the generalization ability and intelligence level of computer vision systems.

Research on Improved Algorithm for Small Object Detection in Intelligent Surveillance Video based on YOLOv7

In order to address the issue of small objects being difficult to detect effectively in intelligent surveillance videos, this study proposes an improved scheme for the YOLOv7-tiny algorithm. This scheme integrates the Convolutional Block Attention Module (CBAM) into YOLOv7-tiny, effectively enhancing the model's feature extraction and small object detection capabilities in complex backgrounds, thereby improving the overall detection precision. Experimental evaluations indicate that the improved algorithm shows enhanced performance in specific small object detection tasks, achieving an accuracy of 85.6%, a recall rate of 85.2%, and a mean average precision (mAP) of 90.2%. These results demonstrate the effectiveness and practical value of the improved scheme in enhancing the performance of YOLOv7-tiny in small object detection tasks.

MFIL-FCOS: A Multi-Scale Fusion and Interactive Learning Method for 2D Object Detection and Remote Sensing Image Detection

Object detection is dedicated to finding objects in an image and estimate their categories and locations. Recently, object detection algorithms suffer from a loss of semantic information in the deeper feature maps due to the deepening of the backbone network. For example, when using complex backbone networks, existing feature fusion methods cannot fuse information from different layers effectively. In addition, anchor-free object detection methods fail to accurately predict the same object due to the different learning mechanisms of the regression and centrality of the prediction branches. To address the above problem, we propose a multi-scale fusion and interactive learning method for fully convolutional one-stage anchor-free object detection, called MFIL-FCOS. Specifically, we designed a multi-scale fusion module to address the problem of local semantic information loss in high-level feature maps which strengthen the ability of feature extraction by enhancing the local information of low-level features and fusing the rich semantic information of high-level features. Furthermore, we propose an interactive learning module to increase the interactivity and more accurate predictions by generating a centrality-position weight adjustment regression task and a centrality prediction task. Following these strategic improvements, we conduct extensive experiments on the COCO and DIOR datasets, demonstrating its superior capabilities in 2D object detection tasks and remote sensing image detection, even under challenging conditions.

Research on multi-task perception network of traffic scene based on feature fusion1

Aiming at the problems of low precision and low real-time performance when deploying to embedded platforms in existing multi-task networks, this paper proposes a traffic scene multi-task perception network model (ETS_YOLOP) based on feature fusion. Firstly, an Efficient Attention Control Aggregation Network Module (EACAN) is constructed to improve the real-time perception of the model, and the Space Pyramid Pool Fast Convolutional Module (SPPFCSPC) is used at the end of the backbone network to increase the receptive field. Finally, a Multiscale Convolution Transformer Fusion Module (CTFM) is designed in the task branch to better capture global information and rich context information. The experimental results show that compared with the YOLOP model, the ETS_YOLOP model has a significant improvement in perception accuracy, 156% in real-time performance, 0.4% increase in mAP on the object detection task, 0.5% increase in mIoU on the drivable area segmentation task, and an 11.4% increase in accuracy on the lane detection task. In order to verify the real-time perception of the model on the embedded platform, the ETS_YOLOP model is deployed on the Huawei MDC300F computing platform. Under the condition of the image input size of 640×640, the average frame rate can reach 55FPS, which can realize real-time perception on the embedded platform.

Multi-perspective analysis on data augmentation in knowledge distillation

Knowledge distillation stands as a capable technique for transferring knowledge from a larger to a smaller model, thereby notably enhancing the smaller model’s performance. In the recent past, data augmentation has been employed in contrastive learning based knowledge distillation techniques yielding superior results. Despite the significant role of data augmentation, its value remains underappreciated within the domain of knowledge distillation, with no in-depth analysis in the literature thus far. To make up for this oversight, we conduct a multi-perspective theoretical and experimental analysis on the role that data augmentation can play in knowledge distillation. We summarize the properties of data augmentation and list the core findings as follows. (a) Our investigations validate that data augmentation significantly boosts the performance of knowledge distillation on the tasks of image classification and object detection. And this holds true even if the teacher model lacks comprehensive information about the augmented samples. Moreover, our novel Joint Data Augmentation (JDA) approach outperforms single data augmentation in knowledge distillation. (b) The pivotal role of data augmentation in knowledge distillation can be theoretically explained via Sharpness-Aware Minimization. (c) The compatibility of data augmentation with various knowledge distillation methods can enhance their performance. In light of these observations, we propose a new method called Cosine Confidence Distillation (CCD) for more reasonable knowledge transfer from augmented samples. Experimental results not only demonstrate that CCD becomes the state-of-the-art method with less storage requirement on CIFAR-100 and ImageNet-1k, but also validate the superiority of CCD over DIST on the object detection benchmark dataset, MS-COCO.

Class‐Relation Reasoning with Knowledge‐Transfer for Few‐Shot Object Detection

Few‐Shot Object Detection (FSOD) task involves accurately identifying target object classes using only a small set of labeled samples. Most of the current FSOD tasks independently predict class prototype features without considering class relationships and only rely on visual information. To address these challenges, we propose a novel Class‐relational Reasoning Method with Knowledge‐transfer (CRK‐Net), built on the meta‐learning‐based framework. Although data may be scarce, the semantic relationship between classes is invariant, Joint‐feature Fusion Module (JFM) are hence proposed to transfers the semantic information of different categories in the natural language world to integrate with visual information and produce multi‐modality embeddings. Some base classes and novel classes have similar features, so this can be borrowed by modeling the relationship between classes feature. Building upon the observation, we propose a Class‐relational Reasoning Module (CRM) to establish the correlations between categories and enhance prototype representations for each category. After passing through the JFM and CRM modules, a high‐quality class prototype is finally produced for subsequent regression and classification. Extensive experiments on PASCAL VOC demonstrate the effectiveness of our proposed method and provide a new scheme for fusing semantic and visual information. © 2024 Institute of Electrical Engineers of Japan. Published by Wiley Periodicals LLC.

A GAN-based anomaly detector using multi-feature fusion and selection

In numerous applications, abnormal samples are hard to collect, limiting the use of well-established supervised learning methods. GAN-based models which trained in an unsupervised and single feature set manner have been proposed by simultaneously considering the reconstruction error and the latent space deviation between normal samples and abnormal samples. However, the ability to capture the input distribution of each feature set is limited. Hence, we propose an unsupervised and multi-feature model, Wave-GANomaly, trained only on normal samples to learn the distribution of these normal samples. The model predicts whether a given sample is normal or not by its deviation from the distribution of normal samples. Wave-GANomaly fuses and selects from the wave-based features extracted by the WaveBlock module and the convolution-based features. The WaveBlock has proven to efficiently improve the performance on image classification, object detection, and segmentation tasks. As a result, Wave-GANomaly achieves the best average area under the curve (AUC) on the Canadian Institute for Advanced Research (CIFAR)-10 dataset (94.3%) and on the Modified National Institute of Standards and Technology (MNIST) dataset (91.0%) when compared to existing state-of-the-art anomaly detectors such as GANomaly, Skip-GANomaly, and the skip-attention generative adversarial network (SAGAN). We further verify our method by the self-curated real-world dataset, the result show that our method is better than GANomaly which only use single feature set for training the model.

Object Detection in Remote Sensing Images Based on Adaptive Multi-Scale Feature Fusion Method

Multi-scale object detection is critical for analyzing remote sensing images. Traditional feature pyramid networks, which are aimed at accommodating objects of varying sizes through multi-level feature extraction, face significant challenges due to the diverse scale variations present in remote sensing images. This situation often forces single-level features to span a broad spectrum of object sizes, complicating accurate localization and classification. To tackle these challenges, this paper proposes an innovative algorithm that incorporates an adaptive multi-scale feature enhancement and fusion module (ASEM), which enhances remote sensing image object detection through sophisticated multi-scale feature fusion. Our method begins by employing a feature pyramid to gather coarse multi-scale features. Subsequently, it integrates a fine-grained feature extraction module at each level, utilizing atrous convolutions with varied dilation rates to refine multi-scale features, which markedly improves the information capture from widely varied object scales. Furthermore, an adaptive enhancement module is applied to the features of each level by employing an attention mechanism for feature fusion. This strategy concentrates on the features of critical scale, which significantly enhance the effectiveness of capturing essential feature information. Compared with the baseline method, namely, Rotated FasterRCNN, our method achieved an mAP of 74.21% ( 0.81%) on the DOTA-v1.0 dataset and an mAP of 84.90% (+9.2%) on the HRSC2016 dataset. These results validated the effectiveness and practicality of our method and demonstrated its significant application value in multi-scale remote sensing object detection tasks.

Object Detection using an Advanced Deep Learning Algorithm: YOLOv4

This research proposes implementing YOLOv4, a real-time object detection system that can accurately and efficiently recognize objects in photos and videos. The goals include creating the YOLOv4 architecture, generating datasets, training the model, evaluating its performance, deploying it in real-world applications, and providing extensive documentation. Deep learning frameworks such as TensorFlow or PyTorch are used in the implementation, along with advanced techniques such as transfer learning and data augmentation. By curating annotated datasets and refining training techniques, the model hopes to attain high accuracy, precision, and recall in object detection tasks. Performance analysis will compare the model's outcomes to those of cutting-edge systems, assessing its strengths and weaknesses. The deployment phase involves integrating the model into existing systems or creating separate applications for real-world scenarios such as pedestrian and vehicle detection. Comprehensive documentation and a user guide will help developers and users make the best use of the trained model. Overall, this research intends to demonstrate YOLOv4's usefulness and feasibility in developing computer vision technology and supporting the creation of intelligent systems with real-time object identification capabilities.

Optimization of YOLOv7 Based on PConv, SE Attention and Wise-IoU

With the rapid development of deep learning technology, object detection algorithms have made significant breakthroughs in the field of computer vision. However, due to the complexity and computational requirements of deep Convolutional Neural Network (CNN), these models face many challenges in practical applications, especially on resource-constrained edge devices. To address this problem, researchers have proposed many lightweight methods that aim to reduce the model size and computational complexity while maintaining high performance. The popularity of mobile devices and embedded systems has led to an increasing demand for lightweight models. However, existing lightweight methods often lead to accuracy loss, limiting their feasibility in practical applications. Therefore, how to realize the light weight of the model while maintaining high accuracy has become an urgent problem to be solved. To address this challenge, this paper proposes a lightweight YOLOv7 method based on PConv, Squeeze-and-Excitation (SE) attention mechanism and Wise-IoU (WIoU), which we refer to as YOLOv7-PSW. PConv can effectively reduce the number of parameters and computational complexity. The SE can help the model focus on important feature information, thereby improving performance. WIoU is introduced to measure the similarity between the detection box and the Ground Truth, so that the model can effectively reduce the False Positive rate. By applying these advanced techniques to the YOLOv7, we achieve a lightweight model while maintaining a high detection accuracy. Experimental results on PASCAL VOC dataset show that YOLOv7-PSW performs better than the original YOLOv7 on object detection tasks. The number of parameters is reduced by 12.3%, FLOPs is reduced by 18.86%, and the accuracy is improved by about 0.5%. While the detection accuracy is not decreased or even slightly improved, the number of FLOPs and parameters is greatly reduced, which realizes lightweight to a certain extent. The proposed method can provide new ideas and directions for the subsequent research on lightweight object detection, and is expected to promote its application on edge devices. Meanwhile, YOLOv7-PSW can also be applied to other computer vision tasks to improve its performance and efficiency. In summary, the proposed YOLOv7-PSW lightweight method realizes the light weight of the model while maintaining high accuracy. This is of great significance for promoting the application of object detection algorithms on edge devices.

Partial half fine-tuning for object detection with unmanned aerial vehicles

<span lang="EN-US">Deep learning has shown outstanding performance in object detection tasks with unmanned aerial vehicles (UAVs), which involve the fine-tuning technique to improve performance by transferring features from pre-trained models to specific tasks. However, despite the immense popularity of fine-tuning, no works focused on to study of the precise fine-tuning effects of object detection tasks with UAVs. In this research, we conduct an experimental analysis of each existing fine-tuning strategy to answer which is the best procedure for transferring features with fine-tuning techniques. We also proposed a partial half fine-tuning strategy which we divided into two techniques: first half fine-tuning (first half F-T) and final half fine-tuning (Final half F-T). We use the VisDrone dataset for the training and validation process. Here we show that the partial half fine-tuning: Final half F-T can outperform other fine-tuning techniques and are also better than one of the state-of-the-art methods by a difference of 19.7% from the best results of previous studies.</span>

Image detection of aortic dissection complications based on multi-scale feature fusion

BackgroundAortic dissection refers to the true and false two-lumen separation of the aortic wall, in which the blood in the aortic lumen enters the aortic mesomembrane from the tear of the aortic intima to separate the mesomembrane and expand along the long axis of the aorta. PurposeIn view of the problems of individual differences, complex complications and many small targets in clinical aortic dissection detection, this paper proposes a convolution neural network MFF-FPN (Multi-scale Feature Fusion based Feature Pyramid Network) for the detection of aortic dissection complications. MethodsThe proposed model uses Resnet50 as the backbone for feature extraction and builds a pyramid structure to fuse low-level and high-level feature information. We add an attention mechanism to the backbone network, which can establish inter-dependencies between feature graph channels and enhance the representation quality of CNN. ResultsThe proposed method has a mean average precision (MAP) of 99.40% in the task of multi object detection for aortic dissection and complications, which is higher than the accuracy of 96.3% on SSD model and 99.05% on YoloV7 model. It greatly improves the accuracy of small target detection such as cysts, making it more suitable for clinical focus detection. ConclusionsThe proposed deep learning model achieves feature reuse and focuses on local important information. By adding only a small number of model parameters, we are able to greatly improve the detection accuracy, which is effective in detecting small target lesions commonly found in clinical settings, and also performs well on other medical and natural datasets.

Backbone search for object detection for applications in intrusion warning systems

<span>In this work, we propose a novel backbone search method for object detection for applications in intrusion warning systems. The goal is to find a compact model for use in embedded thermal imaging cameras widely used in intrusion warning systems. The proposed method is based on faster region-based convolutional neural network (Faster R-CNN) because it can detect small objects. Inspired by EfficientNet, the sought-after backbone architecture is obtained by finding the most suitable width scale for the base backbone (ResNet50). The evaluation metrics are mean average precision (mAP), number of parameters, and number of multiply–accumulate operations (MACs). The experimental results showed that the proposed method is effective in building a lightweight neural network for the task of object detection. The obtained model can keep the predefined mAP while minimizing the number of parameters and computational resources. All experiments are executed elaborately on the person detection in intrusion warning systems (PDIWS) dataset. </span>

A Comparative Analysis of YOLOv8 and YOLOv5 for Nut Thread Classification - Deep Learning Approach

Abstract: This research provides valuable and compelling insights into the performance comparison of YOLOv8 and YOLOv5 for Thread and No thread detection. The findings underscore the significant impact of architectural improvements and model advancements, resulting in higher precision and accuracy in object detection tasks. Furthermore, the successful implementation of LabelImg and Beautiful Soup demonstrates their remarkable effectiveness in dataset annotation and collection, crucial contributors to the overall success of the study. As a result, these results lay a solid foundation for future advancements in object detection techniques, with wide-ranging applications across diverse domains, encompassing industrial automation, surveillance systems, and beyond.

Deep Learning in Dental Radiographic Imaging

Deep learning algorithms are becoming more prevalent in dental research because they are utilized in everyday activities. However, dental researchers and clinicians find it challenging to interpret deep learning studies. This review aimed to provide an overview of the general concept of deep learning and current deep learning research in dental radiographic image analysis. In addition, the process of implementing deep learning research is described. Deep-learning-based algorithmic models perform well in classification, object detection, and segmentation tasks, making it possible to automatically diagnose oral lesions and anatomical structures. The deep learning model can enhance the decision-making process for researchers and clinicians. This review may be useful to dental researchers who are currently evaluating and assessing deep learning studies in the field of dentistry.