Object Detection Method Research Articles

CRRFNet: An adaptive traffic object detection method based on camera and radar radio frequency fusion

A large number of studies have proved that camera and radar fusion is a useful and economical solution for traffic object detection. However, how to improve the reliability and robustness of fusion methods is still a huge challenge. In this paper, an adaptive traffic object detection method based on a camera and radar radio frequency Network (CRRFNet) is proposed, to solve the problem of robust and reliable traffic object detection in noisy or abnormal scenes. Firstly, two different deep convolution modules are designed for extracting features from the camera and radar; Secondly, the camera and radar features are catenated, and a deconvolution module is built for upsampling; Thirdly, the heatmap module is used to compress redundant channels. Finally, the objects in the Field of View (FoV) are predicted by location-based Non-Maximum Suppression (L-NMS). In addition, a data scrambling technique is proposed to alleviate the problem of overfitting to a single sensor by the fusion method. The existing Washington University Camera Radar (CRUW) dataset is improved and a new dataset named Camera-Radar Nanjing University of Science and Technology Version 1.0 (CRNJUST-v1.0) is collected to verify the proposed method. Experiments show that CRRFNet can detect objects by using the information of radar and camera at the same time, which is far more accurate than a single sensor method. Combined with the proposed data scrambling technology, CRRFNet shows excellent robustness that can effectively detect objects in the case of interference or single sensor failure.

FSPDD: A double-branch attention guided network for few-shot PCB defect detection

AbstractDuring the production of printed circuit board (PCB), there will be defects due to inappropriate operations, which will affect the use of electronic products. Majority defect detection methods cost a large number of annotated samples to train detection models. However, PCB defect samples are difficult to collect. Moreover, existing few-shot object detection methods tend to extracting low-level features from support and query images via the shared backbone such as ResNet-50. However, it is not sufficient to obtain fine-grained prior guidance. To address the above issues, we propose a few-shot PCB defect detection model with double-branch attention. Specifically, the joint attention enhancement (JAE) module is proposed to fully mine effective information of query PCB images in multiple dimensions to enhance the representation of latent defects. Then, the multi-scale guidance (MSG) module is proposed to integrate prior knowledge within support PCB images into vectors to reweight query PCB images. Experiments on the PCB defect dataset demonstrate that AP of FSPDD outperforms state-of-the-art methods under different shot settings (k=1,2,3,5,10,30) and our proposed FSPDD has a good generalization ability, in which AP reachs 0.273 when $$k=30$$ k = 30 and is 5.28% higher than SOTA methods.

OPTICAL DEEP LEARNING LANDMINE DETECTION BASED ON LIMITED DATASET OF AERIAL IMAGERY

Landmine detection is one of the most innovative applications of unmanned aerial vehicles that became possible due to rapid development of both aerial vehicles equipped by different optical cameras and sensors using different physical principles, and object classification and detection methods, including machine learning and especially deep learning. Optical detection is an essential part of the overall landmine detection process that can be performed either separately or in combination with data processing from other types of cameras or sensors. The development of deep convolutional neural networks has dramatically changed the landscape of optical detection by making them de-facto choice number one for the majority of object classification, detection and segmentation tasks. However, the deterrent factor in the case of landmine detection is limited availability of appropriate data for training that different researchers try to overcome in different ways. The assessment of necessary amount of training data for any particular object detection problem still remains an experimental task. Despite several years of development in this area, still there is a shortage of research based on real landmine imagery obtained from unmanned aerial vehicles, so currently any public effort in this direction is valuable and works as an inspiration for new researchers. This paper describes such a study, namely its first iteration in which popular open-source tools are used to build detection pipeline and estimation of their efficiency is done using limited amount of data. It is shown that the problem of limited amount of training data can be effectively overcome by data augmentation and iterational process of training optical landmine detector is demonstrated. The effectiveness of open-source tools and libraries for neural networks training, object detection and dataset preparation is also demonstrated.

HP-YOLOv8: High-Precision Small Object Detection Algorithm for Remote Sensing Images.

YOLOv8, as an efficient object detection method, can swiftly and precisely identify objects within images. However, traditional algorithms encounter difficulties when detecting small objects in remote sensing images, such as missing information, background noise, and interactions among multiple objects in complex scenes, which may affect performance. To tackle these challenges, we propose an enhanced algorithm optimized for detecting small objects in remote sensing images, named HP-YOLOv8. Firstly, we design the C2f-D-Mixer (C2f-DM) module as a replacement for the original C2f module. This module integrates both local and global information, significantly improving the ability to detect features of small objects. Secondly, we introduce a feature fusion technique based on attention mechanisms, named Bi-Level Routing Attention in Gated Feature Pyramid Network (BGFPN). This technique utilizes an efficient feature aggregation network and reparameterization technology to optimize information interaction between different scale feature maps, and through the Bi-Level Routing Attention (BRA) mechanism, it effectively captures critical feature information of small objects. Finally, we propose the Shape Mean Perpendicular Distance Intersection over Union (SMPDIoU) loss function. The method comprehensively considers the shape and size of detection boxes, enhances the model's focus on the attributes of detection boxes, and provides a more accurate bounding box regression loss calculation method. To demonstrate our approach's efficacy, we conducted comprehensive experiments across the RSOD, NWPU VHR-10, and VisDrone2019 datasets. The experimental results show that the HP-YOLOv8 achieves 95.11%, 93.05%, and 53.49% in the mAP@0.5 metric, and 72.03%, 65.37%, and 38.91% in the more stringent mAP@0.5:0.95 metric, respectively.

Prototypical contrastive learning based oriented detector for kitchen waste

Automatic detection of kitchen waste enables the identification and quantification of non-degradable materials, such as plastics, metals, and other substances that cannot be easily decomposed. This approach increases the efficiency of the waste disposal process with significant time and cost savings compared to manual sorting. However, detecting non-degradable waste automatically presents a great challenge due to the deformability and directional uncertainty of the objects in kitchen waste images. Object deformation leads to a lack of structural features, making it more difficult to distinguish objects. Direction uncertainty leads to redundant backgrounds when detecting with the horizontal box detector. To address these issues, we propose a kitchen waste detector (KWDet) for the automatic detection of non-degradable waste in kitchen waste. First, the KWDet introduces the oriented bounding box representation and rotated region of interest (RRoI) learner, which enables the oriented detection of non-degradable waste in kitchen waste. Second, the KWDet constructs a prototypical contrastive learning (PCL) based detection head to improve the classification performance by learning discriminative features of different non-degradable waste in kitchen waste. In the PCL based detection head, the prototype-sample contrastive loss is proposed to pull samples and prototypes from the same classes together while pushing samples and prototypes from the different classes apart. After that, we design the inter-prototype contrastive loss to learn the discriminative prototypes of different non-degradable waste. Finally, to evaluate the proposed KWDet, we construct a large kitchen waste detection dataset (KWDD), containing 13,873 kitchen waste images and 213,565 instances collected from Zhengzhou China with 5 classes. To validate the effectiveness of our proposed method, we performed ablation experiments and comparison experiments on the KWDD dataset. The experiments demonstrate that KWDet achieves a state-of-the-art (SOTA) mAP50 of 71.5%, surpassing other object detection methods by more than 1.6% mAP50. Furthermore, our approach’s Macro-F1 score outperforms the SOTA method by 4.0%. Extensive experiments on the KWDD demonstrate that the prototypical contrastive learning-based oriented detector is able to extract more robust features and better characterize the target location, suggesting that the KWDet is a powerful support for intelligent kitchen waste sorting.

A DL method to detect multi-type hidden objects in tunnel linings using a comprehensive GPR dataset

This study proposes an enhanced detection method for tunnel lining hidden objects based on the comprehensive Ground Penetrating Radar (GPR) dataset and an improved deep learning (DL) algorithm. To mitigate the issue of data scarcity, a tailored dataset consisting of six types of hidden objects was compiled, which contained 7426 on-site collected GPR images and 71,014 well-labeled objects. This bespoke dataset offered substantial support for algorithm training and validation, making it one of the most extensive datasets in the domain for tunnel lining GPR detection. An enhanced Faster R-CNN algorithm was then developed to effectively detect different types of hidden objects. Modifications were made in four key areas: Feature extraction module, Feature Pyramid Network (FPN), customized anchor scheme and modified loss function. This algorithm was subsequently trained and validated using the comprehensive dataset. Evaluation metrics demonstrated the performance of this approach, with a mean average Precision (mAP) of 81.2%. Results also proved particularly effective in detecting rebars, pipelines, and steel arches while demonstrating a shortfall in detecting voids and cavities due to the data limitation. The results underscore the potential of this enhanced algorithm for the automatic detection of diverse hidden objects in tunnel linings, thus stimulating further research in this vital field.

Vision-language pre-training via modal interaction

Existing vision-language pre-training models typically extract region features and conduct fine-grained local alignment based on masked image/text completion or object detection methods. However, these models often design independent subtasks for different modalities, which may not adequately leverage interactions between modalities, requiring large datasets to achieve optimal performance. To address these limitations, this paper introduces a novel pre-training approach that facilitates fine-grained vision-language interaction. We propose two new subtasks — image filling and text filling — that utilize data from one modality to complete missing parts in another, enhancing the model’s ability to integrate multi-modal information. A selector mechanism is also developed to minimize semantic overlap between modalities, thereby improving the efficiency and effectiveness of the pre-trained model. Our comprehensive experimental results demonstrate that our approach not only fosters better semantic associations among different modalities but also achieves state-of-the-art performance on downstream vision-language tasks with significantly smaller datasets.

Small Object Detection in Medium–Low-Resolution Remote Sensing Images Based on Degradation Reconstruction

With the continuous development of space remote sensing technology, the spatial resolution of visible remote sensing images has been continuously improved, which has promoted the progress of remote sensing target detection. However, due to the limitation of sensor lattice size, it is still challenging to obtain a large range of high-resolution (HR) remote sensing images in practical applications, which makes it difficult to carry out target monitoring in a large range of areas. At present, many object detection methods focus on the detection and positioning technology of HR remote sensing images, but there are relatively few studies on object detection methods using medium- and low-resolution (M-LR) remote sensing images. Because of its wide coverage area and short observation period, M-LR remote sensing imagery is of great significance for obtaining information quickly in space applications. However, the small amount of fine-texture information on objects in M-LR images brings great challenges to detection and recognition tasks. Therefore, we propose a small target detection method based on degradation reconstruction, named DRADNet. Different from the previous methods that use super resolution as a pre-processing step and then directly input the image into the detector, we have designed an additional degenerate reconstruction-assisted framework to effectively improve the detector’s performance in detection tasks with M-LR remote sensing images. In addition, we introduce a hybrid parallel-attention feature fusion module in the detector to achieve focused attention on target features and suppress redundant complex backgrounds, thus improving the accuracy of the model in small target localization. The experimental results are based on the widely used VEDAI dataset and Airbus-Ships dataset, and verify the effectiveness of our method in the detection of small- and medium-sized targets in M-LR remote sensing images.

Self-supervised spatial–temporal feature enhancement for one-shot video object detection

One-shot video object detection is a task that aims to locate and identify objects in video sequences given only a single video sample for each class. Exploration in this field is still in its infancy and previous few-shot video object detection methods have limitation in this task. In this paper, we propose the Self-supervised Feature Enhancement (SFE) framework to address one-shot video object detection task. SFE includes two important modules: Hybrid Spatial Self-supervised Feature Enhancement (HSSFE) and Dynamic Temporal Self-supervised Feature Enhancement (DTSFE). HSSFE enhances features from a spatial perspective with spatial self-supervised auxiliary tasks at frame and instance levels. DTSFE on the other hand enhances features from a temporal perspective with memory-based self-supervised constraint for the same object across different frames. We have conducted experiments on multiple benchmarks, and the results demonstrate that our method achieves state-of-the-art performance.

LiDAR-Based 3D Temporal Object Detection via Motion-Aware LiDAR Feature Fusion.

Recently, the growing demand for autonomous driving in the industry has led to a lot of interest in 3D object detection, resulting in many excellent 3D object detection algorithms. However, most 3D object detectors focus only on a single set of LiDAR points, ignoring their potential ability to improve performance by leveraging the information provided by the consecutive set of LIDAR points. In this paper, we propose a novel 3D object detection method called temporal motion-aware 3D object detection (TM3DOD), which utilizes temporal LiDAR data. In the proposed TM3DOD method, we aggregate LiDAR voxels over time and the current BEV features by generating motion features using consecutive BEV feature maps. First, we present the temporal voxel encoder (TVE), which generates voxel representations by capturing the temporal relationships among the point sets within a voxel. Next, we design a motion-aware feature aggregation network (MFANet), which aims to enhance the current BEV feature representation by quantifying the temporal variation between two consecutive BEV feature maps. By analyzing the differences and changes in the BEV feature maps over time, MFANet captures motion information and integrates it into the current feature representation, enabling more robust and accurate detection of 3D objects. Experimental evaluations on the nuScenes benchmark dataset demonstrate that the proposed TM3DOD method achieved significant improvements in 3D detection performance compared with the baseline methods. Additionally, our method achieved comparable performance to state-of-the-art approaches.

Video object detection via space–time feature aggregation and result reuse

AbstractWhen detecting the objects in videos, motion always leads to object deterioration, like blurring and occlusion, as well as the strange state of the object's shape and posture. Consequently, the detection of video frames will lead to a decline in accuracy by using the image object detection model. This paper proposes an online video object detection method based on the one‐stage detector YOLOx. First, the module for space–time feature aggregation is given, which uses the space–time information of past frames to enhance the feature quality of the current frame. Then, the module for result reuse is given, which incorporates the detection results of past frames to improve the detection stability of the current frame. By these two modules, the trade‐off between accuracy and speed of video object detection could be achieved. Experimental results on the ImageNet VID show the improvement of speed and accuracy of the proposed method.

Identifying rice field weeds from unmanned aerial vehicle remote sensing imagery using deep learning

BackgroundRice field weed object detection can provide key information on weed species and locations for precise spraying, which is of great significance in actual agricultural production. However, facing the complex and changing real farm environments, traditional object detection methods still have difficulties in identifying small-sized, occluded and densely distributed weed instances. To address these problems, this paper proposes a multi-scale feature enhanced DETR network, named RMS-DETR. By adding multi-scale feature extraction branches on top of DETR, this model fully utilizes the information from different semantic feature layers to improve recognition capability for rice field weeds in real-world scenarios.MethodsIntroducing multi-scale feature layers on the basis of the DETR model, we conduct a differentiated design for different semantic feature layers. The high-level semantic feature layer adopts Transformer structure to extract contextual information between barnyard grass and rice plants. The low-level semantic feature layer uses CNN structure to extract local detail features of barnyard grass. Introducing multi-scale feature layers inevitably leads to increased model computation, thus lowering model inference speed. Therefore, we employ a new type of Pconv (Partial convolution) to replace traditional standard convolutions in the model.ResultsCompared to the original DETR model, our proposed RMS-DETR model achieved an average recognition accuracy improvement of 3.6% and 4.4% on our constructed rice field weeds dataset and the DOTA public dataset, respectively. The average recognition accuracies reached 0.792 and 0.851, respectively. The RMS-DETR model size is 40.8 M with inference time of 0.0081 s. Compared with three classical DETR models (Deformable DETR, Anchor DETR and DAB-DETR), the RMS-DETR model respectively improved average precision by 2.1%, 4.9% and 2.4%.DiscussionThis model is capable of accurately identifying rice field weeds in complex real-world scenarios, thus providing key technical support for precision spraying and management of variable-rate spraying systems.

Exploring the performance of the topological energy method for object and damage detection from noisy and poor databases.

In this work, we study the performance of numerical methods based on the computation of topological energies to process data from synthetic and real experiments where only noisy limited-view data corresponding to a few frequencies are available. We show numerical experiments in two problems of practical interest. The first one corresponds to experimental measurements of the electromagnetic scattering produced by different objects extracted from the Fresnel database. The second one is related to synthetic experiments in a simplified model where steel welding joints are acoustically tested to identify possible flaws (air bubbles and inclusions) produced during the welding process. This article is part of the theme issue 'Non-smooth variational problems with applications in mechanics'.

A New Approach for Super Resolution Object Detection Using an Image Slicing Algorithm and the Segment Anything Model.

Object detection in high resolution enables the identification and localization of objects for monitoring critical areas with precision. Although there have been improvements in object detection at high resolution, the variety of object scales, as well as the diversity of backgrounds and textures in high-resolution images, make it challenging for detectors to generalize successfully. This study introduces a new method for object detection in high-resolution images. The pre-processing stage of the method includes ISA and SAM to slice the input image and segment the objects in bounding boxes, respectively. In order to improve the resolution in the slices, the first layer of YOLO is designed as SRGAN. Thus, before applying YOLO detection, the resolution of the sliced images is increased to improve features. The proposed system is evaluated on xView and VisDrone datasets for object detection algorithms in satellite and aerial imagery contexts. The success of the algorithm is presented in four different YOLO architectures integrated with SRGAN. According to comparative evaluations, the proposed system with Yolov5 and Yolov8 produces the best results on xView and VisDrone datasets, respectively. Based on the comparisons with the literature, our proposed system produces better results.

PGDIG-YOLO: a lightweight method for airport runway foreign object detection

PGDIG-YOLO: a lightweight method for airport runway foreign object detection

An Improved Lightweight YOLOv5s-Based Method for Detecting Electric Bicycles in Elevators

The increase in fire accidents caused by indoor charging of electric bicycles has raised concerns among people. Monitoring EBs in elevators is challenging, and the current object detection method is a variant of YOLOv5, which faces problems with calculating the load and detection rate. To address this issue, this paper presents an improved lightweight method based on YOLOv5s to detect EBs in elevators. This method introduces the MobileNetV2 module to achieve the lightweight performance of the model. By introducing the CBAM attention mechanism and the Bidirectional Feature Pyramid Network (BiFPN) into the YOLOv5s neck network, the detection precision is improved. In order to better verify that the model can be deployed at the edge of an elevator, this article deploys it using the Raspberry Pi 4B embedded development board and connects it to a buzzer for application verification. The experimental results demonstrate that the model parameters of EBs are reduced by 58.4%, the computational complexity is reduced by 50.6%, the detection precision reaches 95.9%, and real-time detection of electric vehicles in elevators is achieved.

BSMH: Cross‐dataset object detection based on box‐separated multiple‐head

AbstractCross‐dataset object detection methods can adapt to the needs of rapid category expansion in object detection tasks. However, these methods are prone to generate dataset‐aware errors with false alarm objects. This study is aimed to address these issues. A box‐separated multiple‐head module and box‐separated loss function based on the YOLOv8 network are devised to achieve cross‐dataset object detection. Additionally, a sameclass‐aware fusion module to avoid gradient conflicts due to cross‐category conflicts is developed. A multiple‐head fusion module is devised to reduce the number of false alarm objects caused by dataset‐aware errors. A global class‐aware sampler is also designed to adapt to the impact of the imbalanced number of categories and training samples across datasets. The effectiveness of the box‐separated multiple‐head module is verified using cross‐datasets built using the COCO, WiderFace, WiderPerson, and OpenImages V4 datasets. Extensive experiments demonstrate the efficiency and precision of the proposed method.

An Improved Instance Segmentation Method for Fast Assessment of Damaged Buildings Based on Post-Earthquake UAV Images.

Quickly and accurately assessing the damage level of buildings is a challenging task for post-disaster emergency response. Most of the existing research mainly adopts semantic segmentation and object detection methods, which have yielded good results. However, for high-resolution Unmanned Aerial Vehicle (UAV) imagery, these methods may result in the problem of various damage categories within a building and fail to accurately extract building edges, thus hindering post-disaster rescue and fine-grained assessment. To address this issue, we proposed an improved instance segmentation model that enhances classification accuracy by incorporating a Mixed Local Channel Attention (MLCA) mechanism in the backbone and improving small object segmentation accuracy by refining the Neck part. The method was tested on the Yangbi earthquake UVA images. The experimental results indicated that the modified model outperformed the original model by 1.07% and 1.11% in the two mean Average Precision (mAP) evaluation metrics, mAPbbox50 and mAPseg50, respectively. Importantly, the classification accuracy of the intact category was improved by 2.73% and 2.73%, respectively, while the collapse category saw an improvement of 2.58% and 2.14%. In addition, the proposed method was also compared with state-of-the-art instance segmentation models, e.g., Mask-R-CNN and YOLO V9-Seg. The results demonstrated that the proposed model exhibits advantages in both accuracy and efficiency. Specifically, the efficiency of the proposed model is three times faster than other models with similar accuracy. The proposed method can provide a valuable solution for fine-grained building damage evaluation.

Pothole detection in adverse weather: leveraging synthetic images and attention-based object detection methods

Pothole detection in adverse weather: leveraging synthetic images and attention-based object detection methods

Leveraging foreground–background cues for semantically-driven, training-free moving object detection

Deep learning algorithms and semantic cue-based methods are increasingly used to detect moving objects. The semantic label changes in pixel location over time are used for moving object detection in semantic cue-based studies. In this approach, traditional background subtraction is employed only when there is a lack of semantic background. However, the change in semantic information in each pixel is insufficient for accurate detection of moving objects due to similar labels assignments for different objects and limited use of pixel intensity data. Moreover, challenges arise in detecting moving objects, such as a lack of diversity in labeled objects, an uneven distribution of labels or classes, and an insufficient amount of unseen datasets. In this study, we propose the Moving Semantic Object Detector (MSemOD), a train-less moving object detection method based on a traditional method enhanced with semantic information provided by pre-trained semantic segmentation models. The proposed method includes a region-based decision step, utilizing the traditional methods and semantic information as spatial–temporal cues. Threshold values are applied in the method to determine the temporal and spatial movements of each segment in the image. To improve the consistency of the semantic information, the proposed method converts the semantic segmentation output into a semantic probability map by enhancing the temporal and spatial state. The performance of the proposed method is evaluated on the LASIESTA, CDNet2014, and the BMC datasets and the results are compared with the state-of-the-art algorithms. The results show that using region-based semantic information can significantly reduce false positives and enhance foreground–background segmentation in moving object detection.