Object Detection Tasks Research Articles

Learning Omni-Dimensional Spatio-Temporal Dependencies for Millimeter-Wave Radar Perception

Reliable environmental perception capabilities are a prerequisite for achieving autonomous driving. Cameras and LiDAR are sensitive to illumination and weather conditions, while millimeter-wave radar avoids these issues. Existing models rely heavily on image-based approaches, which may not be able to fully characterize radar sensor data or efficiently further utilize them for perception tasks. This paper rethinks the approach to modeling radar signals and proposes a novel U-shaped multilayer perceptron network (U-MLPNet) that aims to enhance the learning of omni-dimensional spatio-temporal dependencies. Our method involves innovative signal processing techniques, including a 3D CNN for spatio-temporal feature extraction and an encoder–decoder framework with cross-shaped receptive fields specifically designed to capture the sparse and non-uniform characteristics of radar signals. We conducted extensive experiments using a diverse dataset of urban driving scenarios to characterize the sensor’s performance in multi-view semantic segmentation and object detection tasks. Experiments showed that U-MLPNet achieves competitive performance against state-of-the-art (SOTA) methods, improving the mAP by 3.0% and mDice by 2.7% in RD segmentation and AR and AP by 1.77% and 2.03%, respectively, in object detection. These improvements signify an advancement in radar-based perception for autonomous vehicles, potentially enhancing their reliability and safety across diverse driving conditions.

Urine Sediment Detection Algorithm Based on Channel Enhancement and Deformable Convolution.

Urine sediment detection is a vital method in clinical urine analysis for evaluating an individual's kidney and urinary system health, as well as identifying potential diseases. Nevertheless, urine sediment images exhibit the characteristic of diverse shapes for the same category of targets. These characteristics pose a considerable challenge to the accurate identification of the visible components within the images. We approach urine sediment detection as an object detection task and have introduced the specialized YOLOv7-CSD algorithm for this purpose. In particular, we have integrated channel enhancement feature pyramid network (CE-FPN) and selective kernel (SK) into the YOLOv7 model to address the issue of model confusion in classifying and identifying tasks caused by the feature aliasing effects of feature pyramid network (FPN). Furthermore, we enhance the efficient layer aggregation networks (ELAN) network by adding a second channel, enabling the model to acquire a more extensive set of feature information. On top of this, we introduce the deformable convolutional v3 (DCNv3) operator, allowing the model to dynamically adjust its receptive field, addressing the issue of variable shapes. Tested on the USE dataset and a dataset for urine crystals, YOLOv7-CSD achieves accuracies of 92.8 and 89.6 , respectively.

Overview of object detection based on deep learning

Abstract. In recent years, computer vision technology has developed rapidly, as one of its important research directions, object detection has received widespread attention due to its high accuracy. Meanwhile, object detection has many application fields, such as intelligent transportation, medical and health, security systems, etc. Traditional object detection methods have limitations when applied to complex real-world scenarios. To improve the shortcomings of conventional methods, deep learning based object detection algorithms have significantly improved the efficiency of object detection and become a research hotspot in object detection. This article summarizes the algorithms into two-stage and one-stage object detection algorithms based on the technical processes and structural differences in handling object detection tasks. Firstly, several common two-stage and one-stage object detection algorithms and their applications in real-world scenarios are introduced. Then their data sets and algorithm performance are analyzed and compared. Finally, according to the results of the comparison, the existing problems of the two-stage object detection algorithm and the one-stage object detection method are discussed, and their future development directions are pointed out.

Boosting Point Set-Based Network with Optimal Transport Optimization for Oriented Object Detection

When handling complex remote sensing scenarios, rotational angle information can improve detection accuracy and enhance algorithm robustness, providing support for fine-grained detection. Point set representation is one of the most commonly used methods in arbitrary-oriented object detection tasks, leveraging discrete feature points to represent oriented targets and achieve high accuracy in angle prediction. However, due to the inherent discreteness of point set representation, it is prone to significant impact from isolated points and representational ambiguity in harsh application scenarios, leading to inaccurate detection. To address this issue, an efficient aerial object detector named BE-Det is proposed, which uses the optimal transport (OT) strategy to constrain the positions of isolated points. Additionally, a candidate point set quality evaluation scheme is designed to effectively assess the quality of candidate point sets. Experimental results on two challenging aerial datasets demonstrate that the proposed method outperforms several advanced detection methods.

Transformer Based Ship Detector: An Improvement on Feature Map and Tiny Training Set

The exponential increment of commodity exchange has raised the need for maritime border security in recent years. One of the most critical tasks for naval border security is ship detection inside and outside the territorial sea. Conventionally, the task requires a substantial human workload. Fortunately, with the rapid growth of the digital camera and deep-learning technique, computer programs can handle object detection tasks well enough to replace human labor. Therefore, this paper studies how to apply recent state-of-the-art deep-learning networks to the ship detection task. We found that with a suitable number of object queries, the Deformable-DETR method will improve the performance compared to the state-of-the-art ship detector. Moreover, comprehensive experiments on different scale datasets prove that the technique can significantly improve the results when the training sample is limited. Last but not least, feature maps given by the method will focus well on key objects in the image.

3D morphometry of Martian craters from HRSC DEMs using a multi-scale semantic segmentation network and morphological analysis

The morphology of impact craters reveals the structure and composition of the Martian surface, especially the subsurface conditions and Martian geological history, which have increasing importance in Mars exploration missions. This work presents a 3D morphometric method for detecting and delineating Martian craters, and a 3D morphological analysis was conducted. Specifically, this work first focused on the segmentation of Martian craters. Based on the segmentation results, clustering of crater instances was then carried out. Finally, with the individual craters that were extracted, a morphological analysis involving the measurements of their diameter, depth, area, RMS height, rim height, circularity, and the statistics thereof was performed. Unlike previous studies, which have mainly used optical images and object detection approaches, this work regards crater extraction as a semantic segmentation task instead of an object detection task to better delineate the precise shape and boundary information. Digital elevation model (DEM) was utilized as primary data to directly obtain 3D information, which was converted into 3D point cloud format and fed to a multi-scale semantic segmentation network. The semantic segmentation results achieved an overall accuracy of 0.932 and mIOU of 0.871 on the test data. We automatically counted 63 craters in Noachis Terra and 40 craters in Terra Cimmeria. The 3D morphological measurements showed that 66% of the impact craters in the first region were larger than 10 km in diameter, while 50% of the impact craters in the second region were larger than 10 km. In both areas, craters could reach a maximum depth of 2000 m. With the proposed method, we can automatically conduct 3D morphological measurements of Martian craters with high efficiency that is improved by 15 times compared with that of manual crater analysis tools. The achieved 3D morphometric results can provide a reference and support for future research on Martian landforms.

Unharvested Palm Fruit Bunch Ripeness Detection with Hybrid Color Correction

Unharvested palm fruit bunch (PFB) ripeness detection is crucial for efficient palm oil production as labor costs rise and worker shortages grow. Challenges remain under varying illumination, affecting detection performance. Although the deep learning-based color constancy model shows promise, the limitation lies in the lack of data. This paper addresses these issues by proposing a hybrid color correction method that combines the learning-based and the physical-based color constancy technique, then trains with YOLOv8 to enhance unharvested PFB detection performance. Specifically, the captured images and their corresponding simultaneous spectra are used to generate the ground truth images, then map the images to other ambient spectra to train the color constancy model, and finally, train the YOLOv8 with the corrected images for ripeness detection. Results demonstrate a 1.5% improvement in the mean Average Precision (mAP) @0.5 of YOLOv8, increasing from 88.2% to 89.7% after applying the proposed hybrid color correction method. For each ripeness category, the mAP@0.5 for unripe and ripe bunches increased by 1.7% and 1.8%, respectively. Underripe bunches exhibit the most increment in mAP of 2.1%, while the overripe bunch gives the least increase of 0.4%. The qualitative results show that the model has enhanced the ability to distinguish subtle differences in PFB ripeness. Consequently, it advances the effectiveness and reliability of PFB ripeness detection in practical palm oil production scenarios, especially for unripe to ripe bunches wherein color gives the most information. This research demonstrates potential in object detection tasks that incorporate illumination-based color correction methods.

Mitigating gradient conflicts via expert squads in multi-task learning

The foundation of multi-task learning lies in the collaboration and interaction among tasks. However, in numerous real-world scenarios, certain tasks usually necessitate distinct, specialized knowledge. The mixing of these different task-specific knowledge often results in gradient conflicts during the optimization process, posing a significant challenge in the design of effective multi-task learning systems. This study proposes a straightforward yet effective multi-task learning framework that employs groups of expert networks to decouple the learning of task-specific knowledge and mitigate such gradient conflicts. Specifically, this approach partitions the feature channels into task-specific and shared components. The task-specific subsets are processed by dedicated experts to distill specialized knowledge. The shared features are captured by a point-wise aggregation layer from the whole outputs of all experts, demonstrating superior performance in capturing inter-task interactions. By considering both task-specific knowledge and shared features, the proposed approach exhibits superior performance in multi-task learning. Extensive experiments conducted on the PASCAL-Context and NYUD-v2 datasets have demonstrated the superiority of the proposed approach compared to other state-of-the-art methods. Furthermore, a benchmark dataset for multi-task learning in underwater scenarios has been developed, encompassing object detection and underwater image enhancement tasks. Comprehensive experiments on this dataset consistently validate the effectiveness of the proposed multi-task learning strategy. The source code is available at https://github.com/chenjie04/Multi-Task-Learning-PyTorch.

Using Convolutional Neural Networks for Image Semantic Segmentation and Object Detection

Convolutional neural networks are widely used for feature extraction in the fields of object detection and image segmentation. However, traditional CNN models often struggle to ensure accuracy in high noise environments. A study proposes an enhanced CNN model to improve its ability to recognize targets of different scales. This model combines multi-scale perceptual aggregation and feature alignment (MPAFA) mechanisms. This new method effectively combines low-level and high-level features, which helps to better identify objects of different sizes. The experimental results show that the proposed model achieved a segmentation accuracy of 99.6% on the Cityscapes dataset, and maintained an accuracy of 97.3% even with increased noise. Further experiments have shown that the model outperforms existing methods in terms of accuracy and recall. The experimental results show that the model exhibits excellent performance in object detection and segmentation tasks. This study provides a more effective strategy for processing complex images by optimizing network structure and enhancing feature fusion.

Small aircraft detection in infrared aerial imagery based on deep neural network

Detection of aerial target is an important part of infrared image processing. Both neural network method and traditional method can be used in infrared object detection. Neural network method has many advantages such as high accuracy and good portability compared with traditional object detection method. Since the features extracted by neural network method can change over detection target, automatic feature extraction makes neural network based detection method more effective. In recent years deep learning method has been also found wide use for object detection in images. In this paper, an object detection model based on the deep learning network YOLO is constructed for solving the infrared aircraft detection problem. We construct the dataset used for training and testing with recognized features being iteratively learned. The task of infrared otject detection is sensitive to model size and detection speed. There is a requirement of using quantization method to reduce the storage space and the computation complexity. We propose a quantized model with appropriate accuracy for infrared object detection task. To solve the detection task for multiple extremely small aircrafts, model adjustment and quantization are used in proposed model and it gets a better performance. Experimental results on the constructed dataset show that the storage space for weight after quantization shrinks to a quarter, and there is no precision loss for extremely small aircrafts compared to the original model. The optimized YOLO-based deep learning model is effective to detect the small aircraft target in infrared aerial imagery.

Representational drift in barrel cortex is receptive field dependent.

Cortical populations often exhibit changes in activity even when behavior is stable. How behavioral stability is maintained in the face of such "representational drift" remains unclear. One possibility is that some neurons are more stable than others. We examined whisker touch responses in layers 2-4 of the primary vibrissal somatosensory cortex (vS1) over several weeks in mice stably performing an object detection task with two whiskers. Although the number of touch neurons remained constant, individual neurons changed with time. Touch-responsive neurons with broad receptive fields were more stable than narrowly tuned neurons. Transitions between functional types were non-random: before becoming broadly tuned, unresponsive neurons first passed through a period of narrower tuning. Broadly tuned neurons in layers 2 and 3 with higher pairwise correlations to other touch neurons were more stable than neurons with lower correlations. Thus, a small population of broadly tuned and synchronously active touch neurons exhibits elevated stability and may be particularly important for behavior.

Adaptive Transformer-Based Multi-Modal Image Fusion for Real-Time Medical Diagnosis and Object Detection

In recent years, medical diagnosis and object detection have been significantly enhanced by the integration of multi-modal image fusion techniques. This study proposes an Adaptive Transformer-Based Multi-Modal Image Fusion (AT-MMIF) framework designed for real-time medical diagnosis and object detection. The framework employs a Transformer architecture to capture both global and local feature correlations across multiple imaging modalities, including MRI, CT, PET, and X-ray, for more accurate diagnostic results and faster object detection in medical imagery. The fusion process incorporates spatial and frequency-domain information to improve the clarity and detail of the output images, enhancing diagnostic accuracy. The adaptive attention mechanism within the Transformer dynamically adjusts to the relevant features of different image types, optimizing fusion in real time. This leads to an improved sensitivity (98.5%) and specificity (96.7%) in medical diagnosis. Additionally, the model significantly reduces false positives and negatives, with an F1 score of 97.2% in object detection tasks. The AT-MMIF framework is further optimized for real-time processing with an average inference time of 120 ms per image and a model size reduction of 35% compared to existing multi-modal fusion models. By leveraging the strengths of Transformer architectures and adaptive learning, the proposed framework offers a highly efficient and scalable solution for real-time medical diagnosis and object detection in various clinical settings, including radiology, oncology, and pathology.

MAR-YOLOv9: A multi-dataset object detection method for agricultural fields based on YOLOv9

With the development of deep learning technology, object detection has been widely applied in various fields. However, in cross-dataset object detection, conventional deep learning models often face performance degradation issues. This is particularly true in the agricultural field, where there is a multitude of crop types and a complex and variable environment. Existing technologies still face performance bottlenecks when dealing with diverse scenarios. To address these issues, this study proposes a lightweight, cross-dataset enhanced object detection method for the agricultural domain based on YOLOv9, named Multi-Adapt Recognition-YOLOv9 (MAR-YOLOv9). The traditional 32x downsampling Backbone network has been optimized, and a 16x downsampling Backbone network has been innovatively designed. A more streamlined and lightweight Main Neck structure has been introduced, along with innovative methods for feature extraction, up-sampling, and Concat connection. The hybrid connection strategy allows the model to flexibly utilize features from different levels. This solves the issues of increased training time and redundant weights caused by the detection neck and auxiliary branch structures in traditional YOLOv9, enabling MAR-YOLOv9 to maintain high performance while reducing the model’s computational complexity and improving detection speed, making it more suitable for real-time detection tasks. In comparative experiments on four plant datasets, MAR-YOLOv9 improved the mAP@0.5 accuracy by 39.18% compared to seven mainstream object detection algorithms, and by 1.28% compared to the YOLOv9 model. At the same time, the model size was reduced by 9.3%, and the number of model layers was decreased, reducing computational costs and storage requirements. Additionally, MAR-YOLOv9 demonstrated significant advantages in detecting complex agricultural images, providing an efficient, lightweight, and adaptable solution for object detection tasks in the agricultural field. The curated data and code can be accessed at the following link: https://github.com/YangxuWangamI/MAR-YOLOv9.

MSSA: Multi-Representation Semantics-Augmented Set Abstraction for 3D Object Detection

Accurate recognition and localization of 3D objects is a fundamental research problem in 3D computer vision. Benefiting from transformation-free point cloud processing and flexible receptive fields, point-based methods have become accurate in 3D point cloud modeling, but still fall behind voxel-based competitors in 3D detection. We observe that the set abstraction module, commonly utilized by point-based methods for downsampling points, tends to retain excessive irrelevant background information, thus hindering the effective learning of features for object detection tasks. To address this issue, we propose MSSA, a Multi-representation Semantics-augmented Set Abstraction for 3D object detection. Specifically, we first design a backbone network to encode different representation features of point clouds, which extracts point-wise features through PointNet to preserve fine-grained geometric structure features, and adopts VoxelNet to extract voxel features and BEV features to enhance the semantic features of key points. Second, to efficiently fuse different representation features of keypoints, we propose a Point feature-guided Voxel feature and BEV feature fusion (PVB-Fusion) module to adaptively fuse multi-representation features and remove noise. At last, a novel Multi-representation Semantic-guided Farthest Point Sampling (MS-FPS) algorithm is designed to help set abstraction modules progressively downsample point clouds, thereby improving instance recall and detection performance with more important foreground points. We evaluate MSSA on the widely used KITTI dataset and the more challenging nuScenes dataset. Experimental results show that compared to PointRCNN, our method improves the AP of “moderate” level for three classes of objects by 7.02%, 6.76%, and 5.44%, respectively. Compared to the advanced point-voxel-based method PV-RCNN, our method improves the AP of “moderate” level by 1.23%, 2.84%, and 0.55% for the three classes, respectively.

SOD-YOLO: A lightweight small object detection framework

Currently, lightweight small object detection algorithms for unmanned aerial vehicles (UAVs) often employ group convolutions, resulting in high Memory Access Cost (MAC) and rendering them unsuitable for edge devices that rely on parallel computing. To address this issue, we propose the SOD-YOLO model based on YOLOv7, which incorporates a DSDM-LFIM backbone network and includes a small object detection branch. The DSDM-LFIM backbone network, which combines Deep-Shallow Downsampling Modules (DSD Modules) and Lightweight Feature Integration Modules (LFI Modules), avoids excessive use of group convolutions and element-wise operations. The DSD Module focuses on extracting both deep and shallow features from feature maps using fewer parameters to obtain richer feature representations. The LFI Module, is a dual-branch feature integration module designed to consolidate feature information. Experimental results demonstrate that the SOD-YOLO model achieves an AP50 of 50.7% and a FPS of 72.5 on the VisDrone validation set. Compared to YOLOv7, our model reduces computational costs by 20.25% and decreases the number of parameters by 17.89%. After scaling the number of channels in the model, it achieves an AP50 of 33.4% with an inference time of 27.3ms on the Atlas 200I DK A2. These experimental results indicate that the SOD-YOLO model can effectively perform small object detection tasks in a large number of aerial images captured by UAVs.

YOLO-DroneMS: Multi-Scale Object Detection Network for Unmanned Aerial Vehicle (UAV) Images

In recent years, research on Unmanned Aerial Vehicles (UAVs) has developed rapidly. Compared to traditional remote-sensing images, UAV images exhibit complex backgrounds, high resolution, and large differences in object scales. Therefore, UAV object detection is an essential yet challenging task. This paper proposes a multi-scale object detection network, namely YOLO-DroneMS (You Only Look Once for Drone Multi-Scale Object), for UAV images. Targeting the pivotal connection between the backbone and neck, the Large Separable Kernel Attention (LSKA) mechanism is adopted with the Spatial Pyramid Pooling Factor (SPPF), where weighted processing of multi-scale feature maps is performed to focus more on features. And Attentional Scale Sequence Fusion DySample (ASF-DySample) is introduced to perform attention scale sequence fusion and dynamic upsampling to conserve resources. Then, the faster cross-stage partial network bottleneck with two convolutions (named C2f) in the backbone is optimized using the Inverted Residual Mobile Block and Dilated Reparam Block (iRMB-DRB), which balances the advantages of dynamic global modeling and static local information fusion. This optimization effectively increases the model’s receptive field, enhancing its capability for downstream tasks. By replacing the original CIoU with WIoUv3, the model prioritizes anchoring boxes of superior quality, dynamically adjusting weights to enhance detection performance for small objects. Experimental findings on the VisDrone2019 dataset demonstrate that at an Intersection over Union (IoU) of 0.5, YOLO-DroneMS achieves a 3.6% increase in mAP@50 compared to the YOLOv8n model. Moreover, YOLO-DroneMS exhibits improved detection speed, increasing the number of frames per second (FPS) from 78.7 to 83.3. The enhanced model supports diverse target scales and achieves high recognition rates, making it well-suited for drone-based object detection tasks, particularly in scenarios involving multiple object clusters.

ACT-FRCNN: Progress Towards Transformer-Based Object Detection

Maintaining a high input resolution is crucial for more complex tasks like detection or segmentation to ensure that models can adequately identify and reflect fine details in the output. This study aims to reduce the computation costs associated with high-resolution input by using a variant of transformer, known as the Adaptive Clustering Transformer (ACT). The proposed model is named ACT-FRCNN. Which integrates ACT with a Faster Region-Based Convolution Neural Network (FRCNN) for a detection task head. In this paper, we proposed a method to improve the detection framework, resulting in better performance for out-of-domain images, improved object identification, and reduced dependence on non-maximum suppression. The ACT-FRCNN represents a significant step in the application of transformer models to challenging visual tasks like object detection, laying the foundation for future work using transformer models. The performance of ACT-FRCNN was evaluated on a variety of well-known datasets including BSDS500, NYUDv2, and COCO. The results indicate that ACT-FRCNN reduces over-detection errors and improves the detection of large objects. The findings from this research have practical implications for object detection and other computer vision tasks.

Point cloud segmentation method based on an image mask and its application verification

Abstract Accurately perceiving three-dimensional (3D) environments or objects is crucial for the advancement of artificial intelligence (AI) interaction technologies. Currently, various types of sensors are employed to obtain point cloud data for 3D object detection or segmentation tasks. While this multi-sensor approach provides more precise 3D data than monocular or stereo cameras, it is also more expensive. The advent of RGB-D cameras, which provide both RGB images and depth information, addresses this issue.
In this study, we propose a point cloud segmentation method based on image masks. By using an RGB-D camera to capture color and depth images, we generate image masks through object recognition and segmentation. Given the mapping relationship between RGB image pixels and point clouds, these image masks can be further used to extract the point cloud data of the target objects. The experimental results revealed that the average accuracy of target segmentation was 84.78%, which was close to that of PointNet++. Compared with three traditional segmentation algorithms, the accuracy was improved by nearly 23.97%. The running time of our algorithm is reduced by 95.76% compared to the PointNet++ algorithm, which has the longest running time; and by 15.65% compared to the LCCP algorithm, which has the shortest running time among traditional methods. Compared with PointNet++, the segmentation accuracy was improved. This method addressed the issues of low robustness and excessive reliance on manual feature extraction in traditional point cloud segmentation methods, providing valuable support and reference for the accurate segmentation of 3D point clouds.

AMW-YOLOv8n: Road Scene Object Detection Based on an Improved YOLOv8

This study introduces an improved YOLOv8 model tailored for detecting objects in road scenes. To overcome the limitations of standard convolution operations in adapting to varying targets, we introduce Adaptive Kernel Convolution (AKconv). AKconv dynamically adjusts the convolution kernel’s shape and size, enhancing the backbone network’s feature extraction capabilities and improving feature representation across different scales. Additionally, we employ a Multi-Scale Dilated Attention (MSDA) mechanism to focus on key target features, further enhancing feature representation. To address the challenge posed by YOLOv8’s large down sampling factor, which limits the learning of small target features in deeper feature maps, we add a small target detection layer. Finally, to improve model training efficiency, we introduce a regression loss function with a Wise-IoU dynamic non-monotonic focusing mechanism. With these enhancements, our improved YOLOv8 model excels in road scene object detection tasks, achieving a 5.6 percentage point improvement in average precision over the original YOLOv8n on real road datasets.

Modular YOLOv8 optimization for real-time UAV maritime rescue object detection.

The task of UAV-based maritime rescue object detection faces two significant challenges: accuracy and real-time performance. The YOLO series models, known for their streamlined and fast performance, offer promising solutions for this task. However, existing YOLO-based UAV maritime rescue object detection methods tend to prioritize high accuracy, often at the expense of real-time performance and ease of implementation and expansion. This study proposes a modular plug-and-play optimization approach based on the YOLOv8 framework, aiming to enhance real-time performance while maintaining high accuracy for UAV maritime rescue object detection. The proposed optimization modules are flexible, easy to implement, and extendable. In experiments on the large-scale publicly available SeaDronesSee dataset, our method achieved a 13.53% improvement in accuracy over YOLOv8x while reducing computational cost by 85.63%. Additionally, it surpassed the detection speed of the SeaDronesSee official code's two-stage detector by over 20 times, while maintaining comparable accuracy. Furthermore, our analysis of the experimental results highlights differences in detection difficulty among various objects and potential biases within the dataset.