Object Detection Research Articles

MAR-YOLOv9: A multi-dataset object detection method for agricultural fields based on YOLOv9

With the development of deep learning technology, object detection has been widely applied in various fields. However, in cross-dataset object detection, conventional deep learning models often face performance degradation issues. This is particularly true in the agricultural field, where there is a multitude of crop types and a complex and variable environment. Existing technologies still face performance bottlenecks when dealing with diverse scenarios. To address these issues, this study proposes a lightweight, cross-dataset enhanced object detection method for the agricultural domain based on YOLOv9, named Multi-Adapt Recognition-YOLOv9 (MAR-YOLOv9). The traditional 32x downsampling Backbone network has been optimized, and a 16x downsampling Backbone network has been innovatively designed. A more streamlined and lightweight Main Neck structure has been introduced, along with innovative methods for feature extraction, up-sampling, and Concat connection. The hybrid connection strategy allows the model to flexibly utilize features from different levels. This solves the issues of increased training time and redundant weights caused by the detection neck and auxiliary branch structures in traditional YOLOv9, enabling MAR-YOLOv9 to maintain high performance while reducing the model’s computational complexity and improving detection speed, making it more suitable for real-time detection tasks. In comparative experiments on four plant datasets, MAR-YOLOv9 improved the mAP@0.5 accuracy by 39.18% compared to seven mainstream object detection algorithms, and by 1.28% compared to the YOLOv9 model. At the same time, the model size was reduced by 9.3%, and the number of model layers was decreased, reducing computational costs and storage requirements. Additionally, MAR-YOLOv9 demonstrated significant advantages in detecting complex agricultural images, providing an efficient, lightweight, and adaptable solution for object detection tasks in the agricultural field. The curated data and code can be accessed at the following link: https://github.com/YangxuWangamI/MAR-YOLOv9.

A Review of Deep Learning Based Object Detection Algorithms

With the rapid development of artificial intelligence technology, the traditional manual feature-based object detection method has been gradually replaced by deep learning-based object detection technology. Object detection is the basis and prerequisite for many computer vision tasks aimed at recognizing targets in an image and determining their class and location. The purpose of object detection algorithm research based on deep learning is to improve the detection accuracy and detection speed of the detection algorithm, so that the object detection algorithm can be more safe and convenient applied to People's Daily production and life. This article reviews the evolution of object detection algorithms, focusing on two-stage, one-stage and object detection algorithms based on the transformer architecture. In addition, the performance, advantages and disadvantages of different algorithms are compared, and the development trend of object detection algorithms based on deep learning is summarized in the end.

Knowledge Distillation and Student-Teacher Learning for Weed Detection in Turf

Abstract Machine vision-based herbicide applications relying on object detection or image classification deep convolutional neural network (DCNN) demand high memory and computational resources, resulting in lengthy inference times. To tackle these challenges, this study assessed the effectiveness of three teacher models, each trained on datasets of varying sizes, including D-20k (comprising 10,000 true positive and true negative images) and D-10k (comprising 5,000 true positive and true negative images). Additionally, knowledge distillation was performed on their corresponding student models across a range of temperature settings. After the process of student-teacher learning, the parameters of all student models were reduced. ResNet18 not only achieved higher accuracy (ACC≥0.989) but also maintained higher frames per second (FPS≥742.9) under its optimal temperature condition (T=1). Overall, the results suggest that employing knowledge distillation on the machine vision models enabled accurate and reliable weed detection in turf while reducing the need for extensive computational resources, thereby facilitating real-time weed detection and contributing to the development of smart, machine vision-based sprayers.

Advanced Image Classification Using Convolutional Neural Networks

Deep learning is essential for computer vision and object detection. This project explores the use of Convolutional Neural Networks (CNNs) for image classification using the CIFAR-10 dataset, which is widely used and provides a robust benchmark for evaluating image classification models. The CNN model, comprising various convolutional, pooling, and fully connected layers, is trained on a portion of the dataset and tested on another portion to evaluate its performance. Techniques such as data augmentation, dropout, and specific activation functions are employed to enhance the model’s efficacy. The study details the CNN architecture and reports the results, including accuracy metrics, to demonstrate the model’s effectiveness.

Review of Convolutional Neural Network

Over the past decade, the domain of deep learning has emerged as a swiftly expanding area of interest among researchers globally. It offers substantial benefits over traditional shallow networks, particularly in the realms of feature extraction and model fitting. Deep learning excels at uncovering intricate, distributed features from raw data, which exhibit robust generalization capabilities. It has triumphantly addressed challenges that were once deemed intractable within the field of artificial intelligence. With the exponential growth in the volume of training data and a marked increase in computational power, deep learning has achieved remarkable strides and found applications across a spectrum of domains, including but not limited to object detection, computer vision, natural language processing, speech recognition, and semantic parsing, thereby accelerating the evolution of artificial intelligence. Deep learning encompasses a series of non-linear transformations organized hierarchically, with deep neural networks representing the predominant form of contemporary deep learning methodologies. Inspired by the neural connections found in the animal visual cortex, these networks are characterized by local connectivity, shared weights, and pooling operations. Such attributes not only simplify the network model and curtail the number of trainable parameters but also engender invariance to shifts, distortions, and scaling, thereby endowing the model with enhanced robustness and fault tolerance. This makes the training and optimization of the network structure more manageable. This paper commences with a retrospective on the evolution of convolutional neural networks (CNNs). Subsequently, it delineates the architectures of neuron models and multilayer perceptron. It proceeds to dissect the CNN architecture, which typically encompasses a sequence of convolutional and pooling layers, culminating in fully connected layers, each serving distinct functions. The paper also elaborates on various enhanced algorithms for CNNs, such as the “Network in Network” and “Spatial Transformer Networks,” while also introducing supervised and unsupervised learning methodologies pertinent to CNNs and highlighting several prevalent open-source tools. Further, the paper scrutinizes the application of CNNs in various fields, including image classification, facial recognition, audio retrieval, electrocardiogram analysis, and object detection. It posits the amalgamation of CNNs with recurrent neural networks as a potential alternative for training datasets. The research culminates in the design of CNN structures with varying parameters and depths, supported by experimental analysis that elucidates the interplay among these parameters and the ramifications of their configuration. Ultimately, the paper synthesizes the merits and lingering challenges associated with CNNs and their practical applications.

MSSA: Multi-Representation Semantics-Augmented Set Abstraction for 3D Object Detection

Accurate recognition and localization of 3D objects is a fundamental research problem in 3D computer vision. Benefiting from transformation-free point cloud processing and flexible receptive fields, point-based methods have become accurate in 3D point cloud modeling, but still fall behind voxel-based competitors in 3D detection. We observe that the set abstraction module, commonly utilized by point-based methods for downsampling points, tends to retain excessive irrelevant background information, thus hindering the effective learning of features for object detection tasks. To address this issue, we propose MSSA, a Multi-representation Semantics-augmented Set Abstraction for 3D object detection. Specifically, we first design a backbone network to encode different representation features of point clouds, which extracts point-wise features through PointNet to preserve fine-grained geometric structure features, and adopts VoxelNet to extract voxel features and BEV features to enhance the semantic features of key points. Second, to efficiently fuse different representation features of keypoints, we propose a Point feature-guided Voxel feature and BEV feature fusion (PVB-Fusion) module to adaptively fuse multi-representation features and remove noise. At last, a novel Multi-representation Semantic-guided Farthest Point Sampling (MS-FPS) algorithm is designed to help set abstraction modules progressively downsample point clouds, thereby improving instance recall and detection performance with more important foreground points. We evaluate MSSA on the widely used KITTI dataset and the more challenging nuScenes dataset. Experimental results show that compared to PointRCNN, our method improves the AP of “moderate” level for three classes of objects by 7.02%, 6.76%, and 5.44%, respectively. Compared to the advanced point-voxel-based method PV-RCNN, our method improves the AP of “moderate” level by 1.23%, 2.84%, and 0.55% for the three classes, respectively.

Design and development of machine vision robotic arm for vegetable crops in hydroponics

Modern agricultural machinery such as harvesting robotic arms are necessary to optimize the semi-automated agricultural processes to meet the ever-growing food needs of the population of the world. Production of low cost and highly efficient robotic arms are needed to meet the needs of the small-scale growers of underdeveloped countries. In this regard, mechanical design, sensor placement, and object identification are essential to optimize the performance of agricultural robots. This research is conducted to design a fully automated robotic arm for fruit picking in hydroponics system and research farm using machine vision to help the smallholders. For this purpose, a robotic arm with 1.3 m in height is designed using aluminum (18-gauge) and mild steel (MS-16) as fabrication material for the mechanical structure. The prototype design is tested for stress estimation at various payloads using SolidWorks simulations. The proposed robotic arm is based on a four degrees of freedom (4-DoF) design with the gripper having an opening range of 5.5 to 12 cm depending upon the task to handle various objects. The robotic arm is installed on an expandable vertical mobile platform that enables an expanded operational workspace with lower energy consumption. The total weight of the robotic arm is 60 kgs housing a payload capacity ranging between 8 and 10 kgs in accordance with industry-standard load-bearing specifications. The inverse kinematic algorithm using DH-table is computed with an accuracy of up to 95 % and target height of gripper is cross checked by mounting the ultrasonic sensor at the base of robotic arm. The YOLOv8 algorithm is used for object detection using the depth sense camera having angle range of 0.3 m to 3 m with the precision of up to 96 %. The time needed for recognition of fruits and pitching was about 15 s, with a success rate of up to 90 %. This research produces a low-cost and efficient solution for stallholders by developing a fully automated robotic arm for fruit picking. It optimizes agricultural productivity through improved mechanical design, sensor accuracy, and object detection which reduces labor costs by increasing efficiency of semi-automated farming systems. Further research is required to optimize the overall weight of the system and automation of the movement of robotic arm system between the consecutive rows to build a fully autonomous fruit picking operation in hydroponics.

Capacity Constraint Analysis Using Object Detection for Smart Manufacturing

The increasing adoption of Deep Learning (DL)-based Object Detection (OD) models in smart manufacturing has opened up new avenues for optimizing production processes. Traditional industries facing capacity constraints require noninvasive methods for in-depth operations analysis to optimize processes and increase revenue. In this study, we propose a novel framework for capacity constraint analysis that identifies bottlenecks in production facilities and conducts cycle time studies using an end-to-end pipeline. This pipeline employs a Convolutional Neural Network (CNN)-based OD model to accurately identify potential objects on the production floor, followed by a CNN-based tracker to monitor their lifecycle in each workstation. The extracted metadata are further processed through the proposed framework. Our analysis of a real-world manufacturing facility over six months revealed that the bottleneck station operated at only 73.1% productivity, falling to less than 40% on certain days; additionally, the processing time of each item increased by 53% during certain weeks due to critical labor and materials shortages. These findings highlight significant opportunities for process optimization and efficiency improvements. The proposed pipeline can be extended to other production facilities where manual labor is used to assemble parts, and can be used to analyze and manage labor and materials over time as well as to conduct audits and improve overall yields, potentially transforming capacity management in smart manufacturing environments.

CWGA-Net: Center-Weighted Graph Attention Network for 3D object detection from point clouds

The precision of 3D object detection from unevenly distributed outdoor point clouds is critical in autonomous driving perception systems. Current point-based detectors employ self-attention and graph convolution to establish contextual relationships between point clouds; however, they often introduce weakly correlated redundant information, leading to blurred geometric details and false detections. To address this issue, a novel Center-weighted Graph Attention Network (CWGA-Net) has been proposed to fuse geometric and semantic similarities for weighting cross-attention scores, thereby capturing precise fine-grained geometric features. CWGA-Net initially constructs and encodes local graphs between foreground points, establishing connections between point clouds from geometric and semantic dimensions. Subsequently, center-weighted cross-attention is utilized to compute the contextual relationships between vertices within the graph, and geometric and semantic similarities between vertices are fused to weight attention scores, thereby extracting strongly related geometric shape features. Finally, a cross-feature fusion Module is introduced to deeply fuse high and low-resolution features to compensate for the information loss during downsampling. Experiments conducted on the KITTI and Waymo datasets demonstrate that the network achieves superior detection capabilities, outperforming state-of-the-art point-based single-stage methods in terms of average precision metrics while maintaining good speed.

Transformative Impact of Convolutional Neural Networks on Healthcare, Autonomous Systems, and Global Technological Advancements

Convolutional Neural Networks (CNNs) have significantly impacted various industries by enabling machines to process visual data with unprecedented accuracy. This paper explores the transformative effects of CNNs on key sectors such as healthcare, autonomous systems, security, and environmental monitoring. In healthcare, CNNs are used in medical imaging for early diagnosis and treatment planning, enhancing telemedicine capabilities. In autonomous systems, CNNs enable real-time object detection and navigation, contributing to the development of safer, more efficient vehicles. In security, CNNs power facial recognition systems, raising both opportunities and ethical concerns regarding privacy and surveillance. Environmental monitoring benefits from CNNs through climate change research and wildlife conservation efforts, where they analyze vast amounts of data for critical insights. The paper also addresses the challenges posed by CNNs, including data privacy, security, and algorithmic bias, emphasizing the need for ethical standards and regulatory frameworks. Future advancements in CNN architectures and their integration with other AI models are expected to expand their applicability further and improve performance across different domains. This analysis underscores the dual importance of technological progress and ethical considerations to ensure that the benefits of CNNs are equitably distributed, contributing positively to global technological development and societal well-being.

Hierarchical evidence aggregation in two dimensions for active water surface object detection

Hierarchical evidence aggregation in two dimensions for active water surface object detection

A Small-Scale Object Detection Algorithm in Intelligent Transportation Scenarios

In response to the problem of poor detection ability of object detection models for small-scale targets in intelligent transportation scenarios, a fusion method is proposed to enhance the features of small-scale targets, starting from feature utilization and fusion methods. The algorithm is based on the YOLOv4 tiny framework and enhances the utilization of shallow and mid-level features on the basis of Feature Pyramid Network (FPN), improving the detection accuracy of small and medium-sized targets. In view of the problem that the background of the intelligent traffic scene image is cluttered, and there is more redundant information, the Convolutional Block Attention Module (CBAM) is used to improve the attention of the model to the traffic target. To address the problem of data imbalance and prior bounding box adaptation in custom traffic data sets that expand traffic images in COCO and VOC, we propose a Copy-Paste method with an improved generation method and a K-means algorithm with improved distance measurement to enhance the model’s detection ability for corresponding categories. Comparative experiments were conducted on a customized 260-thousand traffic data set containing public traffic images, and the results showed that compared to YOLOv4 tiny, the proposed algorithm improved mAP by 4.9% while still ensuring the real-time performance of the model.

ScaledDETR: An alight weight object detection model for autonomous driving

ScaledDETR: An alight weight object detection model for autonomous driving

Research on Deep Learning Detection Model for Pedestrian Objects in Complex Scenes Based on Improved YOLOv7.

Pedestrian detection is very important for the environment perception and safety action of intelligent robots and autonomous driving, and is the key to ensuring the safe action of intelligent robots and auto assisted driving. In response to the characteristics of pedestrian objects occupying a small image area, diverse poses, complex scenes and severe occlusion, this paper proposes an improved pedestrian object detection method based on the YOLOv7 model, which adopts the Convolutional Block Attention Module (CBAM) attention mechanism and Deformable ConvNets v2 (DCNv2) in the two Efficient Layer Aggregation Network (ELAN) modules of the backbone feature extraction network. In addition, the detection head is replaced with a Dynamic Head (DyHead) detector head with an attention mechanism; unnecessary background information around the pedestrian object is also effectively excluded, making the model learn more concentrated feature representations. Compared with the original model, the log-average miss rate of the improved YOLOv7 model is significantly reduced in both the Citypersons dataset and the INRIA dataset. The improved YOLOv7 model proposed in this paper achieved good performance improvement in different pedestrian detection problems. The research in this paper has important reference significance for pedestrian detection in complex scenes such as small, occluded and overlapping objects.

PE-MCAT: Leveraging Image Sensor Fusion and Adaptive Thresholds for Semi-Supervised 3D Object Detection.

Existing 3D object detection frameworks in sensor-based applications heavily rely on large-scale annotated data to achieve optimal performance. However, obtaining such annotations from sensor data-like LiDAR or image sensors-is both time-consuming and costly. Semi-supervised learning offers an efficient solution to this challenge and holds significant potential for sensor-driven artificial intelligence (AI) applications. While it reduces the need for labeled data, semi-supervised learning still depends on a small amount of labeled samples for training. In the initial stages, relying on such limited samples can adversely affect the effective training of student-teacher networks. In this paper, we propose PE-MCAT, a semi-supervised 3D object detection method that generates high-precision pseudo-labels. First, to address the challenges of insufficient local feature capture and poor robustness in point cloud data, we introduce a point enrichment module. This module incorporates information from image sensors and combines multiple feature fusion methods of local and self-features to directly enhance the quality of point clouds and pseudo-labels, compensating for the limitations posed by using only a few labeled samples. Second, we explore the relationship between the teacher network and the pseudo-labels it generates. We propose a multi-class adaptive threshold strategy to initially filter and create a high-quality pseudo-label set. Furthermore, a joint variable threshold strategy is introduced to refine this set further, enhancing the selection of superior pseudo-labels.Extensive experiments demonstrate that PE-MCAT consistently outperforms recent state-of-the-art methods across different datasets. Specifically, on the KITTI dataset and using only 2% of labeled samples, our method improved the mean Average Precision (mAP) by 0.7% for cars, 3.7% for pedestrians, and 3.0% for cyclists.

BGF-YOLOv10: Small Object Detection Algorithm from Unmanned Aerial Vehicle Perspective Based on Improved YOLOv10.

With the rapid development of deep learning, unmanned aerial vehicles (UAVs) have acquired intelligent perception capabilities, demonstrating efficient data collection across various fields. In UAV perspective scenarios, captured images often contain small and unevenly distributed objects, and are typically high-resolution. This makes object detection in UAV imagery more challenging compared to conventional detection tasks. To address this issue, we propose a lightweight object detection algorithm, BGF-YOLOv10, specifically designed for small object detection, based on an improved version of YOLOv10n. First, we introduce a novel YOLOv10 architecture tailored for small objects, incorporating BoTNet, variants of C2f and C3 in the backbone, along with an additional small object detection head, to enhance detection performance for small objects. Second, we embed GhostConv into both the backbone and head, effectively reducing the number of parameters by nearly half. Finally, we insert a Patch Expanding Layer module in the neck to restore the feature spatial resolution. Experimental results on the VisDrone-DET2019 and UAVDT datasets demonstrate that our method significantly improves detection accuracy compared to YOLO series networks. Moreover, when compared to other state-of-the-art networks, our approach achieves a substantial reduction in the number of parameters.

Research on Impact Prediction Model for Corn Ears by Integrating Motion Features Using Machine Learning Algorithms

The mechanical damage of corn kernels during harvest leads to mildew in the kernel storage process, seriously affecting food safety and quality. Impact force is the primary source of mechanical damage in the corn threshing process, and its accurate detection is of great significance for corn threshing with low damage. A method for the impact force detection of corn ears was proposed in this manuscript. The momentum theorem determined the main factors influencing impact force (weight, falling height, and space attitude). Corn ear weight, falling height, and space attitude were used as experimental factors. The bench test was carried out with the impact force of corn ear as the output variable. During the experiment, piezoelectric sensors were used to collect the impact force of corn ears under different motion states. Then, the impact force detection model was constructed using four machine learning algorithms: multiple linear regression, ridge regression, random forest, and support vector regression. The results showed that the RF algorithm was more suitable for constructing the prediction model of average and maximum impact force when corn ears fall, SD, RMSE, and r were, respectively: 0.9526, 1.2685, 0.9855; 3.8389, 3.6071, and 0.8510. Secondly, the weight characteristics had the most significant influence on the impact force detection of the ear. Therefore, this method can be used as an accurate, objective, and efficient online detection method for impact force.

Eyes on nature: Embedded vision cameras for terrestrial biodiversity monitoring

Abstract We need comprehensive information to manage and protect biodiversity in the face of global environmental challenges, and artificial intelligence is required to generate that information from vast amounts of biodiversity data. Currently, vision‐based monitoring methods are heterogenous; they poorly cover spatial and temporal dimensions, overly depend on humans, and are not reactive enough for adaptive management. To mitigate these issues, we present a portable, modular, affordable and low‐power device with embedded vision for biodiversity monitoring of a wide range of terrestrial taxa. Our camera uses interchangeable lenses to resolve barely visible and remote targets, as well as customisable algorithms for blob detection, region‐of‐interest classification and object detection to automatically identify them. We showcase our system in six use cases from ethology, landscape ecology, agronomy, pollination ecology, conservation biology and phenology disciplines. Using the same devices with different setups, we discovered bats feeding on durian tree flowers, monitored flying bats and their insect prey, identified nocturnal insect pests in paddy fields, detected bees visiting rapeseed crop flowers, triggered real‐time alerts for waterfowl and tracked flower phenology over months. We measured classification accuracies (i.e. F1‐scores) between 55% and 95% in our field surveys and used them to standardise observations over highly resolved time scales. Our cameras are amenable to situations where automated vision‐based monitoring is required off the grid, in natural and agricultural ecosystems, and in particular for quantifying species interactions. Embedded vision devices such as this will help addressing global biodiversity challenges and facilitate a technology‐aided agricultural systems transformation.

Bearing defect detection based on the improved YOLOv5 algorithm.

In the field of bearing defect detection, Aiming at the problem of low efficiency in manual inspection and prone to missed detections in scenarios with small target defects, and overlapping targets, an improved YOLOv5-based object detection method is proposed. Firstly, in terms of feature extraction, the C3 modules in the original backbone of YOLOv5 are replaced with the finer-grained Res2Block modules to improve the model's feature extraction ability. Secondly, in terms of feature fusion, a Bidirectional Feature Pyramid Network (BiFPN) is added to the original neck of YOLOv5 to enhance the fusion ability of shallow graphic features and deep semantic features. Finally, the performance of the improved YOLOv5 algorithm is validated through ablation experiments and comparative experiments with other defect detection algorithms, including the Small_obj algorithm the existing method of adding a small target detection head for identifying small target defects. The experimental results demonstrate that the improved YOLOv5 algorithm exhibits high mAP and accuracy in bearing defect detection, enabling more precise identification the types of small target defects on bearings in complex scenarios with multiple coexisting defects and overlapping detection targets, thereby providing valuable reference for practical bearing defect detection.

Estimation of Strawberry Canopy Volume in Unmanned Aerial Vehicle RGB Imagery Using an Object Detection-Based Convolutional Neural Network.

Estimating canopy volumes of strawberry plants can be useful for predicting yields and establishing advanced management plans. Therefore, this study evaluated the spatial variability of strawberry canopy volumes using a ResNet50V2-based convolutional neural network (CNN) model trained with RGB images acquired through manual unmanned aerial vehicle (UAV) flights equipped with a digital color camera. A preprocessing method based on the You Only Look Once v8 Nano (YOLOv8n) object detection model was applied to correct image distortions influenced by fluctuating flight altitude under a manual maneuver. The CNN model was trained using actual canopy volumes measured using a cylindrical case and small expanded polystyrene (EPS) balls to account for internal plant spaces. Estimated canopy volumes using the CNN with flight altitude compensation closely matched the canopy volumes measured with EPS balls (nearly 1:1 relationship). The model achieved a slope, coefficient of determination (R2), and root mean squared error (RMSE) of 0.98, 0.98, and 74.3 cm3, respectively, corresponding to an 84% improvement over the conventional paraboloid shape approximation. In the application tests, the canopy volume map of the entire strawberry field was generated, highlighting the spatial variability of the plant's canopy volumes, which is crucial for implementing site-specific management of strawberry crops.