Average Detection Precision Research Articles

Detecting Logos for Indoor Environmental Perception Using Unsupervised and Few-Shot Learning

Indoor scenes are crucial components of urban spaces, with logos serving as vital information within these environments. The accurate perception of logos is essential for effectively operating mobile robots in indoor environments, which significantly contributes to many upper-level applications. With the rapid development of neural networks, numerous deep-learning-based object-detection methods have been applied to logo detection. However, most of these methods depend on large labeled datasets. Given the fast-changing nature of logos in indoor scenes, achieving reliable detection performance with either the existing large labeled datasets or a limited number of labeled logos remains challenging. In this article, we propose a method named MobileNetV2-YOLOv4-UP, which integrates unsupervised learning with few-shot learning for logo detection. We develop an autoencoder to obtain latent feature representations of logos by pre-training on a public unlabeled logo dataset. Subsequently, we construct a lightweight logo-detection network and embed the encoder weights as prior information. Training is performed on a small dataset of labeled indoor-scene logos to update the weights of the logo-detection network. Experimental results on the public logo625 dataset and our self-collected LOGO2000 dataset demonstrate that our method outperforms classic object-detection methods, achieving a mean average detection precision of 83.8%. Notably, our unsupervised pre-training strategy (UP) has proven effective, delivering a 15.4% improvement.

Domain Feature Decomposition for Efficient Object Detection in Aerial Images

Object detection in UAV aerial images faces domain-adaptive challenges, such as changes in shooting height, viewing angle, and weather. These changes constitute a large number of fine-grained domains that place greater demands on the network’s generalizability. To tackle these challenges, we initially decompose image features into domain-invariant and domain-specific features using practical imaging condition parameters. The composite feature can improve domain generalization and single-domain accuracy compared to the conventional fine-grained domain-detection method. Then, to solve the problem of the overfitting of high-frequency imaging condition parameters, we mixed images from different imaging conditions in a balanced sampling manner as input for the training of the detection network. The data-augmentation method improves the robustness of training and reduces the overfitting of high-frequency imaging parameters. The proposed algorithm is compared with state-of-the-art fine-grained domain detectors on the UAVDT and VisDrone datasets. The results show that it achieves an average detection precision improvement of 5.7 and 2.4, respectively. The airborne experiments validate that the algorithm achieves a 20 Hz processing performance for 720P images on an onboard computer with Nvidia Jetson Xavier NX.

Mask R-CNN-Based Stone Detection and Segmentation for Underground Pipeline Exploration Robots

Stones are one of the primary objects that impede the normal activity of underground pipelines. As human intervention is difficult inside a narrow underground pipe, a robot with a machine vision system is required. In order to remove the stones during regular robotic inspections, precise stone detection, segmentation, and measurement of their distance from the robot are needed. We applied Mask R-CNN to perform an instant segmentation of stones. The distance between the robot and the segmented stones was calculated using spatial information obtained from a lidar camera. Artificial light was used for both image acquisition and testing, as natural light is not available inside the underground pipe. ResNet101 was chosen as the foundation of the Mask R-CNN, and transfer learning was utilized to shorten the training time. The experimental results of our model showed that the average detection precision rate reached 92.0; the recall rate was 90.0%; and the F1 score rate reached 91.0%. The distance values were calculated efficiently with an error margin of 11.36 mm. Moreover, the Mask R-CNN-based stone detection model can detect asymmetrically shaped stones in complex background and lighting conditions.

Monitoring fast‐moving animals—Building a customized camera system and evaluation toolset

Abstract Automated cameras (including camera traps) are an established observation tool, allowing, for example the identification of behaviours and monitoring without harming organisms. However, limitations including imperfect detection, insufficient data storage and power supply restrict the use of camera traps, making inexpensive and customizable solutions desirable. We describe a camera system and evaluation toolset based on Raspberry Pi computers and YOLOv5 that can overcome those shortcomings with its modular properties. We facilitate the set‐up and modification for researchers via detailed step‐by‐step guides. A customized camera system prototype was constructed to monitor fast‐moving organisms on a continuous schedule. For testing and benchmarking, we recorded mason bees (Osmia cornuta) approaching nesting aids on 20 sites. To efficiently process the extensive video material, we developed an evaluation toolset utilizing the convolutional neural network YOLOv5 to detect bees in the videos. In the field test, the camera system performed reliably for more than a week (2 h per day) under varying weather conditions. YOLOv5 detected and classified bees with only 775 original training images. Overall detection reliability varied depending on camera perspective, site and weather conditions, but a high average detection precision (78%) was achieved, which was confirmed by a human observer (80% of algorithm‐based detections confirmed). The customized camera system mitigates several disadvantages of commercial camera traps by using interchangeable components and incorporates all major requirements a researcher has for working in the field including moderate costs, easy assembly and an external energy source. We provide detailed user guides to bridge the gap between ecology, computer science and engineering.

Damage Object Detection of Steel Wire Rope-Core Conveyor Belts Based on the Improved YOLOv5

In response to the challenges in detecting damage features in X-ray images of steel wire rope-cores in conveyor belts, such as complex damage shapes, small sizes, low detection precision, and poor generalization ability, an improved YOLOv5 algorithm was proposed. The aim of the model is to accurately and efficiently identify and locate damage in the X-ray images of steel wire rope-cores in conveyor belts. First, the Adaptive Histogram Equalization (AHE) method is used to preprocess the images, reducing the interference of harsh mining environments and improving the quality of the dataset. Second, to better retain image details and enhance the detection ability of damage features, transpose convolutional upsampling is adopted, and the C3 module in the backbone network is replaced by C2f to ensure lightweight network models, meanwhile, it obtains richer gradient flow information and optimizing the loss function. Finally, the improved algorithm is compared with four classical detection algorithms using the damage feature dataset of steel wire rope-core conveyor belts. The experimental result shows that the proposed algorithm achieves an average detection precision of 91.8% and a detection speed of 40 frames per second (FPS) for images collected in harsh mining environments. The designed detection model provides a reference for the automatic recognition and detection of damage to steel wire rope-core conveyor belts.

Design and experiment of Panax notoginseng root orientation transplanting device based on YOLOv5s.

Consistent root orientation is one of the important requirements of Panax notoginseng transplanting agronomy. In this paper, a Panax notoginseng orientation transplanting method based on machine vision technology and negative pressure adsorption principle was proposed. With the cut-main root of Panax notoginseng roots as the detection object, the YOLOv5s was used to establish a root feature detection model. A Panax notoginseng root orientation transplanting device was designed. The orientation control system identifies the root posture according to the detection results and controls the orientation actuator to adjust the root posture. The detection results show that the precision rate of the model was 94.2%, the recall rate was 92.0%, and the average detection precision was 94.9%. The Box-Behnken experiments were performed to investigate the effects of suction plate rotation speed, servo rotation speed and the angle between the camera and the orientation actuator(ACOA) on the orientation qualification rate and root drop rate. Response surface method and objective optimisation algorithm were used to analyse the experimental results. The optimal working parameters were suction plate rotation speed of 5.73 r/min, servo rotation speed of 0.86 r/s and ACOA of 35°. Under this condition, the orientation qualification rate and root drop rate of the actual experiment were 89.87% and 6.57%, respectively, which met the requirements of orientation transplanting for Panax notoginseng roots. The research method of this paper is helpful to solve the problem of orientation transplanting of other root crops.

An Effective Dataset Preprocessing Method in Tilted Gear Defects Target Detection

Gear defect detection is a crucial component in power automation systems. Methods based on deep learning have exhibited excellent performance in detecting gears. However, the effect of defect detection for tilted gears is not as good. This is due to the traditional horizontal bounding box annotation method, which inevitably produces many overlapping areas in the annotated bounding box when used in tilted gear targets, and a large part of the areas do not belong to defects. In order to address the issue of low precision in the detection of defects in tilted gears based on deep learning, a dataset preprocessing scheme has been proposed. Initially, the images and annotation files of the training set and validation set are automatically rotated. Subsequently, the rotated data are utilized to train the defect detection model. Finally, the test set is subjected to the same image rotation method as the first step, resulting in the generation of high‐precision defect detection results, which are then input into the defect detection model. In order to facilitate comparison with the ground truth, the JSON file obtained from the detection result is rotated in reverse and the result is mapped to the original test set before rotation. In order to verify the effectiveness of the dataset optimization method proposed in this article, the same configuration file is used to train and evaluate the gear defect detection model with the dataset before and after optimization, respectively. The three detection models with the highest comprehensive indicators were employed to assess the dataset before and after optimization, with the average detection precision mAP serving as a quantitative comparison indicator. The findings demonstrate that the data optimization method proposed in this article has markedly enhanced mAP in tilted gear defect detection. Upon testing the proposed method on the PP‐YOLOE + model, mAP (0.5) exhibited an increase of 18.5%, while mAP (0.5 : 0.95) demonstrated a 29.1% improvement. When the method was tested on the RT‐DETR model, mAP (0.5) exhibited an increase of 24.7%, while mAP (0.5 : 0.95) demonstrated an 18.4% improvement. When tested on the few‐short model, mAP (0.5) increased by 23.3% and mAP (0.5 : 0.95) increased by 19.8%. This validates the effectiveness of the dataset processing method proposed in this article in the detection of tilted gear defects.

Segmentation of wheat scab fungus spores based on CRF_ResUNet++

Accurate segmentation of tiny spore pictures is a significant technique in computer-aided evaluation of wheat scab. A CRF_ResUNet++ spore segmentation model using UNet++ as the fundamental framework is proposed to address the issues of uneven brightness, low spore-background contrast, and spore adhesion in microscopically collected Fusarium spore images. ResNet and fully connected conditional random fields (CRF) are combined in this model to address these issues. First, a residual block ResNet is introduced into UNet++ to enhance the propagation of features and extract more spore detail information. Next, the spore image is initially segmented using UNet++ with an encoder-decoder structure. Finally, postprocessing is carried out using a fully connected conditional random field model to obtain more precise edges and complete spore regions. The laboratory collection dataset test results revealed that the F1-score and MIoU reached 0.943 and 0.925, respectively, with an average detection Precision improvement of 3.2 % over the original UNet++ model. In comparison to existing models, the suggested model can segment spore pictures in challenging circumstances, such as when they contain independent and inter-adherent spores, with excellent computer vision processing capabilities effects and objective evaluation metric segmentation effects. In conclusion, the method in this study can reliably detect and segment adherent wheat scab spores in complex backgrounds, which offers critical technical support for automatic detection of adherent wheat scab spores and early spore concentration prediction in complex environments in the field.

Improved YOLOv7 Algorithm for Detecting Occluded Targets at Aircraft Turnaround Milestone

The aircraft and special vehicles in the aircraft turnaround milestone have the characteristics of large-scale difference, small spacing distance, etc., and many external interference factors result in many occlusion situations in the process of services. Aiming at the problem that existing airport detection algorithms are ineffective in detecting occluded targets in milestones, an improved YOLOv7 occluded target detection algorithm for aircraft turnaround milestones is proposed. The algorithm adds an efficient channel attention mechanism to the backbone network to suppress the interference information in the environment and improve the attention to the important channels and replaces the convolution in the feature fusion network with a double-pooled coordinate convolution, which is used to improve the model’s ability to extract the spatial information of the occluded target and the remaining features. The algorithm achieves an average detection precision of 89.7% and a recall rate of 88.3% on the aircraft turnaround milestone dataset covering numerous occlusion scenarios, which are 6.1% and 4.4% higher than the YOLOv7 algorithm, respectively. The experiments show that the algorithm can better cope with the task of occlusion target detection in aircraft turnaround milestones.

Feature extraction and segmentation of pavement distress using an improved hybrid task cascade network

ABSTRACT An improved Hybrid Task Cascade (HTC) instance segmentation model was proposed to segment pavement surface distress intelligently and accurately. First, the automatic color equalization (ACE) algorithm was performed to preprocess the dataset. Then, HTC and the proposed HTC were described. The proposed HTC was modified using the ResNeXt101 network, balanced feature pyramid (BFP) and deformable convolution (DCN). The Soft non-maximum suppression method (Soft-NMS) and balanced L1 loss were used to supplement the above work. Finally, the average detection precision (AP) and the segmentation AP indexes were assessed by comparison and contrast. Experimental results demonstrated that the original HTC has crack miss detection and false detection problems. In contrast, the improved HTC has a recall rate as high as 96.1%, indicating a low miss-detection rate. Compared with other mainstream instance segmentation models, the improved HTC still leads in various indicators on our or public datasets. Comparisons between the predicted and true distress areas were analysed, the relative error of which was reasonable and satisfactory. The results help judge the degree of pavement distress and provide data support for repair and maintenance, which has essential engineering significance.

Rapid Detection of Multi-QR Codes Based on Multistage Stepwise Discrimination and a Compressed MobileNet

Poor real-time performance in multi-QR codes detection has been a bottleneck in QR code decoding based Internet-of-Things (IoT) systems. To tackle this issue, we propose in this paper a rapid detection approach, which consists of Multistage Stepwise Discrimination (MSD) and a Compressed MobileNet. Inspired by the object category determination analysis, the preprocessed QR codes are extracted accurately on a small scale using the MSD. Guided by the small scale of the image and the end-to-end detection model, we obtain a lightweight Compressed MobileNet in a deep weight compression manner to realize rapid inference of multi-QR codes. The Average Detection Precision (ADP), Multiple Box Rate (MBR) and running time are used for quantitative evaluation of the efficacy and efficiency. Compared with a few state-of-the-art methods, our approach has higher detection performance in rapid and accurate extraction of all the QR codes. The approach is conducive to embedded implementation in edge devices along with a bit of overhead computation to further benefit a wide range of real-time IoT applications.

Monocular 3D Object Detection Using Depth Fusion

It is an important task to estimate a 3D bounding box from monocular images for autonomous driving. However, the monocular pictures do not have distance information, so it is difficult to acquire accurate results. For the sake of solving the trouble of low accuracy of the monocular image in 3D target detection because of lacking distance information, an improved monocular three-dimensional target detection algorithm based on GUPNet and neural network was proposed to promote the precision of target detection. First, based on the geometric method proposed by GUPNet, the depth, and uncertainty are obtained by direct regression using a neural network. According to the difference in the accuracy of the two methods, a parameter α was introduced, and their depth scores are obtained from the uncertainty. According to the depth score and parameter α, the depth obtained by the two methods is fused to get the final depth. Test results prove that the proposed algorithm promotes average detection precision of KITTI data set in simple, medium, and difficult cases.

Soft Actor–Critic-Based Multilevel Cooperative Perception for Connected Autonomous Vehicles

Cooperative perception is an effective way for connected autonomous vehicles to extend sensing range, improve detection precision, and thus enhance perception ability by combining their own sensing information with that of other vehicles. The existing cooperation perception schemes share only raw-, feature-, or object-level data, thus lacking the flexibility to adapt to highly dynamic vehicular network conditions, which leads to either bandwidth saturation or bandwidth underutilization, degrading the detection precision in the long run. In this article, we propose ML-Cooper, a multilevel cooperative perception framework, to fully utilize the bandwidth and hence improve detection precision. The key idea of ML-Cooper is to divide each frame of sensing data of the sender vehicle into three parts, and the corresponding raw data, feature data, and object data are transmitted to and fused at the receiver vehicle. We also develop a soft actor–critic (SAC) deep reinforcement learning algorithm to adaptively adjust the proportion of the three parts according to the channel state information of the Vehicle-to-Vehicle (V2V) link. The experimental results on KITTI and our collected data sets on two real vehicles show that ML-Cooper can achieve the highest average detection precision compared to existing single-level cooperative perception schemes.

An Improved Crucible Spatial Bubble Detection Based on YOLOv5 Fusion Target Tracking.

A three-dimensional spatial bubble counting method is proposed to solve the problem of the existing crucible bubble detection only being able to perform two-dimensional statistics. First, spatial video images of the transparent layer of the crucible are acquired by a digital microscope, and a quartz crucible bubble dataset is constructed independently. Secondly, to address the problems of poor real-time and the insufficient small-target detection capability of existing methods for quartz crucible bubble detection, rich detailed feature information is retained by reducing the depth of down-sampling in the YOLOv5 network structure. In the neck, the dilated convolution algorithm is used to increase the feature map perceptual field to achieve the extraction of global semantic features; in front of the detection layer, an effective channel attention network (ECA-Net) mechanism is added to improve the capability of expressing significant channel characteristics. Furthermore, a tracking algorithm based on Kalman filtering and Hungarian matching is presented for bubble counting in crucible space. The experimental results demonstrate that the detector algorithm presented in this paper can effectively reduce the missed detection rate of tiny bubbles and increase the average detection precision from 96.27% to 98.76% while reducing weight by half and reaching a speed of 82 FPS. The excellent detector performance improves the tracker’s accuracy significantly, allowing for real-time and high-precision counting of bubbles in quartz crucibles. It is an effective method for detecting crucible spatial bubbles.

Augmentation Method for High Intra-Class Variation Data in Apple Detection

Deep learning is widely used in modern orchard production for various inspection missions, which helps improve the efficiency of orchard operations. In the mission of visual detection during fruit picking, most current lightweight detection models are not yet effective enough to detect multi-type occlusion targets, severely affecting automated fruit-picking efficiency. This study addresses this problem by proposing the pioneering design of a multi-type occlusion apple dataset and an augmentation method of data balance. We divided apple occlusion into eight types and used the proposed method to balance the number of annotation boxes for multi-type occlusion apple targets. Finally, a validation experiment was carried out using five popular lightweight object detection models: yolox-s, yolov5-s, yolov4-s, yolov3-tiny, and efficidentdet-d0. The results show that, using the proposed augmentation method, the average detection precision of the five popular lightweight object detection models improved significantly. Specifically, the precision increased from 0.894 to 0.974, recall increased from 0.845 to 0.972, and mAP0.5 increased from 0.982 to 0.919 for yolox-s. This implies that the proposed augmentation method shows great potential for different fruit detection missions in future orchard applications.

Deep Learning Based SWIR Object Detection in Long-Range Surveillance Systems: An Automated Cross-Spectral Approach.

SWIR imaging bears considerable advantages over visible-light (color) and thermal images in certain challenging propagation conditions. Thus, the SWIR imaging channel is frequently used in multi-spectral imaging systems (MSIS) for long-range surveillance in combination with color and thermal imaging to improve the probability of correct operation in various day, night and climate conditions. Integration of deep-learning (DL)-based real-time object detection in MSIS enables an increase in efficient utilization for complex long-range surveillance solutions such as border or critical assets control. Unfortunately, a lack of datasets for DL-based object detection models training for the SWIR channel limits their performance. To overcome this, by using the MSIS setting we propose a new cross-spectral automatic data annotation methodology for SWIR channel training dataset creation, in which the visible-light channel provides a source for detecting object types and bounding boxes which are then transformed to the SWIR channel. A mathematical image transformation that overcomes differences between the SWIR and color channel and their image distortion effects for various magnifications are explained in detail. With the proposed cross-spectral methodology, the goal of the paper is to improve object detection in SWIR images captured in challenging outdoor scenes. Experimental tests for two object types (cars and persons) using a state-of-the-art YOLOX model demonstrate that retraining with the proposed automatic cross-spectrally created SWIR image dataset significantly improves average detection precision. We achieved excellent improvements in detection performance in various variants of the YOLOX model (nano, tiny and x).

Deep Learning for Strawberry Canopy Delineation and Biomass Prediction from High-Resolution Images.

Modeling plant canopy biophysical parameters at the individual plant level remains a major challenge. This study presents a workflow for automatic strawberry canopy delineation and biomass prediction from high-resolution images using deep neural networks. High-resolution (5 mm) RGB orthoimages, near-infrared (NIR) orthoimages, and Digital Surface Models (DSM), which were generated by Structure from Motion (SfM), were utilized in this study. Mask R-CNN was applied to the orthoimages of two band combinations (RGB and RGB-NIR) to identify and delineate strawberry plant canopies. The average detection precision rate and recall rate were 97.28% and 99.71% for RGB images and 99.13% and 99.54% for RGB-NIR images, and the mean intersection over union (mIoU) rates for instance segmentation were 98.32% and 98.45% for RGB and RGB-NIR images, respectively. Based on the center of the canopy mask, we imported the cropped RGB, NIR, DSM, and mask images of individual plants to vanilla deep regression models to model canopy leaf area and dry biomass. Two networks (VGG-16 and ResNet-50) were used as the backbone architecture for feature map extraction. The R 2 values of dry biomass models were about 0.76 and 0.79 for the VGG-16 and ResNet-50 networks, respectively. Similarly, the R 2 values of leaf area were 0.82 and 0.84, respectively. The RMSE values were approximately 8.31 and 8.73 g for dry biomass analyzed using the VGG-16 and ResNet-50 networks, respectively. Leaf area RMSE was 0.05 m2 for both networks. This work demonstrates the feasibility of deep learning networks in individual strawberry plant extraction and biomass estimation.

DB-BlendMask: Decomposed Attention and Balanced BlendMask for Instance Segmentation of High-Resolution Remote Sensing Images

Instance segmentation is an important method for high-resolution remote sensing images (HRRSIs) analysis. Traditional instance segmentation algorithms are not suitable to analyze complex HRRSIs that exhibit: 1) various shapes and sizes of targets; 2) a large number of small targets; and 3) data with long tail distribution. Here we introduce DB-BlendMask, an efficient and accurate instance segmentation method that can accommodate complex HRRSIs. It is composed of size balance coefficient (SBC), class balance module (CBM), and decomposed attention blender module (DA-Blender module). SBC consists of a fair weight allocation strategy for positive samples in object detection. CBM combines classification obtained in object detection stage to guide the semantic feature extraction. Complementary to a traditional convolutional neural network (CNN) architecture, DA-Blender module has the ability to considerably compress space complexity of attention and merge attention with semantic feature to generate the instance mask. We compare the performance of DB-BlendMask with a benchmark Mask R-CNN on two typical datasets, iSAID, and ISPRS Postdam. We obtain an average detection precision of 39.2% on iSAID and 63.6% on ISPRS Postdam, which corresponds to an improvement of 2.5% and 2.7%, respectively, compared to the benchmark in a real-time scenario.

Weakly perceived object detection based on an improved CenterNet.

Nowadays, object detection methods based on deep neural networks have been widely applied in autonomous driving and intelligent robot systems. However, weakly perceived objects with a small size in the complex scenes own too few features to be detected, resulting in the decrease of the detection accuracy. To improve the performance of the detection model in complex scenes, the detector of an improved CenterNet was developed via this work to enhance the feature representation of weakly perceived objects. Specifically, we replace the ResNet50 with ResNext50 as the backbone network to enhance the ability of feature extraction of the model. Then, we append the lateral connection structure and the dilated convolution to improve the feature enhancement layer of the CenterNet, leading to enriched features and enlarged receptive fields for the weakly sensed objects. Finally, we apply the attention mechanism in the detection head of the network to enhance the key information of the weakly perceived objects. To demonstrate the effectiveness, we evaluate the proposed model on the KITTI dataset and COCO dataset. Compared with the original model, the average precision of multiple categories of the improved CenterNet for the vehicles and pedestrians in the KITTI dataset increased by 5.37%, whereas the average precision of weakly perceived pedestrians increased by 9.30%. Moreover, the average precision of small objects (AP_S) of the weakly perceived small objects in the COCO dataset increase 7.4%. Experiments show that the improved CenterNet can significantly improve the average detection precision for weakly perceived objects.

A real-time branch detection and reconstruction mechanism for harvesting robot via convolutional neural network and image segmentation

To alleviate the burden of fruit harvesting imposed by rising costs and decreasing labor supply, intelligent robots are highly desired in modern farms. A major problem, however, is how to detect and locate the tree branches for the robots to plan their arm movements during harvesting process. This study addresses the obscured branch detection and reconstruction problem, and proposes a real-time branch detection and reconstruction (RBDR) mechanism using convolutional neural networks (CNNs) and image processing techniques. Firstly, we build a Branch-CNN framework for detecting the bare branches and complete their rough localization, and then, realize the background segmentation in HSV space to obtain the precise branch regions. Finally, with the distance and angle constraints considered, a polynomial fit is conducted onto the precise boxes of the same branch to fill in the obscured areas. The proposed RBDR mechanism is applied onto a harvesting robot platform, and experiments with both lab simulated orchard environment and real pomegranate tree environment are conducted to verify its feasibility. Results show that under the simulation environment, at an Intersection over Union (IOU) threshold of 0.5, Branch-CNN achieves the best overall performance, with the average detection precision, recall rate, and F1-Score being 90.98%, 92%, and 91%, respectively, and the average reconstruction accuracy of RBDR is 88.76%. Under the real pomegranate tree environment, Branch-CNN achieves 90.7% detection precision, 89% recall rate, and 90% F1-Score, respectively. The overall reconstruction speed of RBDR is 22.7 frames per second (FPS) on image with a resolution of 960*720. Such results fully demonstrate the rationality and effectiveness of RBDR.