Detection Boxes Research Articles

Cascaded information enhancement and cross-modal attention feature fusion for multispectral pedestrian detection

Multispectral pedestrian detection is a technology designed to detect and locate pedestrians in Color and Thermal images, which has been widely used in automatic driving, video surveillance, etc. So far most available multispectral pedestrian detection algorithms only achieved limited success in pedestrian detection because of the lacking take into account the confusion of pedestrian information and background noise in Color and Thermal images. Here we propose a multispectral pedestrian detection algorithm, which mainly consists of a cascaded information enhancement module and a cross-modal attention feature fusion module. On the one hand, the cascaded information enhancement module adopts the channel and spatial attention mechanism to perform attention weighting on the features fused by the cascaded feature fusion block. Moreover, it multiplies the single-modal features with the attention weight element by element to enhance the pedestrian features in the single-modal and thus suppress the interference from the background. On the other hand, the cross-modal attention feature fusion module mines the features of both Color and Thermal modalities to complement each other, then the global features are constructed by adding the cross-modal complemented features element by element, which are attentionally weighted to achieve the effective fusion of the two modal features. Finally, the fused features are input into the detection head to detect and locate pedestrians. Extensive experiments have been performed on two improved versions of annotations (sanitized annotations and paired annotations) of the public dataset KAIST. The experimental results show that our method demonstrates a lower pedestrian miss rate and more accurate pedestrian detection boxes compared to the comparison method. Additionally, the ablation experiment also proved the effectiveness of each module designed in this paper.

Three-dimensional object detection network based on geometric information supplement strategy

Three-dimensional object detection from point clouds greatly advances autonomous driving. The real point clouds are unbalanced, where some occluded or distant objects with fewer points suffer from insufficient features, resulting in limited detection accuracy. To address this issue, we propose a novel three-dimensional object detection network based on a Geometric Information Supplement (GIS) strategy. Concretely, we design a novel shape completion model and a point attention fusion module to implement the GIS strategy. The shape completion model based on a structure-aware Transformer first recovers the complete shape points of objects within proposals. Then, the point attention fusion module supplies the recovered geometric shape cues to the original spatial features of objects. By the GIS strategy, the network learns enriched geometric features of objects for proposal refinement, thus, generating accurate detection boxes. Comparative experiments on KITTI dataset show that our method significantly outperforms other state-of-the-art methods while maintaining modest model complexity.

An improved method MSS-YOLOv5 for object detection with balancing speed-accuracy

For deep learning-based object detection, we present a superior network named MSS-YOLOv5, which not only considers the reliability in complex scenes but also promotes its timeliness to better adapt to practical scenarios. First of all, multi-scale information is integrated into different feature dimensions to improve the distinction and robustness of features. The design of the detectors increases the variety of detection boxes to accommodate a wider range of detected objects. Secondly, the pooling method is upgraded to obtain more detailed information. At last, we add the Angle cost and assign new weights to different loss functions to accelerate the convergence and improve the accuracy of network detection. In our network, we explore four variants MSS-YOLOv5s, MSS-YOLOv5m, MSS-YOLOv5x, and MSS-YOLOv5l. Experimental results of MSS-Yolov5s show that our technique improves mAP on the PASCAL VOC2007 and PASCAL 2012 datasets by 2.4% and 2.9%, respectively. Meanwhile, it maintains a fast inference speed. At the same time, the other three models have different degrees of performance improvement in terms of balancing speed and precision in challenging detection regions.

A Real-Time Negative Obstacle Detection Method for Autonomous Trucks in Open-Pit Mines

Negative obstacles such as potholes and road collapses on unstructured roads in open-pit mining areas seriously affect the safe transportation of autonomous trucks. In this paper, we propose a real-time negative obstacle detection method for self-driving trucks in open-pit mines. By analyzing the characteristics of road negative obstacles in open-pit mines, a real-time target detection model based on the Yolov4 network was built. It uses RepVGG as the backbone feature extraction network, applying SimAM space and a channel attention mechanism to negative obstacle multiscale feature fusion. In addition, the classification and prediction modules of the network are optimized to improve the accuracy with which it detects negative obstacle targets. A non-maximum suppression optimization algorithm (CIoU Soft Non-Maximum Suppression, CS-NMS) is proposed in the post-processing stage of negative obstacle detection. The CS-NMS calculates the confidence of each detection frame with weighted optimization to solve the problems of encountering obscure negative obstacles or poor positioning accuracy of the detection boxes. The experimental results show that this research method achieves 96.35% mAP for detecting negative obstacles on mining roads with a real-time detection speed of 69.3 fps, and that it can effectively identify negative obstacles on unstructured roads in open-pit mines with complex backgrounds.

Coupled Global–Local object detection for large VHR aerial images

Object detection in large aerial images generally requires splitting each image into local images in the preprocessing step, and even the state-of-the-art models currently use this preprocessing method. However, image splitting often leads to deficiencies in contextual information and incomplete detection of oversized objects. At present, many object detection methods are designed to deal with large images. However, they require complex additional structures or training steps, and their applicability is limited. To address these problems, we propose the Coupled Global–Local (CGL) network, which can be easily embedded in frequently used detection models, to efficiently capture more information. Specifically, we employ a multiscale feature fusion module to share information between the global and local branches. Furthermore, a new convolution method is proposed to adaptively adjust the receptive field for better feature extraction. In addition, we find that detection results from global branches in the existing global–local architecture hinder the performance improvement on details when the detection results from different-resolution branches are fused. Therefore, on the global branch, a proposal filter and a nonlocal suppression (NLS) algorithm are developed to prevent small positive proposals and remove unqualified detection boxes easily and efficiently, respectively. We conduct extensive experiments on the DOTA-1.0, DOTA-1.5, and DOTA-2.0 data sets. The results demonstrate that CGL can significantly improve the detection performance of various baseline models for large very-high-resolution (VHR) aerial images without bells and whistles.

A Multi-Pedestrian Tracking Algorithm for Dense Scenes Based on an Attention Mechanism and Dual Data Association

Aiming at the problems of frequent identity switches (IDs) and trajectory interruption of multi-pedestrian tracking algorithms in dense scenes, this paper proposes a multi-pedestrian tracking algorithm based on an attention mechanism and dual data association. First, the FairMOT algorithm is used as a baseline to introduce the feature pyramid network in the CenterNet detection network and up-sampling the output multi-scale fused feature maps, effectively reducing the rate of missed detection of small-sized and obscured pedestrians. The improved channel attention mechanism module is embedded in the CenterNet’s backbone network to improve detection accuracy. Then, a re-identification (ReID) branch is embedded in the head of the detection network, and the two sub-tasks of pedestrian detection and pedestrian apparent feature extraction are combined in a multi-task joint learning approach to output the pedestrian apparent feature vectors while detecting pedestrians, which improves the computational efficiency and localization accuracy of the algorithm. Finally, we propose a dual data association tracking model that tracks by associating almost every detection box instead of only the high-scoring ones. For low-scoring detection boxes, we utilize their similarities with trajectories to recover obscured pedestrians. The experiment using the MOT17 dataset shows that the tracking accuracy is improved by 0.6% compared with the baseline FairMOT algorithm, and the number of switches decreases from 3303 to 2056, which indicates that the proposed algorithm can effectively reduce the number of trajectory interruptions and identity switching.

Detecting and tracking using 2D laser range finders and deep learning

Detecting and tracking people using 2D laser rangefinders (LRFs) is challenging due to the features of the human leg motion, high levels of self-occlusion and the existence of objects which are similar to the human legs. Previous approaches use datasets that are manually labelled with support of images of the scenes. We propose a system with a calibrated monocular camera and 2D LRF mounted on a mobile robot in order to generate a dataset of leg patterns through automatic labelling which is valid to achieve a robust and efficient 2D LRF-based people detector and tracker. First, both images and 2D laser data are recorded during the robot navigation in indoor environments. Second, the people detection boxes and keypoints obtained by a deep learning-based object detector are used to locate both people and their legs on the images. The coordinates frame of 2D laser is extrinsically calibrated to the camera coordinates allowing our system to automatically label the leg instances. The automatically labelled dataset is then used to achieve a leg detector by machine learning techniques. To validate the proposal, the leg detector is used to develop a Kalman filter-based people detection and tracking algorithm which is experimentally assessed. The experimentation shows that the proposed system overcomes the Angus Leigh’s detector and tracker which is considered the state of the art on 2D LRF-based people detector and tracker.

Acne Detection by Ensemble Neural Networks.

Acne detection, utilizing prior knowledge to diagnose acne severity, number or position through facial images, plays a very important role in medical diagnoses and treatment for patients with skin problems. Recently, deep learning algorithms were introduced in acne detection to improve detection precision. However, it remains challenging to diagnose acne based on the facial images of patients due to the complex context and special application scenarios. Here, we provide an ensemble neural network composed of two modules: (1) a classification module aiming to calculate the acne severity and number; (2) a localization module aiming to calculate the detection boxes. This ensemble model could precisely predict the acne severity, number, and position simultaneously, and could be an effective tool to help the patient self-test and assist the doctor in the diagnosis.

Corner‐based object detection method for reactivating box constraints

Corner-based detectors usually generate a large number of false detection boxes because of insufficient attention to the detection area. Recent corner-based detectors can achieve good performance, but the training equipment requirements have greatly increased. For example, due to the dense network structure and the large input image size, CenterNet requires expensive equipment for network training (e.g. Tesla V100). Its performance will be greatly reduced when a more mainstream and cheaper device is used for fine-tuning. The high equipment requirements make it difficult for most researchers to follow up these studies. In this work, CenternessNet, a detector that adds additional box-edge length constraints to CenterNet is proposed, thereby allowing the network to be trained on more general devices and obtain a better performance. It simply introduces the box as a constraint into the corner-based network. In this way, the method improves the ability to aggregate corners during training and enhances the model's ability to discriminate the corners of objects in the same category to some extent. The method achieves a better performance than other corner-based detection networks trained on similar low-memory devices.

Fast and accurate multi-class geospatial object detection with large-size remote sensing imagery using CNN and Truncated NMS

Multi-class geospatial object detection with remote sensing imagery has broad prospects in urban planning, natural disaster warning, industrial production, military surveillance and other applications. Accuracy and efficiency are two common measures for evaluating object detection models, and it is often difficult to achieve both at the same time. Developing a practical remote sensing object detection algorithm that balances the accuracy and efficiency is thus a big challenge in the Earth observation community. Here, we propose a comprehensive high-speed multi-class remote sensing object detection method. Firstly, we obtain a multi-volume YOLO (You Only Look Once) v4 model for balancing speed and accuracy, based on a pruning strategy of the convolutional neural network (CNN) and the one-stage object detection network YOLO v4. Moreover, we apply the Manhattan-Distance Intersection of Union (MIOU) loss function to the multi-volume YOLO v4 to further improve the accuracy without additional computational burden.Secondly, mainly due to computing limitations, a remote sensing image that is large-size relative to a natural image must first be divided into multiple smaller tiles, which are then detected separately, and finally, the detection results are spliced back to match the original image. In the process of remote sensing image slicing, a large number of truncated objects appear at the edge of tiles, which will produce a large number of false results in the subsequent detection links. To solve this problem, we propose a Truncated Non-Maximum Suppression (NMS) algorithm to filter out repeated and false detection boxes from truncated targets in the spliced detection results. We compare the proposed algorithm with the state-of-the-art methods on the Dataset for Object deTection in Aerial images (DOTA) and DOTA v2. Quantitative evaluations show that mAP and FPS reach 77.3 and 35 on DOTA, and 61.0 and 74 on DOTA v2. Overall, our method reaches the optimal balance between efficiency and accuracy, and realizes the high-speed remote sensing object detection.

Daedalus: Breaking Nonmaximum Suppression in Object Detection via Adversarial Examples.

This article demonstrates that nonmaximum suppression (NMS), which is commonly used in object detection (OD) tasks to filter redundant detection results, is no longer secure. Considering that NMS has been an integral part of OD systems, thwarting the functionality of NMS can result in unexpected or even lethal consequences for such systems. In this article, an adversarial example attack that triggers malfunctioning of NMS in OD models is proposed. The attack, namely, Daedalus, compresses the dimensions of detection boxes to evade NMS. As a result, the final detection output contains extremely dense false positives. This can be fatal for many OD applications, such as autonomous vehicles and surveillance systems. The attack can be generalized to different OD models, such that the attack cripples various OD applications. Furthermore, a way of crafting robust adversarial examples is developed by using an ensemble of popular detection models as the substitutes. Considering the pervasive nature of model reuse in real-world OD scenarios, Daedalus examples crafted based on an ensemble of substitutes can launch attacks without knowing the parameters of the victim models. The experimental results demonstrate that the attack effectively stops NMS from filtering redundant bounding boxes. As the evaluation results suggest, Daedalus increases the false positive rate in detection results to 99.9% and reduces the mean average precision scores to 0, while maintaining a low cost of distortion on the original inputs. It also demonstrates that the attack can be practically launched against real-world OD systems via printed posters.

Arbitrary-Oriented Detection of Insulators in Thermal Imagery via Rotation Region Network

The precise location of insulators in infrared images is of great significance for insulator condition monitoring and fault diagnosis. Due to the characteristics of insulators themselves and the use of handheld infrared cameras, insulators usually appear in infrared images with different aspect ratios and main axis orientations. Therefore, it is very important and necessary to make full use of the prior knowledge of the insulator itself to accurately locate it. However, most of the existing methods use axial horizontal detection boxes to detect insulators, which cannot take into account the characteristics of the insulator well. When there are large overlapping areas of two horizontal detection boxes, the nonmaximum suppression algorithm may lead to missed detection of the object. To further improve the accuracy of the detection algorithm, this article makes full use of the prior features carried by the insulator itself, and optimizes faster region-based convolutional neural networks (R-CNN) from five aspects: rectangular box representation, feature extraction, candidate box generation, anchor design, and feature alignment. An oriented detection model for infrared images of insulators is constructed. Comparative experiments with a variety of mainstream detection methods were carried out on the constructed infrared dataset. The results show that the proposed method is superior to other models in detection accuracy. When the intersection and union ratio is 0.5, the average precision reaches 95.08%. In addition, it can also effectively predict the shape and angle information of insulators in complex scenes, laying a beneficial foundation for subsequent diagnosis automation tasks.

GCD-YOLOv5: An Armored Target Recognition Algorithm in Complex Environments Based on Array Lidar

For the recognition of armored targets in complex battlefield environments, how to reduce missed and false alarms while achieving real-time is an urgent issue. To this end, the GCD-YOLOv5 algorithm is innovatively proposed. Firstly, array lidar is used to acquire the armor target data. Secondly, the armor target data is expanded with an improved GAN(Generative Adversarial Network) to increase the diversity of training data. Afterward, the expanded dataset is fed into the GCD-YOLv5(You Only Look Once) for training. And the GCD-YOLOv5 is reflected in the following aspects. Firstly, the CBAM(Convolutional Block Attention Module) and the multi-scale feature fusion are added to improve the feature extraction capability and detection efficiency, increasing the recognition capability of small and obscured targets. Secondly, combining with DETR(Detection Transformer) to lighten YOLOv5 to achieve the real-time requirement. Thirdly, the YOLOv5 loss function and prediction box filtering method are improved to increase the detection accuracy and the confidence of the detection boxes. The experimental results show that the GCD-YOLOv5 algorithm has higher accuracy and real-time, the mAP(mean Average Precision) can reach 99.7%, and fps is 68.56% higher compared to YOLOv5, which significantly improves the recognition capability of armored targets in complex battlefield environments.

A Deep Learning-Based System for Monitoring the Number and Height Growth Rates of Moso Bamboo Shoots

The number and growth of new shoots are very important information for bamboo forest cultivation and management. At present, there is no real-time, efficient and accurate monitoring method. In this study, a fixed webcam was applied for image capture, optimized YOLOv4 was used to model the detection of moso bamboo shoots, and a strategy of sorting and screening was proposed to track each moso bamboo shoot. The change in the number and height of moso bamboo shoots was obtained according to the number and height of detection boxes. The experimental results show that the system can remotely and automatically obtain the number of moso bamboo shoots and the pixel height of each bamboo shoot at any given time. The average relative error and variance in the number of moso bamboo shoots were 1.28% and 0.016%, respectively, and those for the corresponding pixel height results were −0.39% and 0.02%. This system can be applied to a series of monitoring purposes, such as the daily or weekly growth rate of moso bamboo shoots at monitoring stations and trends in the height of selected bamboo shoots.

The Stress Detection and Segmentation Strategy in Tea Plant at Canopy Level.

As compared with the traditional visual discrimination methods, deep learning and image processing methods have the ability to detect plants efficiently and non-invasively. This is of great significance in the diagnosis and breeding of plant disease resistance phenotypes. Currently, the studies on plant diseases and pest stresses mainly focus on a leaf scale. There are only a few works regarding the stress detection at a complex canopy scale. In this work, three tea plant stresses with similar symptoms that cause a severe threat to the yield and quality of tea gardens, including the tea green leafhopper [Empoasca (Matsumurasca) onukii Matsuda], anthracnose (Gloeosporium theae-sinensis Miyake), and sunburn (disease-like stress), are evaluated. In this work, a stress detection and segmentation method by fusing deep learning and image processing techniques at a canopy scale is proposed. First, a specified Faster RCNN algorithm is proposed for stress detection of tea plants at a canopy scale. After obtaining the stress detection boxes, a new feature, i.e., RGReLU, is proposed for the segmentation of tea plant stress scabs. Finally, the detection model at the canopy scale is transferred to a field scale by using unmanned aerial vehicle (UAV) images. The results show that the proposed method effectively achieves canopy-scale stress adaptive segmentation and outputs the scab type and corresponding damage ratio. The mean average precision (mAP) of the object detection reaches 76.07%, and the overall accuracy of the scab segmentation reaches 88.85%. In addition, the results also show that the proposed method has a strong generalization ability, and the model can be migrated and deployed to UAV scenarios. By fusing deep learning and image processing technology, the fine and quantitative results of canopy-scale stress monitoring can provide support for a wide range of scouting of tea garden.

Security Monitoring Strategy of Distribution Community Operation Site Based on Intelligent Image Processing Method

With the continuous development of energy infrastructure, a large number of distributed new energy, distributed energy storage, and various power electronic equipment are connected to distribution communities. The access to a large amount of equipment not only increases the workload of power operators, but also leads to a complex field operation environment, which will threaten the security of field operators. Traditional monitoring strategy for distribution community operation site relies on manual operation, which is tedious, labor-intensive, and error-prone. To solve this problem, this paper proposes a security monitoring strategy for distribution community operation sites based on an intelligent image processing method. Firstly, a power image enhancement method based on multi-filtering algorithm is proposed. The filter operators are used to smooth and denoise the image, enhance the edge and detail information, and expand the security monitoring data set of operation sites by merging the filtered images. Secondly, an object detection method for personnel and security protection tools based on the improved Faster R-CNN algorithm is proposed, and a pyramid structure is constructed to improve the detection and localization accuracy of small objects such as safety helmets and safety belts. To simplify the object detection task and the complexity of the labeling system, it is disassembled into four single tasks: operators, non-operators, safety helmets, and safety belts. Finally, the detection frame labels are fused by calculating the overlap of the object detection boxes to describe the security of the operator. Experiments show that the method proposed in this paper can effectively locate the object and accurately describe the security of the operator.

Graph fusion network for multi-oriented object detection

In object detection, non-maximum suppression (NMS) methods are extensively adopted to remove horizontal duplicates of detected dense boxes for generating final object instances. However, due to the degraded quality of dense detection boxes and not explicit exploration of the context information, existing NMS methods via simple intersection-over-union (IoU) metrics tend to underperform on multi-oriented and long-size objects detection. Distinguishing with general NMS methods via duplicate removal, we propose a novel graph fusion network, named GFNet, for multi-oriented object detection. Our GFNet is extensible and adaptively fuse dense detection boxes to detect more accurate and holistic multi-oriented object instances. Specifically, we first adopt a locality-aware clustering algorithm to group dense detection boxes into different clusters. We will construct an instance sub-graph for the detection boxes belonging to one cluster. Then, we propose a graph-based fusion network via Graph Convolutional Network (GCN) to learn to reason and fuse the detection boxes for generating final instance boxes. Extensive experiments both on public available multi-oriented text datasets (including MSRA-TD500, ICDAR2015, ICDAR2017-MLT) and multi-oriented object datasets (DOTA) verify the effectiveness and robustness of our method against general NMS methods in multi-oriented object detection.

An Efficient Lightweight SAR Ship Target Detection Network with Improved Regression Loss Function and Enhanced Feature Information Expression.

It is difficult to identify the ship images obtained by a synthetic aperture radar (SAR) due to the influence of dense ships, complex background and small target size, so a deep learning-based target detection algorithm was introduced to obtain better detection performance. However, in order to achieve excellent performance, most of the current target detection algorithms focus on building deep and high-width neural networks, resulting in bloated network structure and reduced detection speed, which is not conducive to the practical application of target detection algorithms. Thereby, an efficient lightweight network Efficient-YOLO for ship detection in complex situations is proposed in the present work. Firstly, a new regression loss function ECIOU is proposed to enhance the detection boxes localization accuracy and model convergence speed. Secondly, We propose the SCUPA module to enhance the multiplexing of picture feature information and the model generalization performance. Thirdly, The GCHE module is proposed to strengthen the network’s ability to extract feature information. At last, the effectiveness of our method is tested on the specialized ship dataset: SSDD and HRSID datasets. The results show that Efficient-YOLO outperforms other state-of-the-art algorithms in accuracy, recall and detection speed, with smaller model complexity and model size.

Attention Network for Rail Surface Defect Detection via Consistency of Intersection-over-Union(IoU)-Guided Center-Point Estimation

Rail surface defect inspection based on machine vision faces challenges against the complex background with interference and severe data imbalance. To meet these challenges, in this article, we regard defect detection as a key-point estimation problem and present the proposed attention neural network for rail surface defect detection via consistency of Intersection-over-Union(IoU)-guided center-point estimation (CCEANN). The CCEANN contains two crucial components. The two components are the stacked attention Hourglass backbone via cross-stage fusion of multiscale features (CSFA-Hourglass) and the CASIoU-guided center-point estimation head module (CASIoU-CEHM). Furthermore, the CASIoU-guided center-point estimation head module integrating the delicate coordinate compensation mechanism regresses detection boxes flexibly to adapt to defects’ large-scale variation, in which the proposed CASIoU loss, a loss regressing the consistency of intersection-over-union (IoU), central-point distance, area ratio, and scale ratio between the targeted defect and the predicted defect, achieves higher regression accuracy than state-of-the-art IoU-based losses. The experiments demonstrate that the CCEANN outperforms competitive deep learning-based methods in four surface defect datasets.

Influence of the Computer-Aided Decision Support System Design on Ultrasound-Based Breast Cancer Classification.

Simple SummaryThe implementation of artificial intelligence in the computer-aided decision (CAD) support systems holds great promise for future cancer diagnosis. It is crucial to build these algorithms in a structured manner to ensure reproducibility and reliability. In this context, we used a dataset of breast ultrasound (US) images with 252 breast cancer and 253 benign cases to refine the CAD image analysis workflow. Various dataset preparations (i.e., pre-processing, and spatial augmentation) and machine learning algorithms were tested to establish the framework with the best performance in the detection and classification of breast lesions in US images. The efficacy of the proposed workflows was evaluated regarding accuracy, precision, specificity, and sensitivity.Automation of medical data analysis is an important topic in modern cancer diagnostics, aiming at robust and reproducible workflows. Therefore, we used a dataset of breast US images (252 malignant and 253 benign cases) to realize and compare different strategies for CAD support in lesion detection and classification. Eight different datasets (including pre-processed and spatially augmented images) were prepared, and machine learning algorithms (i.e., Viola–Jones; YOLOv3) were trained for lesion detection. The radiomics signature (RS) was derived from detection boxes and compared with RS derived from manually obtained segments. Finally, the classification model was established and evaluated concerning accuracy, sensitivity, specificity, and area under the Receiver Operating Characteristic curve. After training on a dataset including logarithmic derivatives of US images, we found that YOLOv3 obtains better results in breast lesion detection (IoU: 0.544 ± 0.081; LE: 0.171 ± 0.009) than the Viola–Jones framework (IoU: 0.399 ± 0.054; LE: 0.096 ± 0.016). Interestingly, our findings show that the classification model trained with RS derived from detection boxes and the model based on the RS derived from a gold standard manual segmentation are comparable (p-value = 0.071). Thus, deriving radiomics signatures from the detection box is a promising technique for building a breast lesion classification model, and may reduce the need for the lesion segmentation step in the future design of CAD systems.