Single-stage Detector Research Articles

Object detection using YOLO: challenges, architectural successors, datasets and applications.

Object detection is one of the predominant and challenging problems in computer vision. Over the decade, with the expeditious evolution of deep learning, researchers have extensively experimented and contributed in the performance enhancement of object detection and related tasks such as object classification, localization, and segmentation using underlying deep models. Broadly, object detectors are classified into two categories viz. two stage and single stage object detectors. Two stage detectors mainly focus on selective region proposals strategy via complex architecture; however, single stage detectors focus on all the spatial region proposals for the possible detection of objects via relatively simpler architecture in one shot. Performance of any object detector is evaluated through detection accuracy and inference time. Generally, the detection accuracy of two stage detectors outperforms single stage object detectors. However, the inference time of single stage detectors is better compared to its counterparts. Moreover, with the advent of YOLO (You Only Look Once) and its architectural successors, the detection accuracy is improving significantly and sometime it is better than two stage detectors. YOLOs are adopted in various applications majorly due to their faster inferences rather than considering detection accuracy. As an example, detection accuracies are 63.4 and 70 for YOLO and Fast-RCNN respectively, however, inference time is around 300 times faster in case of YOLO. In this paper, we present a comprehensive review of single stage object detectors specially YOLOs, regression formulation, their architecture advancements, and performance statistics. Moreover, we summarize the comparative illustration between two stage and single stage object detectors, among different versions of YOLOs, applications based on two stage detectors, and different versions of YOLOs along with the future research directions.

DCPLD-Net: A diffusion coupled convolution neural network for real-time power transmission lines detection from UAV-Borne LiDAR data

The stable and reliable supply of electric power is strongly related to the normal social production. In recent years, power transmission lines inspection based on remote sensing methods has made great progress, especially using the UAV-borne LiDAR system. The extraction and identification of power transmission lines (i.e., conductors) from 3D point clouds are the basis of LiDAR data-based power grid risk management. However, existing rule-based/traditional machine learning extraction approaches have exposed some limitations, such as the lack of timeliness and generalization. Moreover, the potential of deep learning is seriously overlooked in RoW (Right of Way) LiDAR inspection tasks. Thus, we proposed DCPLD-Net: a diffusion coupled convolution neural network for real-time power transmission lines detection from UAV-borne LiDAR data. To implement efficient 2D convolution on 3D point clouds, we proposed a novel point cloud representation, named Cross Section View (CSV), which transforms the discrete point clouds into 3D tensors constructed by voxels with a deformed geometric shape along the flight trajectory. After the CSV feature generation, the encoded features of each voxel are treated as energy (i.e., heat) signals and diffused in space to generate diffusion feature maps. The feature of the power line points are thus enhanced through this simulated physical process (diffusion). Finally, a single-stage detector named PLDNet is proposed for the multiscale detection of conductors on the diffused CSV representations. The experimental results show that the DCPLD-Net achieves an average F1 score of 97.14 % at 8 Hz detection frequency on RoWs inspection LiDAR datasets collected by both mini-UAV LiDAR and large-scale fully autonomous UAV power lines inspection robots, and surpasses compared methods (i.e. PointNet ++, RandLA-Net) in terms of F1 scores and IoU.

Attention-based domain adaptation for single-stage detectors

While domain adaptation has been used to improve the performance of object detectors when the training and test data follow different distributions, previous work has mostly focused on two-stage detectors. This is because their use of region proposals makes it possible to perform local adaptation, which has been shown to significantly improve the adaptation effectiveness. Here, by contrast, we target single-stage architectures, which are better suited to resource-constrained detection than two-stage ones but do not provide region proposals. To nonetheless benefit from the strength of local adaptation, we introduce an attention mechanism that lets us identify the important regions on which adaptation should focus. Our method gradually adapts the features from global, image level to local, instance level. Our approach is generic and can be integrated into any Single-Shot Detector. We demonstrate this on standard benchmark datasets by applying it to both the single-shot detector (SSD) and a recent variant of the You Only Look Once detector (YOLOv5). Furthermore, for equivalent single-stage architectures, our method outperforms the state-of-the-art domain adaptation techniques even though they were designed for specific detectors.

Evaluation of focal loss based deep neural networks for traffic sign detection

ABSTRACT With advancements in autonomous driving, demand for stringent and computationally efficient traffic sign detection systems has increased. However, bringing such a system to a deployable level requires handling critical accuracy and processing speed issues. A focal loss-based single-stage object detector, i.e RetinaNet, is used as a trade-off between accuracy and processing speed as it handles the class imbalance problem of the single-stage detector and is thus suitable for traffic sign detection (TSD). We assessed the detector’s performance by combining various feature extractors such as ResNet-50, ResNet-101, and ResNet-152 on three publicly available TSD benchmark datasets. Performance comparison of the detector using different backbone includes evaluation parameters like mean average precision (mAP), memory allocation, running time, and floating-point operations. From the evaluation results, we found that the RetinaNet object detector using the ResNet-152 backbone obtains the best mAP, while that using ResNet-101 strikes the best trade-off between accuracy and execution time. The motivation behind benchmarking the detector on different datasets is to analyse the detector’s performance on different TSD benchmark datasets. Among the three feature extractors, the RetinaNet model trained using the ResNet-50 backbone is an excellent model in memory consumption, making it an optimal choice for low-cost embedded devices deployment.

Recognition and statistical analysis of coastal marine aquacultural cages based on R3Det single-stage detector: A case study of Fujian Province, China

Fujian Province, located in the southeastern coast of China, has a highly developed coastal marine net cage aquaculture industry. The scale and density of marine aquaculture, especially net cages aquaculture, in China are increasing, which has seriously affected coastal and nearshore environments. Thus, methods for accurately obtaining the area and density of aquaculture net cages hold great significance. Because of the scattered distribution of net cage aquaculture areas, it is difficult to obtain an accurate net cage aquaculture area with traditional remote-sensing extraction methods. Therefore, this paper proposes a method for accurately identifying of the area of aquaculture net cages by combining R3Det single-stage detector and fishnet segmentation technology. A constructed sensing net cage dataset was used for model training and a fishing net segmentation technology was used to segment the remote-sensing image into R3Det testable image. The experimental results showed that the extraction accuracy of this method for round and square net cages was 92.65% and 98.06%, respectively, and the extraction accuracy of the actual net cage culture area was 97.24%. We concluded that the total area of net cage aquaculture in Fujian Province in 2019–2020 was 4363.58 ha, mainly in square net cages. The area of aquaculture was greater in the northern and southern parts of the province and less in the middle. The density of aquaculture was high in the south and low in the north. Compared with traditional remote-sensing extraction methods and neural networks, this solution reduced the impact of similar features on the classification target, realized the application of high-precision, large-area remote-sensing images on the R3Det single-stage detector, and improved the recognition accuracy and the robustness of the model.

Spatial Alignment for Unsupervised Domain Adaptive Single-Stage Object Detection.

Domain adaptation methods are proposed to improve the performance of object detection in new domains without additional annotation costs. Recently, domain adaptation methods based on adversarial learning to align source and target domain image distributions are effective. However, for object detection tasks, image-level alignment enforces the alignment of non-transferable background regions, which affects the performance of important target regions. Therefore, how to balance the alignment of background and target remains a challenge. In addition, the current research with good effect is based on two-stage detectors, and there are relatively few studies on single-stage detectors. To address these issues, in this paper, we propose a selective domain adaptation framework for the spatial alignment of a single-stage detector. The framework can identify the background and target and pay different attention to them. On the premise that the single-stage detector does not generate region suggestions, it can achieve domain feature alignment and reduce the influence of the background, enabling transfer between different domains. We validate the effectiveness of our method for weather discrepancy, camera angles, synthetic to real-world, and real images to artistic images. Extensive experiments on four representative adaptation tasks show that the method effectively improves the performance of single-stage object detectors in different domains while maintaining good scalability.

Rapid -YOLO: A novel YOLO based architecture for shadow detection

Shadow detection is widely considered as segmentation problem that happens to be time consuming process with lots of duplication of results. The alternative to segmentation can be the applicability of single stage detectors like YOLO (You Only Look Once) that can run on the input image once and perform the detections during single pass. The study proposes a novel deep learning-based architecture namely, Rapid-YOLO which is an extended form of YOLOv3 architecture for detection of shadows. The proposed model is an extension of YOLOv3 architecture with addition of extra YOLO detection layers and convolution layers for aiding detection accuracy. The model exploits use of spatial pyramid pooling for scaling variations, a novel non-maximum suppression using chaotic whale optimization (CW-NMS) for overpowering false bounding box detections. We further propose tangent loss function for confidence and classification loss to improve training time and reduce errors. Drop block regularization is also used to reduce over-fitting and optimize results for shadow detection. Further, lack of proper dataset for single shot detectors encouraged us to create a novel dataset for the use of YOLO based detectors to boost further work in this direction. The model has been verified by rich experiments and results prove that the proposed methodology is superior to the state-of-the-art methods in terms of objective quantization and in subjective vision.

IoU-Balanced loss functions for single-stage object detection

Single-stage object detectors have been widely applied in computer vision applications due to their high efficiency. However, the loss functions adopted by single-stage detectors hurt the localization accuracy seriously. Firstly, the cross-entropy loss for classification is independent of the localization task and drives all the positive examples to learn as high classification scores as possible regardless of localization accuracy. Thus, there exist many detections with high classification scores but low IoU or detections with low classification scores but high IoU. Secondly, for the smooth L1 loss, the gradient is dominated by the outliers with poor localization accuracy. In this work, IoU-balanced loss functions consisting of IoU-balanced classification loss and IoU-balanced localization loss are proposed to solve these problems. IoU-balanced classification loss pays more attention to positive examples with high IoU and enhances the correlation between classification and localization tasks. IoU-balanced localization loss decreases the gradient of examples with low IoU and increases the gradient of examples with high IoU, which improves the localization accuracy of models. Extensive experiments on MS COCO, PASCAL VOC, Cityscapes and WIDERFace demonstrate that IoU-balanced losses can substantially improve the popular single-stage detectors, especially the localization accuracy. On COCO test-dev , the proposed methods can substantially improve AP by 1.0 % ∼ 1.7 % and AP 75 by 1.0 % ∼ 2.4 % . On PASCAL VOC, Cityscape and WIDERFace, it can also substantially improve AP by 1.0 % ∼ 1.5 % and A P 80 , A P 90 by ∼ 3.9 % . The source code will be made publicly available.

Experimental research on occlusion face detection method based on attention mechanism

Abstract Face occlusion is a common problem in real life. Aiming at the problem that the face detection effect is poor when there is occlusion in the natural environment, this paper proposes the RetinaNet occlusion face detection network based on the fusion image enhancement network and the attention mechanism. Traditional RetinaNet, as a typical single-stage detector, does not explicitly give the process of extracting candidate regions and directly obtains the results of object detection. Compared with the two-stage detector, the processing speed is faster, but the accuracy will also be reduced. In this paper, the attention mechanism and image enhancement strategy are introduced to improve the detection accuracy while ensuring the detection speed. The experimental results on the MAFA dataset show that the RetinaNet model based on the attention mechanism proposed in this paper achieves 89.7% accuracy in detecting occluded faces. The feasibility and effectiveness of the model are verified.

멀티스펙트럴 영상기반 전기시설물 이상치 감지

Regular inspection of electrical facilities plays a significant role in preventing deterioration caused by aging equipment, but the dangers associated with these inspections make it difficult for examiners to perform them without first suspending the power. Currently, these inspections cause shutdowns and result in loss of productivity. For this reason, researchers are actively exploring new ways to detect faults in electrical facilities using deep learning algorithms and various modalities such as single-infrared, single-RGB, and multispectral imagery. Existing methods either use a single modality or fuse two modalities with image registration that decreases technology accessibility. This paper will explore the use of a multispectral anomaly pre-detector based on YOLOv5s, a simple yet effective single-stage detector. The anomaly region detector is a real-world application of this technology that detects parts in the potentially anomalous device and finds the anomalous regions within those parts. In order to verify the two detectors, far- and near-distance datasets were obtained and evaluated using the proposed models. The results indicate that the proposed pipeline enables pixel-level electrical fault detection.

Automated Aerial Docking System Using Onboard Vision-Based Deep Learning

This paper proposes an automated aerial docking system for unmanned aerial vehicles. The proposed automated aerial docking system consists of the following two subsystems: 1) docking mechanical system and 2) vision-based deep learning detection/tracking system. A fundamental challenge during the midair docking phase is integration between the leader and follower vehicles with a robust target detection/tracking strategy. This paper presents not only the design of a probe-and-drogue-type aerial docking system, but it also presents the development of an onboard machine learning system for aerial target detection and tracking, which is limited from 0.1 to 3.5 m. The aerial docking system is designed based on a bistable mechanism to increase the robustness of integration and separation between two vehicles. The prototype of the proposed docking mechanism is built by a 3D printer. To employ effective drogue detection in the air, a deep convolutional neural network-based single-stage detector algorithm, YOLOv4-tiny, is applied. Furthermore, to track the moving drogue in the air simultaneously, a point cloud-based tracking algorithm with an RGB-D camera system is developed. The developed deep learning-based detection/tracking system is implemented to be operated in the Arm architecture-based onboard machine learning computer. For performance validation of the proposed automated aerial docking system, a ground test using two robot arms and an indoor flight test using quadcopter drone and an unmanned ground vehicle were conducted.

Anchor constrained refinement network with Intersection-over-Union-aware and scale-aware loss for oriented object detection

Object detection has achieved good progress in the last few years. However, there are many challenges in the field of remote sensing imagery. Objects in remote sensing images usually have arbitrary orientations and various scales. In addition, some objects are easily overwhelmed by a cluttered background. To take advantage of single-stage object detectors that have fast speed, many cascaded structures based on single-stage detectors have been proposed to improve detection performance. However, feature inconsistency in cascade structure results in poor detection performance. To address these problems, we propose an innovative model in terms of both model improvement and loss function refinement. This model consists of an attention module to highlight useful information in cluttered scenes, a multi-scale feature fusion module, and a cascade refinement module with anchor constrained convolution to address feature inconsistency. Furthermore, Intersection-over-Union (IoU) classification loss is proposed to enhance the correlation between classification and localization, and a scale-aware regression loss is proposed to improve the detection performance on objects with different scales. We conducted extensive experiments on both the DOTA dataset and the HRSC2016 dataset, and the experimental results show that our model has advantages compared with current state-of-the-art methods.

AT-BOD: An Adversarial Attack on Fool DNN-Based Blackbox Object Detection Models

Object recognition is a fundamental concept in computer vision. Object detection models have recently played a vital role in various applications, including real-time and safety-critical systems such as camera surveillance and self-driving cars. Scientific research has proven that object detection models are prone to adversarial attacks. Although several proposed methods exist throughout the literature, they either target white-box models or specific-task black-box models and do not generalize on other detectors. In this paper, we proposed a new adversarial attack against Blackbox-based object detectors called AT-BOD. The proposed AT-BOD model can fool the single-stage and multi-stage detectors, where we used an optimization algorithm to generate adversarial examples depending only on the detector predictions. AT-BOD model works in two diverse ways, reducing the confidence score and misleading the model to make the wrong decision or hide the object detection models. Our solution achieved a fooling rate of 97% and a false negative increase of 99% on the YOLOv3 detector, and a fooling rate of 61% false-negative increase of 57% on the Faster R-CNN detector. The detection accuracy of YOLOv3 and Faster R-CNN under AT-BOD was dramatically reduced and reached ≤1% and ≤3%, respectively.

PRNet++: Learning towards generalized occluded pedestrian detection via progressive refinement network

Pedestrian detection has achieved significant progress in recent years. Though promising results have been obtained on standard pedestrians, it remains challenging to detect pedestrians in various occlusion situations. In this paper, we propose Progressive Refinement Network (PRNet), a novel single-stage detector for occluded pedestrian detection. Inspired by human’s progressive process on annotating occluded pedestrians, PRNet perform sequential refinement in a single-stage detection framework by three phases: finding anchors of visible parts with high confidence, calibrating these anchors with a full body template derived from occlusion statistics, and adjusting the calibrated anchors to target full-body regions. Unlike conventional methods that utilize predefined anchors directly for full-body estimation, the proposed confidence-aware anchor calibration offers an adaptive anchor initialization for detection with occlusions, while helps reduce the gap between visible-part and full-body detection. We also propose an occlusion loss, which automatically up-weights heavily occluded pedestrian samples. In addition, a Receptive Field Backfeed (RFB) module is introduced to diversify receptive fields in early layers that commonly fire only on visible parts or small-size full-body regions. To further learn generalized representations of pedestrians in different occlusion states, we propose to establish a new single-stage detector with dual-stream architecture namely PRNet++. An Easy-branch and a Hard-branch are designed to learn complementary representation that are more robust to various occlusions. Moreover, the generalization ability of models for other domains are critical in real-world applications, but it has not attracted too much attention. To address this issue, we introduce the unsupervised domain adaptation setting. To handle occlusion situations in unknown domain better, we especially design a Dynamic Iterative adaptation strategy and a Multi-experts adaptation strategy for our occlusion-aware detectors PRNet and PRNet++. Extensive supervised within-dataset experiments and unsupervised domain adaptation experiments were performed on CityPersons, Caltech, ECP-night, KITTI and INRIA datasets, which can validate the effectiveness of our proposed methods in occluded pedestrian detection.

Single-Stage Detector With Dual Feature Alignment for Remote Sensing Object Detection

As a fundamental vision-based task in the remote sensing filed, object detection has achieved significant progress. However, remote sensing object detection is still an urgent challenge owing to dense distribution, large aspect ratio, and arbitrary orientations. To address this issue, we develop an end-to-end dual align single-stage rotation detector (DA-Net) consisting of two main components: a Rotation Feature Selection (RFS) module and a Rotation Feature Align (RFA) module. Specifically, RFS module can empower neurons with the capability to adjust receptive fields, which achieves the first stage of feature alignment on the image level. Furthermore, RFA module is employed to adaptively align the feature based on the size, shapes, orientations of its corresponding anchors, realizing the second stage of instance-level feature alignment. Extensive experiments have shown that our DA-Net can significantly improve remote sensing detection performance against several start-of-the-art algorithms on two benchmark datasets.

Feature-Aligned Single-Stage Rotation Object Detection With Continuous Boundary

Recently, rotation detection has gained much attention and shown its potential for accurate localization in remote sensing scenes. However, the objects in remote sensing images have a variety of directions, sizes, and aspect ratios, which makes it difficult to locate and classify objects. Therefore, the object detection task is still facing great challenges in the field of remote sensing. In this paper, we propose a novel single-stage detector, which includes feature alignment block (FAB), double regression branches (DRBs), and a circumcircle rotation box (CRB). FAB utilizes the deformable convolution to flexibly obtain the features of different aspect ratio objects, and aligns the regression features with the corresponding classification features through fusion. Consequently, it can make the extracted feature information have stronger discrimination, which is conducive to improving the object position and classification accuracy. DRBs consist of a main regression branch and an auxiliary regression branch. The main regression branch is used to fine-tune the result of the auxiliary regression branch to obtain a more accurate regression result. Moreover, to eliminate the boundary discontinuity problem faced by regression-based detectors, we construct CRB through designing an angle of rotation and the radius of the circumcircle. Extensive experiments and visual analysis are conducted on three public benchmarks, i.e., DOTA, HRSC2016, and DIOR-R. The results show that our proposed method has excellent localization and classification performance for oriented objects on all the representative datasets.

Instance Shadow Detection With a Single-Stage Detector.

This article formulates a new problem, instance shadow detection, which aims to detect shadow instance and the associated object instance that cast each shadow in the input image. To approach this task, we first compile a new dataset with the masks for shadow instances, object instances, and shadow-object associations. We then design an evaluation metric for quantitative evaluation of the performance of instance shadow detection. Further, we design a single-stage detector to perform instance shadow detection in an end-to-end manner, where the bidirectional relation learning module and the deformable maskIoU head are proposed in the detector to directly learn the relation between shadow instances and object instances and to improve the accuracy of the predicted masks. Finally, we quantitatively and qualitatively evaluate our method on the benchmark dataset of instance shadow detection and show the applicability of our method on light direction estimation and photo editing.

Liver tumor detection based on objects as points

The automatic detection of liver tumors by computed tomography is challenging, owing to their wide variations in size and location, as well as to their irregular shapes. Existing detection methods largely rely on two-stage detectors and use CT images marked with bounding boxes for training and detection. In this study, we propose a single-stage detector method designed to accurately detect multiple tumors simultaneously, and provide results demonstrating its increased speed and efficiency compared to prior methods. The proposed model divides CT images into multiple channels to obtain continuity information and implements a bounding box attention mechanism to overcome the limitation of inaccurate prediction of tumor center points and decrease redundant bounding boxes. The model integrates information from various channels using an effective Squeeze-and-Excitation attention module. The proposed model obtained a mean average precision result of 0.476 on the Decathlon dataset, which was superior to that of the prior methods examined for comparison. This research is expected to enable physicians to diagnose tumors very efficiently; particularly, the prediction of tumor center points is expected to enable physicians to rapidly verify their diagnostic judgments. The proposed method is considered suitable for future adoption in clinical practice in hospitals and resource-poor areas because its superior performance does not increase computational cost; hence, the equipment required is relatively inexpensive.

Deep Learning for the Improvement of Object Detection in Augmented Reality

In this article, we give a comprehensive overview of recent methods in object detection using deep learning and their uses in augmented reality. The objective is to present a complete understanding of these algorithms and how augmented reality functions and services can be improved by integrating these methods. We discuss in detail the different characteristics of each approach and their influence on real-time detection performance. Experimental analyses are provided to compare the performance of each method and make meaningful conclusions for their use in augmented reality. Two-stage detectors generally provide better detection performance, while single-stage detectors are significantly more time efficient and more applicable to real-time object detection. Finally, we discuss several future directions to facilitate and stimulate future research on object detection in augmented reality. Keywords: object detection, deep learning, convolutional neural network, augmented reality.

Density Aware 3D Object Single Stage Detector

In 3D object detection, a lot of works are focused on the characteristics of disorder and sparseness of point cloud. However, few of them directly explore the nonuniformity of point cloud which limits the detection precision. To deal with the problem, a method called Density Aware 3D Object Single Stage Detector (DA-3DSSD) is proposed in this paper. A method of defining point density of point cloud and the network processing strategies on the basis of the calculated density are proposed in DA-3DSSD. These strategies include a Density-aware Points Sampling (DPS) strategy and a Center Density Attention (CDA) module. Specifically, DPS adds a density aware factor in farthest points sampling (FPS), which improve the sampling process sensibility to density. DPS can reserve more internal points for sparse objects to gather more context information in the feature extraction process. CDA module models the relationships between center points densities and features, which can stress object features and suppress non-object features. CDA module strengthens the classification ability of the network. In the experiments, the proposed DA-3DSSD is evaluated on KITTI benchmark. Compared with the baseline method, the average precision has a significant improvement which verifies the effectiveness of our methods.