Single-stage Detector Research Articles

A regional attention-based detector for SAR ship detection

ABSTRACT Automatic ship detection in Synthetic Aperture Radar (SAR)imagery has been playing a significant role in the field of marine monitoring. But great challenges still exist in real-time application. Despite the exciting progress made by deep-learning techniques, most detectors failed to yield locations of fairly high quality, especially for small objects under the complicated background. To alleviate the above problem, the author proposes a single-stage detector based on the attention mechanism. First, we degenerate pixel-level semantic segmentation into box-level segmentation to suppress background interference. The attention map generated from weak segmentation roughly locates the region of interest through automatic learning. Besides, it has a top-down feature pyramid structure embedded with the multi-branch fusion module. With more detailed features and richer semantic information, it can detect multi-scale and multi-directional targets more effectively. Experiments on the SAR ship dataset have achieved a promising result.

Equivalent circuit and fundamental limit of multi-stage infrared photodetectors

Based on an equivalent circuit model, a signal current in a multi-stage infrared (IR) photodetector is derived and used to discover a meaningful relation among quantum efficiency, collection efficiency, and particle conversion efficiency. Based on such a relation, it is demonstrated theoretically that the ultimate detectivities in multi-stage interband cascade infrared photodetectors (ICIPs) with identical discrete absorbers are the same as that in a conventional single-absorber detector in the limit of perfect collection (i.e., independent of the number of cascade stages) but higher than that in the single-stage detector with a finite diffusion length. Detailed derivations and calculations, along with relevant discussion, are provided to show how ICIPs are optimized for maximizing the detectivity and to understand the underlying physics. Multi-stage ICIPs with identical discrete absorbers are robust and durable against structural variations without being restricted by current matching and, therefore, are of more significance for practical applications such as those that require high-speed response or circumventing the diffusion length limitation. The results obtained for ICIPs with identical discrete absorbers can also be applied for quantum cascade detectors and photovoltaic quantum well infrared photodetectors. The results and insights gained from this work will further improve the understanding of multi-stage IR photodetectors and generate increased interest in the development of ICIPs and related devices for useful applications.

Realtime single-stage instance segmentation network based on anchors

In this paper, we propose an instance segmentation method uses a single-stage detector. Compared to the two-stage method, the single-stage method is simpler and easier to train. Not rely on the traditional region proposal, it directly uses pixels, which reduces the complexity of the network and significantly increases the speed. Our segmentation method is based on anchor boxes, which performs multi-scale detection by setting anchors of different sizes on multi-scale feature maps. We add a new branch to the prediction head to generate prototype masks and mask coefficients, then linearly combine them to generate mask. In our experiments, the proposed model had better performance, we got 35.12 fps on a single NVIDIA GEFORCE GTX 2080 GPU, which proves that our method is simple, effective, and faster.

RFSOD: a lightweight single-stage detector for real-time embedded applications to detect small-size objects

RFSOD: a lightweight single-stage detector for real-time embedded applications to detect small-size objects

Multi-level refinement enriched feature pyramid network for object detection

Class Imbalance and scales imbalance are common in object detection. A class imbalance occurs due to insufficient inequality between the number of instances with respect to different classes, while an imbalance in scale occurs when object have different scales and a different number of examples of different scales. In order to solve the problem of scale variance (scale imbalance) and class imbalance together, we propose a simple and effective feature enhancement scheme that explicitly uses all information of a multi-level structure to generate a multilevel contextual features pyramid with multiple scales. We also introduce a cascaded refinement scheme that incorporates multi-scale contextual features into the Single Shot Detector (SSD) predictive layers to improve their distinctiveness for multi-scale detection. A stack of multi-scale contextual feature modules is used in a feature enhancement scheme to merge the multi-level and multi-scale features. Then we collect the equivalent scale features over the Multi-layer Feature Fusion (MLFF) unit to construct a feature pyramid in which each feature map is made up of layers from multiple levels. More robustness and contextual information are integrated into the pyramid through chain parallel pooling operation. To improve classification and regression, a cascaded refinement scheme is proposed that effectively captures a large amount of contextual information and refines the anchors to solve the class imbalance problem. The experiments are carried out on two benchmarks datasets: MS COCO and PASCAL VOC 07/12. Our proposed approach achieves state-of-the-art accuracy with an AP of 40.6 in the case of multi-scale inference on MS COCO Test-dev (input size 320 × 320). For 512 × 512 input on the MS COCO Test-dev, our approach leads in an absolute gain in precision of 1.8% compared to the best reported results of single-stage detector (AP: 45.7).

Real-time Automated Detection and Recognition of Nigerian License Plates via Deep Learning Single Shot Detection and Optical Character Recognition

License plate detection and recognition are critical components of the development of a connected Intelligent transportation system, but are underused in developing countries because to the associated costs. Existing license plate detection and recognition systems with high accuracy require the usage of Graphical Processing Units (GPU), which may be difficult to come by in developing nations. Single stage detectors and commercial optical character recognition engines, on the other hand, are less computationally expensive and can achieve acceptable detection and recognition accuracy without the use of a GPU. In this work, a pretrained SSD model and a tesseract tessdata-fast traineddata were fine-tuned on a dataset of more than 2,000 images of vehicles with license plate. These models were combined with a unique image preprocessing algorithm for character segmentation and tested using a general-purpose personal computer on a new collection of 200 automobiles with license plate photos. On this testing set, the plate detection system achieved a detection accuracy of 99.5 % at an IOU threshold of 0.45 while the OCR engine successfully recognized all characters on 150 license plates, one character incorrectly on 24 license plates, and two or more incorrect characters on 26 license plates. The detection procedure took an average of 80 milliseconds, while the character segmentation and identification stages took an average of 95 milliseconds, resulting in an average processing time of 175 milliseconds per image, or 6 photos per second. The obtained results are suitable for real-time traffic applications.

Robust Vehicle Detection in High-Resolution Aerial Images With Imbalanced Data

Vehicle detection in images from unmanned aerial vehicles (UAVs) plays an important role in traffic surveillance and urban planning due to the popularity of UAVs. However, the class imbalance problem is an important factor that restricts the performance of vehicle detectors. There are two types of class imbalance in UAV images, i.e., foreground-background imbalance and foreground–foreground imbalance. For anchor-based single stage detector, as many ground truths cannot be assigned to corresponding anchors because of low intersection over union, it makes the foreground-background imbalance problem more severe. Therefore, we propose a novel bag-based single-stage detector, which treats each position on the feature map as a bag. A simple and adaptive definition of bags is proposed along with the positive sample definition method, which is utilized to ensure more ground truths can be assigned to proper bags. In addition, we utilize online hard example mining method to control the proportion of positive and negative samples during the training process. To address the foreground–foreground imbalance, we propose a novel data augmentation algorithm, which allows us to create appropriate visual context for under-represented class. Extensive experiments demonstrate the superiority of the proposed algorithm, compared with other state-of-the-art solutions. <p xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Impact Statement—Recently, unmanned aerial vehicles (UAVs) are widely used in intelligent transportation due to their low price and high flexibility, which makes vehicle detection in UAV images important for automatically gathering of traffic information. However, the class imbalance problem, which is common in object detection where some classes have far fewer frequencies in the dataset, has an adverse effect on the performance of vehicle detectors. The data augmentation method and deep learning based vehicle detector proposed in this article are able to reduce the negative impact and improve detection performance by at least 1.27% in mean average precision index. In addition, compared with algorithms with similar detection performance, our method is at least 15 ms faster. The proposed method can benefit users in a wide variety of applications including UAV transportation, traffic surveillance, and urban planning.

CIA-SSD: Confident IoU-Aware Single-Stage Object Detector From Point Cloud

Existing single-stage detectors for locating objects in point clouds often treat object localization and category classification as separate tasks, so the localization accuracy and classification confidence may not well align. To address this issue, we present a new single-stage detector named the Confident IoU-Aware Single-Stage object Detector (CIA-SSD). First, we design the lightweight Spatial-Semantic Feature Aggregation module to adaptively fuse high-level abstract semantic features and low-level spatial features for accurate predictions of bounding boxes and classification confidence. Also, the predicted confidence is further rectified with our designed IoU-aware confidence rectification module to make the confidence more consistent with the localization accuracy. Based on the rectified confidence, we further formulate the Distance-variant IoU-weighted NMS to obtain smoother regressions and avoid redundant predictions. We experiment CIA-SSD on 3D car detection in the KITTI test set and show that it attains top performance in terms of the official ranking metric (moderate AP 80.28%) and above 32 FPS inference speed, outperforming all prior single-stage detectors. The code is available at https://github.com/Vegeta2020/CIA-SSD.

R3Det: Refined Single-Stage Detector with Feature Refinement for Rotating Object

Rotation detection is a challenging task due to the difficulties of locating the multi-angle objects and separating them effectively from the background. Though considerable progress has been made, for practical settings, there still exist challenges for rotating objects with large aspect ratio, dense distribution and category extremely imbalance. In this paper, we propose an end-to-end refined single-stage rotation detector for fast and accurate object detection by using a progressive regression approach from coarse to fine granularity. Considering the shortcoming of feature misalignment in existing refined single-stage detector, we design a feature refinement module to improve detection performance by getting more accurate features. The key idea of feature refinement module is to re-encode the position information of the current refined bounding box to the corresponding feature points through pixel-wise feature interpolation to realize feature reconstruction and alignment. For more accurate rotation estimation, an approximate SkewIoU loss is proposed to solve the problem that the calculation of SkewIoU is not derivable. Experiments on three popular remote sensing public datasets DOTA, HRSC2016, UCAS-AOD as well as one scene text dataset ICDAR2015 show the effectiveness of our approach. The source code is available at https://github.com/Thinklab-SJTU/R3Det_Tensorflow and is also integrated in our open source rotation detection benchmark: https://github.com/yangxue0827/RotationDetection.

Local Enhancement and Bidirectional Feature Refinement Network for Single-Shot Detector

Benefit from multi-scale feature pyramid methods, recently single-stage object detectors have achieved promising accuracy and fast inference speed. However, the majority of existing feature pyramid detection techniques only simply describe complex contextual relationships from different scales. Not only are there no effective modules that adaptively extend appropriate semantic information from deeper layers, but the finer spatial localization cues from lower layers are often ignored. In this paper, we present a Local Enhancement and Bidirectional Feature Refinement Network (LFBFR), which includes two optimization methods to achieve remarkable improvements in detection accuracy. Firstly, to make the backbone more suitable for detection task, we modify the pre-trained classification backbone to mitigate the loss of details in small objects due to consecutive decrease of the image resolution. Then we propose a Bidirectional Feature Refinement Pyramid, which can effectively utilize the inter-channel relationship of higher-level features and fine appearance cues from lower-level features by using the attention residual refinement module and the feature reuse module. Ultimately, to assess the performance of the proposed LFBFR, we design a powerful end-to-end single-stage detector called LFBFR-SSD by embedding it into the framework of SSD. Extensive experiments on the PASCAL VOC and MS COCO verify that our LFBFR-SSD outperforms a lot of state-of-the-art detectors while maintaining a real-time speed.

Class-discriminative focal loss for extreme imbalanced multiclass object detection towards autonomous driving

Currently, modern object detection algorithms still suffer the imbalance problems especially the foreground–background and foreground–foreground class imbalance. Existing methods generally adopt re-sampling based on the class frequency or re-weighting based on the category prediction probability, such as focal loss, proposed to rebalance the loss assigned to easy negative examples and hard positive examples for single-stage detectors. However, there are still two critical issues unresolved. In practical applications, such as autonomous driving, the class imbalance will become more extreme due to the increased detection field and target distribution characteristics, needing a more effective way to balance the foreground–background class imbalance. Besides, existing methods typically employ the sigmoid or softmax entropy loss for classification task, which we believe is not capable to realize the foreground–foreground class balance. In this paper, we propose a new form of focal loss by re-designing the re-weighting scheme that can calculate the weight according to the probability as well as widen the weight difference of the examples. Besides, we introduce the extended focal loss to multi-class classification task by reformulating the standard softmax cross-entropy loss for better utilizing the discriminant difference of foreground categories, thereby yielding a class-discriminative focal loss. Comprehensive experiments are conducted on the KITTI and BDD dataset, respectively. The results show that our approach can easily surpass focal loss with no more training and inference time cost. Besides, when trained with the proposed loss function, current state-of-the-art object detectors no matter in one-stage or two-stage paradigms can achieve significant performance gains.

An unsupervised domain adaptation scheme for single-stage artwork recognition in cultural sites

Recognizing artworks in a cultural site using images acquired from the user's point of view (First Person Vision) allows to build interesting applications for both the visitors and the site managers. However, current object detection algorithms working in fully supervised settings need to be trained with large quantities of labeled data, whose collection requires a lot of times and high costs in order to achieve good performance. Using synthetic data generated from the 3D model of the cultural site to train the algorithms can reduce these costs. On the other hand, when these models are tested with real images, a significant drop in performance is observed due to the differences between real and synthetic images. In this study we consider the problem of Unsupervised Domain Adaptation for object detection in cultural sites. To address this problem, we created a new dataset containing both synthetic and real images of 16 different artworks. We hence investigated different domain adaptation techniques based on one-stage and two-stage object detector, image-to-image translation and feature alignment. Based on the observation that single-stage detectors are more robust to the domain shift in the considered settings, we proposed a new method which builds on RetinaNet and feature alignment that we called DA-RetinaNet. The proposed approach achieves better results than compared methods on the proposed dataset and on Cityscapes. To support research in this field we release the dataset at the following link https://iplab.dmi.unict.it/EGO-CH-OBJ-UDA/ and the code of the proposed architecture at https://github.com/fpv-iplab/DA-RetinaNet.

Feature Learning Improved by Location Guidance and Supervision for Object Detection

In recent years, the single-stage detectors have been developed rapidly; however, compared with the multi-stage detectors, their detection precision is still relatively low. Single-stage detectors and multi-stage detectors are analyzes and compared in detail in this paper, which reveals that single-stage detectors suffer from some problems, including feature loss and inaccurate feature extraction. Therefore, this paper proposes a novel detection model, dubbed Optimized Network (OptNet), to alleviate these deficiencies. OptNet consists of three modules: pyramid of attention features, feature alignment and consistency supervision (CS). The pyramid of attention features, based on feature pyramid networks (FPNs), introduces a novel branch named attention FPN (AtFPN), which aggregates the multi-layer features of the backbone network and optimizes the object features by using lightweight attention modules. AtFPN alleviates the loss of the feature pyramid information and the blocking of feature transmission between adjacent layers. Meanwhile, it provides global information for the model. The feature alignment module aligns the anchor box to the feature by using the object location information to guide the network to extract precise object features. Finally, CS accelerates network optimization and reduces semantic differences between the features on different layers. In the detection stage, OptNet optimizes the prediction of the model with the first detection result to improve the accuracy. Experiments on the MS COCO 2017 dataset demonstrate that OptNet yields significant improvement in the detection precision.

A Fast Postprocessing Algorithm for the Overlapping Problem in Wafer Map Detection

Mixed defects have become increasingly popular in defect detection and one of the hottest research areas in wafer maps. Postprocessing methods used to solve the overlapping problem in mass mixed defects have a poor detection speed, which is insufficient for rapid defect detection. In this paper, the fast‐soft nonmaximum suppression (fs‐NMS) method is proposed to solve this problem. The score of the detection box is updated by optimizing the penalty distribution function. Further, this paper analyzes the performance of the fs‐NMS method in wafer defect detection. As a penalty, the logistic function is used, and experiments are conducted using single‐stage and two‐stage detectors. The final results show that, compared to the soft‐NMS, the efficiency for the single‐stage and two‐stage detectors is increased on average by 9.63% and 21.72%, respectively.

A Novel Detection Framework About Conditions of Wearing Face Mask for Helping Control the Spread of COVID-19.

Properly wearing a face mask has become an effective way to limit the COVID-19 transmission. In this work, we target at detecting the fine-grained wearing state of face mask: face without mask, face with wrong mask, face with correct mask. This task has two main challenging points: 1) absence of practical datasets, and 2) small intra-class distance and large inter-class distance. For the first challenging point, we introduce a new practical dataset covering various conditions, which contains 8635 faces with different wearing status. For the second challenging point, we propose a novel detection framework about conditions of wearing face mask, named Context-Attention R-CNN, which enlarge the intra-class distance and shorten inter-class distance by extracting distinguishing features. Specifically, we first extract the multiple context feature for region proposals, and use attention module to weight these context feature from channel and spatial levels. And then, we decoupling the classification and localization branches to extract more appropriate feature for these two tasks respectively. Experiments show that the Context-Attention R-CNN achieves 84.1% mAP on our proposed dataset, outperforming Faster R-CNN by 6.8 points. Moreover, Context-Attention R-CNN still exceed some state-of-the-art single-stage detectors.

Multi-Level Refinement Feature Pyramid Network for Scale Imbalance Object Detection

Object detection becomes a challenge due to diversity of object scales. In general, modern object detectors use feature pyramid to learn multi-scale representation for better results. However, current versions of feature pyramid are insufficient to handle the scale imbalance, as it is inefficient to integrate the semantic information across different scales. In this paper, we reformulate the feature pyramid construction as a feature reconfiguration process. Finally, we propose a novel detection network, Multi-level Refinement Feature pyramid Network (MRFPN), to combine the high-level features (i.e., semantic information), middle-level feature and low-level feature (i.e., boundary information), in a highly-nonlinear yet efficient manner. In particular, a novel contextual features module (chain parallel pooling) is proposed, which consists of global attention and local reconfigurations. It efficiently gathers task-oriented contextual features across different scales and spatial locations (i.e., lightweight local reconfiguration and global attention). To evaluate significance of proposed model, we designed and trained a robust end-to-end single stage detector called MRFDet by assimilating it into a conventional SSD model, and it achieved better detection performance compared to most recent single-stage objects detectors. In particular, MRFDet achieves an AP of 45.2 with MS-COCO and an improvement in the map of 4.5% with VOC compared to conventional SSD. We are releasing the source code for our proposed model MRFDet, to facilitate the research community.

A Refined Single-Stage Detector With Feature Enhancement and Alignment for Oriented Objects

Compared to traditional object detection of horizontal bounding box, detecting rotated objects with arbitrary orientations and various scales is a critical yet challenging task especially in remote sensing images. Although considerable progresses have been made through the utilization of deep CNN, there still exists space for exploration of oriented object detection. In this article, we propose a refined single-stage detector for oriented objects, which is equipped with the enhanced feature extraction network and an adaptive feature alignment module for finer detection. For feature enhancement, a bidirectional and inner residual feature pyramid network and a multiscale feature aggregation module are devised for getting more representative features. Then, to address the problem of feature misalignment, we propose the adaptive feature alignment module to reallocate the sampling locations and weight the positive and negative feature points during the refined detection process. The reallocation and weighting will be adjusted adaptively according to the results from the coarse detection. Experiments conducted on public remote sensing databases show the effectiveness of our method.

ROS-Det: Arbitrary-Oriented Ship Detection in High Resolution Optical Remote Sensing Images via Rotated One-Stage Detector

To address these problems, namely, arbitrary orientations, various sizes, and dense distributions of ship detection, we propose an arbitrary-oriented ship detection method via rotated single-stage detector (ROS-Det), which integrates a feature pyramid network (FPN) based on an improved ResNet50, rotated anchors, a classification network, and a regression network together. Firstly, to improve robustness against various sizes of ships, the FPN is used to fusion multiscale convolutional feature maps. Through several tweaks, the improved ResNet50 can receive more information and reduce the computational cost. Secondly, for the purpose of arbitrary-oriented ship detection, rotated anchors, skew intersection over union (IoU), and skew non-maximum suppression (NMS) are introduced to RetinaNet. Then, on account of the disadvantages that the arbitrary-oriented object detection methods usually cause loss discontinuity problem, we improve the traditional smooth L1 loss function by introducing an IoU constant factor. Finally, based on several techniques such as data augmentation and transfer learning, we achieve ship detection on a public ship dataset HRSC2016. Through comparison experiments, we have analyzed and discussed the validity of our proposed ROS-Det, which achieves the state-of-the-art performance.

A CenterNet++ model for ship detection in SAR images

Ship detection in SAR images is a challenging task due to two difficulties. (1) Because of the long observation distance, ships in SAR images are small with low resolution, leading to high false negative. (2) Because of the complex onshore background, ships are easily confused with other objects with similar appearance. To solve these problems, we propose an effective and stable single-stage detector called CenterNet++. Our model mainly consists of three modules, i.e., feature refinement module, feature pyramids fusion module, and head enhancement module. Firstly, to address small objects detection problem, we design a feature refinement module for extracting multi-scale contextual information. Secondly, feature pyramids fusion module is developed for generating more powerful semantic information. Finally, to alleviate the impact of complex background, head enhancement module is proposed for a balance between foreground and background. To prove the effectiveness and robustness of the proposed method, we make extensive experiments on three popular SAR image datasets, i.e., AIR-SARShip, SSDD, SAR-Ship. The experimental results show that our CenterNet++ reaches state-of-the-art performance on all datasets. In addition, compared with the baseline CenterNet, the proposed method achieves a remarkable accuracy improvement with negligible increase in time cost.

Cascade single stage Detector using full convolution network

For object detection, the two-stage approach always achieves better performance than one-stage. But one-stage is more effective. The main shortcomings of one-stage approach are: class imbalance problem during the training phase and sometimes the correct object score is lower. To address those problems, we propose a novel single-shot detector, called Cascade detection. It is composed of several stages, and each stage is composed of several detection layers. The main point of our method is: we design an anchors to anchors module (A2AM), which gives one-stage approach the ability of training stage by stage. It can not only address the class imbalance problem in the training phase, but also increase the object scores and the Iou between prediction and ground truth in the inference phase. The multitask loss function enables us to train the whole network in an end-to-end way. Experiments on PASCAL VOC 2007, PASCAL VOC 2012 and MS COCO prove that our method achieves state-of-art performance detection accuracy and computation efficiency.