One-stage Object Detection Research Articles

Rethinking prediction alignment in one-stage object detection

Owing to their excellent performance and efficiency, one-stage detectors have been widely used in multimedia tasks, such as temporal action detection, object tracking, and video detection. However, misalignment between classification and regression branches limits the accuracy of the detector. Most existing works add an auxiliary branch or adopt a specific sample assignment strategy to alleviate this problem, but with little effect. In this paper, we attribute this to incomplete branch interactions and propose a comprehensive Predictive Aligned Object Detector (PAOD), which can better correlate two subtasks. Specifically, our proposed PAOD achieves a better trade-off between prediction-interactive and prediction-specific by adopting an Iterative Aggregation Module (IAM) and a Mutual Constraint Module (MCM). We also design an aligned label assignment with an adaptive metric and re-weighting mechanism to further narrow the misalignment between prediction heads. With negligible additional overhead, PAOD achieves 50.4 AP at single-model single-scale testing on the MS-COCO branch, which demonstrates the effectiveness of our proposal. Notably, PAOD consistently outperforms previous sota such as ATSS (47.7 AP), BorderDet (48.0 AP) and GFL (48.2 AP) by a large margin on COCO test-dev dataset, and achieves better performance than various dense detectors on Pascal VOC and CrowdHuman datasets. Code is available at https://github.com/JunruiXiao/PAOD.

Autonomous surface crack identification of concrete structures based on an improved one-stage object detection algorithm

oncrete is a widely used material in the infrastructure system. However, this material is susceptible to several factors that eventually create concrete cracks. Thus, accurately identifying the cracks' size and location in concrete structures is crucial for structural safety evaluation. In this research, an improved YOLOv4 network adopting the pruning technique and the EvoNorm-S0 structure was put forward to better identify concrete cracks from many misleading targets. The pruning technique is used to light-weight the network structure, and the EvoNorm-S0 could improve the detection accuracy. The results indicate that compared with the original YOLOv4, the mAP50 of the improved network is increased from 91.69% to 92.54% when both models are trained for 100 epochs, and the 1-Batch inference time is reduced by 15.9%. Moreover, the weight of the proposed network is just 54.9% of the original ones. The proposed network was also compared with three other leading algorithms in this field (i.e., SSD300, YOLOv3, and YOLO X-L) using the same dataset. The results show that the proposed network can not only correctly classify the largest number of objects with a fast calculation speed, but also has the highest mAP50. Thus, this proposed network exhibits several advantages for detecting concrete cracks and is a desirable tool for practical engineering.

Detection of Tip-Burn Stress on Lettuce Grown in an Indoor Environment Using Deep Learning Algorithms.

Lettuce grown in indoor farms under fully artificial light is susceptible to a physiological disorder known as tip-burn. A vital factor that controls plant growth in indoor farms is the ability to adjust the growing environment to promote faster crop growth. However, this rapid growth process exacerbates the tip-burn problem, especially for lettuce. This paper presents an automated detection of tip-burn lettuce grown indoors using a deep-learning algorithm based on a one-stage object detector. The tip-burn lettuce images were captured under various light and indoor background conditions (under white, red, and blue LEDs). After augmentation, a total of 2333 images were generated and used for training using three different one-stage detectors, namely, CenterNet, YOLOv4, and YOLOv5. In the training dataset, all the models exhibited a mean average precision (mAP) greater than 80% except for YOLOv4. The most accurate model for detecting tip-burns was YOLOv5, which had the highest mAP of 82.8%. The performance of the trained models was also evaluated on the images taken under different indoor farm light settings, including white, red, and blue LEDs. Again, YOLOv5 was significantly better than CenterNet and YOLOv4. Therefore, detecting tip-burn on lettuce grown in indoor farms under different lighting conditions can be recognized by using deep-learning algorithms with a reliable overall accuracy. Early detection of tip-burn can help growers readjust the lighting and controlled environment parameters to increase the freshness of lettuce grown in plant factories.

MiniYOLO: A lightweight object detection algorithm that realizes the trade‐off between model size and detection accuracy

The object detection task is to locate and classify objects in an image. The current state-of-the-art high-accuracy object detection algorithms rely on complex networks and high computational cost. These algorithms have high requirements on the memory resource and computing capability of the deployed device, and are difficult to apply to mobile and embedded devices. Through the depthwise separable convolution and multiple efficient network structures, this paper designs a lightweight backbone network and two different multiscale feature fusion structures, and proposes a lightweight one-stage object detection algorithm—MiniYOLO. With the model size of only 4.2 MB, MiniYOLO still maintains a high detection accuracy, realizing the trade-off between the model size and detection accuracy. Experimental results on MS COCO 2017 data set show that compared to the state-of-the-art PP-YOLO-tiny, MiniYOLO achieves higher mAP with the same model size. Compared with other lightweight object detection algorithms, MiniYOLO has certain advantages in detection accuracy or model size. The code associated with this paper can be downloaded from https://github.com/CaedmonLY/MiniYOLO/.

Visual inspection of surface defects of extreme size based on an advanced FCOS

ABSTRACT Surface defects of industrial products are generally detected through anchor-based object detection methods during manufacturing. However, these methods are prone to missed and false detection for ultra-elongated and ultra-fine defects. An advanced fully convolutional one-stage object detector (FCOS) is proposed. This method is based on an anchor-free FCOS network model. First, a novel type of center-ness is proposed to reduce the suppression of off-centered positions of defects of extreme size. In addition, to eliminate background interference, a self-adaptive center sampling method is proposed as a replacement for the conventional center sampling method. The regularization method and the loss function are also improved according to the defect characteristics. Experimental results show that this advanced-FCOS-based method outperforms anchor-based methodson the surface defect dataset. The proposed method effectively detects defects of extreme size without affecting the detection of normal defects. The performance of the proposed method meets the requirements of real industrial applications.

Ship Detection in SAR Images via Cross-Attention Mechanism

Deep learning has been widely applied to ship detection in Synthetic Aperture Radar (SAR) images. Unlike optical images, the current object detection methods have the problem of weak feature representation due to the low object resolution in SAR images. In addition, disturbed by chaotic noise, the features of classification and location are prone to significant differences, resulting in classification and location task misalignment. Therefore, this paper proposes a novel SAR ship target detection algorithm based on Cross-Attention Mechanism (CAM), which can establish the information interaction between the classification and localization task and strengthen the correlation between features through attention. In addition, to suppress the noise in multi-scale feature fusion, we designed an Attention-based Feature Fusion Module (AFFM), which uses the attention information between channels to perform the re-weighting operation. This operation can enhance useful feature information and suppress noise information. Experimental results show that on a benchmark SAR Ship Detection Dataset (SSDD), the Fully Convolutional One-Stage Object Detector (FCOS) with ResNet-50 backbone network was optimized to improve AP by 6.5% and computational cost by 0.51%. RetinaNet with ResNet-50 backbone network was optimized to improve AP by 1.8% and computational cost by 0.51%.

Artificial intelligence–based method for the rapid detection of fish parasites (Ichthyophthirius multifiliis, Gyrodactylus kobayashii, and Argulus japonicus)

Ichthyophthirius (Ichthyophthirius multifiliis), Monogenea (Gyrodactylus kobayashii) and fish lice (Argulus japonicus) are mainly infectious parasites, representative species of Protozoa, Platyhelminthes and Arthropoda, which cause serious economic losses in aquatic industry. In this research, a visual system that can rapidly detect and count these three kinds of parasites was realized based on a one-stage object detection deep learning algorithm YOLOv4 through python. Firstly, we made a dataset of parasites containing 27,930 images. Secondly, weights of the trained fish lice model were applied as the pre-training weights, and network (backbone indeed) frozen was also applied to obtaining a good performance predicting model with less time and higher accuracy, which showed that Transfer Learning could meet the training requirement for detecting these three fish parasites by using self-made data set. In addition, by comparison of different one-stage algorithms YOLOv4‑tiny, YOLOv3, et al., the best model with a total average accuracy (mAP) of 95.41% was achieved by the YOLOv4. Finally, this model could quickly detect and count mixed infected pictures with a speed of 0.13 s per image measured in GPU time. Further, a visual prediction and counting system equipped with the YOLOv4 was developed by using PyQt which is convenient for real-time video detection. A simple drug-giving system equipped with Praziquantel was also developed based on the thought of the Internet of Things in this study and after using a drug, the number of monogeneans infecting gold fish was reduced. At the same time, we modified YOLOv4 PANet by adding additional detection layers, which achieved greater performance of detecting smaller targets like Monogenea. Together, this Artificial intelligence–based method could realize the rapid detection and diagnosis of fish parasites in images and video.

LNFCOS: Efficient Object Detection through Deep Learning Based on LNblock

In recent deep-learning-based real-time object detection methods, the trade-off between accuracy and computational cost is an important consideration. Therefore, based on the fully convolutional one-stage detector (FCOS), which is a one-stage object detection method, we propose a light next FCOS (LNFCOS) that achieves an optimal trade-off between computational cost and accuracy. In LNFCOS, the loss of low- and high-level information is minimized by combining the features of different scales through the proposed feature fusion module. Moreover, the light next block (LNblock) is proposed for efficient feature extraction. LNblock performs feature extraction with a low computational cost compared with standard convolutions, through sequential operation on a small amount of spatial and channel information. To define the optimal parameters of LNFCOS suggested through experiments and for a fair comparison, experiments and evaluations were conducted on the publicly available benchmark datasets MSCOCO and PASCAL VOC. Additionally, the average precision (AP) was used as an evaluation index for quantitative evaluation. LNFCOS achieved an optimal trade-off between computational cost and accuracy by achieving a detection accuracy of 79.3 AP and 37.2 AP on the MS COCO and PASCAL VOC datasets, respectively, with 36% lower computational cost than the FCOS.

Multi-task feature-aligned head in one-stage object detection

Multi-task feature-aligned head in one-stage object detection

An improved YOLOv5 method for large objects detection with multi-scale feature cross-layer fusion network

SSD and YOLOv5 are the one-stage object detector representative algorithms. An improved one-stage object detector based on the YOLOv5 method is proposed in this paper, named Multi-scale Feature Cross-layer Fusion Network (M-FCFN). Firstly, we extract shallow features and deep features from the PANet structure for cross-layer fusion and obtain a feature scale different from 80 × 80, 40 × 40, and 20 × 20 as output. Then, according to the single shot multi-box detector, we propose the different scale features which are obtained by cross-layer fusion for dimension reduction and use it as another output for prediction. Therefore, two completely different feature scales are added as the output. Features of different scales are necessary for detecting objects of different sizes, which can increase the probability of object detection and significantly improve detection accuracy. Finally, aiming at the Autoanchor mechanism proposed by YOLOv5, we propose an EIOU k-means calculation. We have compared the four model structures of S , M , L , and X of YOLOv5 respectively. The problem of missed and false detections for large objects is improved which has better detection results. The experimental results show that our methods achieve 89.1% and 67.8% mAP @0.5 on the PASCAL VOC and MS COCO datasets. Compared with the YOLOv5_S, our methods improve by 4.4% and 1.4% mAP @ [0.5:0.95] on the PASCAL VOC and MS COCO datasets. Compared with the four models of YOLOv5, our methods have better detection accuracy for large objects. It should be more attention that our method on the large-scale mAP @ [0.5:0.95] is 5.4% higher than YOLOv5_S on the MS COCO datasets. • We proposed Multi-scale Feature Cross-layer Fusion Network (M-FCFN). • Two completely different feature scales are added as the output. • We propose an EIOU k-means Autoanchor calculation. • The problem of missed and false detections for large objects is improved. • Our method on the large-scale mAP @[0.5:0.95] is 5.4% higher than YOLOv5_S.

A novel detection method for hot spots of photovoltaic (PV) panels using improved anchors and prediction heads of YOLOv5 network

Accurate classification and detection of hot spots of photovoltaic (PV) panels can help guide operation and maintenance decisions, improve the power generation efficiency of the PV system, and ensure power stations’ safe and stable operation. Considering that, in this paper, the hot spots of PV panels collected on site are taken as the research object, and their formation mechanism is studied. Based on this, the morphological characteristics possessed by the hot spots of PV panels are classified into circular, linear, and array ones. A novel method for detecting hot spots of PV panels based on improved anchors and prediction heads of the YOLOv5 (AP-YOLOv5) network is proposed. Besides, to improve the detection precision of the YOLOv5 network at different scales in hot spots of PV panels, the K-means clustering algorithm is employed to cluster the length–width ratio of the data annotation frame, and a group of the anchors with smaller values is added so as to realize the detection of small targets by optimizing the cluster number. Apart from that, the corresponding prediction heads are constructed for the new anchor parameters to improve the detection precision concerning hot spots of PV panels. Furthermore, the model is verified by training experimental data and comparing test set results. The results showed that compared with other one-stage object detection models, the mean average precision (mAP) of the proposed network can achieve 87.8%, while the average recall rate is 89.0%, and the F1 score reaches 88.9%. In addition, the precision of this model is better than that of other models while maintaining a high frame rate; the frames per second (FPS) is as high as 98.6, thus laying a foundation for developing rapid detection tool of hot spots of PV panels, improving the safety of power station operation, and providing a method for intelligent operation and maintenance of PV power stations.

Embedded Real-Time Clothing Classifier Using One-Stage Methods for Saving Energy in Thermostats

Energy-saving is a mandatory research topic since the growing population demands additional energy yearly. Moreover, climate change requires more attention to reduce the impact of generating more CO2. As a result, some new research areas need to be explored to create innovative energy-saving alternatives in electrical devices that have high energy consumption. One research area of interest is the computer visual classification for reducing energy consumption and keeping thermal comfort in thermostats. Usually, connected thermostats obrtain information from sensors for detecting persons and scheduling autonomous operations to save energy. However, there is a lack of knowledge of how computer vision can be deployed in embedded digital systems to analyze clothing insulation in connected thermostats to reduce energy consumption and keep thermal comfort. The clothing classification algorithm embedded in a digital system for saving energy could be a companion device in connected thermostats to obtain the clothing insulation. Currently, there is no connected thermostat in the market using complementary computer visual classification systems to analyze the clothing insulation factor. Hence, this proposal aims to develop and evaluate an embedded real-time clothing classifier that could help to improve the efficiency of heating and ventilation air conditioning systems in homes or buildings. This paper compares six different one-stage object detection and classification algorithms trained with a small custom dataset in two embedded systems and a personal computer to compare the models. In addition, the paper describes how the classifier could interact with the thermostat to tune the temperature set point to save energy and keep thermal comfort. The results confirm that the proposed real-time clothing classifier could be implemented as a companion device in connected thermostats to provide additional information to end-users about making decisions on saving energy.

A Deep Learning Approach to Capture the Essence of Candida albicans Morphologies.

ABSTRACTWe present deep learning-based approaches for exploring the complex array of morphologies exhibited by the opportunistic human pathogen Candida albicans. Our system, entitled Candescence, automatically detects C. albicans cells from differential image contrast microscopy and labels each detected cell with one of nine morphologies. This ranges from yeast white and opaque forms to hyphal and pseudohyphal filamentous morphologies. The software is based upon a fully convolutional one-stage (FCOS) object detector, a deep learning technique that uses an extensive set of images that we manually annotated with the location and morphology of each cell. We developed a novel cumulative curriculum-based learning strategy that stratifies our images by difficulty from simple yeast forms to complex filamentous architectures. Candescence achieves very good performance (~85% recall; 81% precision) on this difficult learning set, where some images contain hundreds of cells with substantial intermixing between the predicted classes. To capture the essence of each C. albicans morphology and how they intermix, we used a second technique from deep learning entitled generative adversarial networks. The resultant models allow us to identify and explore technical variables, developmental trajectories, and morphological switches. Importantly, the model allows us to quantitatively capture morphological plasticity observed with genetically modified strains or strains grown in different media and environments. We envision Candescence as a community meeting point for quantitative explorations of C. albicans morphology.IMPORTANCE The fungus Candida albicans can “shape shift” between 12 morphologies in response to environmental variables. The cytoprotective capacity provided by this polymorphism makes C. albicans a formidable pathogen to treat clinically. Microscopy images of C. albicans colonies can contain hundreds of cells in different morphological states. Manual annotation of images can be difficult, especially as a result of densely packed and filamentous colonies and of technical artifacts from the microscopy itself. Manual annotation is inherently subjective, depending on the experience and opinion of annotators. Here, we built a deep learning approach entitled Candescence to parse images in an automated, quantitative, and objective fashion: each cell in an image is located and labeled with its morphology. Candescence effectively replaces simple rules based on visual phenotypes (size, shape, and shading) with neural circuitry capable of capturing subtle but salient features in images that may be too complex for human annotators.

Detecting Endotracheal Tube and Carina on Portable Supine Chest Radiographs Using One-Stage Detector with a Coarse-to-Fine Attention.

In intensive care units (ICUs), after endotracheal intubation, the position of the endotracheal tube (ETT) should be checked to avoid complications. The malposition can be detected by the distance between the ETT tip and the Carina (ETT–Carina distance). However, it struggles with a limited performance for two major problems, i.e., occlusion by external machine, and the posture and machine of taking chest radiographs. While previous studies addressed these problems, they always suffered from the requirements of manual intervention. Therefore, the purpose of this paper is to locate the ETT tip and the Carina more accurately for detecting the malposition without manual intervention. The proposed architecture is composed of FCOS: Fully Convolutional One-Stage Object Detection, an attention mechanism named Coarse-to-Fine Attention (CTFA), and a segmentation branch. Moreover, a post-process algorithm is adopted to select the final location of the ETT tip and the Carina. Three metrics were used to evaluate the performance of the proposed method. With the dataset provided by National Cheng Kung University Hospital, the accuracy of the malposition detected by the proposed method achieves and the ETT–Carina distance errors are less than mm.

Fast and accurate multi-class geospatial object detection with large-size remote sensing imagery using CNN and Truncated NMS

Multi-class geospatial object detection with remote sensing imagery has broad prospects in urban planning, natural disaster warning, industrial production, military surveillance and other applications. Accuracy and efficiency are two common measures for evaluating object detection models, and it is often difficult to achieve both at the same time. Developing a practical remote sensing object detection algorithm that balances the accuracy and efficiency is thus a big challenge in the Earth observation community. Here, we propose a comprehensive high-speed multi-class remote sensing object detection method. Firstly, we obtain a multi-volume YOLO (You Only Look Once) v4 model for balancing speed and accuracy, based on a pruning strategy of the convolutional neural network (CNN) and the one-stage object detection network YOLO v4. Moreover, we apply the Manhattan-Distance Intersection of Union (MIOU) loss function to the multi-volume YOLO v4 to further improve the accuracy without additional computational burden.Secondly, mainly due to computing limitations, a remote sensing image that is large-size relative to a natural image must first be divided into multiple smaller tiles, which are then detected separately, and finally, the detection results are spliced back to match the original image. In the process of remote sensing image slicing, a large number of truncated objects appear at the edge of tiles, which will produce a large number of false results in the subsequent detection links. To solve this problem, we propose a Truncated Non-Maximum Suppression (NMS) algorithm to filter out repeated and false detection boxes from truncated targets in the spliced detection results. We compare the proposed algorithm with the state-of-the-art methods on the Dataset for Object deTection in Aerial images (DOTA) and DOTA v2. Quantitative evaluations show that mAP and FPS reach 77.3 and 35 on DOTA, and 61.0 and 74 on DOTA v2. Overall, our method reaches the optimal balance between efficiency and accuracy, and realizes the high-speed remote sensing object detection.

X-Ray Small Target Security Inspection Based on TB-YOLOv5

Security inspection is extremely important for the safety of public places. In this research, we are trying to propose a novel algorithm and investigated theoretically in the X-ray dataset, which can optimize the relative low detection accuracy and the latent omission detection of smaller objects when using You Only Look Once version 5 (YOLOv5). For one side, the transform detection network is selected to be added at the bottom layer of backbone structure to avoid the loss of useful information during sequential calculation. On another side, we attempt to adjust the existing PANet structural elements of the model, including their connections and other related parameters to improve the detection performance. We integrate an efficient BiFPN with the CA mechanism, which can enhance feature extraction, and named it attention-BiFPN. Experimental consequences demonstrate that the detection accuracy of the proposed model, which we name “TB-YOLOv5,” has obvious advantages in check performance compared with the mainstream one-stage object detection models. Meanwhile, compared with YOLOv5, the data results display an improvement of up to 14.9%, and the average precision at 0.5 IOU even reached 23.4% higher in the region of small object detection. Our purpose was to explore the potential of changing a popular detection algorithm such as YOLO to address specific tasks and provide insights on how specialized adjustments can influence the detection of small objects. Our work can supply an effective method of enhancing the performance of X-ray security inspection and show promising potential for deep learning in related fields.

Inferior and Coordinate Distillation for Object Detectors.

Current distillation methods only distill between corresponding layers, and do not consider the knowledge contained in preceding layers. To solve this problem, we analyzed the guiding effect of the inferior features of a teacher model on the coordinate feature of a student model, and proposed inferior and coordinate distillation for object detectors. The proposed method utilizes the rich information contained in different layers of the teacher model; such that the student model can review the old information and learn the new information, in addition to the dark knowledge in the teacher model. Moreover, the refine module is used to align the features of different layers, distinguish the spatial and channel to extract attention, strengthen the correlation between the features of different stages, and prevent the disorder caused by merging. Exclusive experiments were conducted on different object detectors. The results for the mean average precision (mAP) obtained using Faster R-CNN, RetinaNet, and fully convolutional one-stage object detector (FCOS) with ResNet-50 as its backbone were 40.5%, 39.8%, and 42.8% with regard to the COCO dataset, respectively; which are 2.1%, 2.4%, and 4.3% higher than the benchmark, respectively.

Adaptive weighted multiscale feature fusion for small drone object detection

Drone object detection in low-altitude airspace plays an essential role in many practical applications, such as security and airspace monitoring. Despite the remarkable progress made by many methods, drone object detection still remains challenging due to the complex background and huge differences in scales of drones. To address the above issues, an improved fully convolutional one-stage object detection (FCOS) model based on adaptive weighted feature fusion (AWFF) module is proposed for multiscale drone object detection in complex background. By learning the spatial relevance of feature maps at each scale and improving the scale invariance of features based on the channel attention mechanism, AWFF module could adaptively fuse the features of adjacent scale. In addition, a receptive field enhancement module is designed to reduce the information loss in the feature fusion process. Extensive experiments are conducted to evaluate the effectiveness of the proposed module and method on the constructed low-altitude drone dataset, which concludes that the mean average precision of the AWFF-FCOS is increased by 2.1% compared with the baseline method. And extensive ablation experiments further demonstrate that the proposed AWFF module and REF module could be integrated into the state-of-the-art method to improve the performance of drone object detection.

Automatic detection of defects in concrete structures based on deep learning

The inspection of defects in concrete infrastructure is a costly and time-consuming task. Many image processing techniques (IPTs) have been proposed to replace manual inspection of defects. However, the extensively varying situations in real world (e.g., lighting and shadow changes) pose challenges to the wide adoption of IPTs. To solve these problems, we propose an automated detection method for concrete surface defects based on deep learning. Our model is a one-stage object detection network, which is composed of two parts: the backbone network EfficientNetB0 and the detector. EfficientNetB0 is formed by stacking multiple mobile inverted bottleneck convolution (MBConv) blocks, improving the feature extraction performance while reducing the number of parameters. The detector is inspired by feature pyramid network which extracts feature information from three scales, and fuses low-level features with high-level features via up-sampling operation to improve detection accuracy. The training and testing of the model are carried out on the concrete surface defects dataset. Results show 76.4% and 89.9% average precision (AP) for crack and exposed bars, respectively, with a mean AP of 83.2%.

Transmission Line Object Detection Method Based on Label Adaptive Allocation

Inspection of the integrality of components and connecting parts is an important task to maintain safe and stable operation of transmission lines. In view of the fact that the scale difference of the auxiliary component in a connecting part is large and the background environment of the object is complex, a one-stage object detection method based on the enhanced real feature information and the label adaptive allocation is proposed in this study. Based on the anchor-free detection algorithm FCOS, this method is optimized by expanding the real feature information of the adjacent feature layer fusion and the semantic information of the deep feature layer, as well as adaptively assigning the label through the idea of pixel-by-pixel detection. In addition, the grading ring image is sliced in original data to improve the proportion of bolts in the dataset, which can clear the appearance features of small objects and reduce the difficulty of detection. Experimental results show that this method can eliminate the background interference in the GT (ground truth) as much as possible in object detection process, and improve the detection accuracy for objects with a narrow shape and small size. The evaluation index AP (average precision) increased by 4.1%. Further improvement of detection accuracy lays a foundation for the realization of efficient real-time patrol inspection.