General Object Detector Research Articles

A Defect Detection Method Based on YOLOv7 for Automated Remanufacturing

Remanufacturing of mechanical parts has recently gained much attention due to the rapid development of green technologies and sustainability. Recent efforts to automate the inspection step in the remanufacturing process using artificial intelligence are noticeable. In this step, a visual inspection of the end-of-life (EOL) parts is carried out to detect defective regions for restoration. This operation relates to the object detection process, a typical computer vision task. Many researchers have adopted well-known deep-learning models for the detection of damage. A common technique in the object detection field is transfer learning, where general object detectors are adopted for specific tasks such as metal surface defect detection. One open-sourced model, YOLOv7, is known for real-time object detection, high accuracy, and optimal scaling. In this work, an investigation into the YOLOv7 behavior on various public metal surface defect datasets, including NEU-DET, NRSD, and KolektorSDD2, is conducted. A case study validation is also included to demonstrate the model’s application in an industrial setting. The tiny variant of the YOLOv7 model showed the best performance on the NEU-DET dataset with a 73.9% mAP (mean average precision) and 103 FPS (frames per second) in inference. For the NRSD dataset, the model’s base variant resulted in 88.5% for object detection and semantic segmentation inferences. In addition, the model achieved 65% accuracy when testing on the KolektorSDD2 dataset. Further, the results are studied and compared with some of the existing defect detection models. Moreover, the segmentation performance of the model was also reported.

FocusDet: an efficient object detector for small object

The object scale of a small object scene changes greatly, and the object is easily disturbed by a complex background. Generic object detectors do not perform well on small object detection tasks. In this paper, we focus on small object detection based on FocusDet. FocusDet refers to the small object detector proposed in this paper. It consists of three parts: backbone, feature fusion structure, and detection head. STCF-EANet was used as the backbone for feature extraction, the Bottom Focus-PAN for feature fusion, and the detection head for object localization and recognition.To maintain sufficient global context information and extract multi-scale features, the STCF-EANet network backbone is used as the feature extraction network.PAN is a feature fusion module used in general object detectors. It is used to perform feature fusion on the extracted feature maps to supplement feature information.In the feature fusion network, FocusDet uses Bottom Focus-PAN to capture a wider range of locations and lower-level feature information of small objects.SIOU-SoftNMS is the proposed algorithm for removing redundant prediction boxes in the post-processing stage. SIOU multi-dimension accurately locates the prediction box, and SoftNMS uses the Gaussian algorithm to remove redundant prediction boxes. FocusDet uses SIOU-SoftNMS to address the missed detection problem common in dense tiny objects.The VisDrone2021-DET and CCTSDB2021 object detection datasets are used as benchmarks, and tests are carried out on VisDrone2021-det-test-dev and CCTSDB-val datasets. Experimental results show that FocusDet improves mAP@.5% from 33.6% to 46.7% on the VisDrone dataset. mAP@.5% on the CCTSDB2021 dataset is improved from 81.6% to 87.8%. It is shown that the model has good performance for small object detection, and the research is innovative.

FRED: Towards a Full Rotation-Equivariance in Aerial Image Object Detection

Rotation-equivariance is an essential yet challenging property in oriented object detection. While general object detectors naturally leverage robustness to spatial shifts due to the translation-equivariance of the conventional CNNs, achieving rotation-equivariance remains an elusive goal. Current detectors deploy various alignment techniques to derive rotation-invariant features, but still rely on high capacity models and heavy data augmentation with all possible rotations. In this paper, we introduce a Fully Rotation-Equivariant Oriented Object Detector (FRED), whose entire process from the image to the bounding box prediction is strictly equivariant. Specifically, we decouple the invariant task (object classification) and the equivariant task (object localization) to achieve end-to-end equivariance. We represent the bounding box as a set of rotation-equivariant vectors to implement rotation-equivariant localization. Moreover, we utilized these rotation-equivariant vectors as offsets in the deformable convolution, thereby enhancing the existing advantages of spatial adaptation. Leveraging full rotation-equivariance, our FRED demonstrates higher robustness to image-level rotation compared to existing methods. Furthermore, we show that FRED is one step closer to non-axis aligned learning through our experiments. Compared to state-of-the-art methods, our proposed method delivers comparable performance on DOTA-v1.0 and outperforms by 1.5 mAP on DOTA-v1.5, all while significantly reducing the model parameters to 16%.

AO-DETR: Anti-Overlapping DETR for X-Ray Prohibited Items Detection.

Prohibited item detection in X-ray images is one of the most essential and highly effective methods widely employed in various security inspection scenarios. Considering the significant overlapping phenomenon in X-ray prohibited item images, we propose an anti-overlapping detection transformer (AO-DETR) based on one of the state-of-the-art (SOTA) general object detectors, DETR with improved denoising anchor boxes (DINO). Specifically, to address the feature coupling issue caused by overlapping phenomena, we introduce the category-specific one-to-one assignment (CSA) strategy to constrain category-specific object queries in predicting prohibited items of fixed categories, which can enhance their ability to extract features specific to prohibited items of a particular category from the overlapping foreground-background features. To address the edge blurring problem caused by overlapping phenomena, we propose the look forward densely (LFD) scheme, which improves the localization accuracy of reference boxes in mid-to-high-level decoder layers and enhances the ability to locate blurry edges of the final layer. Similar to DINO, our AO-DETR provides two different versions with distinct backbones, tailored to meet diverse application requirements. Extensive experiments on the PIXray, OPIXray, and HIXray datasets demonstrate that the proposed method surpasses the SOTA object detectors, indicating its potential applications in the field of prohibited item detection. The source code will be available at: https://github.com/Limingyuan001/AO-DETR.

Learning geometric Jensen-Shannon divergence for tiny object detection in remote sensing images.

Tiny objects in remote sensing images only have a few pixels, and the detection difficulty is much higher than that of regular objects. General object detectors lack effective extraction of tiny object features, and are sensitive to the Intersection-over-Union (IoU) calculation and the threshold setting in the prediction stage. Therefore, it is particularly important to design a tiny-object-specific detector that can avoid the above problems. This article proposes the network JSDNet by learning the geometric Jensen-Shannon (JS) divergence representation between Gaussian distributions. First, the Swin Transformer model is integrated into the feature extraction stage as the backbone to improve the feature extraction capability of JSDNet for tiny objects. Second, the anchor box and ground-truth are modeled as two two-dimensional (2D) Gaussian distributions, so that the tiny object is represented as a statistical distribution model. Then, in view of the sensitivity problem faced by the IoU calculation for tiny objects, the JSDM module is designed as a regression sub-network, and the geometric JS divergence between two Gaussian distributions is derived from the perspective of information geometry to guide the regression prediction of anchor boxes. Experiments on the AI-TOD and DOTA datasets show that JSDNet can achieve superior detection performance for tiny objects compared to state-of-the-art general object detectors.

LRTransDet: A Real-Time SAR Ship-Detection Network with Lightweight ViT and Multi-Scale Feature Fusion

In recent years, significant strides have been made in the field of synthetic aperture radar (SAR) ship detection through the application of deep learning techniques. These advanced methods have substantially improved the accuracy of ship detection. Nonetheless, SAR images present distinct challenges, including complex backgrounds, small ship targets, and noise interference, thereby rendering the detectors particularly demanding. In this paper, we introduce LRTransDet, a real-time SAR ship detector. LRTransDet leverages a lightweight vision transformer (ViT) and a multi-scale feature fusion neck to address these challenges effectively. First, our model implements a lightweight backbone that combines convolutional neural networks (CNNs) and transformers, thus enabling it to simultaneously capture both local and global features from input SAR images. Moreover, we boost the model’s efficiency by incorporating the faster weighted feature fusion (Faster-WF2) module and coordinate attention (CA) mechanism within the feature fusion neck. These components optimize computational resources while maintaining the model’s performance. To overcome the challenge of detecting small ship targets in SAR images, we refine the original loss function and use the normalized Wasserstein distance (NWD) metric and the intersection over union (IoU) scheme. This combination improves the detector’s ability to efficiently detect small targets. To prove the performance of our proposed model, we conducted experiments on four challenging datasets (the SSDD, the SAR-Ship Dataset, the HRSID, and the LS-SSDD-v1.0). The results demonstrate that our model surpasses both general object detectors and state-of-the-art SAR ship detectors in terms of detection accuracy (97.8% on the SSDD and 93.9% on the HRSID) and speed (74.6 FPS on the SSDD and 75.8 FPS on the HRSID), all while demanding 3.07 M parameters. Additionally, we conducted a series of ablation experiments to illustrate the impact of the EfficientViT, the Faster-WF2 module, the CA mechanism, and the NWD metric on multi-scale feature fusion and detection performance.

Faster OreFSDet: A lightweight and effective few-shot object detector for ore images

For the ore particle size detection, obtaining a sizable amount of high-quality ore labeled data is time-consuming and expensive. General object detection methods often suffer from severe over-fitting with scarce labeled data. Despite their ability to eliminate over-fitting, existing few-shot object detectors encounter drawbacks such as slow detection speed and high memory requirements, making them difficult to implement in a real-world deployment scenario. To this end, we propose a lightweight and effective few-shot detector to achieve competitive performance with general object detection with only a few samples for ore images. First, the proposed support feature mining block characterizes the importance of location information in support features. Next, the relationship guidance block makes full use of support features to guide the generation of accurate candidate proposals. Finally, the dual-scale semantic aggregation module retrieves detailed features at different resolutions to contribute with the prediction process. Experimental results show that our method consistently exceeds the few-shot detectors with an excellent performance gap on all metrics. Moreover, our method achieves the smallest model size of 19 MB as well as being competitive at 50 FPS detection speed compared with general object detectors.

Long-Distance Person Detection Based on YOLOv7

In the research field of small object detection, most object detectors have been successfully used for pedestrian detection, face recognition, lost and found, and automatic driving, among other applications, and have achieved good results. However, when general object detectors encounter challenging low-resolution images from the TinyPerson dataset, they will produce undesirable detection results because of the dense occlusion between people and different body poses. In order to solve these problems, this paper proposes a tiny object detection method TOD-YOLOv7 based on YOLOv7.First, this paper presents a reconstruction of the YOLOv7 network by adding a tiny object detection layer to enhance its detection ability. Then, we use the recursive gated convolution module to realize the interaction with the higher-order space to accelerate the model initialization process and reduce the reasoning time. Secondly, this paper proposes the integration of a coordinate attention mechanism into the YOLOv7 feature extraction network to strengthen the pedestrian object information and weaken the background information.Additionally, we leverage data augmentation techniques to improve the representation learning of the algorithm. The results show that compared with the baseline model YOLOv7, the detection accuracy of this model on the TinyPerson dataset is improved from 7.1% to 9.5%, and the detection speed reaches 208 frames per second (FPS). The algorithm of this paper is shown to achieve better detection results for tiny object detection.

SRS-Net: Training object detectors from scratch for remote sensing images without pretraining

Most of the current object detection algorithms use pretrained models that are trained on ImageNet and then fine-tuned in the network, which can achieve good performance in terms of general object detectors. However, in the field of remote sensing image object detection, as pretrained models are significantly different from remote sensing data, it is meaningful to explore a train-from-scratch technique for remote sensing images. This paper proposes an object detection framework trained from scratch, SRS-Net, and describes the design of a densely connected backbone network to provide integrated hidden layer supervision for the convolution module. Then, two necessary improvement principles are proposed: studying the role of normalization in the network structure, and improving data augmentation methods for remote sensing images. To evaluate the proposed framework, we performed many ablation experiments on the DIOR, DOTA, and AS datasets. The results show that whether using the improved backbone network, the normalization method or training data enhancement strategy, the performance of the object detection network trained from scratch increased. These principles compensate for the lack of pretrained models. Furthermore, we found that SRS-Net could achieve similar to or slightly better performance than baseline methods, and surpassed most advanced general detectors.

OW-Adapter: Human-Assisted Open-World Object Detection with a Few Examples.

Open-world object detection (OWOD) is an emerging computer vision problem that involves not only the identification of predefined object classes, like what general object detectors do, but also detects new unknown objects simultaneously. Recently, several end-to-end deep learning models have been proposed to address the OWOD problem. However, these approaches face several challenges: a) significant changes in both network architecture and training procedure are required; b) they are trained from scratch, which can not leverage existing pre-trained general detectors; c) costly annotations for all unknown classes are needed. To overcome these challenges, we present a visual analytic framework called OW-Adapter. It acts as an adaptor to enable pre-trained general object detectors to handle the OWOD problem. Specifically, OW-Adapter is designed to identify, summarize, and annotate unknown examples with minimal human effort. Moreover, we introduce a lightweight classifier to learn newly annotated unknown classes and plug the classifier into pre-trained general detectors to detect unknown objects. We demonstrate the effectiveness of our framework through two case studies of different domains, including common object recognition and autonomous driving. The studies show that a simple yet powerful adaptor can extend the capability of pre-trained general detectors to detect unknown objects and improve the performance on known classes simultaneously.

Progressive refined redistribution pyramid network for defect detection in complex scenarios

Due to the rapid development of manufacturing capabilities and the general improvement in requirements for product quality, the role of quality inspection in the industrial production process is becoming increasingly important. Unlike the case for natural objects, detailed information is particularly crucial in defect classification and localization, resulting in poor performance of general object detectors on complex defect detection tasks. Therefore, this paper proposes a progressively refined redistribution pyramid network for visual defect detection in complex images, in which three novel components are designed. (1) The aligned dense feature pyramid network (AD-FPN) refines scale differences and performs efficient alignment, alleviating feature misalignment in FPN-based methods. (2) The phase-wise feature redistribution module (PFRM) enhances the interaction between features across layers, where global information is reassigned in a semantically adaptive manner. (3) The adaptive feature purification module (AFPM) helps the network distinguish defects from complex backgrounds, and its update is directly supervised by an auxiliary branch to accelerate convergence. These ideas are all implemented based on YOLOv5. Extensive experiments on the Tianchi fabric dataset, the publicly available surface defect dataset NEU-DET, and the PCB defect dataset show that our method outperforms other state-of-the-art defect detection methods on most evaluation metrics. In addition, experimental results on the remote sensing dataset RSOD and pothole image dataset also demonstrate the strong generalization ability of our method in other complex scenarios.

MSSDet: Multi-Scale Ship-Detection Framework in Optical Remote-Sensing Images and New Benchmark

Ships comprise the only and most important ocean transportation mode. Thus, ship detection is one of the most critical technologies in ship monitoring, which plays an essential role in maintaining marine safety. Optical remote-sensing images contain rich color and texture information, which is beneficial to ship detection. However, few optical remote-sensing datasets are open publicly due to the issue of sensitive data and copyrights, and only the HRSC2016 dataset is built for the ship-detection task. Moreover, almost all general object detectors suffer from the failure of multi-scale ship detection because of the diversity of spatial resolution and ship size. In this paper, we re-annotate the HRSC2016 dataset and supplement 610 optical remote-sensing images to build a new open source ship-detection benchmark dataset with rich multi-scale ship objects named the HRSC2016-MS dataset. In addition, we further explore the potential of a recursive mechanism in the field of object detection and propose a novel multi-scale ship-detection framework (MSSDet) in optical remote-sensing images. The success of detecting multi-scale objects depends on the hierarchical pyramid structure in the object-detection framework. However, the inherent semantic and spatial gaps among hierarchical pyramid levels seriously affect detection performance. To alleviate this problem, we propose a joint recursive feature pyramid (JRFP), which can generate semantically strong and spatially refined multi-scale features. Extensive experiments were conducted on the HRSC2016-MS, HRSC2016, and DIOR datasets. Detailed ablation studies directly demonstrated the effectiveness of the proposed JRFP architecture and also showed that the proposed method has excellent generalizability. Comparisons with state-of-the-art methods showed that the proposed method achieves competitive performance, i.e., 77.3%, 95.8%, and 73.3% mean average precision accuracy on the HRSC2016-MS, HRSC2016, and DIOR datasets, respectively.

Addressing scale imbalance for small object detection with dense detector

There are severe challenges on small object detection when using general object detector, especially scale imbalance on samples and features. Anchor-based detector performs poorly on small object detection because IoUs are too low to regress the objects. Key-point detector is hard to detect small objects for containing little semantic information. In contrast, a dense detector which directly learns an object via dense points is potential for small object detection. But they only catch fewer small objects because few of them can address scale imbalance brought by label assignment and feature extraction. We propose a dense detector named Libra EBox which includes Libra Ellipse Sampling (LES) and Residual Low-level Feature Enhancement (RiLFE). LES regulates positive regions of various objects by rescaling an ellipse box to catch samples of small object as much as possible. RiLFE is designed by several low-level feature maps to enhance small object’s feature representation. Experimental results show that our Libra EBox outperforms FoveaBox, FCOS and RetinaNet by 2.0% AP, 1.7% AP and 2.2% AP respectively for small objects on MS-COCO test-dev, and also outperforms most popular dense detectors on VisDrone-DET2018 and TinyPerson.

YOLSO: You Only Look Small Object

Small object detection is challenging and far from satisfactory. Most general object detectors suffer from two critical issues with small objects: (1) Feature extractor based on classification network cannot express the characteristics of small objects reasonably due to insufficient appearance information of targets and a large amount of background interference around them. (2) The detector requires a much higher location accuracy for small objects than for general objects. This paper proposes an effective and efficient small object detector YOLSO to address the above problems. For feature representation, we analyze the drawbacks in previous backbones and present a Half-Space Shortcut(HSSC) module to build a background-aware backbone. Furthermore, a coarse-to-fine Feature Pyramid Enhancement(FPE) module is introduced for layer-wise aggregation at a granular level to enhance the semantic discriminability. For loss function, we propose an exponential L1 loss to promote the convergence of regression, and a focal IOU loss to focus on prime samples with high classification confidence and high IOU. Both of them significantly improves the location accuracy of small objects. The proposed YOLSO sets state-of-the-art results on two typical small object datasets, MOCOD and VeDAI, at a speed of over 200 FPS. In the meantime, it also outperforms the baseline YOLOv3 by a wide margin on the common COCO dataset.

Object Detection in Real-World Hazy Scene

<p indent=0mm>Accurate object detection in the real-world hazy scene is very important to some potential visual task, such as video surveillance, smart city, autonomous driving and so on. This paper focuses on two research problems, which are to build a synthetic dataset of object detection in hazy scene and to analyze the effect of prior knowledge and joint learning of model on object detection in real-world hazy scene. Two frameworks are proposed which are the knowledge-guided object detection (KODNet) and the joint learning in dehazing and object detection (DONet). In KODNet, statistical prior knowledge will be used to guide the general object detection network to learn object features in the hazy scene during the training, makes the general object detector better adapt to the special object detection scenario. DONet can effectively solve the problem of structural detail missing and color distortion caused by image dehazing, thereby realizing the improvement of the objects detection accuracy in real-world scene. The experimental results on RTTS show that KODNet and DONet are effective to the object detection in the real-world hazy scene and they achieve the mAP of 70.5% and 66.6%.

Novel up-scale feature aggregation for object detection in aerial images

Object detection is a pivotal task for many unmanned aerial vehicle (UAV) applications. Compared to general scenes, the objects in aerial images are typically much smaller. For this reason, most general object detectors suffer from two critical challenges while dealing with aerial images: 1) The widely exploited Feature Pyramid Network works by integrating high-level features to lower levels progressively. However, this manner does not transfer equivalent information from each level of backbone network to the generated features, and the shared detection head faces an unbalanced sources of information flow, damaging the detection accuracy. 2) Up-sampling is commonly used to expand feature resolution for feature fusion or feature aggregation. However, existing up-sampling methods are ineffective to reconstruct high resolution feature maps. To address these two challenges, two works are proposed: 1) An up-scale feature aggregation framework that fully utilizes multi-scale complementary information, and 2) a novel up-sampling method that further improve detection accuracy. These two proposals are integrated into an end-to-end single-stage object detector namely HawkNet. Extensive experiments are conducted on VisDrone-DET2018, UAVDT and DIOR datasets. Compared to the RetinaNet baseline, our HawkNet achieves absolute gains of 6.0%, 1.2% and 5.9% in average precision (AP) on VisDrone-DET2018, UAVDT and DIOR datasets, respectively. For a 800 × 1333 input on the UAVDT dataset, HawkNet with ResNet-50 backbone surpasses existing methods for single-scale inference and achieves the best performance (37.4 AP), while operating at 10.6 frames per second on a single Nvidia GTX 1080Ti GPU.

Scalable Object Retrieval with Compact Image Representation from Generic Object Regions

In content-based visual object retrieval, image representation is one of the fundamental issues in improving retrieval performance. Existing works adopt either local SIFT-like features or holistic features, and may suffer sensitivity to noise or poor discrimination power. In this article, we propose a compact representation for scalable object retrieval from few generic object regions. The regions are identified with a general object detector and are described with a fusion of learning-based features and aggregated SIFT features. Further, we compress feature representation in large-scale image retrieval scenarios. We evaluate the performance of the proposed method on two public ground-truth datasets, with promising results. Experimental results on a million-scale image database demonstrate superior retrieval accuracy with efficiency gain in both computation and memory usage.