Multi-scale Object Detection Research Articles

ConcentrateNet: Multi-Scale Object Detection Model for Advanced Driving Assistance System Using Real-Time Distant Region Locating Technique.

This paper proposes a deep learning based object detection method to locate a distant region in an image in real-time. It concentrates on distant objects from a vehicular front camcorder perspective, trying to solve one of the common problems in Advanced Driver Assistance Systems (ADAS) applications, which is, to detect the smaller and faraway objects with the same confidence as those with the bigger and closer objects. This paper presents an efficient multi-scale object detection network, termed as ConcentrateNet to detect a vanishing point and concentrate on the near-distant region. Initially, the object detection model inferencing will produce a larger scale of receptive field detection results and predict a potentially vanishing point location, that is, the farthest location in the frame. Then, the image is cropped near the vanishing point location and processed with the object detection model for second inferencing to obtain distant object detection results. Finally, the two-inferencing results are merged with a specific Non-Maximum Suppression (NMS) method. The proposed network architecture can be employed in most of the object detection models as the proposed model is implemented in some of the state-of-the-art object detection models to check feasibility. Compared with original models using higher resolution input size, ConcentrateNet architecture models use lower resolution input size, with less model complexity, achieving significant precision and recall improvements. Moreover, the proposed ConcentrateNet architecture model is successfully ported onto a low-powered embedded system, NVIDIA Jetson AGX Xavier, suiting the real-time autonomous machines.

Review on online operation insulator identification and fault diagnosis based on UAV patrol images and deep learning algorithms

Artificial intelligence has great potential for use in smart grids. Power system image recognition based on artificial intelligence is an important research direction. The insulator is essential equipment for the power grid and is related to operational safety. Online operating insulator location identification and fault diagnosis technologies based on unmanned aerial vehicle (UAV) patrol the images, and deep learning algorithms have been continuously suggested and developed. These technologies have achieved good results in practical application. By compiling the recent literature on insulator detection technology, three common application scenarios and research difficulties are uncovered: The need for increased detection accuracy and real-time speed; faulty image recognition of complex backgrounds and target occlusion; and multiscale object and small object detection improvements. At the same time, the improved algorithms in the literature are comprehensively summarized, and the performance evaluation indices of various algorithms are compared.

Single-Shot Object Detection via Feature Enhancement and Channel Attention.

Features play a critical role in computer vision tasks. Deep learning methods have resulted in significant breakthroughs in the field of object detection, but it is still an extremely challenging obstacle when an object is very small. In this work, we propose a feature-enhancement- and channel-attention-guided single-shot detector called the FCSSD with four modules to improve object detection performance. Specifically, inspired by the structure of atrous convolution, we built an efficient feature-extraction module (EFM) in order to explore contextual information along the spatial dimension, and then pyramidal aggregation module (PAM) is presented to explore the semantic features of deep layers, thus reducing the semantic gap between multi-scale features. Furthermore, we construct an effective feature pyramid refinement fusion (FPRF) to refine the multi-scale features and create benefits for richer object knowledge. Finally, an attention-guided module (AGM) is developed to balance the channel weights and optimize the final integrated features on each level; this alleviates the aliasing effects of the FPN with negligible computational costs. The FCSSD exploits richer information of shallow layers and higher layers by using our designed modules, thus accomplishing excellent detection performance for multi-scale object detection and reaching a better tradeoff between accuracy and inference time. Experiments on PASCAL VOC and MS COCO datasets were conducted to evaluate the performance, showing that our FCSSD achieves competitive detection performance compared with existing mainstream object detection methods.

A Novel Attention-Based Lightweight Network for Multiscale Object Detection in Underwater Images

Underwater images have low quality, and underwater targets have different sizes. The mainstream target detection networks cannot achieve good results in detecting objects from underwater images. In this study, a lightweight underwater multiscale target detection model with an attention mechanism is designed to solve the above problems. In this model, MobileNetv3 is used as the backbone network for preliminary feature extraction. The lightweight feature extraction module (LFEM) pays attention to the feature map at the channel and space levels. The features with large weights are promoted, while the features with small weights are suppressed. Meanwhile, cross-group information exchange enriches the semantic information and location information of the objects. The context aggregation module (CIAM) pools the extracted feature maps to obtain feature pyramids, and it uses the upsampling-feature refinement-cascade addition (URC) method to effectively fuse global context information and enhance the feature representation. The scale normalization for feature pyramids (SNFP) performs adaptive multiscale perception and multianchor detection on feature maps to cover objects of different sizes and realize multiscale object detection in underwater images. The proposed network can realize lightweight feature extraction, effectively handle the global relationship between the underwater scene and the object while expanding the receptive field, traverse the objects of different scales, and achieve adaptive multianchor detection of multiscale objects in underwater images. The experimental results indicate that our method achieves an average accuracy of 81.94% and a detection speed of 44.3 FPS on a composite dataset. Also, our method is better than the mainstream object detection networks in terms of detection accuracy, lightweight design, and real-time performance.

Multiscale Object Detection Method for Track Construction Safety Based on Improved YOLOv5

In track construction, it is an important and necessary guarantee for production safety to check the number of workers and tools before and after the track maintenance. In view of time-consuming, laborious, and low detection efficiency of traditional manual inspection way, we propose an improved YOLOv5 multiscale object detection algorithm for track construction safety in this paper. We improve, from Generalized Intersection over Union (GIoU) to Distance Intersection over Union (DIoU), the loss function for Bounding Box (BBox) regression to speed up the convergence of the model. And we also improve, from the Non-Maximum Suppression (NMS) to DIoU-NMS, the post-processing method to enhance the model’s detection ability for occluded objects and small objects. The experiment results on the Track Maintenance dataset (our self-prepared dataset) and MS COCO dataset show that the mAP value of our improved YOLOv5 algorithm is 94.8% and 38.7%, respectively. Compared with the original YOLOv5 algorithm, the mAP values on above datasets are increased by 5.1% and 5.4%, respectively. The validation experimental results on MS COCO dataset and Track Maintenance dataset indicate that the detection ability of our improved YOLOv5 algorithm for occluded objects and small objects is enhanced. The proposed algorithm can provide technical support for the real-time accurate detection of track construction workers and tools.

Learning Discriminated Features Based on Feature Pyramid Networks and Attention for Multi-scale Object Detection

Learning Discriminated Features Based on Feature Pyramid Networks and Attention for Multi-scale Object Detection

Multi-Scale Object Detection with the Pixel Attention Mechanism in a Complex Background

The object detection task is usually affected by complex backgrounds. In this paper, a new image object detection method is proposed, which can perform multi-feature selection on multi-scale feature maps. By this method, a bidirectional multi-scale feature fusion network was designed to fuse semantic features and shallow features to improve the detection effects of small objects in complex backgrounds. When the shallow features are transferred to the top layer, a bottom-up path is added to reduce the number of network layers experienced by the feature fusion network, reducing the loss of shallow features. In addition, a multi-feature selection module based on the attention mechanism is used to minimize the interference of useless information in subsequent classification and regression, allowing the network to adaptively focus on appropriate information for classification or regression to improve detection accuracy. Because the traditional five-parameter regression method has severe boundary problems when predicting objects with large aspect ratios, the proposed network treats angle prediction as a classification task. The experimental results on the DOTA dataset, the self-made DOTA-GF dataset and the HRSC 2016 dataset show that, compared with several popular object detection algorithms, the proposed method has certain advantages in detection accuracy.

Attention-Based Multi-Level Feature Fusion for Object Detection in Remote Sensing Images

We study the problem of object detection in remote sensing images. As a simple but effective feature extractor, Feature Pyramid Network (FPN) has been widely used in several generic vision tasks. However, it still faces some challenges when used for remote sensing object detection, as the objects in remote sensing images usually exhibit variable shapes, orientations, and sizes. To this end, we propose a dedicated object detector based on the FPN architecture to achieve accurate object detection in remote sensing images. Specifically, considering the variable shapes and orientations of remote sensing objects, we first replace the original lateral connections of FPN with Deformable Convolution Lateral Connection Modules (DCLCMs), each of which includes a 3×3 deformable convolution to generate feature maps with deformable receptive fields. Additionally, we further introduce several Attention-based Multi-Level Feature Fusion Modules (A-MLFFMs) to integrate the multi-level outputs of FPN adaptively, further enabling multi-scale object detection. Extensive experimental results on the DIOR dataset demonstrated the state-of-the-art performance achieved by the proposed method, with the highest mean Average Precision (mAP) of 73.6%.

MS-IAF: Multi-Scale Information Augmentation Framework for Aircraft Detection

Aircrafts have been an important object of study in the field of multi-scale image object detection due to their important strategic role. However, the multi-scale detection of aircrafts and their key parts from remote sensing images can be a challenge, as images often present complex backgrounds and obscured conditions. Most of today’s multi-scale datasets consist of independent objects and lack mixed annotations of aircrafts and their key parts. In this paper, we contribute a multi-scale aircraft dataset (AP-DATA) consisting of 7000 aircraft images that were taken in complex environments and obscured conditions. Our dataset includes mixed annotations of aircrafts and their key parts. We also present a multi-scale information augmentation framework (MS-IAF) to recognize multi-scale aircrafts and their key parts accurately. First, we propose a new deep convolutional module ResNeSt-D as the backbone, which stacks scattered attention in a multi-path manner and makes the receptive field more suitable for the object. Then, based on the combination of Faster R-CNN with ResNeSt-D, we propose a multi-scale feature fusion module called BFPCAR. BFPCAR overcomes the attention imbalance problem of the non-adjacent layers of the FPN module by reducing the loss of information between different layers and including more semantic features during information fusion. Based on AP-DATA, a dataset with three types of features, the average precision (AP) of MS-IAF reached 0.884, i.e., 2.67% higher than that of the original Faster R-CNN. The APs of these two modules were improved by 2.32% and 1.39%, respectively. The robustness of our proposed model was validated using the open sourced RSOD remote sensing image dataset, and the best accuracy was achieved.

Multiscale object detection based on channel and data enhancement at construction sites

Multiscale object detection based on channel and data enhancement at construction sites

Research on Intelligent Video Detection of Small Targets Based on Deep Learning Intelligent Algorithm.

Compared with the traditional object detection algorithm, the object detection algorithm based on deep learning has stronger robustness to complex scenarios, which is the hot direction of current research. According to the process characteristics of the object detection algorithm based on deep learning, it is divided into two-stage object detection algorithm and single-stage object detection algorithm, focusing on the problems solved by some classical algorithms and their advantages and disadvantages. In view of the problem of object detection, especially small object detection, the commonly used data sets and performance evaluation indicators are summarized; the characteristics, advantages, and detection difficulties of various common data sets are compared; the challenges faced by commonly used object detection methods and small object detection are systematically summarized; the latest work of small object detection methods based on deep learning is sorted out; and the small object detection methods based on multiscale and small object detection methods based on super-resolution are introduced. At the same time, the lightweight strategy for target detection methods and the performance of some lightweight models are introduced; the characteristics, advantages, and limitations of various methods are summarized; and the future development direction of small object detection methods based on deep learning is prospected.

Content‐augmented feature pyramid network with light linear spatial transformers for object detection

As one of the prevalent components, feature pyramid network (FPN) is widely used in current object detection models for improving multi-scale object detection performance. However, its feature fusion mode is still in a misaligned and local manner, thus limiting the representation power. To address the inherited defects of FPN, a novel architecture termed content-augmented feature pyramid network (CA-FPN) is proposed in this paper. Firstly, a global content extraction module (GCEM) is proposed to extract multi-scale context information. Secondly, lightweight linear spatial Transformer connections are added in the top-down pathway to augment each feature map with multi-scale features, where a linearized approximate self-attention function is designed for reducing model complexity. By means of the self-attention mechanism in Transformer, it is no longer needed to align feature maps during feature fusion, thus solving the misaligned defect. By setting the query scope to the entire feature map, the local defect can also be solved. Extensive experiments on COCO and PASCAL VOC datasets demonstrated that the CA-FPN outperforms other FPN-based detectors without bells and whistles and is robust in different settings.

Adaptive weighted multiscale feature fusion for small drone object detection

Drone object detection in low-altitude airspace plays an essential role in many practical applications, such as security and airspace monitoring. Despite the remarkable progress made by many methods, drone object detection still remains challenging due to the complex background and huge differences in scales of drones. To address the above issues, an improved fully convolutional one-stage object detection (FCOS) model based on adaptive weighted feature fusion (AWFF) module is proposed for multiscale drone object detection in complex background. By learning the spatial relevance of feature maps at each scale and improving the scale invariance of features based on the channel attention mechanism, AWFF module could adaptively fuse the features of adjacent scale. In addition, a receptive field enhancement module is designed to reduce the information loss in the feature fusion process. Extensive experiments are conducted to evaluate the effectiveness of the proposed module and method on the constructed low-altitude drone dataset, which concludes that the mean average precision of the AWFF-FCOS is increased by 2.1% compared with the baseline method. And extensive ablation experiments further demonstrate that the proposed AWFF module and REF module could be integrated into the state-of-the-art method to improve the performance of drone object detection.

Semantics reused context feature pyramid network for object detection in remote sensing images

Object detection plays an important role in the field of remote sensing (RS) images analysis. The advancement of object detection task for RS images is extremely challenging due to object scale variation and complex background. Almost all detection frameworks use the neck network to fuse the feature maps extracted by the backbone to obtain better features, among which feature pyramid network (FPN) is the most widely used. Although traditional FPN has shown great potential in multi-scale object detection based on deep learning, it has unsatisfactory detection accuracy for small objects and confusing objects in RS images because of the lack of rich semantic and contextual information. We propose an architecture, called semantics reused context FPN that is portable to any FPN-based detectors to boost the detection performance in RS images without parameters increasing significantly. It includes two blocks, namely, context feature enhanced block, which uses dense connection and a learnable branch structure to extract rich context features with multiple receptive fields, and semantic feature reused block, which enhances semantic information of shallow feature maps by reusing later-layer features. Comprehensive evaluations on three benchmark datasets of geospatial object detection demonstrate that our method is superior to the existing methods.

Construct Effective Geometry Aware Feature Pyramid Network for Multi-Scale Object Detection

Feature Pyramid Network (FPN) has been widely adopted to exploit multi-scale features for scale variation in object detection. However, intrinsic defects in most of the current methods with FPN make it difficult to adapt to the feature of different geometric objects. To address this issue, we introduce geometric prior into FPN to obtain more discriminative features. In this paper, we propose Geometry-aware Feature Pyramid Network (GaFPN), which mainly consists of the novel Geometry-aware Mapping Module and Geometry-aware Predictor Head.The Geometry-aware Mapping Module is proposed to make full use of all pyramid features to obtain better proposal features by the weight-generation subnetwork. The weights generation subnetwork generates fusion weight for each layer proposal features by using the geometric information of the proposal. The Geometry-aware Predictor Head introduces geometric prior into predictor head by the embedding generation network to strengthen feature representation for classification and regression. Our GaFPN can be easily extended to other two-stage object detectors with feature pyramid and applied to instance segmentation task. The proposed GaFPN significantly improves detection performance compared to baseline detectors with ResNet-50-FPN: +1.9, +2.0, +1.7, +1.3, +0.8 points Average Precision (AP) on Faster-RCNN, Cascade R-CNN, Dynamic R-CNN, SABL, and AugFPN respectively on MS COCO dataset.

Visual Inspection Method for Metal Rolls Based on Multi-Scale Spatial Location Feature

Metal rolls in a non-ferrous-metal manufacturing workshop manifest the characteristics of symmetry, multiple scales and mutual covering, which poses great challenges for metal roll detection. To solve this problem, firstly, an efficient attention mechanism algorithm named ECLAM (efficient capture location attendant model) is proposed for capturing spatial position features efficiently, to obtain complete location information for metal rolls in a complex environment. ECLAM can improve the ability to extract the spatial features of backbone networks and reduce the influence of the non-critical background. In addition, in order to give feature maps a larger receptive field and improve the weight of location information in multi-scale feature maps, a nonlinear feature fusion module named LFFM (location feature fusion module) is used to fuse two adjacent feature images. Finally, a multi-scale object detection network named L-MSNet (location-based multi-scale object detection network) based on the combination of ECLAM and LFFM is proposed and used to accurately detect multi-scale metal rolls. In the experiments, multi-scale metal roll images are collected from an actual non-ferrous-metal manufacturing workshop. On this basis, a pixel-level image dataset is constructed. Comparative experiments show that, compared with other object detection methods, L-MSNet can detect multi-scale metal rolls more accurately. The average accuracy is improved by 2% to 5%, and the average accuracy of small and medium-sized objects is also significantly improved by 3% to 6%.

DPSSD: Dual-Path Single-Shot Detector.

Object detection is one of the most important and challenging branches of computer vision. It has been widely used in people’s lives, such as for surveillance security and autonomous driving. We propose a novel dual-path multi-scale object detection paradigm in order to extract more abundant feature information for the object detection task and optimize the multi-scale object detection problem, and based on this, we design a single-stage general object detection algorithm called Dual-Path Single-Shot Detector (DPSSD). The dual path ensures that shallow features, i.e., residual path and concatenation path, can be more easily utilized to improve detection accuracy. Our improved dual-path network is more adaptable to multi-scale object detection tasks, and we combine it with the feature fusion module to generate a multi-scale feature learning paradigm called the “Dual-Path Feature Pyramid”. We trained the models on PASCAL VOC datasets and COCO datasets with 320 pixels and 512 pixels input, respectively, and performed inference experiments to validate the structures in the neural network. The experimental results show that our algorithm has an advantage over anchor-based single-stage object detection algorithms and achieves an advanced level in average accuracy. Researchers can replicate the reported results of this paper.

SLMS-SSD: Improving the balance of semantic and spatial information in object detection

With the development of deep learning technology, the research on convolutional neural network-based object detection is becoming more and more mature. However, most methods are unsatisfactory in dealing with the issue of semantic and spatial information imbalance. In this article, we extend the single-shot multibox detector SSD and propose a self-learning multi-scale object detection network by balancing the semantic information and spatial information, named SLMS-SSD. We first construct a shallow feature enhancement module to enhance the representation of small objects by extracting richer context information. Second, in terms of feature connectivity, we design a multi-scale feature selection module for intermediate layer features with a combination of top-down and direct up-samplings. Finally, in terms of feature strength, we design a self-learning feature fusion module for measuring the feature importance. We validate our model on the PASCAL VOC and MS COCO datasets, and the results demonstrate that it can effectively improve the accuracy of object detection, especially the accuracy of small object detection.

NLFFTNet: A non-local feature fusion transformer network for multi-scale object detection

Recent object detection studies attempt to implement multi-scale feature fusion through complicated hierarchical structures. However, the existing feature fusion methods only focus on the interaction between the same local positions and fail to describe the long-distance dependencies of features. In this study, a novel non-local feature fused transformer convolutional network is proposed for object detection. This model can focus on global semantic information by calculating the attention of different positions to capture the long-distance dependency. Meanwhile, a dynamic data augment method called configurable mix-splicing is introduced to solve the problem of data imbalance between different classes. The experimental results indicate that attributed to the feature fusion and data augment method, our model achieves better performance than state-of-the-art models on two authoritative public datasets.

A Scale-Aware Pyramid Network for Multi-Scale Object Detection in SAR Images

Multi-scale object detection within Synthetic Aperture Radar (SAR) images has become a research hotspot in SAR image interpretation. Over the past few years, CNN-based detectors have advanced sharply in SAR object detection. However, the state-of-the-art detection methods are continuously limited in Feature Pyramid Network (FPN) designing and detection anchor setting aspects due to feature misalignment and targets’ appearance variation (i.e., scale change, aspect ratio change). To address the mentioned limitations, a scale-aware feature pyramid network (SARFNet) is proposed in this study, which comprises a scale-adaptive feature extraction module and a learnable anchor assignment strategy. To be specific, an enhanced feature pyramid sub-network is developed by introducing a feature alignment module to estimate the pixel offset and contextually align the high-level features. Moreover, a scale-equalizing pyramid convolution is built through 3-D convolution within the feature pyramid to improve inter-scale correlation at different feature levels. Furthermore, a self-learning anchor assignment is set to update hand-crafted anchor assignments to learnable anchor/feature configuration. By using the dynamic anchors, the detector of this study is capable of flexibly matching the target with different appearance changes. According to extensive experiments on public SAR image data sets (SSDD and HRSID), our algorithm is demonstrated to outperform existing boat detectors.