Small-size Object Detection Research Articles

CSSDet: small object detection via cross-scale feature enhancement on drone-view images

ABSTRACT Object detection on drone-view images is vital for applications like intelligent transportation, abnormal behavior detection, and urban surveillance. However, the diverse perspectives and altitudes from which Unmanned Aerial Vehicle (UAV) capture scenes pose challenges due to multi-scale differences and small object sizes. To address this, we propose CSSDet, emphasizing multi-scale object detection and small object enhancement. CSSDet integrates an additional detection head for small objects and employs a bidirectional weighted feature pyramid network for effective cross-scale feature fusion, enhancing global perceptual capability. Additionally, we introduce coordinate attention and Involution modules to mitigate information loss. Experiments are conducted on two publicly available datasets, VisDrone and UAVDT. Quantitative results from the experiments indicate that the proposed CSSDet achieves optimal performance across a wide range of evaluation metrics, with particular strengths demonstrated in multi-scale object detection and small-size object detection. Qualitative results showcase its remarkable effectiveness in various aspects of object detection, including small object detection, objects at different scales and categories, multi-perspective scenarios, and complex scenes. The substantial experimental findings confirm that CSSDet attains comprehensive object detection performance on drone-view images. The source code is available at https://github.com/Gui-Cheng/CSSDet.

AeroDetectNet: a lightweight, high-precision network for enhanced detection of small objects in aerial remote sensing imagery

Object detection in remote sensing imagery exhibits difficulties due to complex backgrounds, diverse object scales, and intricate spatial context relationships. Motivated by the problems mentioned above, this paper introduces AeroDetectNet, a novel lightweight and high-precision object detection network custom-designed for aerial remote sensing scenarios, building upon the YOLOv7-tiny algorithm. It enhances performance through four key improvements: the normalized Wasserstein distance for consistent object size sensitivity, the Involution module for reduced background noise, a self-designed RCS-biformer module for better spatial context interpretation, and a self-designed WF-CoT SPPCSP feature pyramid for improved feature map weighting and context capture. Ablation studies conducted on a hybrid dataset composed of three open-source remote sensing datasets (including NWPU VHR-10 remote sensing images, RSOD remote sensing images, and VisDrone UAV images) have demonstrated the effectiveness of four improvements specifically for small-size object detection. Visualizations through Grad-CAM further demonstrate AeroDetectNet’s capacity to extract and focus on key object features. Upon individual testing across three open-source datasets, AeroDetectNet has successfully demonstrated its ability to identify objects in images with a smaller pixel area. Through experimental comparisons with other related studies, the AeroDetectNet achieved a competitive mAP while maintaining fewer model parameters, highlighting its highly accurate and lightweight properties.

SPD-YOLOv8: an small-size object detection model of UAV imagery in complex scene

SPD-YOLOv8: an small-size object detection model of UAV imagery in complex scene

DeLiVoTr: Deep and light-weight voxel transformer for 3D object detection

The image-based backbone (feature extraction) networks downsample the feature maps not only to increase the receptive field but also to efficiently detect objects of various scales. The existing feature extraction networks in LiDAR-based 3D object detection tasks follow the feature map downsampling similar to image-based feature extraction networks to increase the receptive field. But, such downsampling of LiDAR feature maps in large-scale autonomous driving scenarios hinder the detection of small size objects, such as pedestrians. To solve this issue we design an architecture that not only maintains the same scale of the feature maps but also the receptive field in the feature extraction network to aid for efficient detection of small size objects. We resort to attention mechanism to build sufficient receptive field and we propose a Deep and Light-weight Voxel Transformer (DeLiVoTr) network with voxel intra- and inter-region transformer modules to extract voxel local and global features respectively. We introduce DeLiVoTr block that uses transformations with expand and reduce strategy to vary the width and depth of the network efficiently. This facilitates to learn wider and deeper voxel representations and enables to use not only smaller dimension for attention mechanism but also a light-weight feed-forward network, facilitating the reduction of parameters and operations. In addition to model scaling, we employ layer-level scaling of DeLiVoTr encoder layers for efficient parameter allocation in each encoder layer instead of fixed number of parameters as in existing approaches. Leveraging layer-level depth and width scaling we formulate three variants of DeLiVoTr network. We conduct extensive experiments and analysis on large-scale Waymo and KITTI datasets. Our network surpasses state-of-the-art methods for detection of small objects (pedestrians) with an inference speed of 20.5 FPS.

Effect of TDI Channels with Anomalous Noise on the Probability of Detection of Small-Size Objects by FPAs

Effect of TDI Channels with Anomalous Noise on the Probability of Detection of Small-Size Objects by FPAs

Multi-Scale Feature Selective Matching Network for Object Detection

Numerous deep learning-based object detection methods have achieved excellent performance. However, the performance on small-size object detection and positive and negative sample imbalance problems is not satisfactory. We propose a multi-scale feature selective matching network (MFSMNet) to improve the performance of small-size object detection and alleviate the positive and negative sample imbalance problems. First, we construct a multi-scale semantic enhancement module (MSEM) to compensate for the information loss of small-sized targets during down-sampling by obtaining richer semantic information from features at multiple scales. Then, we design the anchor selective matching (ASM) strategy to alleviate the training dominated by negative samples caused by the imbalance of positive and negative samples, which converts the offset values of the localization branch output in the detection head into localization scores and reduces negative samples by discarding low-quality anchors. Finally, a series of quantitative and qualitative experiments on the Microsoft COCO 2017 and PASCAL VOC 2007 + 2012 datasets show that our method is competitive compared to nine other representative methods. MFSMNet runs on a GeForce RTX 3090.

DC-YOLOv8: Small-Size Object Detection Algorithm Based on Camera Sensor

Traditional camera sensors rely on human eyes for observation. However, human eyes are prone to fatigue when observing objects of different sizes for a long time in complex scenes, and human cognition is limited, which often leads to judgment errors and greatly reduces efficiency. Object recognition technology is an important technology used to judge the object’s category on a camera sensor. In order to solve this problem, a small-size object detection algorithm for special scenarios was proposed in this paper. The advantage of this algorithm is that it not only has higher precision for small-size object detection but also can ensure that the detection accuracy for each size is not lower than that of the existing algorithm. There are three main innovations in this paper, as follows: (1) A new downsampling method which could better preserve the context feature information is proposed. (2) The feature fusion network is improved to effectively combine shallow information and deep information. (3) A new network structure is proposed to effectively improve the detection accuracy of the model. From the point of view of detection accuracy, it is better than YOLOX, YOLOR, YOLOv3, scaled YOLOv5, YOLOv7-Tiny, and YOLOv8. Three authoritative public datasets are used in these experiments: (a) In the Visdron dataset (small-size objects), the map, precision, and recall ratios of DC-YOLOv8 are 2.5%, 1.9%, and 2.1% higher than those of YOLOv8s, respectively. (b) On the Tinyperson dataset (minimal-size objects), the map, precision, and recall ratios of DC-YOLOv8 are 1%, 0.2%, and 1.2% higher than those of YOLOv8s, respectively. (c) On the PASCAL VOC2007 dataset (normal-size objects), the map, precision, and recall ratios of DC-YOLOv8 are 0.5%, 0.3%, and 0.4% higher than those of YOLOv8s, respectively.

A novel attention-based enhancement framework for face mask detection in complicated scenarios

In the context of COVID-19 pandemic prevention and control, it is of vital significance to realize accurate face mask detection via computer vision technique. In this paper, a novel attention improved Yolo (AI-Yolo) model is proposed, which can handle existing challenges in the complicated real-world scenarios with dense distribution, small-size object detection and interference of similar occlusions. In particular, a selective kernel (SK) module is set to achieve convolution domain soft attention mechanism with split, fusion and selection operations; a spatial pyramid pooling (SPP) module is applied to enhance the expression of local and global features, which enriches the receptive field information; and a feature fusion (FF) module is utilized to promote sufficient fusions of multi-scale features from each resolution branch, which adopts basic convolution operators without excessive computational complexity. In addition, the complete intersection over union (CIoU) loss function is adopted in the training stage for accurate positioning. Experiments are carried out on two challenging public face mask detection datasets, and the results demonstrate the superiority of the proposed AI-Yolo against other seven state-of-the-art object detection algorithms, which achieves the best results in terms of mean average precision and F1 score on both datasets. Furthermore, effectiveness of the meticulously designed modules in AI-Yolo is validated through extensive ablation studies. In a word, the proposed AI-Yolo is competent to accomplish face mask detection tasks under extremely complex situations with precise localization and accurate classification.

Efficient three-dimensional point cloud object detection based on improved Complex-YOLO.

Lidar-based 3D object detection and classification is a critical task for autonomous driving. However, inferencing from exceedingly sparse 3D data in real-time is a formidable challenge. Complex-YOLO solves the problem of point cloud disorder and sparsity by projecting it onto the bird's-eye view and realizes real-time 3D object detection based on LiDAR. However, Complex-YOLO has no object height detection, a shallow network depth, and poor small-size object detection accuracy. To address these issues, this paper has made the following improvements: (1) adds a multi-scale feature fusion network to improve the algorithm's capability to detect small-size objects; (2) uses a more advanced RepVGG as the backbone network to improve network depth and overall detection performance; and (3) adds an effective height detector to the network to improve the height detection. Through experiments, we found that our algorithm's accuracy achieved good performance on the KITTI dataset, while the detection speed and memory usage were very superior, 48FPS on RTX3070Ti and 20FPS on GTX1060, with a memory usage of 841Mib.

SFGNet detecting objects via spatial fine-grained feature and enhanced RPN with spatial context

Object detection, which is one of the most fundamental visual recognition tasks, has been a hotspot in computer vision. CNN (Convolutional Neural Networks) have been widely employed for building detector. Due to the success of RPN (Region Proposal Network), the two-stage detectors get both classification accuracy and precise regression bounding boxes. However, they still struggle in small-size object detection. In this paper, we present a deep network, namely Spatial Fine-Grained Network (SFGN). The SFGN that exploits Spatial Fine-Grained Features (SFGF) concatenates the higher resolution features, which is fine-grained with the low resolution features and high-level semantic by stacking spatial features for fine-grained features. An enhanced region proposal generator is proposed to get the objectless for small object to obtain a small set of proposal. The contextual information surrounding the region of interest is embedded using local spatial information for increasing the useful information and discriminating the background. For improving the detection performance, we use a simple yet surprisingly effective online hard example mining (OHEM) algorithm for training region proposal generator. It embeds an efficiently implemented soft non-maximum suppression (soft-NMS) for replacing with tradition NMS to obtain consistent improvements without increasing the computational complexity in inference. On PASCAL VOC 2007 and PASCAL VOC 2012 datasets, our SFGN improves baseline model from 81.2% mAP to 80.6% mAP. On MS COCO dataset, SFGN also performs better than baseline model. As intuition suggests, our detection results provide strong evidence that our SFGN improves detection accuracy, especially in small object test.

3D object detection combining semantic and geometric features from point clouds

Background: 3D object detection based on point clouds in road scenes has attracted much attention recently. The voxel-based methods voxelize the scene to regular grids, which can be processed with the advanced feature learning frameworks based on convolutional layers for semantic feature learning. The point-based methods can extract the geometric feature of the point due to the coordinate reservations. The combination of the two is effective for 3D object detection. However, the current methods use a voxel-based detection head with anchors for classification and localization. Although the preset anchors cover the entire scene, it is not suitable for detection tasks with larger scenes and multiple categories of objects, due to the limitation of the voxel size. Additionally, the misalignment between the predicted confidence and proposals in the Regions of the Interest (ROI) selection bring obstacles to 3D object detection. Methods: We investigate the combination of voxel-based methods and point-based methods for 3D object detection. Additionally, a voxel-to-point module that captures semantic and geometric features is proposed in the paper. The voxel-to-point module is conducive to the detection of small-size objects and avoids the presets of anchors in the inference stage. Moreover, a confidence adjustment module with the center-boundary-aware confidence attention is proposed to solve the misalignment between the predicted confidence and proposals in the regions of the interest selection. Results: The proposed method has achieved state-of-the-art results for 3D object detection in the Karlsruhe Institute of Technology and Toyota Technological Institute (KITTI) object detection dataset. Actually, as of September 19, 2021, our method ranked 1st in the 3D and Bird Eyes View (BEV) detection of cyclists tagged with difficulty level ‘easy’, and ranked 2nd in the 3D detection of cyclists tagged with ‘moderate’. Conclusions: We propose an end-to-end two-stage 3D object detector with voxel-to-point module and confidence adjustment module.

ACF Based Region Proposal Extraction for YOLOv3 Network Towards High-Performance Cyclist Detection in High Resolution Images.

You Only Look Once (YOLO) deep network can detect objects quickly with high precision and has been successfully applied in many detection problems. The main shortcoming of YOLO network is that YOLO network usually cannot achieve high precision when dealing with small-size object detection in high resolution images. To overcome this problem, we propose an effective region proposal extraction method for YOLO network to constitute an entire detection structure named ACF-PR-YOLO, and take the cyclist detection problem to show our methods. Instead of directly using the generated region proposals for classification or regression like most region proposal methods do, we generate large-size potential regions containing objects for the following deep network. The proposed ACF-PR-YOLO structure includes three main parts. Firstly, a region proposal extraction method based on aggregated channel feature (ACF) is proposed, called ACF based region proposal (ACF-PR) method. In ACF-PR, ACF is firstly utilized to fast extract candidates and then a bounding boxes merging and extending method is designed to merge the bounding boxes into correct region proposals for the following YOLO net. Secondly, we design suitable YOLO net for fine detection in the region proposals generated by ACF-PR. Lastly, we design a post-processing step, in which the results of YOLO net are mapped into the original image outputting the detection and localization results. Experiments performed on the Tsinghua-Daimler Cyclist Benchmark with high resolution images and complex scenes show that the proposed method outperforms the other tested representative detection methods in average precision, and that it outperforms YOLOv3 by average precision and outperforms SSD by average precision.

Cross-layer fusion feature network for material defect detection

Object detection involves solving two main problems: identifying the object and its location. This transforms the problem into a classification and localization problem. Currently, research shows that a convolutional neural network (CNN) can be used to solve the classification problem, and the object detection module incorporating a region proposal method can be used to locate the object. Although a CNN based on region proposals can achieve high recall, which improves detection accuracy, its performance cannot meet the actual requirements for small-size object detection and precise localization. This is mainly due to the feature maps extracted from the CNN and the quality of the region proposals. We present a cross-layer fusion feature network (CLFF-Net) for both high-quality region proposal generation and accurate object detection. The CLFF-Net is based on the cross-layer fusion feature that extracts hierarchical feature maps and then aggregates them into a unified space. The fused feature map appropriately combines deep layer semantic information, middle layer supplemental information, and shallow layer location information for an image to build the CLFF-Net, which is shared for both generating region proposals and detecting objects via end-to-end training. Extensive experiments using a casting dataset demonstrate its promising performance compared to state-of-the-art approaches.

Efficient Small Object Detection with an Improved Region Proposal Networks

Although the state-of-the-art object detection methods, which depend on region proposal algorithms to hypothesize object locations, have achieved high detection accuracy, it still struggles in small-size object detection. In this paper, we present a novel method with a multi-scale and multi-tasking region proposal method to effectively detect small object. In the proposed method, multi-scale features and high-level features are employed to locate object position and identify object category, respectively. The main contributions of the proposed approach are two-fold: (1) A simpler way is used to improve the accuracy performance of small object detection, instead of the complex image pyramids and the complex combination framework, and make the object detection task more flexible. (2) Based on multi-scale and multi-tasking approaches, object location information in low layers and object semantic information in deep layers are made fully advantage respectively. The experimental results on the PASCAL VOC dataset show that the proposed method achieves the state-of-the art object detection accuracy.

Robust Vehicle Detection in Aerial Images Based on Cascaded Convolutional Neural Networks.

Vehicle detection in aerial images is an important and challenging task. Traditionally, many target detection models based on sliding-window fashion were developed and achieved acceptable performance, but these models are time-consuming in the detection phase. Recently, with the great success of convolutional neural networks (CNNs) in computer vision, many state-of-the-art detectors have been designed based on deep CNNs. However, these CNN-based detectors are inefficient when applied in aerial image data due to the fact that the existing CNN-based models struggle with small-size object detection and precise localization. To improve the detection accuracy without decreasing speed, we propose a CNN-based detection model combining two independent convolutional neural networks, where the first network is applied to generate a set of vehicle-like regions from multi-feature maps of different hierarchies and scales. Because the multi-feature maps combine the advantage of the deep and shallow convolutional layer, the first network performs well on locating the small targets in aerial image data. Then, the generated candidate regions are fed into the second network for feature extraction and decision making. Comprehensive experiments are conducted on the Vehicle Detection in Aerial Imagery (VEDAI) dataset and Munich vehicle dataset. The proposed cascaded detection model yields high performance, not only in detection accuracy but also in detection speed.

Exemplar-Based Recognition of Human–Object Interactions

Human action can be recognized from a single still image by modeling human–object interactions (HOIs), which infers the mutual spatial structure information between human and the manipulated object as well as their appearance. Existing approaches rely heavily on accurate detection of human and object and estimation of human pose; they are thus sensitive to large variations of human poses, occlusion, and unsatisfactory detection of small size objects. To overcome this limitation, a novel exemplar-based approach is proposed in this paper. Our approach learns a set of spatial pose–object interaction exemplars, which are probabilistic density functions describing spatially how a person is interacting with a manipulated object for different activities. Specifically, a new framework consisting of an exemplar-based HOI descriptor and an associated matching model is formulated for robust human action recognition in still images. In addition, the framework is extended to perform HOI recognition in videos, where the proposed exemplar representation is used for implicit frame selection to negate irrelevant or noisy frames by temporal structured HOI modeling. Extensive experiments are carried out on two image action datasets and two video action datasets. The results demonstrate the effectiveness of our proposed methods and show that our approach is able to achieve state-of-the-art performance, compared with several recently proposed competitors.