Object Detection Problem Research Articles

Small object detection in remote sensing images based on attention mechanism and multi-scale feature fusion

ABSTRACT Due to the influence of dense distribution of detection objects and complex background, there are many small objects, which are difficult to detect in remote sensing images. In order to solve the difficult problem of small object detection in remote sensing images, we propose an object detection algorithm named CotYOLO-v3 in this paper. First, we redesign the residual blocks in the backbone Darknet-53, and we replace it with Contextual Transformer (Cot) blocks with contextual information in the backbone Darknet-53 to extract contextual information for small objects and enhance visual representation; Second, we introduce the shallow information with attention mechanism before the feature fusion of YOLO-v3 to reduce the influence of background interference factors and improve the expression ability of the network. Then, we optimize the feature fusion process, we replace the up-sampling method with sub-pixel convolution, and we replace the first convolution layer of the prediction branch with a residual block. Finally, we use K-Medians clustering algorithm to regenerate the anchors suitable for the remote sensing image datasets. In this paper, we set up a comparative experiment of CotYOLO-v3 and commonly used object detection algorithms to detect small objects in DIOR datasets. The experimental results show that, compared with other commonly used object detection algorithms, CotYOLO-v3 object detection algorithm has obvious advantages in detecting small objects in remote sensing images. Compared with the original object detection algorithm YOLO-v3, the mean Average Precision (mAP) of CotYOLO-v3 improved by 5.07%.

ODEM-GAN: An Object Deformation Enhancement Model Based on Generative Adversarial Networks

Object detection has attracted great attention in recent years. Many experts and scholars have proposed efficient solutions to address object detection problems and achieve perfect performance. For example, coordinate-based anchor-free (CBAF) module was proposed recently to predict the category and the adjustments to the box of the object by its feature part and its contextual part features, which are based on feature maps divided by spatial coordinates. However, these methods do not work very well for some particular situations (e.g., small object detection, scale variation, deformations, etc.), and the accuracy of object detection still needs to be improved. In this paper, to address these problems, we proposed ODEM-GAN based on CBAF, which utilizes generative adversarial networks to implement the detection of a deformed object. Specifically, ODEM-GAN first generates the object deformation features and then uses these features to enhance the learning ability of CBFA for improving the robustness of the detection. We also conducted extensive experiments to validate the effectiveness of ODEM-GAN in the simulation of a parachute opening process. The experimental results demonstrate that, with the assistance of ODEM-GAN, the AP score of CBAF for parachute detection is 88.4%, thereby the accuracy of detecting the deformed object by CBAF significantly increases.

Intelligent Fault Diagnosis Tool For Power Line Systems

The paper focuses on the intelligent fault diagnosis system for automatic analysis of a huge amount of images with 4K resolution collected during flight inspection of the overhead power lines with the use of aerial platforms. The developed system can be used to detect and locate anomalies of power line structures e.g. anomalies of towers, conductors, dampers, insulators, etc. The purpose of the paper is to present the general architecture of the system and the results obtained during inspection of thousands of kilometer flights of power lines. The proposed system is based on the predefined deep neural network called Faster R-CNN which is dedicated for solving real-time object detection problems. The faster R-CNN-based neural model created on COCO dataset was additionally retrained by the authors applying images gathered as a result of flight inspections in order to obtain the high performance of the whole system. The comprehensive verification tests were carried out to prove the merits of the proposed solution.

VARIANTS OF NEURAL NETWORKS: A REVIEW

Machine learning (ML) techniques are part of artificial intelligence. ML involves imitating human behavior in solving different problems, such as object detection, text handwriting recognition, and image classification. Several techniques can be used in machine learning, such as Neural Networks (NN). The expansion in information technology enables researchers to collect large amounts of various data types. The challenging issue is to uncover neural network parameters suitable for object detection problems. Therefore, this paper presents a literature review of the latest proposed and developed components in neural network techniques to cope with different sizes and data types. A brief discussion is also introduced to demonstrate the different types of neural network parameters, such as activation functions, loss functions, and regularization methods. Moreover, this paper also uncovers parameter optimization methods and hyperparameters of the model, such as weight, the learning rate, and the number of iterations. From the literature, it is notable that choosing the activation function, loss function, number of neural network layers, and data size is the major factor affecting NN performance. Additionally, utilizing deep learning NN resulted in a significant improvement in model performance for a variety of issues, which became the researcher's attention.

Object Detection Algorithm Based on Context Information and Self-Attention Mechanism

Pursuing an object detector with good detection accuracy while ensuring detection speed has always been a challenging problem in object detection. This paper proposes a multi-scale context information fusion model combined with a self-attention block (CSA-Net). First, an improved backbone network ResNet-SA is designed with self-attention to reduce the interference of the image background area and focus on the object region. Second, this work introduces a receptive field feature enhancement module (RFFE) to combine local and global features while increasing the receptive field. Then this work adopts a spatial feature fusion pyramid with a symmetrical structure, which fuses and transfers semantic information and feature information. Finally, a sibling detection head using an anchor-free detection mechanism is applied to increase the accuracy and speed of detection at the end of the model. A large number of experiments support the above analysis and conclusions. Our model achieves an average accuracy of 46.8% on the COCO 2017 test set.

MobileYOLO: Real-Time Object Detection Algorithm in Autonomous Driving Scenarios.

Object detection is one of the key tasks in an automatic driving system. Aiming to solve the problem of object detection, which cannot meet the detection speed and detection accuracy at the same time, a real-time object detection algorithm (MobileYOLO) is proposed based on YOLOv4. Firstly, the feature extraction network is replaced by introducing the MobileNetv2 network to reduce the number of model parameters; then, part of the standard convolution is replaced by depthwise separable convolution in PAnet and the head network to further reduce the number of model parameters. Finally, by introducing an improved lightweight channel attention modul—Efficient Channel Attention (ECA)—to improve the feature expression ability during feature fusion. The Single-Stage Headless (SSH) context module is introduced to the small object detection branch to increase the receptive field. The experimental results show that the improved algorithm has an accuracy rate of 90.7% on the KITTI data set. Compared with YOLOv4, the parameters of the proposed MobileYOLO model are reduced by 52.11 M, the model size is reduced to one-fifth, and the detection speed is increased by 70%.

Joint optic disk and cup segmentation for glaucoma screening using a region-based deep learning network.

To develop and validate an end-to-end region-based deep convolutional neural network (R-DCNN) to jointly segment the optic disc (OD) and optic cup (OC) in retinal fundus images for precise cup-to-disc ratio (CDR) measurement and glaucoma screening. In total, 2440 retinal fundus images were retrospectively obtained from 2033 participants. An R-DCNN was presented for joint OD and OC segmentation, where the OD and OC segmentation problems were formulated into object detection problems. We compared R-DCNN's segmentation performance on our in-house dataset with that of four ophthalmologists while performing quantitative, qualitative and generalization analyses on the publicly available both DRISHIT-GS and RIM-ONE v3 datasets. The Dice similarity coefficient (DC), Jaccard coefficient (JC), overlapping error (E), sensitivity (SE), specificity (SP) and area under the curve (AUC) were measured. On our in-house dataset, the proposed model achieved a 98.51% DC and a 97.07% JC for OD segmentation, and a 97.63% DC and a 95.39% JC for OC segmentation, achieving a performance level comparable to that of the ophthalmologists. On the DRISHTI-GS dataset, our approach achieved 97.23% and 94.17% results in DC and JC results for OD segmentation, respectively, while it achieved a 94.56% DC and an 89.92% JC for OC segmentation. Additionally, on the RIM-ONE v3 dataset, our model generated DC and JC values of 96.89% and 91.32% on the OD segmentation task, respectively, whereas the DC and JC values acquired for OC segmentation were 88.94% and 78.21%, respectively. The proposed approach achieved very encouraging performance on the OD and OC segmentation tasks, as well as in glaucoma screening. It has the potential to serve as a useful tool for computer-assisted glaucoma screening.

Balanced-YOLOv3: Addressing the Imbalance Problem of Object Detection in PCB Assembly Scene

The object detection algorithm of the PCB (Printed Circuit Board) assembly scene based on CNN (Convolutional Neural Network) can significantly improve the production capacity of intelligent manufacturing of electronic products. However, the object class imbalance in the PCB assembly scene, the multi-scale feature imbalance, and the positive/negative sample imbalance in the CNN have become critical problems restricting object detection performance. Based on YOLOv3, this paper proposes a class-balanced Train/Val (Training set/Validation set) split method for object class imbalance, an additional feature fusion strategy for multi-scale feature imbalance, and an efficient anchor concept for positive/negative sample imbalance. These three contributions are Balanced-YOLOv3. After experimental verification, compared with other YOLOv3 series algorithms, the mAP@.5 (Mean Average Precision at Intersection over Union threshold 0.5) and mAP@.5:.95 (average mAP over different Intersection over Union thresholds, from 0.5 to 0.95, step 0.05) of Balanced-YOLOv3 have achieved the best results and ranked third in the metrics of parameter and inference time. Compared with other current anchor-based object detection algorithms, Balanced-YOLOv3 has excellent detection performance and low computational complexity, which effectively solves the problem of imbalanced object detection in PCB assembly scenarios.

Underwater object detection algorithm based on channel attention and feature fusion

Due to the color deviation, low contrast and fuzzy object in underwater optical images, there are some problems in underwater object detection, such as missed detection and false detection. In order to solve the above-mentioned problems, an underwater object detection algorithm is proposed based on the channel attention and feature fusion for underwater optical images. The excitation residual module is designed based on the channel attention, and the forward propagation feature information is adaptively allocated weights to highlight the salience of different channel feature maps, which improves the network ability to extract high-frequency information from the underwater images. The multi-scale feature fusion module is designed to add a large scale feature map for object detection, which improves the detection performance of the network for small size objects by using its corresponding small size receptive field, and further improves the detection accuracy of the network for different size objects in the underwater environment. To improve the generalization performance of the network to the underwater environment, the data augmentation method based on the stitching and fusion is designed to simulate the overlap, occlusion and blurring of underwater objects, which improves the adaptability of the network to the underwater environment. Through experiments on the public dataset URPC, the algorithm in this paper improves the mean average precision by 5.42%, 3.20% and 0.9% compared with YOLOv3, YOLOv4 and YOLOv5, respectively, effectively improving the missed and false detection of objects of different sizes in complex underwater environments.

A Novel Object Recognition Algorithm Based on Improved YOLOv5 Model for Patient Care Robots

The rapid development of computer vision raises a new research area involving patient care robots. Such robotic systems require fast target recognition at long ranges, where detecting smaller objects is notoriously challenging due to the cameras’ low resolution and noise. Spurred by these concerns, this paper develops a novel object recognition algorithm that solves these problems. Specifically, we amend YOLOv5 with our proposed sparse detection algorithm aiming to improve detection efficiency by separating the most significant context features and constructing smaller and less computational expensive models. Furthermore, we extend FReLU and suggest a novel activation function to improve recognition accuracy, which presents an extended nonlinearity increasing the expressiveness of the activation function. Finally, we propose sine annealing, which affords a trajectory that tends to cross over barriers and escape from local minima during training phase, addressing the challenging small object detection problem. The experimental results highlight that our algorithm has a lower memory consumption (Mem) value than the traditional YOLOv5 with a 5% boost down. Additionally, our method runs twice as fast as the traditional YOLOv5 while preserving accuracy, achieving more than 14.5 FPS on a medium-capability CPU. Overall, the detection results prove that our method can faster and accurately classifies and localizes most small-scaled objects.

MIYOLO: Modification of Improved YOLO-v3

In the world of detection and classification, You Only Look Once (YOLO)-v3 has been maintaining its position as the best classifier and detector and is due to its fastness, accuracy, less complex structure, continuous modifications, improvements, etc. Several approaches have been proposed in this paper to modify improved YOLO-v3. Our main contributions are (1) increasing the number of anchor boxes; (2) replacement of K-means clustering with density-based spatial clustering of applications with noise (DBSCAN); (3) introducing final bounding box alignment (FBBA) technique; and (4) replacement of multi-scaling with a depth calculation algorithm. At first, we increased the number of anchor boxes from 9 to 11 which resulted in a higher performance of 6% for both visual object classes and common objects in context datasets, where increment in computational cost was negligible. Similarly, it has been analyzed that a true number of anchor boxes can increase efficiency. So, instead of generating a static number of anchor boxes with k-means clustering, DBSCAN has been proposed for generating a dynamic and accurate number of anchor boxes. For minimizing the localization error, a new FBBA method has been introduced which would exactly overlap the object. In the same way, depth calculation algorithm can solve the small and large size object detection and classification problem without doing multi-scaling.

An anchor-free object detector based on soften optimized bi-directional FPN

We propose an anchor-free object detector that combines a weighted bi-directional Feature Pyramid Network (BiFPN) and Soft Anchor Point Detector to address the object detection problem in a pixel-wise paradigm. The current mainstream object detection methods are anchor-based, which require to set hyper parameters such as scale and aspect ratio. This requires strong prior knowledge and can be difficult to design. Therefore, we propose an anchor-free detector that completely avoids the complex calculations and all the hyper parameters related to the anchor box by eliminating the predefined set of anchor boxes in an anchor-free way. Anchor-free detectors are essentially dense prediction methods. Although the huge solution space can yield high recall, simple anchor-free methods tend to return too many false positives, which leads to the problem of semantic ambiguity caused by the high overlap of object centers. Therefore, we propose BiFPN to alleviate the impact of high overlap which also effectively addresses the problems related to multi-scale features. Moreover, in order to utilize the power of feature pyramid better, we tackle the issues with a novel training strategy that involves two soften optimization techniques, i.e., soft-weighted anchor points and soft-selected pyramid levels. This training strategy further re-weights the quality of the detection results to make our detection results more stable.

Focus-and-Detect: A small object detection framework for aerial images

Despite recent advances, object detection in aerial images is still a challenging task. Specific problems in aerial images makes the detection problem harder, such as small objects, densely packed objects, objects in different sizes and with different orientations. To address small object detection problem, we propose a two-stage object detection framework called “Focus-and-Detect”. The first stage which consists of an object detector network supervised by a Gaussian Mixture Model, generates clusters of objects constituting the focused regions. The second stage, which is also an object detector network, predicts objects within the focal regions. Incomplete Box Suppression (IBS) method is also proposed to overcome the truncation effect of region search approach. Results indicate that the proposed two-stage framework achieves an AP score of 42.06 on VisDrone validation dataset, surpassing all other state-of-the-art small object detection methods reported in the literature, to the best of authors’ knowledge.

A Feature Pyramid Based Multi-stage Framework for Object Detection in Low-altitude UAV Images

This paper proposes a high-performance framework for accurate multi-stage object detection in low-altitude based UAV images. The proposed system employs a cascade style architecture with increasing thresholds for achieving accurate detection. The framework makes use of highly efficient Feature Pyramid Networks (FPNs) to detect objects of small sizes, and various scales which are the main challenge in low-altitude aerial images. FPNs aim to resolve scale variation problems in object detection by combining features of multiple levels. The experiments have been performed on a complex low-altitude aerial dataset VisDrone which has multiple categories of classes. The FPN-Cascade detector has been supported by slicing the data horizontally and vertically that resulted in an advancement of 8% mAP when compared with the base detector. The experiments compare the FPN-Cascade performance on the standard as well as augmented VisDrone dataset. A concrete methodology about the training process, hyperparameter tuning, and performance evaluation methods for Cascade RCNN on the VisDrone dataset is highlighted. The proposed framework achieves state of the art 30.04% mAP value on the VisDrone dataset.

NaGAN: Nadir-like Generative Adversarial Network for Off-Nadir Object Detection of Multi-View Remote Sensing Imagery

Detecting off-nadir objects is a well-known challenge in remote sensing due to the distortion and mutable representation. Existing methods mainly focus on a narrow range of view angles, and they ignore broad-view pantoscopic remote sensing imagery. To address the off-nadir object detection problem in remote sensing, a new nadir-like generative adversarial network (NaGAN) is proposed in this paper by narrowing the representation differences between the off-nadir and nadir object. NaGAN consists of a generator and a discriminator, in which the generator learns to transform the off-nadir object to a nadir-like one so that they are difficult to discriminate by the discriminator, and the discriminator competes with the generator to learn more nadir-like features. With the progressive competition between the generator and discriminator, the performances of off-nadir object detection are improved significantly. Extensive evaluations on the challenging SpaceNet benchmark for remote sensing demonstrate the superiority of NaGAN to the well-established state-of-the-art in detecting off-nadir objects.

Flower Mapping in Grasslands With Drones and Deep Learning.

Manual assessment of flower abundance of different flowering plant species in grasslands is a time-consuming process. We present an automated approach to determine the flower abundance in grasslands from drone-based aerial images by using deep learning (Faster R-CNN) object detection approach, which was trained and evaluated on data from five flights at two sites. Our deep learning network was able to identify and classify individual flowers. The novel method allowed generating spatially explicit maps of flower abundance that met or exceeded the accuracy of the manual-count-data extrapolation method while being less labor intensive. The results were very good for some types of flowers, with precision and recall being close to or higher than 90%. Other flowers were detected poorly due to reasons such as lack of enough training data, appearance changes due to phenology, or flowers being too small to be reliably distinguishable on the aerial images. The method was able to give precise estimates of the abundance of many flowering plant species. In the future, the collection of more training data will allow better predictions for the flowers that are not well predicted yet. The developed pipeline can be applied to any sort of aerial object detection problem.

An efficient framework for zero-shot sketch-based image retrieval

Zero-shot sketch-based image retrieval (ZS-SBIR) has recently attracted the attention of the computer vision community due to its real-world applications, and the more realistic and challenging setting that it presents over SBIR. ZS-SBIR inherits the main challenges of multiple computer vision problems including content-based Image Retrieval (CBIR), zero-shot learning and domain adaptation. The majority of previous studies using deep neural networks have achieved improved results by either projecting sketch and images into a common low-dimensional space, or transferring knowledge from seen to unseen classes. However, those approaches are trained with complex frameworks composed of multiple deep convolutional neural networks (CNNs) and are dependent on category-level word labels. This increases the requirements for training resources and datasets. In comparison, we propose a simple and efficient framework that does not require high computational training resources, and learns the semantic embedding space from a vision model rather than a language model, as is done by related studies. Furthermore, at training and inference stages our method only uses a single CNN. In this work, a pre-trained ImageNet CNN (i.e., ResNet50) is fine-tuned with three proposed learning objects: domain-balanced quadruplet loss, semantic classification loss, and semantic knowledge preservation loss. The domain-balanced quadruplet and semantic classification losses are introduced to learn discriminative, semantic and domain invariant features by considering ZS-SBIR as an object detection and verification problem. To preserve semantic knowledge learned with ImageNet and exploit it for unseen categories, the semantic knowledge preservation loss is proposed. To reduce computational cost and increase the accuracy of the semantic knowledge distillation process, ground-truth semantic knowledge is prepared in a class-oriented fashion prior to training. Extensive experiments are conducted on three challenging ZS-SBIR datasets: Sketchy Extended, TU-Berlin Extended and QuickDraw Extended. The proposed method achieves state-of-the-art results, and outperforms the majority of related works by a substantial margin.

Evaluation of Automated Measurement of Hair Density Using Deep Neural Networks.

Recently, deep learning has been employed in medical image analysis for several clinical imaging methods, such as X-ray, computed tomography, magnetic resonance imaging, and pathological tissue imaging, and excellent performance has been reported. With the development of these methods, deep learning technologies have rapidly evolved in the healthcare industry related to hair loss. Hair density measurement (HDM) is a process used for detecting the severity of hair loss by counting the number of hairs present in the occipital donor region for transplantation. HDM is a typical object detection and classification problem that could benefit from deep learning. This study analyzed the accuracy of HDM by applying deep learning technology for object detection and reports the feasibility of automating HDM. The dataset for training and evaluation comprised 4492 enlarged hair scalp RGB images obtained from male hair-loss patients and the corresponding annotation data that contained the location information of the hair follicles present in the image and follicle-type information according to the number of hairs. EfficientDet, YOLOv4, and DetectoRS were used as object detection algorithms for performance comparison. The experimental results indicated that YOLOv4 had the best performance, with a mean average precision of 58.67.

Intelligent System for Estimation of the Spatial Position of Apples Based on YOLOv3 and Real Sense Depth Camera D415

Despite the great possibilities of modern neural network architectures concerning the problems of object detection and recognition, the output of such models is the local (pixel) coordinates of objects bounding boxes in the image and their predicted classes. However, in several practical tasks, it is necessary to obtain more complete information about the object from the image. In particular, for robotic apple picking, it is necessary to clearly understand where and how much to move the grabber. To determine the real position of the apple relative to the source of image registration, it is proposed to use the Intel Real Sense depth camera and aggregate information from its depth and brightness channels. The apples detection is carried out using the YOLOv3 architecture; then, based on the distance to the object and its localization in the image, the relative distances are calculated for all coordinates. In this case, to determine the coordinates of apples, a transition to a symmetric coordinate system takes place by means of simple linear transformations. Estimating the position in a symmetric coordinate system allows estimating not only the magnitude of the shift but also the location of the object relative to the camera. The proposed approach makes it possible to obtain position estimates with high accuracy. The approximate root mean square error is 7–12 mm, depending on the range and axis. As for precision and recall metrics, the first is 100% and the second is 90%.

Towards densely clustered tiny pest detection in the wild environment

Our life is populated with many small-size objects, such as human in aerial images and tiny pests in agriculture. Current generic and small object detection methods are only focus on tackling their sizes rather than distribution. Considering this limitation, we state a Densely Clustered Tiny (DCT) object detection problem using a novel metric Object Density Level (ODL) to measure the object distribution in an image. The DCT problem allows varied densely distributed objects in the real-world captured images. In dealing with the DCT problem, we select two kinds of aphids that usually gather into cliques in the real-world agricultural environment, and build an aphid dataset APHID-4K in our task. Accompanying the DCT task, we propose a novel DCT detection network (DCTDet) to address this challenge. Specifically, a Cluster Region Proposal Network (ClusRPN) is trained to select appropriate densely distributed object cluster regions from images. These candidates are classified into different groups according to their density. A Density Merging and Partition module (DMP) merges and partitions them respectively and finally outputs cluster regions with uniform size and density to a subsequent Local Detector Group (LDG). In addition, we also use Composited Cluster data Generation (CCG) to present a large-scale dataset for ClusRPN optimization for robust training procedure and theoretically analyze their effects in detail. Experiments on APHID-4K and another clustered small object detection dataset VisDrone show that our DCTDet achieves state-of-the-art performance.