Tiny Object Detection Research Articles

Detecting tiny objects in aerial images: A normalized Wasserstein distance and a new benchmark

Tiny object detection (TOD) in aerial images is challenging since a tiny object only contains a few pixels. State-of-the-art object detectors do not provide satisfactory results on tiny objects due to the lack of supervision from discriminative features. Our key observation is that the Intersection over Union (IoU) metric and its extensions are very sensitive to the location deviation of the tiny objects, which drastically deteriorates the quality of label assignment when used in anchor-based detectors. To tackle this problem, we propose a new evaluation metric dubbed Normalized Wasserstein Distance (NWD) and a new RanKing-based Assigning (RKA) strategy for tiny object detection. The proposed NWD-RKA strategy can be easily embedded into all kinds of anchor-based detectors to replace the standard IoU threshold-based one, significantly improving label assignment and providing sufficient supervision information for network training. Tested on four datasets, NWD-RKA can consistently improve tiny object detection performance by a large margin. Besides, observing prominent noisy labels in the Tiny Object Detection in Aerial Images (AI-TOD) dataset, we are motivated to meticulously relabel it and release AI-TOD-v2 and its corresponding benchmark. In AI-TOD-v2, the missing annotation and location error problems are considerably mitigated, facilitating more reliable training and validation processes. Embedding NWD-RKA into DetectoRS, the detection performance achieves 4.3 AP points improvement over state-of-the-art competitors on AI-TOD-v2. Datasets, codes, and more visualizations are available at: https://chasel-tsui.github.io/AI-TOD-v2/.

Multi-Site and Multi-Scale Unbalanced Ship Detection Based on CenterNet

Object detection plays an essential role in the computer vision domain, especially the machine learning-based approach, which has developed rapidly in the past decades. However, the development of convolutional neural networks in the marine field is relatively slow, such as in ship classification and tracking. In this paper, ship detection is considered as a central point classification and regression task but discards the non-maximum suppression operation. We first improved the deep layer aggregation network to enhance the feature extraction capability of tiny targets, then reduced the number of parameters through the lightweight convolution module, and finally employed a unique activation function to enhance the nonlinearity of the model. By doing this, the improved network not only suits unbalanced sample ratios in classifying, but is more robust in scenarios where both the number and resolution of samples are unstable. Experimental results demonstrate that the proposed approach obtains outstanding performance and especially suits tiny object detection compared with current advanced methods. Furthermore, in contrast to the original CenterNet framework, the mAP of the proposed approach increased by 5.6%.

TOD-CNN: An effective convolutional neural network for tiny object detection in sperm videos

The detection of tiny objects in microscopic videos is a problematic point, especially in large-scale experiments. For tiny objects (such as sperms) in microscopic videos, current detection methods face challenges in fuzzy, irregular, and precise positioning of objects. In contrast, we present a convolutional neural network for tiny object detection (TOD-CNN) with an underlying data set of high-quality sperm microscopic videos (111 videos, > 278,000 annotated objects), and a graphical user interface (GUI) is designed to employ and test the proposed model effectively. TOD-CNN is highly accurate, achieving 85.60% AP50 in the task of real-time sperm detection in microscopic videos. To demonstrate the importance of sperm detection technology in sperm quality analysis, we carry out relevant sperm quality evaluation metrics and compare them with the diagnosis results from medical doctors.

Generative Adversarial Networks for Unmanned Aerial Vehicle Object Detection with Fusion Technology

Unmanned aerial vehicles (UAVs) also called as a drone comprises of a controller from the base station along with a communications system with the UAV. The UAV plane can be precisely controlled by a machine operator, similar to remotely directed aircraft, or with increasing grades of autonomy, as like autopilot assistance, up to completely self-directed aircraft that require no human input. Obstacle detection and avoidance is important for UAVs, particularly lightweight micro aerial vehicles, but it is a difficult problem to solve because pay load restrictions limit the number of sensors that can be mounted onto the vehicle. Lidar uses Laser for finding the distance between objects and vehicle. The speed and direction of the moving objects are detected and tracked with the help of radar. When many sensors are deployed, both thermal and electro-optro cameras have great clustering capabilities as well as accurate localization and ranging. The purpose of the proposed architecture is to create a fusion system that is cost-effective, lightweight, modular, and robust as well. Also, for tiny object detection, we recommend a novel Perceptual Generative Adversarial Network method that bridges the representation gap between small and large objects. It employs the Generative Adversarial Networks (GAN) algorithm, which iimproves object detection accuracy above benchmark models at the same time maintaining real-time efficiency in an embedded computer for UAVs. Its generator, in particular, learns to turn unsatisfactory tiny object representations into super-resolved items that are similar to large objects to deceive a rival discriminator. At the same time, its discriminator contests with the generator to classify the engendered representation, imposing a perceptual restriction on the generator: created representations of tiny objects must be helpful for detection. With three different obstacles, we were able to successfully identify and determine the magnitude of the barriers in the first trial. The accuracy of proposed models is 83.65% and recall is 81% which is higher than the existing models.

A Novel Approach for Small Object Detection in Medical Images through Deep Ensemble Convolution Neural Network

Small objects detection in medical image becomes an interesting field of research that helps the medical practitioners to focus on in-depth evaluation of diseases. The accurate localization and classification of objects face tremendous difficulty due to lower intensity of the images and distraction of pixel points that vary the decision on identifying the shape, structure etc. In many real-time cases, detection and classification of tiny objects in the medically treated images becomes mandatory. The proposed system is designed in the same criteria in which the semantic segmentation of tiny objects in the medical images is considered. The system design focused on implementing the model for different kinds of human organs such as lung and liver. The axial CT or PET images of Lung and Liver are considered as the prime input for the given system. Detection of tiny objects in the CT-PET images, segmenting it from the background and classification of segmented part as Tumor or Nodule is discussed. The preprocessed images are feature extracted after the morphology segmentation that determines the structural features of the tiny object being segmented. The feature vectors are nothing but the feature points from Kaze feature extraction and Morphology segmented image. These two inputs are fetched to the Deep ensemble Convolution neural network (DECNN) to obtain the dual classification results. Performing the quantitative measurements to evaluate the decision making system for nodule or tumor class is determined. The performance measure is done using accuracy, precision, recall and F1Score.

Insect detection from imagery using YOLOv3-based adaptive feature fusion convolution network

Context Insects are a major threat to crop production. They can infect, damage, and reduce agricultural yields. Accurate and fast detection of insects will help insect control. From a computer algorithm point of view, insect detection from imagery is a tiny object detection problem. Handling detection of tiny objects in large datasets is challenging due to small resolution of the insects in an image, and other nuisances such as occlusion, noise, and lack of features. Aims Our aim was to achieve a high-performance agricultural insect detector using an enhanced artificial intelligence machine learning technique. Methods We used a YOLOv3 network-based framework, which is a high performing and computationally fast object detector. We further improved the original feature pyramidal network of YOLOv3 by integrating an adaptive feature fusion module. For training the network, we first applied data augmentation techniques to regularise the dataset. Then, we trained the network using the adaptive features and optimised the hyper-parameters. Finally, we tested the proposed network on a subset dataset of the multi-class insect pest dataset Pest24, which contains 25 878 images. Key results We achieved an accuracy of 72.10%, which is superior to existing techniques, while achieving a fast detection rate of 63.8 images per second. Conclusions We compared the results with several object detection models regarding detection accuracy and processing speed. The proposed method achieved superior performance both in terms of accuracy and computational speed. Implications The proposed method demonstrates that machine learning networks can provide a foundation for developing real-time systems that can help better pest control to reduce crop damage.

FSANet: Feature-and-Spatial-Aligned Network for Tiny Object Detection in Remote Sensing Images

FSANet: Feature-and-Spatial-Aligned Network for Tiny Object Detection in Remote Sensing Images

ET-YOLOv5s: Toward Deep Identification of Students’ in-Class Behaviors

Recognizing students’ behaviors in classes based on videos of their activity plays an important role in improving teaching quality and paying attention to the healthy growth of students. The existing student behavior recognition methods mainly focus on the behavior of the single student and have low performance and efficiency. To recognize the behaviors of multiple students in the classroom at the same time, we propose a fast and effective solution, called ET-YOLOv5s, which is an improved YOLOv5s with ESRGAN (Enhanced Super-Resolution Generative Adversarial Network) and a tiny object detection module in this paper. First, the ESRGAN is adopted to generate high-definition images from original images of real classroom environments in colleges to improve the recognition accuracy. Then a tiny object detection module is added to the YOLOv5s to detect the smaller objects on high-definition images, such as phones, books, and students sitting at the back of the classroom. Finally, the experimental results show that the proposed solution can detect 11 kinds of students’ behaviors in classrooms, including playing mobile phones, having class, sitting with hands on face, sitting turn right, sitting turn left, bowing, sleeping, and so on. Compared with YOLOv4, YOLOv5s, and other tiny target detection algorithms, it has better detection performance and can effectively deal with the behavior recognition of multiple students.

Vision-Based Maritime Object Detection Covering Far and Tiny Obstacles

Though general object detection with deep learning has achieved great success in the maritime domain, recognizing far-away and tiny objects is still challenging. However, an autonomous vessel must be aware of objects even several nautical miles away for early planning. In this paper, we propose a far-away and tiny object detection framework with deep learning by utilizing a high-resolution image. The framework is two-fold: Small Object Detector (SOD) and Large Object Detector (LOD). SOD investigates a region around the horizon with a fine-resolution image aiming to recognize far-away and small objects. LOD targets large objects that are identifiable in an even coarse image. We evaluate our approach on datasets including objects of various sizes inshore or offshore. The results verify that our proposed scheme is effective against even far-away and tiny objects while keeping computational costs low.

A MOLECULAR DYNAMICS STUDY OF NANOWIRE RESONATOR BIO-OBJECT DETECTION

This paper presents a comprehensive analysis, carried out by the molecular dynamics (MD) simulations, of the vibrations of silicon nanowire (SiNW) resonators, having diverse applications including biological and medical fields. The chosen approach allows us to obtain a better understanding of the nanowire (NW) materials’ characteristics, providing a more detailed insight into the behavior of nanostructures, especially when the topic of interest is relevant to their dynamics, interatomic interactions, and atoms trajectories’ prediction. We first simulate a SiNW to study its frequency of vibrations using MD simulations. Then, we add a molecule of human immunodeficiency virus as an example to investigate the potential of the SiNW resonator for the detection of tiny bio-objects. The developed technique and its application to the detection of tiny objects, such as viruses, are discussed in the context of several key effects pertinent to the design of SiNW.

Dual Architecture Deep Learning Based Object Detection System for Autonomous Driving

Object detection of autonomous vehicles presents a big challenge for researchers due to the requirements of accuracy and precision in real-time. This work presents a deep learning approach based on a dual architecture design of the network. A highly accurate multi-class network of convolutional neural networks (CNN) is presented for input data classification. A Region-Based Convolutional Neural Networks (Faster R-CNN) network with a modified Feature Pyramid Networks (FPN) is used for better detection of tiny objects and You Only Look Once (YOLOv3) network is used for general detection. Each network independently detects the existence of an object. The decision maps are then fused and compared to decide whether an object is present or not. Faster R-CNN with FPN model reported a higher intersection over Union (IoU) and mean average precision (mAP) than the YOLOv3. This approach is reliable demonstrating an upgrade on the existing state-of-the-art methods of fully connected networks. Index Terms— autonomous driving, computer vision, deep learning, object detection

Detection and localization of hand fractures based on GA_Faster R-CNN

X-ray imaging is the primary diagnostic tool for clinical diagnosis of suspected fracture. Hand fracture (HF) is a world-leading health problem for children, adolescents and the elderly. A missed diagnosis of hand fracture on radiography may lead to severe consequences for patients, resulting in delayed treatment and poor recovery of function. Nevertheless, many hand fractures are fairly insidious, which often lead to misdiagnosis. In this dissertation, we propose GA_Faster R-CNN in which a guided anchoring method (GA) of GA_RPN is applied to detect and localize hand fractures in radiographs. Our new guided anchoring method makes the anchor generation more accurate and efficient, greatly improves the network performance, and saves computing power. In our work, Feature Pyramid Network (FPN) is used to solve the problem of tiny object detection which mostly appears at the joint of fingertips and knuckles. In addition, Balanced L1 Loss is applied to adapt to the imbalance of learning tasks. We evaluate the proposed algorithm on a HF dataset containing 3,067 X-ray radiographs, 2,453 of which are assigned as the training dataset and 614 as the testing dataset. The present framework achieved accuracies of 97%–99% and an average precision (AP) of 70.7%, thereby outperforming the previous state-of-the-art methods for detecting HF. As a consequence, the GA_Faster R-CNN has great potential for clinical applications.

FSoD-Net: Full-Scale Object Detection From Optical Remote Sensing Imagery

Object detection is an essential task in computer vision. Recently, several convolution neural network (CNN)-based detectors have achieved a great success in natural scenes. However, for optical remote sensing images with a large scale of view, lower proportion of foreground target pixels and drastic differences in object scale present considerable challenges. To address these problems, we propose a novel one-stage detector called the full-scale object detection network (FSoD-Net) which consists of proposed multiscale enhancement network (MSE-Net) backbone cascaded with scale-invariant regression layers (SIRLs). First, MSE-Net provides the multiscale description enhancement by integrated the Laplace kernel with fewer parallel multiscale convolution layers. Second, SIRLs contain three different isolated regression branch layers (i.e., corresponding to small, medium, and large scales), which make default discrete scale bounding boxes (bboxes) cover full-scale object information in regression procedure. A novel specific scale joint loss is also designed that uses the softmax function combined with a strong <inline-formula> <tex-math notation="LaTeX">$L_{1}$ </tex-math></inline-formula>-norm constraint in each regression branch layer. It can further speed up the convergence and improve the classification scores of predicted bboxes. Finally, extensive experiments are carried on challenge data sets of large-scale dataset for object detection in aerial images (DOTA) and object detection in optical remote sensing images (DIOR) which contain multiple instances from different imaging platforms, and these results demonstrate that FSoD-Net can achieve better performance than other state-of-the-art one-stage detectors, and it can reach a mean average precision (mAP) of 75.33&#x0025; on DOTA and 71.80&#x0025; mAP on DIOR, respectively. Especially, the average precision (AP) of tiny object detection can improve 10&#x0025;&#x2013;20&#x0025; approximately.

Feature Rescaling and Fusion for Tiny Object Detection

Recent years have witnessed rapid developments on computer vision, however, there are still challenges in detecting tiny objects in a large-scale background. The tiny objects knowledge become sparse and weak due to their tiny size, which makes the tiny objects difficult to be detected with the common approaches. In this paper, a new network named Specific Characteristics based Feature Rescaling and Fusion (SFRF) is designed to detect tiny persons in a broad horizon and massive background. Different from the methods in general, a Nonparametric Adaptive Dense Perceiving Algorithm (NADPA) is designed to automatically select and generate a new resized feature map with the high density distribution of tiny objects. Then, a method called Many-For-One strategy is used for feature fusion of the feature pyramid network (FPN) layers to improve the feature representation and detection. Finally, an ensemble model method named hierarchical Coarse-to-fine mechanism is designed based on the proposed methods to further improve the performance. The experiments demonstrate that the proposed approach achieves an obvious performance improvement on tiny object detection than the existing approaches, and our approach has been awarded as the 1st-place in the first large-scale Tiny Object Detection (TOD) challenge.

A Learning Framework for Target Detection and Human Face Recognition in Real Time

Inspired by the function, mechanism and efficiency of the visual nerve system of human beings, a revolutionary detection and reorganization method named YOLO is present to provide an accurate, stable and fast arithmetic for a variety of targets, be it target detection for unmanned vehicle, car license recognition and optimization for surveillance. The traditional method for object detection is to reuse the classifier to implement detection, in contrast, the method named YOLOV2 process this problem by considering it in the mathematical area as a regression of spatially discrete bordered areas and relative class probability. However, as a cost of stable and fast response of this arithmetic, inaccurate detection maybe caused by YOLOV2 when the detected object is tiny (e.g., face recognition in surveillance). In this article, the authors provide a new method to further improve the performance of YOLOV2 by utilizing the accurate, stable and fast properties of YOLOV2 and editing the original code of YOLOV2 to eliminate the inaccuracy of tiny object detection, and implement this method on an embedded system.

A COARSE-TO-FINE MODEL FOR AIRPLANE DETECTION FROM LARGE REMOTE SENSING IMAGES USING SALIENCY MODLE AND DEEP LEARNING

Abstract. High resolution remote sensing images are bearing the important strategic information, especially finding some time-sensitive-targets quickly, like airplanes, ships, and cars. Most of time the problem firstly we face is how to rapidly judge whether a particular target is included in a large random remote sensing image, instead of detecting them on a given image. The problem of time-sensitive-targets target finding in a huge image is a great challenge: 1) Complex background leads to high loss and false alarms in tiny object detection in a large-scale images. 2) Unlike traditional image retrieval, what we need to do is not just compare the similarity of image blocks, but quickly find specific targets in a huge image. In this paper, taking the target of airplane as an example, presents an effective method for searching aircraft targets in large scale optical remote sensing images. Firstly, we used an improved visual attention model utilizes salience detection and line segment detector to quickly locate suspected regions in a large and complicated remote sensing image. Then for each region, without region proposal method, a single neural network predicts bounding boxes and class probabilities directly from full images in one evaluation is adopted to search small airplane objects. Unlike sliding window and region proposal-based techniques, we can do entire image (region) during training and test time so it implicitly encodes contextual information about classes as well as their appearance. Experimental results show the proposed method is quickly identify airplanes in large-scale images.