Small Target Size Research Articles

Multiple Ship Tracking in Remote Sensing Images Using Deep Learning

In remote sensing images, small target size and diverse background cause difficulty in locating targets accurately and quickly. To address the lack of accuracy and inefficient real-time performance of existing tracking algorithms, a multi-object tracking (MOT) algorithm for ships using deep learning was proposed in this study. The feature extraction capability of target detectors determines the performance of MOT algorithms. Therefore, you only look once (YOLO)-v3 model, which has better accuracy and speed than other algorithms, was selected as the target detection framework. The high similarity of ship targets will cause poor tracking results; therefore, we used the multiple granularity network (MGN) to extract richer target appearance information to improve the generalization ability of similar images. We compared the proposed algorithm with other state-of-the-art multi-object tracking algorithms. Results show that the tracking accuracy is improved by 2.23%, while the average running speed is close to 21 frames per second, meeting the needs of real-time tracking.

Practical Approaches to Target Detection in Long Range and Low Quality Infrared Videos

It is challenging to detect vehicles in long range and low quality infrared videos using deep learning techniques such as You Only Look Once (YOLO) mainly due to small target size. This is because small targets do not have detailed texture information. This paper focuses on practical approaches for target detection in infrared videos using deep learning techniques. We first investigated a newer version of You Only Look Once (YOLO v4). We then proposed a practical and effective approach by training the YOLO model using videos from longer ranges. Experimental results using real infrared videos ranging from 1000 m to 3500 m demonstrated huge performance improvements. In particular, the average detection percentage over the six ranges of 1000 m to 3500 m improved from 54% when we used the 1500 m videos for training to 95% if we used the 3000 m videos for training.

Practical Approaches to Target Detection in Long Range and Low Quality Infrared Videos

It is challenging to detect vehicles in long range and low quality infrared videos using deep learning techniques such as You Only Look Once (YOLO) mainly due to small target size. This is because small targets do not have detailed texture information. This paper focuses on practical approaches for target detection in infrared videos using deep learning techniques. We first investigated a newer version of You Only Look Once (YOLO v4). We then proposed a practical and effective approach by training the YOLO model using videos from longer ranges. Experimental results using real infrared videos ranging from 1000 m to 3500 m demonstrated huge performance improvements. In particular, the average detection percentage over the six ranges of 1000 m to 3500 m improved from 54% when we used the 1500 m videos for training to 95% if we used the 3000 m videos for training.

Implementation of a Modified Faster R-CNN for Target Detection Technology of Coastal Defense Radar

The traditional method of constant false-alarm rate detection is based on the assumption of an echo statistical model. The target recognition accuracy rate and the high false-alarm rate under the background of sea clutter and other interferences are very low. Therefore, computer vision technology is widely discussed to improve the detection performance. However, the majority of studies have focused on the synthetic aperture radar because of its high resolution. For the defense radar, the detection performance is not satisfactory because of its low resolution. To this end, we herein propose a novel target detection method for the coastal defense radar based on faster region-based convolutional neural network (Faster R-CNN). The main processing steps are as follows: (1) the Faster R-CNN is selected as the sea-surface target detector because of its high target detection accuracy; (2) a modified Faster R-CNN based on the characteristics of sparsity and small target size in the data set is employed; and (3) soft non-maximum suppression is exploited to eliminate the possible overlapped detection boxes. Furthermore, detailed comparative experiments based on a real data set of coastal defense radar are performed. The mean average precision of the proposed method is improved by 10.86% compared with that of the original Faster R-CNN.

Research on Tower Bolt Identification Technology Based on Convolution Network

Aiming at the fact that tower bolts are easily affected by factors such as occlusion during the identification process, an improved FasterR-CNN bolt identification method is proposed. In this method, in view of the small size of the detection target and the low resolution of the image, a residual network is added to the traditional FasterR-CNN to improve the efficiency of image recognition. Finally, the above scheme was proved through verification. The result shows that the above method is feasible.

Infrared small target detection using tensor based least mean square

Infrared small target detection is an important topic in infrared image processing and pattern recognition. It plays an important role in reconnaissance, early warning system, aircraft tracking and missile guidance. Because of the low signal-to-noise ratio, small target size, obvious shape structure and texture information available, infrared small target detection is a very difficult task. Inspired by the observation that infrared small targets usually exist in image background and the image has matrix structure, we propose to observe the infrared small target in main view and develop a Tensor based least mean square (TLMS) method to detect infrared small target. In TLMS, the neighborhood of the target is firstly utilized to predict the gray value of the central pixel. The predicted background image and the difference image are derived from the principle of mean square minimum error. Finally, the target is obtained by performing the adaptive thresholding. We conduct the experiments on four datasets to compare the performance of TLMS with Max-median/Max-mean, Top-hat, left-TDLMS and right-TDLMS. The experimental results show that TLMS is an effective infrared small target detection method which can well suppress the background and highlight the infrared small targets.

Fast and Robust Infrared Image Small Target Detection Based on the Convolution of Layered Gradient Kernel

Infrared (IR) small target detection is challenging because the IR imaging lacks detailed features, weak shape features, and a low signal-to-noise ratio (SNR). The existing small IR target detection methods usually focus on improving their high detective performance without considering the execution time. However, high-speed detection is vital for various applications, such as early warning systems, military surveillance, infrared search and track (IRST), etc. This paper proposes a fast and robust single-frame IR small target detection algorithm with a low computational cost while maintaining excellent detection performance. We propose a layered gradient kernel (LGK) based on the contrast properties of the human visual system (HVS) and model it through a three-layer patch image model. The layered gradient kernel is used to convolute with the input IR frame to obtain its gradient map. The target detection is further performed on the acquired gradient map with an adaptive threshold method. This method is compared with eight representative small target detection algorithms to evaluate the performance. Experimental results demonstrate that the algorithm is fast and suitable for real-time applications, and it is very effective even when the small target size is as small as 2×2.

DAR-Net: Dense Attentional Residual Network for Vehicle Detection in Aerial Images.

With the rapid development of deep learning and the wide usage of Unmanned Aerial Vehicles (UAVs), CNN-based algorithms of vehicle detection in aerial images have been widely studied in the past several years. As a downstream task of the general object detection, there are some differences between the vehicle detection in aerial images and the general object detection in ground view images, e.g., larger image areas, smaller target sizes, and more complex background. In this paper, to improve the performance of this task, a Dense Attentional Residual Network (DAR-Net) is proposed. The proposed network employs a novel dense waterfall residual block (DW res-block) to effectively preserve the spatial information and extract high-level semantic information at the same time. A multiscale receptive field attention (MRFA) module is also designed to select the informative feature from the feature maps and enhance the ability of multiscale perception. Based on the DW res-block and MRFA module, to protect the spatial information, the proposed framework adopts a new backbone that only downsamples the feature map 3 times; i.e., the total downsampling ratio of the proposed backbone is 8. These designs could alleviate the degradation problem, improve the information flow, and strengthen the feature reuse. In addition, deep-projection units are used to reduce the impact of information loss caused by downsampling operations, and the identity mapping is applied to each stage of the proposed backbone to further improve the information flow. The proposed DAR-Net is evaluated on VEDAI, UCAS-AOD, and DOTA datasets. The experimental results demonstrate that the proposed framework outperforms other state-of-the-art algorithms.

Enhancing Small Moving Target Detection Performance in Low-Quality and Long-Range Infrared Videos Using Optical Flow Techniques

The detection of small moving objects in long-range infrared videos is challenging due to background clutter, air turbulence, and small target size. In this paper, we summarize the investigation of efficient ways to enhance the performance of small target detection in long-range and low-quality infrared videos containing moving objects. In particular, we focus on unsupervised, modular, flexible, and efficient methods for target detection performance enhancement using motion information extracted from optical flow methods. Three well-known optical flow methods were studied. It was found that optical flow methods need to be combined with contrast enhancement, connected component analysis, and target association in order to be effective for target detection. Extensive experiments using long-range mid-wave infrared (MWIR) videos from the Defense Systems Information Analysis Center (DSIAC) dataset clearly demonstrated the efficacy of our proposed approach.

Research on Airplane and Ship Detection of Aerial Remote Sensing Images Based on Convolutional Neural Network.

The wide range, complex background, and small target size of aerial remote sensing images results in the low detection accuracy of remote sensing target detection algorithms. Traditional detection algorithms have low accuracy and slow speed, making it difficult to achieve the precise positioning of small targets. This paper proposes an improved algorithm based on You Only Look Once (YOLO)-v3 for target detection of remote sensing images. Due to the difficulty in obtaining the datasets, research on small targets for complex images, such as airplanes and ships, is the focus of research. To make up for the problem of insufficient data, we screen specific types of training samples from the DOTA (Dataset of Object Detection in Aerial Images) dataset and select small targets in two different complex backgrounds of airplanes and ships to jointly evaluate the optimization degree of the improved network. We compare the improved algorithm with other state-of-the-art target detection algorithms. The results show that the performance indexes of both datasets are ameliorated by 1–3%, effectively verifying the superiority of the improved algorithm.

Research on Detection Technology of Autonomous Landing Based on Airborne Vision

In this paper, aiming at the cooperative target detection problem in the process of unmanned helicopter sliding down, a detection method based on complementary filtering is proposed, which fuses improved SSD algorithm and related filtering KCF algorithm. The improved deep learning SSD model redesigns the feature extraction structure to improve the detection effect of small and medium targets for the small target size and large scale change in the landing scene. Then use the detection results of the SSD model to correct the KCF detection, adjust the weight parameters, and output the final fusion detection results. The test results show that the improved model detection accuracy is significantly improved, the detection accuracy in various environments reaches 93.3%, which is higher than 86.1% of the classic SSD model and 87.5% of the Faster-rcnn model. The final proposed fusion detection algorithm has a success rate of 91.1% and a processing speed of 91 hz, which basically satisfies the requirements of the ship.

High confidence detection for moving target in aerial video

The moving target detection and tracking in aerial video is a challenge task because of its moving background, smaller target sizes, lower resolution and limited onboard computing resources. In this study, a high confidence detection method based on background compensation and three-frame-difference method is designed, which can detect moving objects in a dynamic background accurately. First, the authors use local feature extraction and matching for image registration and demonstrate that speed-up robust feature key points are suitable for the stabilisation task. Then, they estimate the global camera motion parameters using affine transformation which are obtained by the random sample consensus algorithm. Finally, they detect moving object by three-frame-difference method. As the detection results of the frame-difference method generally exists ‘empty’ and noise, in order to select the two higher-quality differential images to perform the logic AND operation, they add image quality assessment to the three-frame-difference method to obtain more accurate moving objects. Moreover, the edge detection algorithm and morphological processing are integrated together to further boost the overall detecting performance. The extensive empirical evaluations on aerial videos demonstrate that the proposed detector is very promising for the various challenging scenarios.

Sig-NMS-Based Faster R-CNN Combining Transfer Learning for Small Target Detection in VHR Optical Remote Sensing Imagery

Small target detection is a challenging task in very-high-resolution (VHR) optical remote sensing imagery, because small targets occupy a minuscule number of pixels and are easily disturbed by backgrounds or occluded by others. Although current convolutional neural network (CNN)-based approaches perform well when detecting normal objects, they are barely suitable for detecting small ones. Two practical problems stand in their way. First, current CNN-based approaches are not specifically designed for the minuscule size of small targets (~15 or ~10 pixels in extent). Second, no well-established data sets include labeled small targets and establishing one from scratch is labor-intensive and time-consuming. To address these two issues, we propose an approach that combines Sig-NMS-based Faster R-CNN with transfer learning. Sig-NMS replaces traditional non-maximum suppression (NMS) in the stage of region proposal network and decreases the possibility of missing small targets. Transfer learning can effectively label remote sensing images by automatically annotating both object classes and object locations. We conduct an experiment on three data sets of VHR optical remote sensing images, RSOD, LEVIR, and NWPU VHR-10, to validate our approach. The results demonstrate that the proposed approach can effectively detect small targets in the VHR optical remote sensing images of about $10\times 10$ pixels and automatically label small targets as well. In addition, our method presents better mean average precisions than other state-of-the-art methods: 1.5% higher when performing on the RSOD data set, 17.8% higher on the LEVIR data set, and 3.8% higher on NWPU VHR-10.

Multiple-Object-Tracking Algorithm Based on Dense Trajectory Voting in Aerial Videos

In recent years, UAV technology has developed rapidly. Due to the mobility, low cost, and variable monitoring altitude of UAVs, multiple-object detection and tracking in aerial videos has become a research hotspot in the field of computer vision. However, due to camera motion, small target size, target adhesion, and unpredictable target motion, it is still difficult to detect and track targets of interest in aerial videos, especially in the case of a low frame rate where the target position changes too much. In this paper, we propose a multiple-object-tracking algorithm based on dense-trajectory voting in aerial videos. The method models the multiple-target-tracking problem as a voting problem of the dense-optical-flow trajectory to the target ID, which can be applied to aerial-surveillance scenes and is robust to low-frame-rate videos. More specifically, we first built an aerial video dataset for vehicle targets, including a training dataset and a diverse test dataset. Based on this, we trained the neural network model by using a deep-learning method to detect vehicles in aerial videos. Thereafter, we calculated the dense optical flow in adjacent frames, and generated effective dense-optical-flow trajectories in each detection bounding box at the current time. When target IDs of optical-flow trajectories are known, the voting results of the optical-flow trajectories in each detection bounding box are counted. Finally, similarity between detection objects in adjacent frames was measured based on the voting results, and tracking results were obtained by data association. In order to evaluate the performance of this algorithm, we conducted experiments on self-built test datasets. A large number of experimental results showed that the proposed algorithm could obtain good target-tracking results in various complex scenarios, and performance was still robust at a low frame rate by changing the video frame rate. In addition, we carried out qualitative and quantitative comparison experiments between the algorithm and three state-of-the-art tracking algorithms, which further proved that this algorithm could not only obtain good tracking results in aerial videos with a normal frame rate, but also had excellent performance under low-frame-rate conditions.

Glance and Stare: Trapping Flying Birds in Aerial Videos by Adaptive Deep Spatio-Temporal Features

Flying bird detection has recently attracted increasing attention in computer vision, which becomes an urgent task with the opening up of the low-altitude airspace. However, compared to conventional object detection tasks, it is much more challenging to trap flying birds in aerial videos due to small target sizes, complex backgrounds of great variations and disturbances of bird-like objects. In this paper, we propose a unified framework termed glance-and-stare detection (GSD) to trap flying birds in aerial videos. The GSD is inspired by the fact that human beings first glance at the whole image and then stare at the areas where the suspected object is most likely to appear until the confirmation is obtained. Specifically, we propose the zooming-in algorithm to generate region proposals for accurate localization of flying birds; to represent region proposal sequences of different lengths, we propose adaptive deep spatio-temporal features by leveraging the strength of 3D convolutional neural networks, based on which classification is conducted to achieve final detection. In contrast to conventional methods, the GSD enables localization and classification to be conducted jointly in an alternating iterative way, which mutually enhances each other to improve their performance. In order to validate the proposed GSD algorithm, we build flying bird data sets including images and videos, which provide new benchmarks for evaluation of flying bird detection systems. Experiments on the data sets demonstrate that the GSD can achieve high detection accuracy and largely outperform the state-of-the-art detection methods.

Learning behaviour recognition based on multi-object image in single viewpoint

Though greatly significant for education evaluation and improvement the analysis of learning behaviour of students is, recognition of behaviour of students in the classroom surveillance based on one single image remains a challenging work due to problems including heavy occlusion, low resolution, small target size, large variability of camera viewpoints and significant perspective effect. In this paper, an innovative multiple-model method for recognizing students’ learning behaviour and counting the number of students in classroom scenes via detecting individuals’ head, which divides students’ learning behaviour into three categories according to the head posture, is proposed. Through viewing heads at different postures as targets of different categories, behaviour recognition problem is transformed into multi-target detection problem. As the density of students in different images varies dramatically, a multiple model consisting of three detection networks with different sized receptive fields and one switch net is applied that each detection network selects a large number of sub-windows with small stride size from input images to optimize the performance of the model in classroom scenes while the switch net is trained to predict the density of students in the input image and relays the classroom image to the corresponding detection network according to the students’ density prediction. This method is obtained to outperform other methods in terms of accuracy and speed in comparison with state-of-the-art methods on a real classroom surveillance dataset.

Heterogeneous Gray-Temperature Fusion-Based Deep Learning Architecture for Far Infrared Small Target Detection

This paper proposes the end-to-end detection of a deep network for far infrared small target detection. The problem of detecting small targets has been a subject of research for decades and has been applied mainly in the field of surveillance. Traditional methods focus on filter design for each environment, and several steps are needed to obtain the final detection result. Most of them work well in a given environment but are vulnerable to severe clutter or environmental changes. This paper proposes a novel deep learning-based far infrared small target detection method and a heterogeneous data fusion method to solve the lack of semantic information due to the small target size. Heterogeneous data consists of radiometric temperature data (14-bit) and gray scale data (8-bit), which includes the physical meaning of the target, and compares the effects of the normalization method to fuse heterogeneous data. Experiments were conducted using an infrared small target dataset built directly on the cloud backgrounds. The experimental results showed that there is a significant difference in performance according to the various fusion methods and normalization methods, and the proposed detector showed approximately 20% improvement in average precision (AP) compared to the baseline constant false alarm rate (CFAR) detector.

Detection and Recognition of SAR Small Ship Objects Using Deep Neural Network

Synthetic aperture radar(SAR) ship target detection plays an increasingly important role in marine monitoring. Aimed at the problems of recognizing small size of ship targets in SAR images and the inability of traditional methods to extract fine target features due to external disturbances, we propose an improved SAR small target detection model based on the deep learning technology. The proposed model mainly consists of two parts:region proposal network(RPN) and object detection network. Firstly, a CNN model is designed and trained to accurately identify small ship targets. Then, the model is used to initialize the parameters of the shared feature extraction layer. Last, we train the proposed object detection model using a self-collected Sentinel-1 SAR small target dataset. The experimental results show that the proposed target detection model has better detection and recognition performance and anti-interference ability for small ship scalable targets in SAR images, and has certain reference value for the research of small target detection in SAR images.

An Adaptive Framework for Multi-Vehicle Ground Speed Estimation in Airborne Videos

With the rapid development of unmanned aerial vehicles (UAVs), UAV-based intelligent airborne surveillance systems represented by real-time ground vehicle speed estimation have attracted wide attention from researchers. However, there are still many challenges in extracting speed information from UAV videos, including the dynamic moving background, small target size, complicated environment, and diverse scenes. In this paper, we propose a novel adaptive framework for multi-vehicle ground speed estimation in airborne videos. Firstly, we build a traffic dataset based on UAV. Then, we use the deep learning detection algorithm to detect the vehicle in the UAV field of view and obtain the trajectory in the image through the tracking-by-detection algorithm. Thereafter, we present a motion compensation method based on homography. This method obtains matching feature points by an optical flow method and eliminates the influence of the detected target to accurately calculate the homography matrix to determine the real motion trajectory in the current frame. Finally, vehicle speed is estimated based on the mapping relationship between the pixel distance and the actual distance. The method regards the actual size of the car as prior information and adaptively recovers the pixel scale by estimating the vehicle size in the image; it then calculates the vehicle speed. In order to evaluate the performance of the proposed system, we carry out a large number of experiments on the AirSim Simulation platform as well as real UAV aerial surveillance experiments. Through quantitative and qualitative analysis of the simulation results and real experiments, we verify that the proposed system has a unique ability to detect, track, and estimate the speed of ground vehicles simultaneously even with a single downward-looking camera. Additionally, the system can obtain effective and accurate speed estimation results, even in various complex scenes.

Examining associations between action game play and motor control

The effects of playing action video games have been investigated across a wide range of attentional and cognitive abilities. However, few studies have examined the association between motor control and action gaming experience. We report data from two discrete pointing tasks, manipulating the index of difficulty (ID) by movement distance and target size, respectively. Participants' gaming experience ranged from non-players to individuals who played several hours a night. Our results indicated greater experience playing action games, but not video games in general, was associated with shorter movement times (MT), higher velocities, and shallower ID-MT slopes when difficulty was manipulated across increasingly further distances and smaller target sizes. Additionally casual players, those who only play action games a couple times a week, were able to achieve a similar level of performance as more experienced players.