Detection In Remote Sensing Images Research Articles

Local Attention Networks for Occluded Airplane Detection in Remote Sensing Images

Despite the great progress of deep learning and target detection in recent years, the accurate detection of the occluded targets in remote sensing images still remains a challenge. In this letter, we propose a new detection method called local attention networks to improve the detection of occluded airplanes. Following the idea of “divide and conquer,” the proposed method is designed by first dividing an airplane target into four visual parts: head, left/right wings, body, and tail, and then considering the detection as the prediction of the individual key points in each of the visual parts. We further introduce an additional attention branch in the standard detection pipeline to enhance the features and make the model focus on individual parts of a target even if it is only partially visible in the image. Detection results and ablation studies on three remote sensing target detection data sets (including two publicly available ones) demonstrate the effectiveness of our method, especially for occluded airplane targets. In addition, our method outperforms the other state-of-the-art detection methods on these data sets.

A multiscale object detection approach for remote sensing images based on MSE-DenseNet and the dynamic anchor assignment

ABSTRACTObject detection has played an important role in the remote sensing (RS) image understanding domain, but it still suffers from challenges, including the complexity of the image background, the variety of objects and differences of target scales. Although many studies applying deep learning methods to object detection have been presented, effective methods of multiscale object detection in RS images are still lacking. In this paper, we propose a novel object detection approach based on the Multiscale-SELU-DenseNet (MSE-DenseNet) and the dynamic anchor assignment (DAA) strategy. We fuse a highly modified DenseNet and a feature pyramid network into the MSE-DenseNet to extract multi-level feature descriptions of RS images, and then we use them to detect objects on various scales. To further improve the detection performance, we propose a DAA strategy for assigning prior anchor boxes. The experimental results show that our proposed method outperforms state-of-the-art detection methods on the VHR-10 dataset and maintains a real-time detection speed.

A2RMNet: Adaptively Aspect Ratio Multi-Scale Network for Object Detection in Remote Sensing Images

Object detection is a significant and challenging problem in the study area of remote sensing and image analysis. However, most existing methods are easy to miss or incorrectly locate objects due to the various sizes and aspect ratios of objects. In this paper, we propose a novel end-to-end Adaptively Aspect Ratio Multi-Scale Network (A 2 RMNet) to solve this problem. On the one hand, we design a multi-scale feature gate fusion network to adaptively integrate the multi-scale features of objects. This network is composed of gate fusion modules, refine blocks and region proposal networks. On the other hand, an aspect ratio attention network is leveraged to preserve the aspect ratios of objects, which alleviates the excessive shape distortions of objects caused by aspect ratio changes during training. Experiments show that the proposed A 2 RMNet significantly outperforms the previous state of the arts on the DOTA dataset, NWPU VHR-10 dataset, RSOD dataset and UCAS-AOD dataset by 5.73 % , 7.06 % , 3.27 % and 2.24 % , respectively.

Effective Airplane Detection in Remote Sensing Images Based on Multilayer Feature Fusion and Improved Nonmaximal Suppression Algorithm

Aiming at the problem of insufficient representation ability of weak and small objects and overlapping detection boxes in airplane object detection, an effective airplane detection method in remote sensing images based on multilayer feature fusion and an improved nonmaximal suppression algorithm is proposed. Firstly, based on the common low-level visual features of natural images and airport remote sensing images, region-based convolutional neural networks are chosen to conduct transfer learning for airplane images using a limited amount of data. Then, the L2 norm normalization, feature connection, scale scaling, and feature dimension reduction are introduced to achieve effective fusion of low- and high-level features. Finally, a nonmaximal suppression method based on a soft decision function is proposed to solve the overlap problem of detection boxes. The experimental results show that the proposed method can effectively improve the representation ability of weak and small objects, as well as quickly and accurately detect airplane objects in the airport area.

Small Aircraft Detection in Remote Sensing Images Based on YOLOv3

Accurate and rapid detection in remote sensing images is of great significance in military navigation, environmental monitoring, and civil applications. Due to the small object detection problem of remote sensing images, higher requirements and challenges are put forward for object detection technology. In recent years, the one-stage object detector and the two-stage object detector based on convolutional neural network have made great achievements in the field of image classification and detection. The one-stage object detector is generally superior to the two-stage object detector in detection speed, but the detection accuracy is inferior to the two-stage target detector. In this paper, YOLOv3 is used as a one- stage target detector for small aircraft detection of remote sensing images. We chose appropriate anchors to cover the size distribution in our experimental data by using dimension clusters. The experimental results show that YOLOv3 not only exceeds the conventional one-stage object detector in speed, but also well match the accuracy of the two-stage object detector. The YOLOv3 achieved excellent detection accuracy and low processing time at the same time for small aircraft detection.

Change Detection in Remote Sensing Images Based on Image Mapping and a Deep Capsule Network

Homogeneous image change detection research has been well developed, and many methods have been proposed. However, change detection between heterogeneous images is challenging since heterogeneous images are in different domains. Therefore, direct heterogeneous image comparison in the way that we do it is difficult. In this paper, a method for heterogeneous synthetic aperture radar (SAR) image and optical image change detection is proposed, which is based on a pixel-level mapping method and a capsule network with a deep structure. The mapping method proposed transforms an image from one feature space to another feature space. Then, the images can be compared directly in a similarly transformed space. In the mapping process, some image blocks in unchanged areas are selected, and these blocks are only a small part of the image. Then, the weighted parameters are acquired by calculating the Euclidean distances between the pixel to be transformed and the pixels in these blocks. The Euclidean distance calculated according to the weighted coordinates is taken as the pixel gray value in another feature space. The other image is transformed in a similar manner. In the transformed feature space, these images are compared, and the fusion of the two different images is achieved. The two experimental images are input to a capsule network, which has a deep structure. The image fusion result is taken as the training labels. The training samples are selected according to the ratio of the center pixel label and its neighboring pixels’ labels. The capsule network can improve the detection result and suppress noise. Experiments on remote sensing datasets show the final detection results, and the proposed method obtains a satisfactory performance.

Object Detection in Remote Sensing Images Based on a Scene-Contextual Feature Pyramid Network

Object detection has attracted increasing attention in the field of remote sensing image analysis. Complex backgrounds, vertical views, and variations in target kind and size in remote sensing images make object detection a challenging task. In this work, considering that the types of objects are often closely related to the scene in which they are located, we propose a convolutional neural network (CNN) by combining scene-contextual information for object detection. Specifically, we put forward the scene-contextual feature pyramid network (SCFPN), which aims to strengthen the relationship between the target and the scene and solve problems resulting from variations in target size. Additionally, to improve the capability of feature extraction, the network is constructed by repeating a building aggregated residual block. This block increases the receptive field, which can extract richer information for targets and achieve excellent performance with respect to small object detection. Moreover, to improve the proposed model performance, we use group normalization, which divides the channels into groups and computes the mean and variance for normalization within each group, to solve the limitation of the batch normalization. The proposed method is validated on a public and challenging dataset. The experimental results demonstrate that our proposed method outperforms other state-of-the-art object detection models.

End-to-End Airport Detection in Remote Sensing Images Combining Cascade Region Proposal Networks and Multi-Threshold Detection Networks

Fast and accurate airport detection in remote sensing images is important for many military and civilian applications. However, traditional airport detection methods have low detection rates, high false alarm rates and slow speeds. Due to the power convolutional neural networks in object-detection systems, an end-to-end airport detection method based on convolutional neural networks is proposed in this study. First, based on the common low-level visual features of natural images and airport remote sensing images, region-based convolutional neural networks are chosen to conduct transfer learning for airport images using a limited amount of data. Second, to further improve the detection rate and reduce the false alarm rate, the concepts of “divide and conquer” and “integral loss’’ are introduced to establish cascade region proposal networks and multi-threshold detection networks, respectively. Third, hard example mining is used to improve the object discrimination ability and the training efficiency of the network during sample training. Additionally, a cross-optimization strategy is employed to achieve convolution layer sharing between the cascade region proposal networks and the subsequent multi-threshold detection networks, and this approach significantly decreases the detection time. The results show that the method established in this study can accurately detect various types of airports in complex backgrounds with a higher detection rate, lower false alarm rate, and shorter detection time than existing airport detection methods.

Rapid Airplane Detection in Remote Sensing Images Based on Multilayer Feature Fusion in Fully Convolutional Neural Networks.

To address the issues encountered when using traditional airplane detection methods, including the low accuracy rate, high false alarm rate, and low detection speed due to small object sizes in aerial remote sensing images, we propose a remote sensing image airplane detection method that uses multilayer feature fusion in fully convolutional neural networks. The shallow layer and deep layer features are fused at the same scale after sampling to overcome the problems of low dimensionality in the deep layer and the inadequate expression of small objects. The sizes of candidate regions are modified to fit the size of the actual airplanes in the remote sensing images. The fully connected layers are replaced with convolutional layers to reduce the network parameters and adapt to different input image sizes. The region proposal network shares convolutional layers with the detection network, which ensures high detection efficiency. The simulation results indicate that, when compared to typical airplane detection methods, the proposed method is more accurate and has a lower false alarm rate. Additionally, the detection speed is considerably faster and the method can accurately and rapidly complete airplane detection tasks in aerial remote sensing images.

Automatic Ship Detection in Remote Sensing Images from Google Earth of Complex Scenes Based on Multiscale Rotation Dense Feature Pyramid Networks

Ship detection has been playing a significant role in the field of remote sensing for a long time, but it is still full of challenges. The main limitations of traditional ship detection methods usually lie in the complexity of application scenarios, the difficulty of intensive object detection, and the redundancy of the detection region. In order to solve these problems above, we propose a framework called Rotation Dense Feature Pyramid Networks (R-DFPN) which can effectively detect ships in different scenes including ocean and port. Specifically, we put forward the Dense Feature Pyramid Network (DFPN), which is aimed at solving problems resulting from the narrow width of the ship. Compared with previous multiscale detectors such as Feature Pyramid Network (FPN), DFPN builds high-level semantic feature-maps for all scales by means of dense connections, through which feature propagation is enhanced and feature reuse is encouraged. Additionally, in the case of ship rotation and dense arrangement, we design a rotation anchor strategy to predict the minimum circumscribed rectangle of the object so as to reduce the redundant detection region and improve the recall. Furthermore, we also propose multiscale region of interest (ROI) Align for the purpose of maintaining the completeness of the semantic and spatial information. Experiments based on remote sensing images from Google Earth for ship detection show that our detection method based on R-DFPN representation has state-of-the-art performance.

A new difference image creation method based on deep neural networks for change detection in remote-sensing images

ABSTRACTIn this article, we propose a novel difference image (DI) creation method for unsupervised change detection in multi-temporal multi-spectral remote-sensing images based on deep learning theory. First, we apply deep belief network to learn local and high-level features from the local neighbour of a given pixel in an unsupervised manner. Second, a back propagation algorithm is improved to build a DI based on selected training samples, which can highlight the difference on changed regions and suppress the false changes on unchanged regions. Finally, we get the change trajectory map using simple clustering analysis. The proposed scheme is tested on three remote-sensing data sets. Qualitative and quantitative evaluations show its superior performance compared to the traditional pixel-level and texture-level-based approaches.

Aircraft Recognition Based on Landmark Detection in Remote Sensing Images

Aircraft type recognition of remote sensing images is critical both in civil and military applications. In this letter, we propose a novel landmark-based aircraft recognition method which is highly accurate and efficient. First, we propose a new idea to address the aircraft type recognition problem by aircraft’s landmark detection. Its advantages are two folds. On the one hand, it needs fewer labeled data and alleviates the work of human annotation. On the other hand, a trained model has strong expansibility because it can be used for any type of aircraft that not contained in training data set without retraining. Then, we use a variant of a convolutional neural network called vanilla network for all landmarks regression at the same time. Therefore, it can avoid bad local minimum effectively by encoding the geometric constraints among landmarks implicitly. To handle aircrafts in myriads of poses, rotation jittering is used for data augmentation in preprocessing and multicrop fusion is used in postprocessing. Thus, an 80% reduction in error rate could be reached. Finally, we use the landmark template matching to recognize the aircraft. Our method shows a competitive performance both in accuracy and efficiency.

Multilevel Cloud Detection in Remote Sensing Images Based on Deep Learning

Cloud detection is one of the important tasks for remote sensing image processing. In this paper, a novel multilevel cloud detection method based on deep learning is proposed for remote sensing images. First, the simple linear iterative clustering (SLIC) method is improved to segment the image into good quality superpixels. Then, a deep convolutional neural network (CNN) with two branches is designed to extract the multiscale features from each superpixel and predict the superpixel as one of three classes including thick cloud, thin cloud, and noncloud. Finally, the predictions of all the superpixels in the image yield the cloud detection result. In the proposed cloud detection framework, the improved SLIC method can obtain accurate cloud boundaries by optimizing initial cluster centers, designing dynamic distance measure, and expanding search space. Moreover, different from traditional cloud detection methods that cannot achieve multilevel detection of cloud, the designed deep CNN model can not only detect cloud but also distinguish thin cloud from thick cloud. Experimental results indicate that the proposed method can detect cloud with higher accuracy and robustness than compared methods.

Aircraft Detection in Remote Sensing Images via CNN Multi-scale Feature Representation

Aircraft detection in remote sensing images is an intractable challenge. The current aircraft detection methods have limited representative capabilities and heavy computational costs. This paper studies how to apply multi-scale feature representation of Convolutional Neural Networks (CNN) to aircraft detection by qualitatively and quantitatively analyzing the performance of Single Shot Detection (SSD) approach. At first, we find that low-level detectors are not robust enough to detect as the semantic gap issue. Therefore we propose a data driven hyper-parameter selection method to alleviate this problem by determining appropriate hyper-parameters of sliding window and default box shape. Besides, we employ a multi-scale training strategy to enhance low-level predictive detectors. Finally, we propose an accurate and efficient aircraft detection framework. Experimental results illustrate that our method could achieve 96.84% AP at 20 FPS on NVIDIA TITAN X. Compared with original SSD method, our proposed approach achived 2.13% AP improvement

Inshore Ship Detection in Remote Sensing Images via Weighted Pose Voting

Inshore ship detection from high-resolution satellite images is a useful yet challenging task in remote surveillance and military reconnaissance. It is difficult to detect the inshore ships with high precision because various interferences are present in the harbor scene. An inshore ship detection method based on the weighted voting and rotation–scale-invariant pose is proposed to improve the detection performance. The proposed method defines the rotation angle pose and the scaling factor of the detected ship to detect the ship with different directions and different sizes. For each pixel on the ship template, the possible poses of a detection window are estimated according to all possible pose-related pixels. To improve robustness to the shape-similar distractor and various interferences, the score of the detection window is obtained by designing a pose weighted voting method. Moreover, the values of some parameters such as similarity threshold and the weight of “V” are investigated. The experimental results on actual satellite images demonstrate that the proposed method is invariant to rotation and scale and robust in the inshore ship detection. In addition, better detection performance is observed in comparison with the existing inshore ship detection algorithms in terms of precision rate and recall rate. The target pose of the detected ship can also be obtained as a byproduct of the ship detection.

An Effective Method Based on ACF for Aircraft Detection in Remote Sensing Images

Detecting artificial targets, such as aircraft, in satellite images is significant in military and civil applications. Although the performance has improved with the use of more complicated features and better learning methods, effectively handling aircraft with variations of type, pose, and size is still very challenging. To solve this problem, we propose a multiscale sliding-window framework based on aggregate channel features, well-designed features that contain rich information. We also employ a fast feature pyramids algorithm to accelerate multiscale aircraft detection. In this framework, features are trained by Cascade AdaBoost including multiple rounds of bootstrapping that leads to improved overall accuracy. A two-step nonmaximum suppression algorithm is carefully designed based on a given set of detections. Our method shows a competitive performance on the QuickBird images of 0.6 m resolution.

Different colours of shadows: classification of UAV images

ABSTRACTDue to their low light conditions, shadows reduce the accuracy of feature extraction and change detection in remote-sensing images. Unmanned aerial vehicles (UAVs) are capable of acquiring images that have a resolution of several centimetres and removing shadows is a challenge. In this study, the Maximum Likelihood (ML) and Support Vector Machine (SVM) classifiers were used to classify a UAV image acquired using a red–green–blue (RGB) camera over the Old Woman Creek National Estuarine Research Reserve in Ohio, USA. The impact of shadows on the classification process was explored for different pixel sizes ranging from 0.03 to 1.00 m. The SVM generated higher overall accuracy (OA) at finer spatial resolution (0.25–0.50 m), while the optimal spatial resolution for the ML classifier was 1.00 m. The percentage of shadow coverage increased with spatial resolution for both classifiers (1.71% for ML and 6.63% for SVM). Shadows were detected and extracted using two approaches: (a) as a separate class using regions of interests (ROIs) observed in the image, and (b) by applying a segmentation threshold of 0.3 to visible atmospherically resistant index (VARI). The extracted shadows were separately classified using ROIs selected from shaded surfaces, and then removed using the fusion of RGB reflectance, VARI, and digital surface model (DSM) images. The OA of classified shadows reached 91.50%. OAs of merged sunlit and shadow classified images improved for 18.48% for SVM, and 17.62% for the ML classifier. VARI accurately captures shadows, and when fused with RGB reflectance and DSM, it intensifies their low signal and enhances classification. Whether used to capture or to remove shadows, VARI serves as an effective ‘shadow index’. Shadows create obstacles to remote-sensing processing; however, their spectral information should not be neglected as both shadows and sunlit areas are important for ecological processes such as photosynthesis, carbon balance, evapotranspiration, fish abundance, and more.

Dynamic Evidential Reasoning for Change Detection in Remote Sensing Images

Theories of evidence have already been applied more or less successfully in the fusion of remote sensing images. These attempts were based on the classical evidential reasoning which works under the condition that all sources of evidence and their fusion results are related to the same invariable (static) frame of discernment.

Weakly supervised target detection in remote sensing images based on transferred deep features and negative bootstrapping

Target detection in remote sensing images (RSIs) is a fundamental yet challenging problem faced for remote sensing images analysis. More recently, weakly supervised learning, in which training sets require only binary labels indicating whether an image contains the object or not, has attracted considerable attention owing to its obvious advantages such as alleviating the tedious and time consuming work of human annotation. Inspired by its impressive success in computer vision field, in this paper, we propose a novel and effective framework for weakly supervised target detection in RSIs based on transferred deep features and negative bootstrapping. On one hand, to effectively mine information from RSIs and improve the performance of target detection, we develop a transferred deep model to extract high-level features from RSIs, which can be achieved by pre-training a convolutional neural network model on a large-scale annotated dataset (e.g. ImageNet) and then transferring it to our task by domain-specifically fine-tuning it on RSI datasets. On the other hand, we integrate negative bootstrapping scheme into detector training process to make the detector converge more stably and faster by exploiting the most discriminative training samples. Comprehensive evaluations on three RSI datasets and comparisons with state-of-the-art weakly supervised target detection approaches demonstrate the effectiveness and superiority of the proposed method.

Accurate Urban Area Detection in Remote Sensing Images

Automatic urban area detection in remote sensing images is an important application in the field of earth observation. Most of the existing methods employ feature classifiers and thereby contain a data training process. Moreover, some methods cannot detect urban areas in complex scenes accurately. This letter proposes an automatic urban area detection method that uses multiple features that have different resolutions. First, a downsampled low-resolution image is used to segment the candidate area. After the corner points of the urban area are extracted, a weighted Gaussian voting matrix technique is employed to integrate the corner points into the candidate area. Then, the edge features and homogeneous region are extracted by using the original high-resolution image. Using these results as the input, the processes of guided filtering and contrast enhancement can finally detect accurately the urban areas. This method combines multiple features, such as corner, edge, and regional characteristics, to detect the urban areas. The experimental results show that the proposed method has better detection accuracy for urban areas than the existing algorithms.