Very High Resolution Aerial Images Research Articles

Contextual Learning in Fourier Complex Field for VHR Remote Sensing Images.

Very high-resolution (VHR) remote sensing (RS) image classification is the fundamental task for RS image analysis and understanding. Recently, Transformer-based models demonstrated outstanding potential for learning high-order contextual relationships from natural images with general resolution ( ≈ 224 × 224 pixels) and achieved remarkable results on general image classification tasks. However, the complexity of the naive Transformer grows quadratically with the increase in image size, which prevents Transformer-based models from VHR RS image ( ≥ 500 × 500 pixels) classification and other computationally expensive downstream tasks. To this end, we propose to decompose the expensive self-attention (SA) into real and imaginary parts via discrete Fourier transform (DFT) and, therefore, propose an efficient complex SA (CSA) mechanism. Benefiting from the conjugated symmetric property of DFT, CSA is capable to model the high-order contextual information with less than half computations of naive SA. To overcome the gradient explosion in Fourier complex field, we replace the Softmax function with the carefully designed Logmax function to normalize the attention map of CSA and stabilize the gradient propagation. By stacking various layers of CSA blocks, we propose the Fourier complex Transformer (FCT) model to learn global contextual information from VHR aerial images following the hierarchical manners. Universal experiments conducted on commonly used RS classification datasets demonstrate the effectiveness and efficiency of FCT, especially on VHR RS images. The source code of FCT will be available at https://github.com/Gao-xiyuan/FCT.

Coupled Global–Local object detection for large VHR aerial images

Object detection in large aerial images generally requires splitting each image into local images in the preprocessing step, and even the state-of-the-art models currently use this preprocessing method. However, image splitting often leads to deficiencies in contextual information and incomplete detection of oversized objects. At present, many object detection methods are designed to deal with large images. However, they require complex additional structures or training steps, and their applicability is limited. To address these problems, we propose the Coupled Global–Local (CGL) network, which can be easily embedded in frequently used detection models, to efficiently capture more information. Specifically, we employ a multiscale feature fusion module to share information between the global and local branches. Furthermore, a new convolution method is proposed to adaptively adjust the receptive field for better feature extraction. In addition, we find that detection results from global branches in the existing global–local architecture hinder the performance improvement on details when the detection results from different-resolution branches are fused. Therefore, on the global branch, a proposal filter and a nonlocal suppression (NLS) algorithm are developed to prevent small positive proposals and remove unqualified detection boxes easily and efficiently, respectively. We conduct extensive experiments on the DOTA-1.0, DOTA-1.5, and DOTA-2.0 data sets. The results demonstrate that CGL can significantly improve the detection performance of various baseline models for large very-high-resolution (VHR) aerial images without bells and whistles.

Deep Multimodal Fusion Network for Semantic Segmentation Using Remote Sensing Image and LiDAR Data

Extracting semantic information from very-high-resolution (VHR) aerial images is a prominent topic in the Earth observation research. An increasing number of different sensor platforms are appearing in remote sensing, each of which can provide corresponding multimodal supplemental or enhanced information, such as optical images, light detection and ranging (LiDAR) point clouds, infrared images, or inertial measurement unit (IMU) data. However, these current deep networks for LiDAR and VHR images have not fully utilized the complete potential of multimodal data. The stacked multimodal fusion network (MFNet) ignores the structural differences between the modalities and the manual statistical characteristics within the modalities. For multimodal remote sensing data and its corresponding carefully designed handcrafted features, we designed a novel deep MFNet that can use multimodal VHR aerial images and LiDAR data and the corresponding intramodal features, such as LiDAR-derived features [slope and normalized digital surface model (NDSM)] and imagery-derived features [infrared–red–green (IRRG), normalized difference vegetation index (NDVI), and difference of Gaussian (DoG)]. Technically, we introduce the attention mechanism and multimodal learning to adaptively fuse intermodal and intramodal features. Specifically, we designed a multimodal fusion mechanism, pyramid dilation blocks, and a multilevel feature fusion module. Through these modules, our network realized the adaptive fusion of multimodal features, improved the receptive field, and enhanced the global-to-local contextual fusion effect. Moreover, we used a multiscale supervision training scheme to optimize the network. Extensive experimental results and ablation studies on the ISPRS semantic dataset and IEEE GRSS DFC Zeebrugge dataset show the effectiveness of our proposed MFNet.

Road network extraction using multi-layered filtering and tensor voting from aerial images

Road network extraction from high-resolution aerial images is a predominant research area in remote sensing due to road network applications in various applications like transportation and industrialization disaster management. In this work, an integrated method is proposed to delineate a smooth and accurate road centerline from very high resolution (VHR) aerial images. The proposed approach incorporates Gabor filtering, hysteresis thresholding, road filtering using shape features, and tensor voting (TV). The initial road map is generated by detecting the road features that lie on the edges in the starting phase, using Gabor filter and hysteresis thresholding. In the next phase, the initial road map's exactitude is enhanced by removing non-road components from the initial road network using filtering based on shape features and morphological operations. Further, the Euclidean distance transform is applied for the extraction of the road centerline. The centerlines extracted by the method are broken at some places due to occlusions caused by different factors. Finally, the fractured road network is reconstructed by the TV technique. The experimental results tested on VHR aerial images demonstrate the accurate centerline extraction of high quality and higher assessable results as compared with contemporary methods.

Estimating Traffic in Urban Areas from Very-High Resolution Aerial Images

ABSTRACT Traffic estimation from very-high-resolution remote-sensing imagery has received increasing interest during the last few years. In this article, we propose an automatic system for estimation of the annual average daily traffic (AADT) using very-high-resolution optical remote-sensing imagery of urban areas in combination with high-quality, but very spatially limited, ground-based measurements. The main part of the system is the vehicle detection, which is based on the deep learning object detection architecture mask region-based convolutional neural network (Mask R-CNN), modified with an image normalization strategy to make it more robust for test images of various conditions and the use of a precise road mask to assist the filtering of driving vehicles from parked ones. Furthermore, to include the high-quality ground-based measurements and to make the traffic estimates more consistent across neighbouring road links, we propose a graph smoothing strategy that utilizes the road network. The fully automatic processing chain has been validated on a set of aerial images covering the city of Narvik, Norway. The precision and recall rate of detecting driving vehicles was 0.74 and 0.66, respectively, and the AADT was estimated with a root mean squared error (RMSE) of 2279 and bias of −383. We conclude that separating driving vehicles from parked ones may be challenging if vehicles are parked along the roads and that for urban environment with short road links several remote-sensing images covering the road links at different time instances are necessary in order to benefit from the remote-sensing images.

Multi-Scale Feature Integrated Attention-Based Rotation Network for Object Detection in VHR Aerial Images.

Accurate and robust detection of multi-class objects in very high resolution (VHR) aerial images has been playing a significant role in many real-world applications. The traditional detection methods have made remarkable progresses with horizontal bounding boxes (HBBs) due to CNNs. However, HBB detection methods still exhibit limitations including the missed detection and the redundant detection regions, especially for densely-distributed and strip-like objects. Besides, large scale variations and diverse background also bring in many challenges. Aiming to address these problems, an effective region-based object detection framework named Multi-scale Feature Integration Attention Rotation Network (MFIAR-Net) is proposed for aerial images with oriented bounding boxes (OBBs), which promotes the integration of the inherent multi-scale pyramid features to generate a discriminative feature map. Meanwhile, the double-path feature attention network supervised by the mask information of ground truth is introduced to guide the network to focus on object regions and suppress the irrelevant noise. To boost the rotation regression and classification performance, we present a robust Rotation Detection Network, which can generate efficient OBB representation. Extensive experiments and comprehensive evaluations on two publicly available datasets demonstrate the effectiveness of the proposed framework.

Object Detection in Very High-Resolution Aerial Images Using One-Stage Densely Connected Feature Pyramid Network.

Object detection in very high-resolution (VHR) aerial images is an essential step for a wide range of applications such as military applications, urban planning, and environmental management. Still, it is a challenging task due to the different scales and appearances of the objects. On the other hand, object detection task in VHR aerial images has improved remarkably in recent years due to the achieved advances in convolution neural networks (CNN). Most of the proposed methods depend on a two-stage approach, namely: a region proposal stage and a classification stage such as Faster R-CNN. Even though two-stage approaches outperform the traditional methods, their optimization is not easy and they are not suitable for real-time applications. In this paper, a uniform one-stage model for object detection in VHR aerial images has been proposed. In order to tackle the challenge of different scales, a densely connected feature pyramid network has been proposed by which high-level multi-scale semantic feature maps with high-quality information are prepared for object detection. This work has been evaluated on two publicly available datasets and outperformed the current state-of-the-art results on both in terms of mean average precision (mAP) and computation time.

Road Centerline Extraction from Very-High-Resolution Aerial Image and LiDAR Data Based on Road Connectivity

The road networks provide key information for a broad range of applications such as urban planning, urban management, and navigation. The fast-developing technology of remote sensing that acquires high-resolution observational data of the land surface offers opportunities for automatic extraction of road networks. However, the road networks extracted from remote sensing images are likely affected by shadows and trees, making the road map irregular and inaccurate. This research aims to improve the extraction of road centerlines using both very-high-resolution (VHR) aerial images and light detection and ranging (LiDAR) by accounting for road connectivity. The proposed method first applies the fractal net evolution approach (FNEA) to segment remote sensing images into image objects and then classifies image objects using the machine learning classifier, random forest. A post-processing approach based on the minimum area bounding rectangle (MABR) is proposed and a structure feature index is adopted to obtain the complete road networks. Finally, a multistep approach, that is, morphology thinning, Harris corner detection, and least square fitting (MHL) approach, is designed to accurately extract the road centerlines from the complex road networks. The proposed method is applied to three datasets, including the New York dataset obtained from the object identification dataset, the Vaihingen dataset obtained from the International Society for Photogrammetry and Remote Sensing (ISPRS) 2D semantic labelling benchmark and Guangzhou dataset. Compared with two state-of-the-art methods, the proposed method can obtain the highest completeness, correctness, and quality for the three datasets. The experiment results show that the proposed method is an efficient solution for extracting road centerlines in complex scenes from VHR aerial images and light detection and ranging (LiDAR) data.

Semiautomatic right-angle building extraction from very high-resolution aerial images using graph cuts with star shape constraint and regularization

This paper proposes a method of semiautomatic right-angle building extraction from very high resolution (VHR) aerial images, based on graph cuts with star shape constraint and regularization. In the proposed method, first, the image block containing the target building is obtained by a seed line from humans. Next, the image block is preprocessed by bilateral filtering and then the simple linear iterative clustering oversegmentation method was used to segment the image into small regions. Then regions that contain the pixels belonging to the seed line are merged as the foreground area, and the background area is automatically composed by annular regions near the imagery boundary. With the obtained foreground and background areas, the graph cuts model with the star shape prior is used to acquire the building object. Finally, we defined a method of detecting the building orientation and proposed a regularizing building method based on the corners and the building orientation. The experiments performed on two VHR aerial images demonstrate the accuracy and stability of the proposed method.

Dynamic Post-Earthquake Image Segmentation with an Adaptive Spectral-Spatial Descriptor

The region merging algorithm is a widely used segmentation technique for very high resolution (VHR) remote sensing images. However, the segmentation of post-earthquake VHR images is more difficult due to the complexity of these images, especially high intra-class and low inter-class variability among damage objects. Herein two key issues must be resolved: the first is to find an appropriate descriptor to measure the similarity of two adjacent regions since they exhibit high complexity among the diverse damage objects, such as landslides, debris flow, and collapsed buildings. The other is how to solve over-segmentation and under-segmentation problems, which are commonly encountered with conventional merging strategies due to their strong dependence on local information. To tackle these two issues, an adaptive dynamic region merging approach (ADRM) is introduced, which combines an adaptive spectral-spatial descriptor and a dynamic merging strategy to adapt to the changes of merging regions for successfully detecting objects scattered globally in a post-earthquake image. In the new descriptor, the spectral similarity and spatial similarity of any two adjacent regions are automatically combined to measure their similarity. Accordingly, the new descriptor offers adaptive semantic descriptions for geo-objects and thus is capable of characterizing different damage objects. Besides, in the dynamic region merging strategy, the adaptive spectral-spatial descriptor is embedded in the defined testing order and combined with graph models to construct a dynamic merging strategy. The new strategy can find the global optimal merging order and ensures that the most similar regions are merged at first. With combination of the two strategies, ADRM can identify spatially scattered objects and alleviates the phenomenon of over-segmentation and under-segmentation. The performance of ADRM has been evaluated by comparing with four state-of-the-art segmentation methods, including the fractal net evolution approach (FNEA, as implemented in the eCognition software, Trimble Inc., Westminster, CO, USA), the J-value segmentation (JSEG) method, the graph-based segmentation (GSEG) method, and the statistical region merging (SRM) approach. The experiments were conducted on six VHR subarea images captured by RGB sensors mounted on aerial platforms, which were acquired after the 2008 Wenchuan Ms 8.0 earthquake. Quantitative and qualitative assessments demonstrated that the proposed method offers high feasibility and improved accuracy in the segmentation of post-earthquake VHR aerial images.

COMPARISON OF UNSUPERVISED VEGETATION CLASSIFICATION METHODS FROM VHR IMAGES AFTER SHADOWS REMOVAL BY INNOVATIVE ALGORITHMS

Abstract. The recognition of vegetation by the analysis of very high resolution (VHR) aerial images provides meaningful information about environmental features; nevertheless, VHR images frequently contain shadows that generate significant problems for the classification of the image components and for the extraction of the needed information. The aim of this research is to classify, from VHR aerial images, vegetation involved in the balance process of the environmental biochemical cycle, and to discriminate it with respect to urban and agricultural features. Three classification algorithms have been experimented in order to better recognize vegetation, and compared to NDVI index; unfortunately all these methods are conditioned by the presence of shadows on the images. Literature presents several algorithms to detect and remove shadows in the scene: most of them are based on the RGB to HSI transformations. In this work some of them have been implemented and compared with one based on RGB bands. Successively, in order to remove shadows and restore brightness on the images, some innovative algorithms, based on Procrustes theory, have been implemented and applied. Among these, we evaluate the capability of the so called “not-centered oblique Procrustes” and “anisotropic Procrustes” methods to efficiently restore brightness with respect to a linear correlation correction based on the Cholesky decomposition. Some experimental results obtained by different classification methods after shadows removal carried out with the innovative algorithms are presented and discussed.

Image-based mapping of surface fissures for the investigation of landslide dynamics

The development of surface fissures is an important indicator for understanding and forecasting slope movements. Landslide investigations therefore frequently include the elaboration and interpretation of maps representing their spatial distribution, typically comprising intensive field work and instrumentation. It is only recently that aerial photography with sub-decimetre spatial resolution is becoming more commonly available and opens a window to analyse such features from a remote sensing perspective. While these data are in principle helpful to elaborate maps from image interpretation techniques, there is still no image processing technique available to extract efficiently these geomorphological features. This work proposes a largely automated technique for the mapping of landslide surface fissures from very-high resolution aerial images. The processing chain includes the use of filtering algorithms and post-processing of the filtered images using object-oriented analysis. The accuracy of the resulting maps is assessed by comparisons with several expert maps in terms of affected area, fissure density and fissure orientation. Under homogenous illumination conditions, true positive rates up to 65% and false positive rates generally below 10% are achieved. The resulting fissure maps provide sufficient detail to infer mechanical processes at the slope scale and to prioritize areas for more detailed ground investigations or monitoring.

A Marked Point Process Model Including Strong Prior Shape Information Applied to Multiple Object Extraction From Images

Object extraction from images is one of the most important tasks in remote sensing image analysis. For accurate extraction from very high resolution (VHR) images, object geometry needs to be taken into account. A method for incorporating strong yet flexible prior shape information into a marked point process model for the extraction of multiple objects of complex shape is presented. To control the computational complexity, the objects considered are defined using the image data and the prior shape information. To estimate the optimal configuration of objects, the process is sampled using a Markov chain based on a stochastic birth-and-death process on the space of multiple objects. The authors present several experimental results on the extraction of tree crowns from VHR aerial images.