Geospatial Objects Research Articles

Efficient Semantic Enrichment Process for Spatiotemporal Trajectories

The increasing availability of location‐acquisition technologies has enabled collecting large‐scale spatiotemporal trajectories, from which we can derive semantic information in urban environments, including location, time, direction, speed, and point of interest. Such semantic information can give us a semantic interpretation of movement behaviors of moving objects. However, existing semantic enrichment process approaches, which can produce semantic trajectories, are generally time‐consuming. In this paper, we propose an efficient semantic enrichment process framework to annotate spatiotemporal trajectories by using geographic and application domain knowledge. The framework mainly includes preannotated semantic trajectory storage phase, spatiotemporal similarity measurement phase, and semantic information matching phase. Having observed the common trajectories in the same geospatial object scenes, we propose a semantic information matching algorithm to match semantic information in preannotated semantic trajectories to new spatiotemporal trajectories. In order to improve the efficiency of this approach, we build a spatial index to enhance the preannotated semantic trajectories. Finally, the experimental results based on a real dataset demonstrate the effectiveness and efficiency of our proposed approaches.

FRPNet: A Feature-Reflowing Pyramid Network for Object Detection of Remote Sensing Images

As a significant and fundamental task in the remote sensing field, object detection has received increasing attention and research studies. However, geospatial object detection is still a challenge owing to the dramatic variation in object scales, intraclass differences, and interclass similarity from multiscale and multiclass objects. To deal with these problems, an end-to-end feature-reflowing pyramid network (FRPNet) is proposed in this letter. FRPNet has two advantages that contribute to improve object detection accuracy. First, we embed a nonlocal block into the backbone in order to get the relevancy between different regions of the geospatial image for obtaining discriminative features. Furthermore, a feature-reflowing pyramid structure is proposed to generate high-quality feature presentation for each scale through fusing fine-grained features from the adjacent lower level, which improves the detection capability for multiscale and multiclass objects. Experiments on a public remote sensing data set DIOR illustrate that FRPNet can significantly improve the performance when compared to several state-of-the-art detection approaches in terms of mean average precision (mAP).

ANALYSIS OF METHODS OF UNIQUE IDENTIFICATION OF OBJECTS IN GEOSPATIAL DATA SETS

The subject of analysis is methods and means of unique identification of objects in geospatial datasets, created, registered and used in geographic information systems for various purposes and territorial coverage. Systems for unique identification of geospatial objects are classified according to basic technologies and the presence of semantic content in the composition of unique identifiers, in particular, these systems are based on the use of: 1) universal unique identifiers such as UUID; 2) direct geocoding methods using object coordinates; 3) indirect geocoding methods using geographic names and addresses. The analysis of the advantages and limitations of each of the three groups of systems for the unique identification of geospatial objects is carried out. The results of a computational experiment confirmed the possibility of reliable unique identification of buildings based on the use of an 11-digit open code location code (OLC, Google plus code). The global uniqueness of OLC-based identifiers for geospatial objects of any type requires additional research, but its uniqueness within objects of the same type is practically achievable due to the variable length of the code and the possibility of its extension with additional attributes.

AF-SSD: An Accurate and Fast Single Shot Detector for High Spatial Remote Sensing Imagery.

There are a large number of studies on geospatial object detection. However, many existing methods only focus on either accuracy or speed. Methods with both fast speed and high accuracy are of great importance in some scenes, like search and rescue, and military information acquisition. In remote sensing images, there are some targets that are small and have few textures and low contrast compared with the background, which impose challenges on object detection. In this paper, we propose an accurate and fast single shot detector (AF-SSD) for high spatial remote sensing imagery to solve these problems. Firstly, we design a lightweight backbone to reduce the number of trainable parameters of the network. In this lightweight backbone, we also use some wide and deep convolutional blocks to extract more semantic information and keep the high detection precision. Secondly, a novel encoding–decoding module is employed to detect small targets accurately. With up-sampling and summation operations, the encoding–decoding module can add strong high-level semantic information to low-level features. Thirdly, we design a cascade structure with spatial and channel attention modules for targets with low contrast (named low-contrast targets) and few textures (named few-texture targets). The spatial attention module can extract long-range features for few-texture targets. By weighting each channel of a feature map, the channel attention module can guide the network to concentrate on easily identifiable features for low-contrast and few-texture targets. The experimental results on the NWPU VHR-10 dataset show that our proposed AF-SSD achieves superior detection performance: parameters 5.7 M, mAP 88.7%, and 0.035 s per image on average on an NVIDIA GTX-1080Ti GPU.

CRPN-SFNet: A High-Performance Object Detector on Large-Scale Remote Sensing Images.

Limited by the GPU memory, the current mainstream detectors fail to directly apply to large-scale remote sensing images for object detection. Moreover, the scale range of objects in remote sensing images is much wider than that of general images, which also greatly hinders the existing methods to effectively detect geospatial objects of various scales. For achieving high-performance object detection on large-scale remote sensing images, this article proposes a much faster and more accurate detecting framework, called cropping region proposal network-based scale folding network (CRPN-SFNet). In our framework, the CRPN includes a weak semantic RPN for quickly locating interesting regions and a strategy of generating cropping regions to effectively filter out meaningless regions, which can greatly reduce the computation and storage burden. Meanwhile, the proposed SFNet leverages the scale folding-based training and testing methods to extend the valid detection range of existing detectors, which is beneficial for detecting remote sensing objects of various scales, including very small and very large geospatial objects. Extensive experiments on the public Dataset for Object deTection in Aerial images data set indicate that our CRPN can help our detector deal the larger image faster with the limited GPU memory; meanwhile, the SFNet is beneficial to achieve more accurate detection of geospatial objects with wide-scale range. For large-scale remote sensing images, the proposed detection framework outperforms the state-of-the-art object detection methods in terms of accuracy and speed.

Mask R-CNN for Geospatial Object Detection

Geospatial imaging technique has opened a door for researchers to implement multiple beneficial applications in many fields, including military investigation, disaster relief, and urban traffic control. As the resolution of geospatial images has increased in recent years, the detection of geospatial objects has attracted a lot of researchers. Mask R-CNN had been designed to identify an object outlines at the pixel level (instance segmentation), and for object detection in natural images. This study describes the Mask R-CNN model and uses it to detect objects in geospatial images. This experiment was prepared an existing dataset to be suitable with object segmentation, and it shows that Mask R-CNN also has the ability to be used in geospatial object detection and it introduces good results to extract the ten classes dataset of Seg-VHR-10.

Automatic classification of fine-scale mountain vegetation based on mountain altitudinal belt.

Vegetation mapping is of considerable significance to both geoscience and mountain ecology, and the improved resolution of remote sensing images makes it possible to map vegetation at a finer scale. While the automatic classification of vegetation has gradually become a research hotspot, real-time and rapid collection of samples has become a bottleneck. How to achieve fine-scale classification and automatic sample selection at the same time needs further study. Stratified sampling based on appropriate prior knowledge is an effective sampling method for geospatial objects. Therefore, based on the idea of stratified sampling, this paper used the following three steps to realize the automatic selection of representative samples and classification of fine-scale mountain vegetation: 1) using Mountain Altitudinal Belt (MAB) distribution information to stratify the study area into multiple vegetation belts; 2) selecting and correcting samples through iterative clustering at each belt automatically; 3) using RF (Random Forest) classifier with strong robustness to achieve automatic classification. The average sample accuracy of nine vegetation formations was 0.933, and the total accuracy of the classification result was 92.2%, with the kappa coefficient of 0.910. The results showed that this method could automatically select high-quality samples and obtain a high-accuracy vegetation map. Compared with the traditional vegetation mapping method, this method greatly improved the efficiency, which is of great significance for the fine-scale mountain vegetation mapping in large-scale areas.

Aerial Scene Classification through Fine-Tuning with Adaptive Learning Rates and Label Smoothing

Remote Sensing (RS) image classification has recently attracted great attention for its application in different tasks, including environmental monitoring, battlefield surveillance, and geospatial object detection. The best practices for these tasks often involve transfer learning from pre-trained Convolutional Neural Networks (CNNs). A common approach in the literature is employing CNNs for feature extraction, and subsequently train classifiers exploiting such features. In this paper, we propose the adoption of transfer learning by fine-tuning pre-trained CNNs for end-to-end aerial image classification. Our approach performs feature extraction from the fine-tuned neural networks and remote sensing image classification with a Support Vector Machine (SVM) model with linear and Radial Basis Function (RBF) kernels. To tune the learning rate hyperparameter, we employ a linear decay learning rate scheduler as well as cyclical learning rates. Moreover, in order to mitigate the overfitting problem of pre-trained models, we apply label smoothing regularization. For the fine-tuning and feature extraction process, we adopt the Inception-v3 and Xception inception-based CNNs, as well the residual-based networks ResNet50 and DenseNet121. We present extensive experiments on two real-world remote sensing image datasets: AID and NWPU-RESISC45. The results show that the proposed method exhibits classification accuracy of up to 98%, outperforming other state-of-the-art methods.

EDense: a convolutional neural network with ELM-based dense connections

The explosive growth of geospatial data is increasing requirements for automatic and efficient data learning abilities. Many deep learning methods have been widely applied for geospatial data understanding tasks, such as road networks and geospatial object detection. However, the demands for more accurate learning of high-level features require the use of deeper neural networks. To further improve the learning efficiency of deep neural networks, in this paper, we propose an improved convolutional neural network named EDense. First, we use its dense connectivity to integrate a CNN with an extreme learning machine. Then, we expand the kernels in the convolutional layers to increase the width of the network model. Furthermore, we propose one-feature EDense (OF-EDense), which is a simplified version of EDense, to fit conditions in which the number of parameters is strictly limited. Finally, the experimental results fully demonstrate the strong learning ability and high learning efficiency of EDense.

HyNet: Hyper-scale object detection network framework for multiple spatial resolution remote sensing imagery

Faced with the problem of the large scale variation, geospatial object detection in multiple spatial resolution (MSR) remote sensing imagery is a challenging task. To avoid the scale problem, the current convolutional neural network (CNN) based object detectors use multi-scale structures in the convolutional layer level to improve the detection performance by utilizing different receptive fields in the convolutional layers with different scales to capture objects with different scales. Examples of such methods are the image pyramid, pyramidal feature hierarchy, and the feature pyramid network. However, in MSR imagery, it is still difficult to model the large scale variation of geospatial objects for the existing multi-scale structures as their receptive fields are limited due to the fixed number of layers. In this paper, to solve the problem, a hyper-scale object detection framework for MSR imagery, namely HyNet, is proposed to alleviate the extreme scale-variation problem by learning hyper-scale feature representation. Differing from the previous multi-scale structure operation in the level of the convolutional layer, HyNet uses a hyper-scale block as the core structure, namely the HyBlock, in the sub-layer group level. In the HyBlock, each convolutional layer in the multi-scale structure is first divided into sub-layer groups with an equal size. In the sub-layer group level, hyper-scale features are obtained by a multi-scale sub-layer group operation with pyramidal receptive fields in the convolutional layers of each scale, which means that HyBlock is a fine-grained multi-scale structure. To effectively aggregate the hyper-scale features, group connection in the sub-layer level is used for intra-layer message passing. By promoting the intra-layer message passing to capture the scale-invariance of the hyper-scale features, the group connection can alleviate the scale-variation issue for object detection in MSR imagery. To better utilize the hyper-scale features, adaptive feature selection is proposed to select more effective hyper-scale features via adaptively weighting the different hyper-scale features. The experimental results obtained using three object detection datasets demonstrate that HyNet can learn a robust scale-invariant feature representation and can outperform the previous algorithms, and hence provides an effective new option for object detection in MSR remote sensing imagery.

Graph convolutional autoencoder model for the shape coding and cognition of buildings in maps

ABSTRACT The shape of a geospatial object is an important characteristic and a significant factor in spatial cognition. Existing shape representation methods for vector-structured objects in the map space are mainly based on geometric and statistical measures. Considering that shape is complicated and cognitively related, this study develops a learning strategy to combine multiple features extracted from its boundary and obtain a reasonable shape representation. Taking building data as example, this study first models the shape of a building using a graph structure and extracts multiple features for each vertex based on the local and regional structures. A graph convolutional autoencoder (GCAE) model comprising graph convolution and autoencoder architecture is proposed to analyze the modeled graph and realize shape coding through unsupervised learning. Experiments show that the GCAE model can produce a cognitively compliant shape coding, with the ability to distinguish different shapes. It outperforms existing methods in terms of similarity measurements. Furthermore, the shape coding is experimentally proven to be effective in representing the local and global characteristics of building shape in application scenarios such as shape retrieval and matching.

Siamese Graph Embedding Network for Object Detection in Remote Sensing Images

Multiclass geospatial object detection is a vital fundamental task for many remote sensing applications. However, it still faces several challenges in very high-resolution (VHR) images in remote sensing, such as the ambiguity of object appearance and the complexity of spatial distribution. In this letter, we propose a novel Siamese graph embedding network (SGEN) that leverages the spatial and semantic information to jointly extract the high-level feature representation for object detection. The main purpose of our SGEN is to learn an embedding discriminative feature space that strengthens the interclass compactness while alleviating the intraclass separability. Specifically, we first design a novel contrastive loss in terms of spatial dependence and semantic correspondence for graph similarity metric learning (ML). Then, the SGEN architecture is adopted for spatial and semantic similarity learning by training the novel contrastive loss function. The SGEN model contains two-stream graph convolutional networks (GCNs) for ML, which is helpful to capture the discriminative features. At last, these extracted features with high spatial and semantic discrimination are served to improve the performance of object detection. The comprehensive evaluations on a combined data set consisting of two public object detection data sets demonstrate the effectiveness of the proposed method.

HQ-ISNet: High-Quality Instance Segmentation for Remote Sensing Imagery

Instance segmentation in high-resolution (HR) remote sensing imagery is one of the most challenging tasks and is more difficult than object detection and semantic segmentation tasks. It aims to predict class labels and pixel-wise instance masks to locate instances in an image. However, there are rare methods currently suitable for instance segmentation in the HR remote sensing images. Meanwhile, it is more difficult to implement instance segmentation due to the complex background of remote sensing images. In this article, a novel instance segmentation approach of HR remote sensing imagery based on Cascade Mask R-CNN is proposed, which is called a high-quality instance segmentation network (HQ-ISNet). In this scheme, the HQ-ISNet exploits a HR feature pyramid network (HRFPN) to fully utilize multi-level feature maps and maintain HR feature maps for remote sensing images’ instance segmentation. Next, to refine mask information flow between mask branches, the instance segmentation network version 2 (ISNetV2) is proposed to promote further improvements in mask prediction accuracy. Then, we construct a new, more challenging dataset based on the synthetic aperture radar (SAR) ship detection dataset (SSDD) and the Northwestern Polytechnical University very-high-resolution 10-class geospatial object detection dataset (NWPU VHR-10) for remote sensing images instance segmentation which can be used as a benchmark for evaluating instance segmentation algorithms in the high-resolution remote sensing images. Finally, extensive experimental analyses and comparisons on the SSDD and the NWPU VHR-10 dataset show that (1) the HRFPN makes the predicted instance masks more accurate, which can effectively enhance the instance segmentation performance of the high-resolution remote sensing imagery; (2) the ISNetV2 is effective and promotes further improvements in mask prediction accuracy; (3) our proposed framework HQ-ISNet is effective and more accurate for instance segmentation in the remote sensing imagery than the existing algorithms.

RESEARCH ON CONSTRUCTION METHOD OF SPATIO-TEMPORAL DATA MODEL BASED ON BEHAVIOR-EVENT

Abstract. Starting from the object-oriented idea, this paper analyses the existing event-based models and the logical relationship between behavioral cognition and events, and discusses the continuity of behavioral cognition on the time axis from the perspective of temporal and spatial cognition. A geospatial data model based on behavioral-event is proposed. The physical structure and logical structure of the model are mainly designed, and the four-dimensional model of “time, space, attribute and event” is constructed on the axis. The organic combination of the four models can well describe the internal mechanism and rules of geographical objects. The expression of data model based on behavior-event not only elaborates the basic information of geospatial objects, but also records the changes of related events caused by the changes of geographic Entities' behavior, and expresses the relationship between spatial and temporal objects before and after the changes of behavior cognition. This paper also designs an effective method to organize spatio-temporal data, so as to realize the effective management and analysis of spatio-temporal data and meet the requirements of storage, processing and mining of large spatio-temporal data.

Automation of geospatial objects converting into the classifiers according to the European data standards

Automation of geospatial objects converting into the classifiers according to the European data standards

Object Detection with Low Capacity GPU Systems Using Improved Faster R-CNN

Object detection in remote sensing images has been frequently used in a wide range of areas such as land planning, city monitoring, traffic monitoring, and agricultural applications. It is essential in the field of aerial and satellite image analysis but it is also a challenge. To overcome this challenging problem, there are many object detection models using convolutional neural networks (CNN). The deformable convolutional structure has been introduced to eliminate the disadvantage of the fixed grid structure of the convolutional neural networks. In this study, a multi-scale Faster R-CNN method based on deformable convolution is proposed for single/low graphics processing unit (GPU) systems. Weight standardization (WS) is used instead of batch normalization (BN) to make the proposed model more efficient for a small batch size (1 img/per GPU) on single GPU systems. Experiments were conducted on the publicly available 10-class geospatial object detection (NWPU-VHR 10) dataset to evaluate the object detection performance of the proposed model. Experiment results show that our model achieved a 92.3 mAP. This is a 1.7% mAP increase when compared to the best results in the models using the same dataset.

Orientation guided anchoring for geospatial object detection from remote sensing imagery

Object detection from remote sensing imagery plays a significant role in a wide range of applications, including urban planning, intelligent transportation systems, ecology and environment analysis, etc. However, scale variations, orientation variations, illumination changes, and partial occlusions, as well as image qualities, bring great challenges for accurate geospatial object detection. In this paper, we propose an efficient orientation guided anchoring based geospatial object detection network based on convolutional neural networks. To handle objects of varying sizes, the feature extraction subnetwork extracts a pyramid of semantically strong features at different scales. Based on orientation guided anchoring, the anchor generation subnetwork generates a small set of high-quality, oriented anchors as object proposals. After orientation region of interest pooling, objects of interest are detected from the object proposals through the object detection subnetwork. The proposed method has been tested on a large geospatial object detection dataset. Quantitative evaluations show that an overall completeness, correctness, quality, and F1-measure of 0.9232, 0.9648, 0.8931, and 0.9435, respectively, are obtained. In addition, the proposed method achieves a processing speed of 8 images per second on a GPU on the cloud computing platform. Comparative studies with the existing object detection methods also demonstrate the advantageous detection accuracy and computational efficiency of our proposed method.

RADC-Net: A residual attention based convolution network for aerial scene classification

With rapid development of satellite and airplane platforms, aerial image has become more and more accessible. Aerial image scene classification plays an important role in many remote sensing and geo-science applications. Deep learning methods, especially convolutional neural network (CNN), have boosted the performance of aerial image scene classification significantly because of the strong feature representation ability. However, the distribution of geo-spatial objects and the spatial arrangement of aerial image scenes is often more complicated than natural image scenes. While current CNNs usually highlight the global semantics, more local semantics and more discriminative features need to be preserved to deal with the aforementioned challenges in aerial scenes. In this paper, we propose a residual attention based dense connected convolutional neural network (RADC-Net) to tackle this problem. This framework contains three dense block each followed by an attention block and an enhanced classification layer. Firstly, we simplify the current dense connection structure so that our dense block has much fewer parameters while maintaining discriminative convolutional feature representation ability. Then, we propose a novel residual attention block for our framework to highlight the local semantics relevant to the aerial scenes. Finally, we introduce an enhanced classification layer in our framework to further refine the extracted convolutional features and highlight local semantic information. We comprehensively evaluate the performance of our proposed RADC-Net on three publicly available benchmark datasets. Experimental results demonstrate that our proposed RADC-Net outperforms some state-of-the-art methods with much fewer parameters.

DRBox-v2: An Improved Detector With Rotatable Boxes for Target Detection in SAR Images

Convolutional neural network (CNN)-based methods have been successfully applied to SAR target detection. Different from prevalently used detection approaches with rectangle bounding box, rotatable bounding box (RBox)-based methods, such as DRBox-v1, can effectively reduce the interference of background pixels and locate the targets more finely for geospatial object detection. Although DRBox-v1 has achieved impressive detected performance, there still exist some remaining problems and room for improvement. In this paper, an improved RBox-based target detection framework is proposed to boost precision and recall rates of detection, and we refer to the method as DRBox-v2 and apply it to target detection in SAR images. The main improvements of DRBox-v2 as well as the contributions of this paper are fourfold. First, a multi-layer prior box generation strategy is designed for detecting small-scale targets. Since shallow layers lack strong sematic information, the feature pyramid network (FPN) module is applied. Second, a modified encoding scheme for RBox is proposed for more precisely estimating the position of RBox and orientation of targets. Third, a focal loss (FL) combined with hard negative mining (HNM) technique is proposed to mitigate the issue of the imbalance between positive and negative samples, which produces better results than solely employing either one. Fourth, comprehensive ablation studies are conducted to reveal the effect of each improvement on detected results. The results of the target detection on three data sets are illustrated and our method obtains 0.135, 0.081, 0.115 gains in average precision compared with three state-of-the-art methods, respectively.

Multi-Scale Geospatial Object Detection Based on Shallow-Deep Feature Extraction

Multi-class detection in remote sensing images (RSIs) has garnered wide attention and introduced several service applications in many fields, including civil and military fields. However, several reasons make detection from aerial images very challenging and more difficult than nature scene images: Objects do not have a fixed size, often appear at very various scales and sometimes appear in dense groups, like vehicles and storage tanks, and have different surroundings or background areas. Furthermore, all of this makes the manual annotation of objects very complex and costly. The powerful effect of the feature extraction methods on object detection and the successes of deep convolutional neural networks (CNN) extract deep features more than traditional methods. This study introduced a novel network structure and designed a unique feature extraction which employs squeeze and excitation network (SENet) and residual network (ResNet) to obtain feature maps, named a shallow-deep feature extraction (SDFE), that improves the resolution and the localization at the same time. Furthermore, this novel model reduces the loss of dense groups and small objects, and provides higher and more stable detection accuracy which is not significantly affected by changing the value of the threshold of the intersection over union (IoU) and overcomes the difficulties of RSIs. Moreover, this study introduced strong evidence about the factors that affect the detection of RSIs. The proposed shallow-deep and multi-scale (SD-MS) method outperforms other approaches for the given ten classes of the NWPU VHR-10 dataset.