Large Satellite Images Research Articles

DSHFS: a new hybrid approach that detects structures with their spatial location from large volume satellite images using CNN, GeoServer and TileCache

DSHFS: a new hybrid approach that detects structures with their spatial location from large volume satellite images using CNN, GeoServer and TileCache

Two-Speed Deep-Learning Ensemble for Classification of Incremental Land-Cover Satellite Image Patches

High-velocity data streams present a challenge to deep learning-based computer vision models due to the resources needed to retrain for new incremental data. This study presents a novel staggered training approach using an ensemble model comprising the following: (i) a resource-intensive high-accuracy vision transformer; and (ii) a fast training, but less accurate, low parameter-count convolutional neural network. The vision transformer provides a scalable and accurate base model. A convolutional neural network (CNN) quickly incorporates new data into the ensemble model. Incremental data are simulated by dividing the very large So2Sat LCZ42 satellite image dataset into four intervals. The CNN is trained every interval and the vision transformer trained every half interval. We call this combination of a complementary ensemble with staggered training a “two-speed” network. The novelty of this approach is in the use of a staggered training schedule that allows the ensemble model to efficiently incorporate new data by retraining the high-speed CNN in advance of the resource-intensive vision transformer, thereby allowing for stable continuous improvement of the ensemble. Additionally, the ensemble models for each data increment out-perform each of the component models, with best accuracy of 65% against a holdout test partition of the RGB version of the So2Sat dataset.

Radiometric Normalization Using a Pseudo−Invariant Polygon Features−Based Algorithm with Contemporaneous Sentinel−2A and Landsat−8 OLI Imagery

As sensor parameters and atmospheric conditions create uncertainties for at−sensor radiation detection, radiometric consistency among satellite images is difficult to achieve. Relative radiometric normalization is a method that can improve multi−image consistency with accurate pseudo−invariant features (PIFs), especially over large areas or long time series satellite images. Although there are algorithms that manually or automatically select PIFs, the spatial mismatch of satellite images can affect PIF extraction, particularly with artificial pixels. To alleviate this problem, we proposed to use Landsat−8 OLI as the reference image and Sentinel−2A as the subject image, to apply pseudo−invariant features−based algorithms with polygon features through the single−band and multiple−band regression. Compared to pseudo−invariant point features, hyperspectral library, and histogram matching approaches, the results demonstrate the superiority of the proposed algorithms with correlation coefficients of 0.9948 and 0.9945, and an RMSE of 0.0097 and 0.0095 with multiple− and single−band regression, respectively. We also found more accurate linear fitting and better shape matching through band scattering and reflectance frequency analysis. The proposed algorithms are a significant improvement in radiometric normalization, within artificial pixels, achieving spectral signature consistency.

Adversarial Learning Based Discriminative Domain Adaptation for Geospatial Image Analysis

The ability of supervised image analysis methods to provide state-of-the-art performance is limited by availability of high-quality labeled data in large quantities. Domain adaptation approaches propose a solution to this problem by leveraging quality labeled information from auxiliary data sources. In this work, we use adversarial learning for domain adaptation for remote sensing applications. First, we approached the problem of unavailable target domain labels with unsupervised domain adaptation and then extended our method for semisupervised domain adaptation to use a few available labels as well. We are using adversarial learning to extract discriminative target domain features that are aligned with source domain. We test our framework for two very different applications of remote sensing imagery, multiclass classification in hyperspectral images and semantic segmentation in large scale satellite images. For hyperspectral image analysis two datasets were used: the University of Houston shadow data was used for quantifying the efficacy of our approach to varying illumination, and the Botswana data was used to quantify the efficacy of our approach under multitemporal spectral shifts. Multisensor high-resolution images from National Agriculture Imagery Program and SpaceNet-Rio datasets were used as the source and target for the task of building extraction for large scale semantic segmentation based domain adaptation.

Deep semantic segmentation for detecting eucalyptus planted forests in the Brazilian territory using sentinel-2 imagery

This research aims to analyze the use of deep semantic segmentation to detect eucalyptus afforestation areas using Sentinel-2 images. The study compared six architectures (U-net, DeepLabv3+, FPN, MANet, PSPNet, LinkNet) with four encoders (ResNet-101, ResNeXt-101, Efficient-net-b3 and Efficient-net-b7), using 10 spectral bands. Even though the differences were not large among the different models, we found that the Efficient-net-b7 was the best backbone among all architectures, and the best overall model was DeepLabv3+ with the Efficient-net-b7 backbone, achieving an IoU of 76.57. Moreover, we compared the mapping of large satellite images with the sliding window technique with overlapping pixels considering six stride values. We found that sliding windows with lower stride values significantly minimized errors in the frame edge both visually and quantitively (metrics). Semantic segmentation allows an evident distinction between the afforestation and the natural vegetation, being fast and efficient for spatial distribution analysis of afforestation changes in Brazil.

Multiscale Cloud Detection in Remote Sensing Images Using a Dual Convolutional Neural Network

Semantic segmentation by convolutional neural networks (CNN) has advanced the state of the art in pixel-level classification of remote sensing images. However, processing large images typically requires analyzing the image in small patches, and hence features that have large spatial extent still cause challenges in tasks such as cloud masking. To support a wider scale of spatial features while simultaneously reducing computational requirements for large satellite images, we propose an architecture of two cascaded CNN model components successively processing undersampled and full resolution images. The first component distinguishes between patches in the inner cloud area from patches at the cloud's boundary region. For the cloud-ambiguous edge patches requiring further segmentation, the framework then delegates computation to a fine-grained model component. We apply the architecture to a cloud detection dataset of complete Sentinel-2 multispectral images, approximately annotated for minimal false negatives in a land use application. On this specific task and data, we achieve a 16\% relative improvement in pixel accuracy over a CNN baseline based on patching.

BT-RoadNet: A boundary and topologically-aware neural network for road extraction from high-resolution remote sensing imagery

Automatic road extraction from high-resolution remote sensing imagery has various applications like urban planning and automatic navigation. Existing methods for automatic road extraction however, focus on regional accuracy but not on the boundary quality; and most of these road extraction methods yield discontinuous results due to noise and occlusions. To address these two problems, a Boundary and Topological-aware Road extraction Network (BT-RoadNet) is proposed. BT-RoadNet is a coarse-to-fine architecture composed of two encoder-to-decoder networks, a Coarse Map Predicting Module (CMPM) and Fine Map Predicting Module (FMPM). The CMPM learns to predict coarse road segmentation maps, in which a Spatial Context Module (SCM) is employed as a bridge to solve discontinuous problems. The FMPM is used to refine the coarse road maps by learning the difference between the coarse road extraction result and the ground truth. Experiments were conducted on the open Massachusetts Road Dataset, a newly annotated Wuhan University (WHU) Road Dataset, and three large satellite images. Quantitative and qualitative analysis demonstrate that the proposed BT-RoadNet can enhance road network extraction to deal with interruptions caused by shadows and occlusions, extract roads with different scales and materials, and handle roads under construction that have incomplete spectral and geometric properties.

Land cover classification in Thailand's Eastern Economic Corridor (EEC) using convolutional neural network on satellite images

Land cover plays an integral role in urban management as a source of information to support authorities’ decision making. Recently, computer vision methods, machine learning algorithms in particular, are increasingly used for land cover and land use mapping, which can help make these processes more efficient and more affordably. To this end, this paper focuses on leveraging artificial intelligence using convolutional neural network (CNN) to propose a new method for land cover mapping. A first CNN model is trained with a large number of 4 image classes to obtain a land cover model. We then apply the model directly on satellite images that are cropped into images of the same size as the training images. The results of the direct application of the model is not satisfactory; in particular, it confuses water and forested areas. The paper propose a different approach where land cover model is obtained by combining 2 models in series. The first model is a binary-class CNN model which contains classification of two classes (water, land) while the second model is a three-class CNN model containing classification of land, excluding non-water areas. Global accuracy obtained is 98% and 91% for the binary- and three-class CNN model respectively. This approach is used successfully on large area satellite images.

The digital Earth Observation Librarian: a data mining approach for large satellite images archives

ABSTRACT Throughout the years, various Earth Observation (EO) satellites have generated huge amounts of data. The extraction of latent information in the data repositories is not a trivial task. New methodologies and tools, being capable of handling the size, complexity and variety of data, are required. Data scientists require support for the data manipulation, labeling and information extraction processes. This paper presents our Earth Observation Image Librarian (EOLib), a modular software framework which offers innovative image data mining capabilities for TerraSAR-X and EO image data, in general. The main goal of EOLib is to reduce the time needed to bring information to end-users from Payload Ground Segments (PGS). EOLib is composed of several modules which offer functionalities such as data ingestion, feature extraction from SAR (Synthetic Aperture Radar) data, meta-data extraction, semantic definition of the image content through machine learning and data mining methods, advanced querying of the image archives based on content, meta-data and semantic categories, as well as 3-D visualization of the processed images. EOLib is operated by DLR’s (German Aerospace Center’s) Multi-Mission Payload Ground Segment of its Remote Sensing Data Center at Oberpfaffenhofen, Germany.

UNSUPERVISED ZONING OF CULTIVATION AREAS WITH SIMILAR CULTIVATION PATTERN IN GOLESTAN PROVINCE BASED ON THE VEGETATION PRODUCTS OF MODIS SENSOR

Abstract. The estimation of cultivation area and categorizing the agricultural product types is one of the prerequisites for achieving sustainable development in the agricultural studies. In this study, an unsupervised zoning the cultivation areas with the same cultivation pattern in Golestan province is on the agenda. Therefore, due to wide spatial range, high temporal resolution and easy access of 16-day products of the vegetation of the MODIS sensor which acquired in a year (From November 2017 to October 2018), these images are used in this research. In the proposed method, after the generating of NDVI vegetation time series as a hyper-cube and separating farmlands’ boundaries in Golestan province using the land-use map; the Sequential Maximum Angle Convex Cone (SMACC) endmember extraction algorithm and the maximum number of product variation using the statistical information of the region (Obtained from the statistics centre of Iran) are used to extract endmembers of the hyper-cube. In the following, the timing responses of the NDVI, identified as endmembers, will be refined in the second phase. In this process, identifying and eliminating noise signals (unrelated to cultivating patterns) and integrating the same cultivating patterns will be on the agenda. At the last stage of the proposed method and after refinement of the endmembers, the hyper-cube is clustered by Spectral Angle Mapper (SAM) algorithm and the mapping of regions with the same cultivation pattern is produced. In the proposed method, the zoning of agricultural land is based solely on the statistical knowledge of the variety of cultivation and the results have led to the production of interconnected spatial parts. This is consistent with the reality of the spatial occurrence of similar cultivating patterns in a geographic area. On the other hand, the visual comparison of results with large scale satellite images illustrates that there is a significant relationship between clustering results and ground truth in terms of cultivating pattern. Obviously, such products can be used as initial layers of information to produce the results of a supervised classification with the aim of applying the cultivation area of a variety of agricultural products.

Search method for changes of the earth’s surface state through multi-temporal satellite images

Subject of Research. We study the approach to the underlying surface interpretation automation for satellite images obtained by the onboard optical-electronic equipment of the Earth remote sensing systems. The topicality of research is determined by the necessity of introduction of computer vision methods aimed at solving the search problem of the earth’s surface state changes through multi-temporal satellite monitoring data. The goal of research is reducing of the time spent on processing of large area satellite images. Method. The method is based on the idea of comparing the contrast of different-time satellite images. For method implementation, a mathematical apparatus is formed for calculating the contrast values of the analyzed images in the normalized interval from 0 to 1. The effectiveness of automated processing of satellite images is ensured by their pre- segmentation and zoning. Segmentation parameters are selected taking into account the size of the objects to be detected. The efficiency of the proposed method is confirmed by the high correlation of the automated processing results with the results of visual analysis of test satellite images. Main Results. The results of calculating the contrast of test images using the formulated mathematical apparatus are presented. The necessity of image segmentation is proved to solve the problem of detecting changes in the terrain on the example of processing images consisting of different number of fragments. An approach is developed for reducing the redundancy of data on terrain changes by performing a preliminary zoning procedure. The essence of this procedure is to determine the researched area boundaries in order to limit the zones for search of changes. Practical Relevance. The proposed method of data processing on the Earth remote sensing provides interpretation of the underlying surface images in an automated mode without operator participation. At that, the interpretation of images, when observing large areas, can be accelerated.

Parallel Processing of k-Means Clustering Algorithm for Unsupervised Classification of Large Satellite Images: A Hybrid Method Using Multicores and a PC-Cluster

Parallel Processing of k-Means Clustering Algorithm for Unsupervised Classification of Large Satellite Images: A Hybrid Method Using Multicores and a PC-Cluster

Scale Matters: Spatially Partitioned Unsupervised Segmentation Parameter Optimization for Large and Heterogeneous Satellite Images

To classify Very-High-Resolution (VHR) imagery, Geographic Object Based Image Analysis (GEOBIA) is the most popular method used to produce high quality Land-Use/Land-Cover maps. A crucial step in GEOBIA is the appropriate parametrization of the segmentation algorithm prior to the classification. However, little effort has been made to automatically optimize GEOBIA algorithms in an unsupervised and spatially meaningful manner. So far, most Unsupervised Segmentation Parameter Optimization (USPO) techniques, assume spatial stationarity for the whole study area extent. This can be questionable, particularly for applications in geographically large and heterogeneous urban areas. In this study, we employed a novel framework named Spatially Partitioned Unsupervised Segmentation Parameter Optimization (SPUSPO), which optimizes segmentation parameters locally rather than globally, for the Sub-Saharan African city of Ouagadougou, Burkina Faso, using WorldView-3 imagery (607 km2). The results showed that there exists significant spatial variation in the optimal segmentation parameters suggested by USPO across the whole scene, which follows landscape patterns—mainly of the various built-up and vegetation types. The most appropriate automatic spatial partitioning method from the investigated techniques, was an edge-detection cutline algorithm, which achieved higher classification accuracy than a global optimization, better predicted built-up regions, and did not suffer from edge effects. The overall classification accuracy using SPUSPO was 90.5%, whilst the accuracy from undertaking a traditional USPO approach was 89.5%. The differences between them were statistically significant (p < 0.05) based on a McNemar’s test of similarity. Our methods were validated further by employing a segmentation goodness metric, Area Fit Index (AFI)on building objects across Ouagadougou, which suggested that a global USPO was more over-segmented than our local approach. The mean AFI values for SPUSPO and USPO were 0.28 and 0.36, respectively. Finally, the processing was carried out using the open-source software GRASS GIS, due to its efficiency in raster-based applications.

A Coarse-to-Fine Geometric Scale-Invariant Feature Transform for Large Size High Resolution Satellite Image Registration.

Large size high resolution (HR) satellite image matching is a challenging task due to local distortion, repetitive structures, intensity changes and low efficiency. In this paper, a novel matching approach is proposed for the large size HR satellite image registration, which is based on coarse-to-fine strategy and geometric scale-invariant feature transform (SIFT). In the coarse matching step, a robust matching method scale restrict (SR) SIFT is implemented at low resolution level. The matching results provide geometric constraints which are then used to guide block division and geometric SIFT in the fine matching step. The block matching method can overcome the memory problem. In geometric SIFT, with area constraints, it is beneficial for validating the candidate matches and decreasing searching complexity. To further improve the matching efficiency, the proposed matching method is parallelized using OpenMP. Finally, the sensing image is rectified to the coordinate of reference image via Triangulated Irregular Network (TIN) transformation. Experiments are designed to test the performance of the proposed matching method. The experimental results show that the proposed method can decrease the matching time and increase the number of matching points while maintaining high registration accuracy.

Land-use scene classification based on a CNN using a constrained extreme learning machine

ABSTRACTClassifying land-use scenes from high-resolution remote-sensing imagery with high quality and accuracy is of paramount interest for science and land management applications. In this article, we proposed a new model for land-use scene classification by integrating the recent success of convolutional neural network (CNN) and constrained extreme learning machine (CELM). In the model, the fully connected layers of a pretrained CNN have been removed. Then, CNN works as a deep and robust convolutional feature extractor. After normalization, deep convolutional features are fed to the CELM classifier. To analyse the performance, the proposed method has been evaluated on two challenging high-resolution data sets: (1) the aerial image data set consisting of 30 different aerial scene categories with sub-metre resolution and (2) a Sydney data set that is a large high spatial resolution satellite image. Experimental results show that the CNN-CELM model improves the generalization ability and reduces the training time compared to state-of-the-art methods.

Integrated Localization and Recognition for Inshore Ships in Large Scene Remote Sensing Images

Automatic inshore ship recognition, which includes target localization and type recognition, is an important and challenging task. However, existing ship recognition methods mainly focus on the classification of ship samples or clips. These methods rely deeply on the detection algorithm to complete localization and recognition in large scene images. In this letter, we present an integrated framework to automatically locate and recognize inshore ships in large scene satellite images. Different from traditional object recognition methods using two steps of detection-classification, the proposed framework could locate inshore ships and identify types without the detection step. Considering ship size is a useful feature, a novel multimodel method is proposed to utilize this feature. And an Euclidean-distance-based fusion strategy is used to combine candidates given by models. This fusion strategy could effectively separate side-by-side ships. To handle large scene images efficiently, scale-invariant feature transform registration is also integrated into the framework to utilize geographic information. All of these make the framework an end-to-end fashion which could automatically recognize inshore ships in large scene satellite images. Experiments on Quickbird images show that this framework could achieve the actual applied requirements.

Digital Images for Plant Phenology Documentation

Plant phenology is a key indicator of changing climatic conditions and, thus, automated documentation of phenology using cameras has recently been advocated. The present study, therefore, aimed at identifying phenological stages in Sapium sebiferum using digital images of a full calendar year. Red, Green and Blue channel information from these images was analyzed. Across the three channels (i.e. R, G, and B) the DN values ranged from a minimum of 71.42 in blue channel (BDN) to 156.16 in green channel (GDN). In general, the RDN, GDN, and BDN values increased till 66 DOY after which there was a steep decline in DN values of R, G, and B channels till 224 DOY. After this, a slight increase in the DN values of all the three channels was observed. The value of Red Fraction ranged between 0.297348281 and 0.352612071. The minimum value for the same was on 3 DOY while the maximum was on 135 DOY. On the other hand, the value of Green Fraction ranged between 0.330045461 and 0.39841947. The values of Green Excess Index (2G-RBi) ranged between −0.009863618 and 0.19525841. It reported minimum value on 42 DOY and the maximum on 139 DOY. Green excess index were found to be reliable indicator of greening up in the species. Red Faction was a good indicator of senescence. Photographs provide an ideal tool for documenting phenology of a species and can provide inputs for large scale satellite image analyses.

When is it time to move to the next map? Optimal foraging in guided visual search.

Suppose that you are looking for visual targets in a set of images, each containing an unknown number of targets. How do you perform that search, and how do you decide when to move from the current image to the next? Optimal foraging theory predicts that foragers should leave the current image when the expected value from staying falls below the expected value from leaving. Here, we describe how to apply these models to more complex tasks, like search for objects in natural scenes where people have prior beliefs about the number and locations of targets in each image, and search is guided by target features and scene context. We model these factors in a guided search task and predict the optimal time to quit search. The data come from a satellite image search task. Participants searched for small gas stations in large satellite images. We model quitting times with a Bayesian model that incorporates prior beliefs about the number of targets in each map, average search efficiency (guidance), and actual search history in the image. Clicks deploying local magnification were used as surrogates for deployments of attention and, thus, for time. Leaving times (measured in mouse clicks) were well-predicted by the model. People terminated search when their expected rate of target collection fell to the average rate for the task. Apparently, people follow a rate-optimizing strategy in this task and use both their prior knowledge and search history in the image to decide when to quit searching.

Satellite image collection modeling for large area hazard emergency response

Timely collection of critical hazard information is the key to intelligent and effective hazard emergency response decisions. Satellite remote sensing imagery provides an effective way to collect critical information. Natural hazards, however, often have large impact areas – larger than a single satellite scene. Additionally, the hazard impact area may be discontinuous, particularly in flooding or tornado hazard events. In this paper, a spatial optimization model is proposed to solve the large area satellite image acquisition planning problem in the context of hazard emergency response. In the model, a large hazard impact area is represented as multiple polygons and image collection priorities for different portion of impact area are addressed. The optimization problem is solved with an exact algorithm. Application results demonstrate that the proposed method can address the satellite image acquisition planning problem. A spatial decision support system supporting the optimization model was developed. Several examples of image acquisition problems are used to demonstrate the complexity of the problem and derive optimized solutions.

Production potential of photosynthesis in forest ecosystems of the low mountain Pokuttya (Ukrainian Carpathians)

The aim of the study was testing on the example of a model region a method of estimation of the production potential of forest ecosystems and the consequences of anthropogenic changes there. The object of study is a typical Carpathian lower mountain forest in the basin of the river Lyuchka, an area of 14,806 ha. It has long undergone considerable agricultural transformations. Studies were based on cartographic modeling of modern anthropogenically transformed biogeocenotic cover using large scale satellite images. The main types of biogeocenotical cover were defined according to the altitudinal zonation of vegetation of the parts of the mountain terrain and the prevailing types of soil and hydrological conditions. For analytical procedures a database of materials describing the biometric features of the forests was created. It is possible to perform calculations of average and potential biometrical parameters of stands growing in different climatic, soil and hydrological conditions. The structure and the biological diversity of different vegetation types was determined by construction of mapping models of spatial structures of the basic types of biogeocenotic cover. The biological productivity of the main types of forest ecosystems was determined on base of the volume of timber stands. The mass of dry wood was determined taking into account its size and standard density of wood of different tree species. Calculation of the total volume of forest biomass was performed using the conversion factors of weight relative to the trunk timber volume. The mass of carbon deposited accounted for 50% of the total biomass. The average annual growth of biomass and carbon deposited was determined by dividing the volume of the stands by their average age. Calculation of phytocenosis consumed as a result of photosynthesis reaction of CO2, H2O and light energy was performed taking into account corresponding material and energy ratios. In general, in the course of one year the biogeocenotic cover of the model lowland area could deposit as a result of photosynthesis for the restoration of potential vegetation cover 43.3 ths. tons of carbon, while consuming 159 ths. t of CO2 and 65.2 ths. t of H2O and 1,724 ∙ 103 GJ of light energy, which is equivalent to 479 GW ∙ hour. During this process O2 – 115.7 ths. t would be emitted into the atmosphere. In terms of 1 hectare, this is equal to C – 2.92 t ∙ ha–1, CO2 – 10.7 t ∙ ha–1, H20 – 4.4 t ∙ha–1, O2 – 7.8 t ∙ ha–1, E – 116.4 GJ ∙ ha–1, which is equivalent to 32.3 MW ∙ h ∙ ha–1. The total production capacity of photosynthesis of the modern biogeocenotic cover model area is 38% of the potential. As a result, the energy loss is 20 MW ∙ h–1 ∙ ha–1 light energy to 1.9 t ∙ ha–1 less than the deposited carbon 6.7 t ∙ ha–1 less carbon dioxide used, 2.8 t ∙ ha–1 water is not used, 3.9 t ha–1 oxygen is not returned to the atmosphere. The large specific amount of unused resources of productivity of biogeocenotic cover, carbon dioxide, light energy, untranspired moisture in the air and unemitted oxygen can cause a significant impact on local climatic conditions.