Very High Resolution Images Research Articles

Geo-parcel-based crop classification in very-high-resolution images via hierarchical perception

ABSTRACTThe basic application of remote sensing is classifying surface objects in images. Traditional pixel-based or object-based classification methods are poorly suited to very high-resolution (VHR) images captured by remote sensors with high spatial resolutions. In the field of computer vision, deep learning has recently achieved great advances in natural image processing. Inspired by this, we propose a methodology guided by hierarchical perception to classify crops in VHR images based on geo-parcels. Geo-parcel-based crop classification is used in agriculture and in refined farmland management. The proposed methodology can be divided into three steps: zoning, location and quality. In the first step, the image is divided into blocks based on the road network. In the second step, geographical entities are extracted from every block defined in the zoning step. In the last step, the geographical entity types are identified based on the texture information. These steps provide mutual constraints. In each step, the information is extracted by neural networks that have been adapted to the VHR images. The experimental results indicate that our methodology performs well, with a precision greater than 90%. Furthermore, our methodology combines deep learning techniques and theory regarding image perception by humans, providing a valuable method for processing remote sensing information.

Improved Piecewise Linear Transformation for Precise Warping of Very-High-Resolution Remote Sensing Images

A large number of evenly distributed conjugate points (CPs) in entirely overlapping regions of the images are required to achieve successful co-registration between very-high-resolution (VHR) remote sensing images. The CPs are then used to construct a non-linear transformation model that locally warps a sensed image to a reference image’s coordinates. Piecewise linear (PL) transformation is largely exploited for warping VHR images because of its superior performance as compared to the other methods. The PL transformation constructs triangular regions on a sensed image from the CPs by applying the Delaunay algorithm, after which the corresponding triangular regions in a reference image are constructed using the same CPs on the image. Each corresponding region in the sensed image is then locally warped to the regions of the reference image through an affine transformation estimated from the CPs on the triangle vertices. The warping performance of the PL transformation shows reliable results, particularly in regions inside the triangles, i.e., within the convex hulls. However, the regions outside the triangles, which are warped when the extrapolated boundary planes are extended using CPs located close to the regions, incur severe geometric distortion. In this study, we propose an effective approach that focuses on the improvement of the warping performance of the PL transformation over the external area of the triangles. Accordingly, the proposed improved piecewise linear (IPL) transformation uses additional pseudo-CPs intentionally extracted from positions on the boundary of the sensed image. The corresponding pseudo-CPs on the reference image are determined by estimating the affine transformation from CPs located close to the pseudo-CPs. The latter are simultaneously used with the former to construct the triangular regions, which are enlarged accordingly. Experiments on both simulated and real datasets, constructed from Worldview-3 and Kompsat-3A satellite images, were conducted to validate the effectiveness of the proposed IPL transformation. That transformation was shown to outperform the existing linear/non-linear transformation models such as an affine, third and fourth polynomials, local weighted mean, and PL. Moreover, we demonstrated that the IPL transformation improved the warping performance over the PL transformation outside the triangular regions by increasing the correlation coefficient values from 0.259 to 0.304, 0.603 to 0.657, and 0.180 to 0.338 in the first, second, and third real datasets, respectively.

Using Vehicle Synthesis Generative Adversarial Networks to Improve Vehicle Detection in Remote Sensing Images

Vehicle detection based on very high-resolution (VHR) remote sensing images is beneficial in many fields such as military surveillance, traffic control, and social/economic studies. However, intricate details about the vehicle and the surrounding background provided by VHR images require sophisticated analysis based on massive data samples, though the number of reliable labeled training data is limited. In practice, data augmentation is often leveraged to solve this conflict. The traditional data augmentation strategy uses a combination of rotation, scaling, and flipping transformations, etc., and has limited capabilities in capturing the essence of feature distribution and proving data diversity. In this study, we propose a learning method named Vehicle Synthesis Generative Adversarial Networks (VS-GANs) to generate annotated vehicles from remote sensing images. The proposed framework has one generator and two discriminators, which try to synthesize realistic vehicles and learn the background context simultaneously. The method can quickly generate high-quality annotated vehicle data samples and greatly helps in the training of vehicle detectors. Experimental results show that the proposed framework can synthesize vehicles and their background images with variations and different levels of details. Compared with traditional data augmentation methods, the proposed method significantly improves the generalization capability of vehicle detectors. Finally, the contribution of VS-GANs to vehicle detection in VHR remote sensing images was proved in experiments conducted on UCAS-AOD and NWPU VHR-10 datasets using up-to-date target detection frameworks.

Extraction of Earthquake-Induced Collapsed Buildings From Bi-Temporal VHR Images Using Object-Level Homogeneity Index and Histogram

The availability of very high resolution (VHR) satellite images has made the analysis of remotely sensed data an increasingly effective tool for the extraction of urban building damage caused by earthquakes. In this study, we proposed a novel method to extract collapsed buildings from bi-temporal VHR images using object-level homogeneity index (OHI) and object histogram. The OHI, which is calculated using an improved spatial homogeneity index of pixel, quantifies internal homogeneity of image object generated from image segmentation. The object histogram quantifies the spectral variability of image object. Ground objects that are intact and significantly different from collapsed buildings, such as vegetation and non-vegetated homogeneous areas, were first extracted from post-event VHR image using OHI and normalized difference vegetation index and were masked out. Collapsed buildings were then extracted from bi-temporal images of the remaining areas using object histogram and a curve matching method, multi-reference spectral angle mapper. The proposed method was evaluated and compared to two existing methods using bi-temporal QuickBird images over Bam, Iran, which was heavily hit by an earthquake in 2003. The experimental results showed that the proposed method outperformed the two comparative methods, with the increase of 11.38 and 5.65% in overall accuracy, and 14.27 and 7.83% in F-score, respectively. The proposed method provides a fast and reliable method for extraction of collapsed buildings.

Transferable Object-Based Framework Based on Deep Convolutional Neural Networks for Building Extraction

Automated building extraction from single, very-high-resolution (VHR) images is still one of the most challenging tasks for urban planning, estimating population, understanding urban dynamics, and many other applications. The complexities of building objects have caused the images of buildings to be oversegmented into multiple segments in the object-based image analysis (OBIA) method. Selecting the appropriate segmentation scale parameter is a major challenge in OBIA that influences the discriminative features extraction, especially for building objects. Furthermore, transferability of OBIA method is another challenge. Presently, convolutional neural networks (CNNs) are a well-understood tool for images scene classification. However, scene classification based on CNNs is still difficult due to the scale variation of the objects in VHR images. To meet these challenges, we propose a novel object-based deep CNN (OCNN) framework for VHR images. The datasets used for testing were Vaihingen (Germany) aerial images and a Tunis Worldview-2 (WV2) satellite imagery. Experimental results prove that our framework is extensible to different types of the image with the same sensor or another sensor (for example WV2) with once-fine-tuning. In addition, our framework extracts the different types of building with respect to size, color, material, spectral similarity to roads, and complex backgrounds. Quantitative evaluation at the object level demonstrated that the proposed framework could yield promising results (average precision 0.88, recall 0.92, quality 0.82, F-score 0.90, overall accuracy 0.95, and Kappa coefficient 0.90). Comparative experimental results indicate that our proposed OCNN significantly outperforms the traditional method for building extraction.

Semantic Segmentation of Urban Buildings from VHR Remote Sensing Imagery Using a Deep Convolutional Neural Network

Urban building segmentation is a prevalent research domain for very high resolution (VHR) remote sensing; however, various appearances and complicated background of VHR remote sensing imagery make accurate semantic segmentation of urban buildings a challenge in relevant applications. Following the basic architecture of U-Net, an end-to-end deep convolutional neural network (denoted as DeepResUnet) was proposed, which can effectively perform urban building segmentation at pixel scale from VHR imagery and generate accurate segmentation results. The method contains two sub-networks: One is a cascade down-sampling network for extracting feature maps of buildings from the VHR image, and the other is an up-sampling network for reconstructing those extracted feature maps back to the same size of the input VHR image. The deep residual learning approach was adopted to facilitate training in order to alleviate the degradation problem that often occurred in the model training process. The proposed DeepResUnet was tested with aerial images with a spatial resolution of 0.075 m and was compared in performance under the exact same conditions with six other state-of-the-art networks—FCN-8s, SegNet, DeconvNet, U-Net, ResUNet and DeepUNet. Results of extensive experiments indicated that the proposed DeepResUnet outperformed the other six existing networks in semantic segmentation of urban buildings in terms of visual and quantitative evaluation, especially in labeling irregular-shape and small-size buildings with higher accuracy and entirety. Compared with the U-Net, the F1 score, Kappa coefficient and overall accuracy of DeepResUnet were improved by 3.52%, 4.67% and 1.72%, respectively. Moreover, the proposed DeepResUnet required much fewer parameters than the U-Net, highlighting its significant improvement among U-Net applications. Nevertheless, the inference time of DeepResUnet is slightly longer than that of the U-Net, which is subject to further improvement.

Modelling the Spectral Uncertainty of Geographic Features in High-Resolution Remote Sensing Images: Semi-Supervising and Weighted Interval Type-2 Fuzzy C-Means Clustering

The spectral uncertainty refers to the diversity and variations of spectral characteristics within a single geographic object or across different objects of the same class. Usually, existing methods represent the spectral characteristics as precise single-valued curves. Thus, the spectral variations cannot be modeled, which further restricts the analysis and classification performance of remote sensing images. On the other hand, unsupervised methods have poor performance in classification and modeling uncertainty, while supervised methods need a large number of samples with high quality. Fuzzy semi-supervised clustering (FSSC) methods achieve a high accuracy with limited labelled samples. Thus, currently, FSSC methods attract more and more attention. This paper proposes a novel method to model the spectral uncertainty for very-high-resolution (VHR) images based on interval type-2 fuzzy sets (IT2 FSs), namely the hierarchical semi-supervising and weighted interval type-2 fuzzy c-means for objects (hierarchical SSW-IT2FCM-O) clustering method. In this method, the VHR image is segmented into image objects to reduce spectral uncertainty within objects. Spectral values, spectral indices and textures were weighted for object-based image classification. To further reduce spectral uncertainty across different objects of the same class, the spectral characteristics of land cover types were represented as banded curves with certain widths instead of precise single-valued spectral curves. The experimental results show that the banded spectral curves produced by the hierarchical SSW-IT2FCM-O can effectively model the spectral uncertainty of geographic objects. From the perspective of classification, four typical validity indices along with the confusion matrix and kappa coefficient were used to test the effectiveness of the hierarchical SSW-IT2FCM-O method, and these indices show that the presented method SSW-IT2FCM-O has greater classification accuracy than the existing FSSC methods and, more importantly, it requires smaller training samples than the existing methods.

BUILDING SEGMENTATION FROM AIRBORNE VHR IMAGES USING MASK R-CNN

Abstract. Up-to-date 3D building models are important for many applications. Airborne very high resolution (VHR) images often acquired annually give an opportunity to create an up-to-date 3D model. Building segmentation is often the first and utmost step. Convolutional neural networks (CNNs) draw lots of attention in interpreting VHR images as they can learn very effective features for very complex scenes. This paper employs Mask R-CNN to address two problems in building segmentation: detecting different scales of building and segmenting buildings to have accurately segmented edges. Mask R-CNN starts from feature pyramid network (FPN) to create different scales of semantically rich features. FPN is integrated with region proposal network (RPN) to generate objects with various scales with the corresponding optimal scale of features. The features with high and low levels of information are further used for better object classification of small objects and for mask prediction of edges. The method is tested on ISPRS benchmark dataset by comparing results with the fully convolutional networks (FCN), which merge high and low level features by a skip-layer to create a single feature for semantic segmentation. The results show that Mask R-CNN outperforms FCN with around 15% in detecting objects, especially in detecting small objects. Moreover, Mask R-CNN has much better results in edge region than FCN. The results also show that choosing the range of anchor scales in Mask R-CNN is a critical factor in segmenting different scale of objects. This paper provides an insight into how a good anchor scale for different dataset should be chosen.

Identifying a Slums’ Degree of Deprivation from VHR Images Using Convolutional Neural Networks

In the cities of the Global South, slum settlements are growing in size and number, but their locations and characteristics are often missing in official statistics and maps. Although several studies have focused on detecting slums from satellite images, only a few captured their variations. This study addresses this gap using an integrated approach that can identify a slums’ degree of deprivation in terms of socio-economic variability in Bangalore, India using image features derived from very high resolution (VHR) satellite images. To characterize deprivation, we use multiple correspondence analysis (MCA) and quantify deprivation with a data-driven index of multiple deprivation (DIMD). We take advantage of spatial features learned by a convolutional neural network (CNN) from VHR satellite images to predict the DIMD. To deal with a small training dataset of only 121 samples with known DIMD values, insufficient to train a deep CNN, we conduct a two-step transfer learning approach using 1461 delineated slum boundaries as follows. First, a CNN is trained using these samples to classify slums and formal areas. The trained network is then fine-tuned using the 121 samples to directly predict the DIMD. The best prediction is obtained by using an ensemble non-linear regression model, combining the results of the CNN and models based on hand-crafted and geographic information system (GIS) features, with R2 of 0.75. Our findings show that using the proposed two-step transfer learning approach, a deep CNN can be trained with a limited number of samples to predict the slums’ degree of deprivation. This demonstrates that the CNN-based approach can capture variations of deprivation in VHR images, providing a comprehensive understanding of the socio-economic situation of slums in Bangalore.

Automated Coregistration of Multisensor Orthophotos Generated from Unmanned Aerial Vehicle Platforms

Image coregistration is a key preprocessing step to ensure the effective application of very-high-resolution (VHR) orthophotos generated from multisensor images acquired from unmanned aerial vehicle (UAV) platforms. The most accurate method to align an orthophoto is the installation of air-photo targets at a test site prior to flight image acquisition, and these targets were used as ground control points (GCPs) for georeferencing and georectification. However, there are time and cost limitations related to installing the targets and conducting field surveys on the targets during every flight. To address this problem, this paper presents an automated coregistration approach for orthophotos generated from VHR images acquired from multisensors mounted on UAV platforms. Spatial information from the orthophotos, provided by the global navigation satellite system (GNSS) at each image’s acquisition time, is used as ancillary information for phase correlation-based coregistration. A transformation function between the multisensor orthophotos is then estimated based on conjugate points (CPs), which are locally extracted over orthophotos using the phase correlation approach. Two multisensor datasets are constructed to evaluate the proposed approach. These visual and quantitative evaluations confirm the superiority of the proposed method.

Detecting Large-Scale Urban Land Cover Changes from Very High Resolution Remote Sensing Images Using CNN-Based Classification

The study investigates land use/cover classification and change detection of urban areas from very high resolution (VHR) remote sensing images using deep learning-based methods. Firstly, we introduce a fully Atrous convolutional neural network (FACNN) to learn the land cover classification. In the FACNN an encoder, consisting of full Atrous convolution layers, is proposed for extracting scale robust features from VHR images. Then, a pixel-based change map is produced based on the classification map of current images and an outdated land cover geographical information system (GIS) map. Both polygon-based and object-based change detection accuracy is investigated, where a polygon is the unit of the GIS map and an object consists of those adjacent changed pixels on the pixel-based change map. The test data covers a rapidly developing city of Wuhan (8000 km2), China, consisting of 0.5 m ground resolution aerial images acquired in 2014, and 1 m ground resolution Beijing-2 satellite images in 2017, and their land cover GIS maps. Testing results showed that our FACNN greatly exceeded several recent convolutional neural networks in land cover classification. Second, the object-based change detection could achieve much better results than a pixel-based method, and provide accurate change maps to facilitate manual urban land cover updating.

Mosaicking Opportunistically Acquired Very High-Resolution Helicopter-Borne Images over Drifting Sea Ice Using COTS Sensors.

Observing sea ice by very high-resolution (VHR) images not only improves the quality of lower-resolution remote sensing products (e.g., sea ice concentration, distribution of melt ponds and pressure ridges, sea ice surface roughness, etc.) by providing details on the ground truth of sea ice, but also assists sea ice fieldwork. In this study, two fieldwork-based methods are proposed, one for the practical acquisition of VHR images over drifting Arctic sea ice using low-cost commercial off-the-shelf (COTS) sensors equipped on a helicopter, and the other for quantifying the compensating effect from continuously drifting sea ice that reduces geolocation uncertainty in the image mosaicking procedure. The drifting trajectory of the target ice was yielded from that recorded by an icebreaker that was tightly anchored to the floe and was then used to reversely compensate the locations of acquired VHR images. After applying the compensation, three-dimensional geolocation errors of the VHR images were decreased by 79.3% and 24.2% for two pre-defined image groups, respectively. The enhanced accuracy of the imaging locations was affected by imaging duration causing variable drifting distances of individual images. Further applicability of the mosaicked VHR image was discussed by comparing it with a TerraSAR-X synthetic aperture radar image containing the target ice, suggesting that the proposed methods can be used for precise comparison with satellite remote sensing products.

Semantic Segmentation of Building Roof in Dense Urban Environment with Deep Convolutional Neural Network: A Case Study Using GF2 VHR Imagery in China.

This paper presents a novel approach for semantic segmentation of building roofs in dense urban environments with a Deep Convolution Neural Network (DCNN) using Chinese Very High Resolution (VHR) satellite (i.e., GF2) imagery. To provide an operational end-to-end approach for accurately mapping build roofs with feature extraction and image segmentation, a fully convolutional DCNN with both convolutional and deconvolutional layers is designed to perform building roof segmentation. We selected typical cities with dense and diverse urban environments in different metropolitan regions of China as study areas, and sample images were collected over cities. High performance GPU-mounted workstations are employed to perform the model training and optimization. With the building roof samples collected over different cities, the predictive model with convolution layers is developed for building roof segmentation. The validation shows that the overall accuracy (OA) and the mean Intersection Over Union (mIOU) of DCNN-based semantic segmentation results are 94.67% and 0.85, respectively, and the CRF-refined segmentation results achieved OA of 94.69% and mIOU of 0.83. The results suggest that the proposed approach is a promising solution for building roof mapping with VHR images over large areas in dense urban environments with different building patterns. With the operational acquisition of GF2 VHR imagery, it is expected to develop an automated pipeline of operational built-up area monitoring, and the timely update of building roof map could be applied in urban management and assessment of human settlement-related sustainable development goals over large areas.

An Automatic Morphological Attribute Building Extraction Approach for Satellite High Spatial Resolution Imagery

A new morphological attribute building index (MABI) and shadow index (MASI) are proposed here for automatically extracting building features from very high-resolution (VHR) remote sensing satellite images. By investigating the associated attributes in morphological attribute filters (AFs), the proposed method establishes a relationship between AFs and the characteristics of buildings/shadows in VHR images (e.g., high local contrast, internal homogeneity, shape, and size). In the pre-processing step of the proposed work, attribute filtering was conducted on the original VHR spectral reflectance data to obtain the input, which has a high homogeneity, and to suppress elongated objects (potential non-buildings). Then, the MABI and MASI were calculated by taking the obtained input as a base image. The dark buildings were considered separately in the MABI to reduce the omission of the dark roofs. To better detect buildings from the MABI feature image, an object-oriented analysis and building-shadow concurrence relationships were utilized to further filter out non-building land covers, such as roads and bare ground, that are confused for buildings. Three VHR datasets from two satellite sensors, i.e., Worldview-2 and QuickBird, were tested to determine the detection performance. In view of both the visual inspection and quantitative assessment, the results of the proposed work are superior to recent automatic building index and supervised binary classification approach results.

Automatic Shadow Detection in Urban Very-High-Resolution Images Using Existing 3D Models for Free Training

Up-to-date 3D city models are needed for many applications. Very-high-resolution (VHR) images with rich geometric and spectral information and a high update rate are increasingly applied for the purpose of updating 3D models. Shadow detection is the primary step for image interpretation, as shadow causes radiometric distortions. In addition, shadow itself is valuable geometric information. However, shadows are often complicated and environment-dependent. Supervised learning is considered to perform well in detecting shadows when training samples selected from these images are available. Unfortunately, manual labeling of images is expensive. Existing 3D models have been used to reconstruct shadows to provide free, computer-generated training samples, i.e., samples free from intensive manual labeling. However, accurate shadow reconstruction for large 3D models consisting of millions of triangles is either difficult or time-consuming. In addition, due to inaccuracy and incompleteness of the model, and different acquisition time between 3D models and images, mislabeling refers to training samples that are shadows but labeled as non-shadows and vice versa. We propose a ray-tracing approach with an effective KD tree construction to feasibly reconstruct accurate shadows for a large 3D model. An adaptive erosion approach is first provided to remove mislabeling effects near shadow boundaries. Next, a comparative study considering four classification methods, quadratic discriminant analysis (QDA) fusion, support vector machine (SVM), K nearest neighbors (KNN) and Random forest (RF), is performed to select the best classification method with respect to capturing the complicated properties of shadows and robustness to mislabeling. The experiments are performed on Dutch Amersfoort data with around 20% mislabels and the Toronto benchmark by simulating mislabels from inverting shadows to non-shadows. RF is tested to give robust and best results with 95.38% overall accuracy (OA) and a value of 0.9 for kappa coefficient (KC) for Amersfoort and around 96% OA and 0.92 KC for Toronto benchmarks when no more than 50% of shadows are inverted. QDA fusion and KNN are tested to be robust to mislabels but their capability to capture complicated properties of shadows is worse than RF. SVM is tested to have a good capability to separate shadow and non-shadows but is largely affected by mislabeled samples. It is shown that RF with free-training samples from existing 3D models is an automatic, effective, and robust approach for shadow detection from VHR images.

Combined multiscale segmentation convolutional neural network for rapid damage mapping from postearthquake very high-resolution images

Classifying land use from postearthquake very high-resolution (VHR) images is challenging due to the complexity of objects in Earth surface after an earthquake. Convolutional neural network (CNN) exhibits satisfied performance in differentiating complex postearthquake objects, thanks to its automatic extraction of high-level features and accurate identification of target geo-objects. Nevertheless, in view of the scale variance of natural objects, the fact that CNN suffers from the fixed receptive field, the reduced feature resolution, and the insufficient training sample has severely contributed to its limitation in the rapid damage mapping. Multiscale segmentation technique is considered as a promising solution as it can generate the homogenous regions and provide the boundary information. Therefore, we propose a combined multiscale segmentation convolutional neural network (CMSCNN) method for postearthquake VHR image classification. First, multiscale training samples are selected based on segments derived from the multiscale segmentation. Then, CNN is directly trained to classify the original image to further produce the preliminary classification maps. To enhance the localization accuracy, the output of CNN is further refined using multiscale segmentations from fine to coarse iteratively to obtain the multiscale classification maps. As a result, the combination strategy is able to capture objects and image context simultaneously. Experimental results show that the proposed CMSCNN method can reflect the multiscale information of complex scenes and obtain satisfied classification results for mapping postearthquake damage using VHR remote sensing images.

Fusion High-and-Low-Level Features via Ridgelet and Convolutional Neural Networks for Very High-Resolution Remote Sensing Imagery Classification

Ridgelet can theoretically approximate low-level image features, and the ridgelet filter is constructed independently of the training sample, but it usually requires to preset a lot of parameters to achieve an ideal representation of complex scenes. Convolutional neural networks (CNNs) can adaptively exploit the high-level image features, but the training process depends on the selection of training samples. Hence, in this paper, the advantages of both the two aspects are fully considered for image feature extraction and a multi-resolution CNNs framework (MRCNNs) is proposed by fusing high-and-low-level features via ridgelet and CNNs. In the proposed method, multi-resolution low-level features are captured by the constructed ridgelet filters, and multi-resolution high-level features are exploited by the trainable convolutional filters. Then the ridgelet features and convolutional features are fused to reduce the dependence of CNNs on training samples, and consequently, improve the classification ability of CNNs. The proposed MRCNNs method is conducted on three very high-resolution (VHR) remote sensing images and compared with several deep-learning approaches. Experimental results present the effectiveness of the proposed method for VHR images classification.

Recurrent Multiresolution Convolutional Networks for VHR Image Classification

Classification of very high resolution (VHR) satellite images has three major challenges: 1) inherent low intra-class and high inter-class spectral similarities, 2) mismatching resolution of available bands, and 3) the need to regularize noisy classification maps. Conventional methods have addressed these challenges by adopting separate stages of image fusion, feature extraction, and post-classification map regularization. These processing stages, however, are not jointly optimizing the classification task at hand. In this study, we propose a single-stage framework embedding the processing stages in a recurrent multiresolution convolutional network trained in an end-to-end manner. The feedforward version of the network, called FuseNet, aims to match the resolution of the panchromatic and multispectral bands in a VHR image using convolutional layers with corresponding downsampling and upsampling operations. Contextual label information is incorporated into FuseNet by means of a recurrent version called ReuseNet. We compared FuseNet and ReuseNet against the use of separate processing steps for both image fusion, e.g. pansharpening and resampling through interpolation, and map regularization such as conditional random fields. We carried out our experiments on a land cover classification task using a Worldview-03 image of Quezon City, Philippines and the ISPRS 2D semantic labeling benchmark dataset of Vaihingen, Germany. FuseNet and ReuseNet surpass the baseline approaches in both quantitative and qualitative results.

MOSAICKING VERY-HIGH-RESOLUTION HELICOPTER-BORNE IMAGES ACQUIRED OVER DRIFTING ARCTIC SEA ICE USING COTS SENSORS

Abstract. In order to observe and record conditions of the sea ice efficiently and specifically during in-situ investigation with the support of icebreaker research vessel (IBRV), the very-high-resolution (VHR) imaging systems have been used in recent past. The VHR images are generally acquired lower altitude than cloud height, therefore, the images can be acquired even in unfavourable weather conditions for optical satellite image acquisition, and can be applied to comparison with various kinds of remote sensing datasets. However, producing mosaicked image using the VHR images have suffered from drift of sea ice. The sea ice drift interrupts simultaneous geotagging in overall study area as geographic locations of sea ice moves continuously; therefore, the mosaicked image generated from improperly geotagged individual image depicts a scene of ambiguous time. In this study, we present a case study of VHR sea ice image acquisition using a helicopter equipped with commercial off-the-shelf (COTS) geotagging and imaging sensors with a support of IBRV Araon in East Siberian Sea, Arctic Ocean. We also propose an image mosaicking strategy using the improperly geotagged VHR images acquired over drifting sea ice to decrease temporal and spatial ambiguity.

Very high resolution remote sensing image classification with SEEDS-CNN and scale effect analysis for superpixel CNN classification

ABSTRACTPixel-based convolutional neural network (CNN) has demonstrated good performance in the classification of very high resolution images (VHRI) from which abstract deep features are extracted. However, conventional pixel-based CNN demands large resources in terms of processing time and disk space. Therefore, superpixel CNN classification has recently become a focus of attention. We therefore propose a CNN based deep learning method combining superpixels extracted via energy-driven sampling (SEEDS) for VHRI classification. The approach consists of three main steps. First, based on the concept of geographic object-based image analysis (GEOBIA), the image is segmented into homogeneous superpixels using the SEEDS based superpixel segmentation method thereby decreasing the number of processing units. Second, the training data and testing data are extracted from the image and concatenated on a superpixel level at a variety of scales for CNN. Third, the training data are input to train the parameters of CNN and abstract deep features are extracted from the VHRI. Using these extracted deep features, we classify two VHRI data sets at single scales and multiple scales. To verify the effectiveness of SEEDS based CNN classification, the performance of SEEDS and three others superpixel segmentation algorithms are compared, and the superpixel extraction via SEEDS method was found to be the optimal superpixel segmentation approach for CNN classification. The scale effect on CNN classification accuracy was investigated by comparing the four superpixel segmentation methods. We found that (1) There is no strong evidence that using scales combinations is better than a single scale in some specific situations; (2) Natural objects with low complexity are not as sensitive to scale as artificial objects; (3) For a simple VHRI that contains clear artificial objects and simple texture, the classification result with multiple scales performs better a the single scale; (4) In contrast, for the complex VHRI containing a large number of complex objects, the classification result with a single small-scale best.