Sub-pixel Mapping Research Articles

Unsupervised Bayesian Subpixel Mapping Autoencoder Network for Hyperspectral Images

Unsupervised subpixel mapping (SPM) of hyperspectral image (HSI) is a challenging task due to the difficulties to integrate different prior information and model constraints into a coherent framework. This paper presents a Bayesian neural network for unsupervised HSI SPM, which has the following characteristics. First, the deep image prior (DIP) achieved by a fully convolutional neural network (FCNN) is used to model the spatial correlation efficiently and adaptively in the subpixel label domain. Second, a discrete spectral mixture model (DSMM) is designed to leverage the forward model for enhanced SPM. Third, an auto-encoder architecture is designed to integrate the FCNN and the DSMM to allow efficient unsupervised representational learning using both data and knowledge. Fourth, an expectation-maximization approach is designed to solve the resulting maximum a posteriori problem, where a purified means approach extracts endmembers, and the gradient descent approach updates FCNN parameters for subpixel label estimation. Comparative experiments on both real and simulated HSIs demonstrate that the proposed method outperforms other state-of-the-art methods in terms of both numerical accuracies and visual subpixel mapping results.

Sub-pixel Resolution Thematic Map Based on Multi-source Data Fusion

Sub-pixel Resolution Thematic Map Based on Multi-source Data Fusion

Subpixel Mapping of Hyperspectral Image Based on Multiscale and Multifeature

Subpixel Mapping of Hyperspectral Image Based on Multiscale and Multifeature

A Spatial–Temporal Depth-Wise Residual Network for Crop Sub-Pixel Mapping from MODIS Images

To address the problem caused by mixed pixels in MODIS images for high-resolution crop mapping, this paper presents a novel spatial–temporal deep learning-based approach for sub-pixel mapping (SPM) of different crop types within mixed pixels from MODIS images. High-resolution cropland data layer (CDL) data were used as ground references. The contributions of this paper are summarized as follows. First, we designed a novel spatial–temporal depth-wise residual network (ST-DRes) model that can simultaneously address both spatial and temporal data in MODIS images in efficient and effective manners for improving SPM accuracy. Second, we systematically compared different ST-DRes architecture variations with fine-tuned parameters for identifying and utilizing the best neural network architecture and hyperparameters. We also compared the proposed method with several classical SPM methods and state-of-the-art (SOTA) deep learning approaches. Third, we evaluated feature importance by comparing model performances with inputs of different satellite-derived metrics and different combinations of reflectance bands in MODIS. Last, we conducted spatial and temporal transfer experiments to evaluate model generalization abilities across different regions and years. Our experiments show that the ST-DRes outperforms the other classical SPM methods and SOTA backbone-based methods, particularly in fragmented categories, with the mean intersection over union (mIoU) of 0.8639 and overall accuracy (OA) of 0.8894 in Sherman County. Experiments in the datasets of transfer areas and transfer years also demonstrate better spatial–temporal generalization capabilities of the proposed method.

Mapping of ferric (Fe3+) and ferrous (Fe2+) iron oxides distribution using ASTER and Landsat 8 OLI data, in Negash Lateritic iron deposit, Northern Ethiopia

ABSTRACT Iron plays an important role in industrial and engineering fields development of a country and as such there is an enormous demand for iron in Ethiopia. However, a search for this valuable primary mineral resource exploration remains challenging and costly. Therefore, this study aims to map iron oxide minerals using Landsat-8/operational land imager (OLI) and advanced space-borne thermal emission and reflection (ASTER) satellite imagery in Negash Lateritic iron deposit, Northern Ethiopia to ease the costs and reduce the time. Different image processing techniques such as band ratio, selective principal component analysis, linear spectral unmixing, and mixture-tuned matched filter were used to produce iron oxide maps. Minimum noise fraction (MNF), pixel purity index (PPI), and N-dimensional visualizer were also applied to extract endmembers in the automated spectral hourglass wizard. In addition to this, the enhanced image thresholding and scatter plot were used to map the potential areas. Ferric iron oxide band ratio of ASTER mapped maximum area of 62.1 km2 followed by a laterite band ratio of ASTER covering 57.8 km2. The result was validated using existing iron oxide polygons and the outcome obtained from selective PCA shows a strong match with the existing iron oxide polygons. The sub-pixel mapping techniques show poor accuracy in mapping goethite and hematite relative to the pixel level. Thus, it is evident from the results that ASTER mapped better than Landsat 8 OLI for band ratios of selective PCA, unmixing, MTMF, and mineralized areas while characterizing with limited fieldwork.

Spatio-temporal subpixel mapping with cloudy images

Spatio-temporal subpixel mapping (STSPM) has shown great potential for monitoring land surfaces, by generating land cover maps with both fine spatial and temporal resolutions. Selecting cloud-free fine spatial resolution images as ancillary data for STSPM can ensure that the temporal dependence term is measured for all subpixels, as in all current STSPM methods. However, such images are generally limited by cloud contamination, thereby resulting in great land cover changes between the available clear image and the desired fine spatial resolution land cover map. This research proposes a cloud-independent STSPM (C-STSPM) method to reconstruct the fine spatial resolution land cover maps by using cloudy images directly, which are assumed to have fewer land cover changes than temporally distant clear images. Cloud-independent spatio-temporal dependence was proposed in the presence of cloudy pixels. Experiments were performed under various cloud conditions involving 21 × 21 pairs of simulated cloudy images. The results demonstrate that by utilizing land cover information of clear pixels in cloudy images, more accurate prediction can be produced by C-STSPM compared to directly discarding those cloudy images, even if the number of cloud pixels increases to 95%. The advantage of C-STSPM is more evident when the clouds are distributed sparsely, which benefits from the increased number of clear pixels at the edge of the cloudy areas. Furthermore, a negative linear correlation was detected between the prediction accuracy and the ratio of overlapping cloudy pixels in the cloudy images. Moreover, the C-STSPM method helps to deal with abrupt changes occurred in the temporally distant cloud-free images by utilizing the temporally adjacent cloudy images with gradual land cover changes. Overall, the C-STSPM method provides a completely new solution to make fuller use of the widely existing cloudy images in multi-scale time-series images.

Bias of area counted from sub-pixel map: Origin and correction

With the increasingly widespread use of sub-pixel mapping techniques in land cover/use mapping, more accurate area information is often required for a specific land cover type in a particular study region. However, the bias of area counted from sub-pixel maps (called area bias below), and the inadequate understanding of the area bias's origin and influential factors pose a challenge to using this information accurately. Traditional model-assisted estimators combining the map and the reference sample showed unreliable performances in the case of small sample sizes collected in target regions. This work presented a theoretical analysis of the origin of area bias. It then proposed a novel bias-adjusted estimator which can effectively deal with the small sample sizes. The theoretical analysis illustrated that area bias mainly originates from two terms, i.e., the abundance-dependent error and the probability distribution of abundances. We next developed a stratified bias-adjusted area estimator named the two-term method (TTM) by incorporating the sub-pixel map and a reference sample obtained from both target and external regions. We validated the effects of different sub-pixel mapping methods, different spatial resolutions, the varying spatial structures of statistical units on area bias, and the performance of TTM in correcting the biased areas in multiple cases. The results showed that area bias varied from zero to approximately 20% with the variation of three influential factors. TTM effectively corrected the biased area values to nearly the true values, showing approximate equivalence with the traditional stratified regression estimator (STRE) when adequate reference samples are collected sorely inside target regions. However, in cases of small samples from target regions, TTM showed significant superiority over STRE in reducing the variance and MSE due to the incorporation of external reference samples. We conclude that the theoretical analysis resulted in a better understanding of area bias counted from sub-pixel maps and an improved area estimator for dealing with the cases of small sample sizes inside target regions.

Remote Sensing Image Information Extraction and Application Based on Improved Pixel Exchange Algorithm

High resolution images can better reflect the size, shape and structural characteristics of ground objects, but due to factors such as purchase cost and observation period, it is often difficult to meet the practical application needs. Sub-pixel mapping technology can effectively improve the resolution of the results, which solves the above problem to some extent. The relevant algorithms of sub-pixel mapping include mixed pixel decomposition, end pixel extraction, sub-pixel positioning and other sub fields. The most classical sub-pixel positioning methods often use spatial correlation to define the positioning criteria. In this paper, NDWI and OTSU segmentation are used to constrain sub-pixel decomposition to some extent, which improves the performance of PSA algorithm, so as to more accurately extract river shoreline information, improve the resolution of monitoring on shoreline erosion collapse and dynamic change in the bank collapse area, and enhance the applicability of satellite remote sensing in the bank collapse monitoring field.

Development of a Novel Burned-Area Subpixel Mapping (BASM) Workflow for Fire Scar Detection at Subpixel Level

The accurate detection of burned forest area is essential for post-fire management and assessment, and for quantifying carbon budgets. Therefore, it is imperative to map burned areas accurately. Currently, there are few burned-area products around the world. Researchers have mapped burned areas directly at the pixel level that is usually a mixture of burned area and other land cover types. In order to improve the burned area mapping at subpixel level, we proposed a Burned Area Subpixel Mapping (BASM) workflow to map burned areas at the subpixel level. We then applied the workflow to Sentinel 2 data sets to obtain burned area mapping at subpixel level. In this study, the information of true fire scar was provided by the Department of Emergency Management of Hunan Province, China. To validate the accuracy of the BASM workflow for detecting burned areas at the subpixel level, we applied the workflow to the Sentinel 2 image data and then compared the detected burned area at subpixel level with in situ measurements at fifteen fire-scar reference sites located in Hunan Province, China. Results show the proposed method generated successfully burned area at the subpixel level. The methods, especially the BASM-Feature Extraction Rule Based (BASM-FERB) method, could minimize misclassification and effects due to noise more effectively compared with the BASM-Random Forest (BASM-RF), BASM-Backpropagation Neural Net (BASM-BPNN), BASM-Support Vector Machine (BASM-SVM), and BASM-notra methods. We conducted a comparison study among BASM-FERB, BASM-RF, BASM-BPNN, BASM-SVM, and BASM-notra using five accuracy evaluation indices, i.e., overall accuracy (OA), user’s accuracy (UA), producer’s accuracy (PA), intersection over union (IoU), and Kappa coefficient (Kappa). The detection accuracy of burned area at the subpixel level by BASM-FERB’s OA, UA, IoU, and Kappa is 98.11%, 81.72%, 74.32%, and 83.98%, respectively, better than BASM-RF’s, BASM-BPNN’s, BASM-SVM’s, and BASM-notra’s, even though BASM-RF’s and BASM-notra’s average PA is higher than BASM-FERB’s, with 89.97%, 91.36%, and 89.52%, respectively. We conclude that the newly proposed BASM workflow can map burned areas at the subpixel level, providing greater accuracy in regards to the burned area for post-forest fire management and assessment.

Sub-pixel building area mapping based on synthetic training data and regression-based unmixing using Sentinel-1 and -2 data

ABSTRACT The identification of buildings has become a major research focus of settlement mapping with Earth Observation data. Building area or building footprint data is particularly required in research related to population, such as disaster risk management or urban health. This study examined the suitability of machine learning regression-based unmixing for quantifying the pixel-wise share of building area with decametre resolution Copernicus Sentinel-1 and Sentinel-2 imagery. Compared to using a single-step approach directly estimating building area, leading to an over-estimation of building area compared to non-building impervious surface area due to feature similarity, the introduction of a hierarchical approach considerably improved mapping results. While the original mapping resolution was 10 m, we found that building area was most accurately mapped starting at a spatial resolution of 100 m – a resolution well suitable for many urban applications. The proposed approach is widely transferable in space as it used spatially robust spectral-temporal metrics from time series imagery and as its requirements for training data are very limited.

Generating continuous fine-scale land cover mapping by edge-guided maximum a posteriori based spatiotemporal sub-pixel mapping

Global land cover mapping activities are of great important for retrieving the environment we are living in. However, the trade-off between spatial and temporal resolution makes it difficult to obtain the continuous fine-scale land cover product for detail and frequent land surface analysis. To overcome the difficulty, this study proposed a modified maximum a posteriori (MAP) based spatiotemporal sub-pixel mapping (SSM), for a long-term and fine-scale land cover mapping. The proposed approach consists of three parts, i.e., data fidelity term, spatial regularization term, and enhanced temporal regularization term, which can make use of historical fine-scale image for a better super-resolution of the current coarse image. One regional experiment was implemented for the algorithm validation, and one real experiment was investigated, by generating continuous land cover mapping products (2010–2015) at Landsat-like spatial resolution from time series of MODIS images. Compared with traditional methods, the proposed approach reconstructed more pleasant spatial details and had greater quantitative accuracy, outperforming the state-of-the-art spatiotemporal sub-pixel mapping method by averagely 2.58% for all the inspected dates in OA metric. This study not only provides a generalized research framework for generating continuous fine-scale land cover product from daily available MODIS images, but also validates the usability of the time series products for detail land cover monitoring, and reveals a finding that the study area of Wuhan is a well practitioner of the sustainable development policy during its urbanization process.

Spectral–Spatial Fusion Sub-Pixel Mapping Based on Deep Neural Network

Sub-pixel mapping (SPM) has been widely adopted to alleviate the mixed pixel problem in hyperspectral image, as an extension of spectral unmixing (SU), providing a way to observe the spatial location of the endmember within mixed pixel. However, most of the SPM methods are unmixing-then-mapping (UTM), i.e., SPM process relies on the abundance images generated from SU, in which process uncertainty inherently exists and would be propagated to SPM. Furthermore, the prior knowledge toward the sub-pixel scale distribution is mainly model-driven/handcrafted, which has limitation for geographical-realistic distribution representation. In this letter, we proposed spectral–spatial fusion SPM based on deep neural network (SSNET), to realize the integrative modeling of SU and SPM problem in a unified network fashion to avoid uncertainty accumulation in UTM process, and it can simultaneously generate SU result and SPM result. Besides, SSNET provides a supervised manner to learn prior knowledge with external exemplar pairs of low- and high-resolution images for a geographical-realistic distribution representation. The experiment with two hyperspectral images validated the superiority of the proposed SSNET.

A Joint Spectral Unmixing and Subpixel Mapping Framework Based on Multiobjective Optimization

Conventional subpixel mapping (SPM) is performed based on the abundance maps obtained by spectral unmixing (SU), to interpret the mixed pixels and improve the mapping resolution for hyperspectral remote-sensing imagery. However, the SU and SPM tasks are separately conducted, so that the unmixing error is propagated to SPM, and the mapping result is strongly reliant on the quality of the abundance maps. In this article, a novel joint SPM and SU framework (MO_SUSM) based on multiobjective optimization is proposed to simultaneously perform unmixing and mapping. Specifically, the multiobjective joint optimization model with a data fidelity term and a Laplacian prior term is constructed for SU and SPM. For the data fidelity term, since the unmixing result can be recovered by downsampling the mapping result, the unmixing model is joined with the mapping model by the downsampling matrix, so that the reconstruction errors of the unmixing and mapping results can be minimized together. Meanwhile, the Laplacian prior term is used to maximize the spatial dependence of the mapping result and provide the spatial constraint for SU. In addition, the multiobjective optimization algorithm with local search is designed to search for the optimal unmixing and mapping results that can balance the objective terms. Since the two objective terms are dynamically integrated during optimization, there is no need to set sensitive weights for the objectives combination. Four experiments were conducted on hyperspectral images of various data sources, including ground, airborne, and satellite images. The unmixing results show that MO_SUSM can reduce the unmixing error and can improve the quality of the abundance maps. The mapping results show that MO_SUSM can alleviate the dependence of SPM on the abundance maps and can improve the mapping accuracy.

Subpixel Change Detection Based on Improved Abundance Values for Remote Sensing Images

To achieve land cover change detection (LCCD) with both fine spatial and temporal resolutions from remote sensing images, subpixel mapping-based approaches have been widely studied in recent years. The fine spatial but coarse temporal resolution image and the coarse spatial but fine temporal image are used to accomplish LCCD by combining their advantages. However, the performance of subpixel mapping is easily affected by the accuracy of spectral unmixing, thereby reducing the reliability of LCCD. In this paper, a novel subpixel change detection scheme based on improved abundance values is proposed to tackle the aforementioned problem, in which the spatial distribution of fine spatial resolution image is borrowed to promote the accuracy of spectral unmixing. First, the coarse spatial resolution image is used to generate the original abundance image by the spectral unmixing method. Second, the spatial distribution information of the fine spatial resolution image is incorporated into the original abundance image to obtain improved abundance values. Third, the fine spatial resolution subpixel map can be generated by the subpixel mapping method using the improved abundance values. At last, the fine resolution change map can be obtained by comparing the subpixel map with the fine spatial resolution image. Experiments are conducted on a simulated dataset based on Landsat-7 images and two real datasets based on Landsat-8 and MODIS images. The results of the real datasets showed that the proposed method can effectively improve the performance of LCCD with an overall accuracy of approximately 1.26% and 0.79% to the existing methods.

Bayesian Subpixel Mapping of Hyperspectral Imagery via Discrete Endmember Variability Mixture Model and Markov Random Field

Although Bayesian methods have been very effective for spatial-spectral analysis of hyperspectral imagery (HSI), they had not been fully explored for enhanced sub-pixel mapping (SPM) by simultaneously considering several key issues i.e., endmember variability, the discrete nature of subpixel class labels and the spatial information in HSI. Therefore, we propose a new Bayesian SPM method based on discrete endmember variability mixture model (DEMM) and Markov random field (MRF), which has three main characteristics. First, DEMM allows the advanced SPM by completely accounting for the endmember-abundance patterns of each pixel to accommodate the endmember variability, the discrete hidden class label field of subpixels while taking into account the noise heterogeneity effect. Second, the discrete class label field modeled by MRF together with the DEMM, which can be integrated into a novel Bayesian model to better exploit the spatial contextual and spectral information. Third, the resulting Bayesian model can be efficiently solved by a designed expectation-maximization (EM) iteration, where E-step estimates the subpixel class label field using a simulated annealing (SA) algorithm and M-step estimates the endmembers for each pixel in HSI using alternating nonnegative least squares (ANLS) approach. The experimental results on three HSI datasets demonstrate that the proposed approach outperforms previously available SPM techniques.

Subpixel Mapping Based on Multisource Remote Sensing Fusion Data for Land-Cover Classes

Subpixel mapping (SPM) based on multisource remote sensing fusion data (MRSFD) for land-cover classes, called SPM-MRSFD, is proposed in this letter. First, the original hyperspectral image and the auxiliary panchromatic image are fused to produce the high spatial and spectral resolution fused image by pan-sharpening technology. Second, the fused image with spatial-spectral information and the auxiliary digital surface model (DSM) of light detection and ranging (LiDAR) with elevation information are fused to obtain the MRSFD with spatial-spectral elevation information by feature fusion. Finally, the fractional images with the proportions of subpixels belonging to land-cover classes are derived by unmixing the MRSFD, and the classes labels are allocated to subpixels to obtain the final SPM result according to these proportions' information. The main contribution of this work is that multiple types of auxiliary information (i.e., spatial-spectral elevation information) from MRSFD is fully utilized and the accuracy of SPM result is improved. Experimental results show that SPM-MRSFD obtains more accurate mapping results than state-of-the-art SPM methods.

Fast and Slow Changes Constrained Spatio-Temporal Subpixel Mapping

Fast and Slow Changes Constrained Spatio-Temporal Subpixel Mapping

Generating 2m fine-scale urban tree cover product over 34 metropolises in China based on deep context-aware sub-pixel mapping network

Contrast to the global forest, few trees live in cities but contribute significantly to urban environment and human health. However, the classical satellite-derived land cover/forest cover products with limited resolution are not fine enough for the identification of urban tree, which is usually appeared in small size and intersected with infrastructure. To relieve the dilemma, this study developed an urban tree specific sub-pixel mapping (SPM) architecture with deep learning approach, which aimed to generate 2m fine-scale urban tree cover product from 10 m Sentinel-2 images for large-scale area of 34 metropolises in China. The proposed approach has remarkable reconstruction ability for delineating the contextual characteristic of the urban tree patterns, and reliable generalization ability to large-scale area. In addition, this study creates a large-volume urban tree cover dataset (UTCD) with 0.13 billion urban tree samples at 2 m resolution, which fills the deficiency of standard dataset in urban tree cover research field. Quantitative analysis of our products was conducted on two typical study sites of Beijing and Wuhan. The results show that our products recover averagely more than 58.72% of urban tree covers that have been underestimated in the existing land cover/forest cover products, and outperforms the state-of-the-art approach both visually and quantitatively, by averagely 11.31% improvement in overall accuracy. From our annual products during 2016–2020, we found an evolution characteristic of urban tree cover: it is more stable in developed cities like Beijing, while more fluctuated in developing cities like Wuhan, and the alteration are usually concentrated at the outer-ring of downtown, which may be caused by the municipal planning and the land development of real estate industry.

Optimization scheme of stereogram synthesis algorithm with wide-activated deep residual network super-resolution for cylindrical grating display

An optimization scheme based on wide-activated deep residual network super-resolution (WDSR) is proposed for the stereoscopic image synthesis algorithm of a cylindrical grating display. Unlike the traditional synthesis scheme, the WDSR algorithm is used to replace, partially or fully, the interpolation algorithm to reduce the antialiasing effect of the image. Thereafter, the viewpoint mapping process is accelerated based on the subpixel mapping table by making masks. Subsequently, a stereoscopic image with a better quality is obtained. The experimental results demonstrate that the proposed scheme can improve the signal-to-noise ratio and quality of the stereoscopic composite image when the same subviewpoint image is used for verification.

Interpolation sub-pixel mapping based on multiple subpixel shifted images based on a consideration of the point spread function effect

ABSTRACT In this paper, the interpolation sub-pixel mapping (ISM) based on multiple sub-pixel shifted images (MSSIs) based on a consideration of the point spread function (PSF) effect, named as MSSI-PSF, is proposed. In the proposed MSSI-PSF, MSSIs are first unmixed to produce the coarse fractional images. Second, under considering the PSF effect, the improved coarse fractional images are obtained by applying the area-to-point kriging (ATPK) and then the ideal square-wave filter to the coarse fractional images. Third, the improved coarse fractional images are upsampled by interpolation to obtain the upsampled fractional images, and then the upsampled fractional images are integrated to obtain a fine fractional image. Finally, according to the fine fractional images from all MSSIs, the class allocation method is utilized to assign class labels to sub-pixels to obtain the final mapping result. The experimental results show that the proposed MSSI-PSF model produces better mapping result than the existing ISM models.