Final Image Resolution Research Articles

Gap-Filling Sentinel-1 Offshore Wind Speed Image Time Series Using Multiple-Point Geostatistical Simulation and Reanalysis Data

Offshore wind is expected to play a key role in future energy systems. Wind energy resource studies often call for long-term and spatially consistent datasets to assess the wind potential. Despite the vast amount of available data sources, no current means can provide relevant sub-daily information at a fine spatial scale (~1 km). Synthetic aperture radar (SAR) delivers wind field estimates over the ocean at fine spatial resolution but suffers from partial coverage and irregular revisit times. Physical model outputs, which are the basis of reanalysis products, can be queried at any time step but lack fine-scale spatial variability. To combine the advantages of both, we use the framework of multiple-point geostatistics to realistically reconstruct wind speed patterns at time instances for which satellite information is absent. Synthetic fine-resolution wind speed images are generated conditioned to coregistered regional reanalysis information at a coarser scale. Available simultaneous data sources are used as training data to generate the synthetic image time series. The latter are then evaluated via cross validation and statistical comparison against reference satellite data. Multiple realizations are also generated to assess the uncertainty associated with the simulation outputs. Results show that the proposed methodology can realistically reproduce fine-scale spatiotemporal variability while honoring the wind speed patterns at the coarse scale and thus filling the satellite information gaps in space and time.

An enhanced spatiotemporal fusion model with degraded fine-resolution images via relativistic generative adversarial networks

Limited by the sensor technology and budget, spatiotemporal fusion (STF) of remote sensing images (RSIs) is an effective strategy to obtain products with dense time and high spatial resolution. The STF methods generally require at least three observed images, and the predicted results depend on the reference date. The lack of high-quality fine-resolution images in practice makes the STF model difficult to apply, and few models consider the bias caused by different sensors. To solve the above problems, an enhanced STF model with degraded fine-resolution images via relativistic generative adversarial networks is proposed, called EDRGAN-STF, which is an end-to-end network with all bands trained simultaneously. To reduce the bias caused by different sensors, a degraded resolution version of the Landsat image with an arbitrary date is introduced into the STF model. The inputs of EDRGAN-STF only contain a MODIS image of the predicted date, a reference Landsat image and degraded version with an arbitrary date. EDRGAN-STF consists of a generator and a relativistic average least squares discriminator (RaLSD). A dual-stream residual dense block is designed to fully obtain the local and global spatial details and low-frequency information in the generator. A multihierarchical feature fusion block is designed to fuse global information. A spectral-spatial attention mechanism is employed in the generator, which focuses on important spectral bands and spatial features and enhances the reconstruction of critical regions. A new composite loss function is introduced to better optimize the designed STF model. To verify the capability of the EDRGAN-STF, extensive experiments are conducted on two typical Landsat-MODIS datasets, and results illustrate that EDRGAN-STF improves the STF accuracy and has great prospects for practical applications. Key policy highlights: The proposed EDRGAN-STF improves spatiotemporal fusion accuracy and has great prospects for practical applications. A degraded resolution version of the Landsat image with an arbitrary date is introduced into the spatiotemporal fusion model. A dual-stream residual dense block is designed to fully obtain the local and global spatial details and low-frequency information. A spectral-spatial attention mechanism focuses on important spectral bands and spatial features and enhances the reconstruction of critical regions. A multihierarchical feature fusion block fuses global information and a new composite loss function is introduced to better optimize the model.

Comparative Analysis and Comprehensive Trade-Off of Four Spatiotemporal Fusion Models for NDVI Generation

It is still difficult to obtain high-resolution and fast-updated NDVI data, and spatiotemporal fusion is an effective means to solve this problem. The purpose of this study is to carry out the comparative analysis and comprehensive trade-off of spatiotemporal fusion models for NDVI generation and to provide references for scholars in this field. In this study, four spatiotemporal fusion models (STARFM, ESTARFM, FSDAF, and GF-SG) were selected to carry out NDVI image fusion in grassland, forest, and farmland test areas, and three indicators of root mean square error (RMSE), average difference (AD), and edge feature richness difference (EFRD) were used. A detailed evaluation and analysis of the fusion results and comprehensive trade-off were carried out. The results show that: (1) all four models can predict fine-resolution NDVI images well, but the phenomenon of over-smoothing generally exists, which is more serious in high-heterogeneity areas; (2) GF-SG performed well in the evaluation of the three indicators, with the highest comprehensive trade-off score (CTS) of 0.9658. Followed by ESTARFM (0.9050), FSDAF (0.8901), and STARFM (0.8789); (3) considering the comparative analysis and comprehensive trade-off results of the three test areas and the three indicators, among the four models, GF-SG has the best accuracy in generating NDVI images. GF-SG is capable of constructing NDVI time series data with high spatial and temporal resolution.

Agricultural Field Boundary Delineation with Satellite Image Segmentation for High-Resolution Crop Mapping: A Case Study of Rice Paddy

Parcel-level cropland maps are an essential data source for crop yield estimation, precision agriculture, and many other agronomy applications. Here, we proposed a rice field mapping approach that combines agricultural field boundary extraction with fine-resolution satellite images and pixel-wise cropland classification with Sentinel-1 time series SAR (Synthetic Aperture Radar) imagery. The agricultural field boundaries were delineated by image segmentation using U-net-based fully convolutional network (FCN) models. Meanwhile, a simple decision-tree classifier was developed based on rice phenology traits to extract rice pixels with time series SAR imagery. Agricultural fields were then classified as rice or non-rice by majority voting from pixel-wise classification results. The evaluation indicated that SeresNet34, as the backbone of the U-net model, had the best performance in agricultural field extraction with an IoU (Intersection over Union) of 0.801 compared to the simple U-net and ResNet-based U-net. The combination of agricultural field maps with the rice pixel detection model showed promising improvement in the accuracy and resolution of rice mapping. The produced rice field map had an IoU score of 0.953, while the User‘s Accuracy and Producer‘s Accuracy of pixel-wise rice field mapping were 0.824 and 0.816, respectively. The proposed model combination scheme merely requires a simple pixel-wise cropland classification model that incorporates the agricultural field mapping results to produce high-accuracy and high-resolution cropland maps.

Spatio-temporal subpixel mapping with cloudy images

Spatio-temporal subpixel mapping (STSPM) has shown great potential for monitoring land surfaces, by generating land cover maps with both fine spatial and temporal resolutions. Selecting cloud-free fine spatial resolution images as ancillary data for STSPM can ensure that the temporal dependence term is measured for all subpixels, as in all current STSPM methods. However, such images are generally limited by cloud contamination, thereby resulting in great land cover changes between the available clear image and the desired fine spatial resolution land cover map. This research proposes a cloud-independent STSPM (C-STSPM) method to reconstruct the fine spatial resolution land cover maps by using cloudy images directly, which are assumed to have fewer land cover changes than temporally distant clear images. Cloud-independent spatio-temporal dependence was proposed in the presence of cloudy pixels. Experiments were performed under various cloud conditions involving 21 × 21 pairs of simulated cloudy images. The results demonstrate that by utilizing land cover information of clear pixels in cloudy images, more accurate prediction can be produced by C-STSPM compared to directly discarding those cloudy images, even if the number of cloud pixels increases to 95%. The advantage of C-STSPM is more evident when the clouds are distributed sparsely, which benefits from the increased number of clear pixels at the edge of the cloudy areas. Furthermore, a negative linear correlation was detected between the prediction accuracy and the ratio of overlapping cloudy pixels in the cloudy images. Moreover, the C-STSPM method helps to deal with abrupt changes occurred in the temporally distant cloud-free images by utilizing the temporally adjacent cloudy images with gradual land cover changes. Overall, the C-STSPM method provides a completely new solution to make fuller use of the widely existing cloudy images in multi-scale time-series images.

A simple method for estimating macroalgae area under clouds on MODIS imagery

The presence of clouds interferes with optical remote sensing monitoring of macroalgae blooms. To solve this problem, we propose a simple method for estimating macroalgae area under clouds (Area_cloud_GT) on MODIS imagery using the principle behind the lowpass filter. The method is based on a rectangle with clouds and eight identical adjacent rectangles surrounding it that contain macroalgae. The cloud rectangle is a central ‘pixel’ (Cloud) and the eight adjacent rectangles are ‘pixels’ GT1–GT8. The core operation is to calculate the central ‘pixel’ value, i.e., the macroalgae coverage rate in the Cloud rectangle. The macroalgae area detected by semi-simultaneous fine resolution images in the same region was taken as the ‘real’ value. A comparison of the estimation results and the ‘real’ value shown that the mean relative difference between them (MRD) was 30.09% when the time interval of the images within 10 minutes. When the time interval was over 3 hours, the MRD was more than 60%. The MRD increased significantly with increasing time interval because of the constant movement of the macroalgae and the limitations of the remote sensing image. The results indicate that this simple method is effective to a certain extent. These results can provide a reference for the quantitative analysis of green tide.

The Interplay between Spatial Urban Expansion and Morphologic Landscapes East of Cairo, Egypt Using Time Series Satellite Imagery

This study aims to monitor the magnitudes, rates, and directions of spatial urban expansions east of Cairo and their interactions with the initial morphologic landscapes. The approach relies on using CORONA satellite images acquired in 1969 with fine spatial resolution and time series images of Landsat and ASTER from 1984 to 2020. The CORONA images enable retrieval of the initial morphologic components, whereas the Landsat and ASTER images enable the spatial urban expansions to be mapped. The magnitudes of spatial urban expansions have been massive, in the order of 165 km2. These expansions have occurred through four main temporal phases with different spatial extents, rates, and directions in response to common urban policies and socioeconomic settings. Assessing the interactions between urban expansions and the morphology of watersheds in the study area indicates that the directions of urban expansion have been opposite to the geospatial orientations of the watersheds. In addition, significant urban areas in the order of ~8 km2 are under the direct threat of flash floods because they are misplaced within the valley floors of the studied watersheds. The study concludes that successful spatial urban expansion should consider the morphologic characteristics of the initial landscape for the purpose of maximizing interests and avoiding or reducing potential hazards against urban settlements.

A Novel Machine Learning–Based Gap-Filling of Fine-Resolution Remotely Sensed Snow Cover Fraction Data by Combining Downscaling and Regression

Abstract Satellite-based remotely sensed observations of snow cover fraction (SCF) can have data gaps in spatially distributed coverage from sensor and orbital limitations. We mitigate these limitations in the example fine-resolution Moderate Resolution Imaging Spectroradiometer (MODIS) data by gap-filling using auxiliary 1-km datasets that either aid in downscaling from coarser-resolution (5 km) MODIS SCF wherever not fully covered by clouds, or else by themselves via regression wherever fully cloud covered. This study’s prototype predicts a 1-km version of the 500-m MOD10A1 SCF target. Due to noncollocatedness of spatial gaps even across input and auxiliary datasets, we consider a recent gap-agnostic advancement of partial convolution in computer vision for both training and predictive gap-filling. Partial convolution accommodates spatially consistent gaps across the input images, effectively implementing a two-dimensional masking. To overcome reduced usable data from noncollocated spatial gaps across inputs, we innovate a fully generalized three-dimensional masking in this partial convolution. This enables a valid output value at a pixel even if only a single valid input variable and its value exist in the neighborhood covered by the convolutional filter zone centered around that pixel. Thus, our gap-agnostic technique can use significantly more examples for training (∼67%) and prediction (∼100%), instead of only less than 10% for the previous partial convolution. We train an example simple three-layer legacy super-resolution convolutional neural network (SRCNN) to obtain downscaling and regression component performances that are better than baseline values of either climatology or MOD10C1 SCF as relevant. Our generalized partial convolution can enable multiple Earth science applications like downscaling, regression, classification, and segmentation that were hindered by data gaps.

Fusing or filling: Which strategy can better reconstruct high-quality fine-resolution satellite time series?

High-quality fine-resolution satellite time series data are important for monitoring land surface dynamics in heterogeneous areas. However, the quality of raw satellite time series is affected by clouds and the revisit frequency. Currently, there are two major strategies to reconstruct high-quality fine-resolution time series: the interpolation of the missing pixels using auxiliary data from the same satellite (known as filling) and the fusion of fine-resolution and coarse-resolution images (known as fusing). These two strategies use different principles and input data to reach the same goal, but which one is superior in different scenarios is not known. Therefore, this study fills this research gap by comparing two representative methods from filling and fusing: the Neighborhood Similar Pixel Interpolator (NSPI) for filling and the Flexible Spatiotemporal DAta Fusion (FSDAF) for fusing. The potential factors affecting the accuracy of the two methods were investigated using two simulated experiments. The results show that (1) the accuracy of both methods decreases with the time interval between the image to be reconstructed and the auxiliary image; (2) NSPI is generally better than FSDAF for reconstructing images with small cloud patches but this superiority is insignificant in homogeneous areas; (3) the accuracy of NSPI significantly decreases with cloud size, and NSPI is worse than FSDAF for reconstructing images with large clouds; and (4) the performance of FSDAF is significantly affected by the scale difference between the fine- and coarse-resolution images, especially for heterogeneous areas. The findings of this study can help users select the appropriate method to reconstruct satellite time series for their specific applications.

An object-based sparse representation model for spatiotemporal image fusion

Many algorithms have been proposed for spatiotemporal image fusion on simulated data, yet only a few deal with spectral changes in real satellite images. An innovative spatiotemporal sparse representation (STSR) image fusion approach is introduced in this study to generate global dense high spatial and temporal resolution images from real satellite images. It aimed to minimize the data gap, especially when fine spatial resolution images are unavailable for a specific period. The proposed approach uses a set of real coarse- and fine-spatial resolution satellite images acquired simultaneously and another coarse image acquired at a different time to predict the corresponding unknown fine image. During the fusion process, pixels located between object classes with different spectral responses are more vulnerable to spectral distortion. Therefore, firstly, a rule-based fuzzy classification algorithm is used in STSR to classify input data and extract accurate edge candidates. Then, an object-based estimation of physical constraints and brightness shift between input data is utilized to construct the proposed sparse representation (SR) model that can deal with real input satellite images. Initial rules to adjust spatial covariance and equalize spectral response of object classes between input images are introduced as prior information to the model, followed by an optimization step to improve the STSR approach. The proposed method is applied to real fine Sentinel-2 and coarse Landsat-8 satellite data. The results showed that introducing objects in the fusion process improved spatial detail, especially over the edge candidates, and eliminated spectral distortion by preserving the spectral continuity of extracted objects. Experiments revealed the promising performance of the proposed object-based STSR image fusion approach based on its quantitative results, where it preserved almost 96.9% and 93.8% of the spectral detail over the smooth and urban areas, respectively.

Assessment of image reconstruction algorithm coupled with fine-resolution array of Cherenkov detectors

The ability to reconstruct fine-resolution images in a high-count-rate environment is an ongoing challenge to the fields of nuclear security, medicine, and high energy physics. This study presents the characterization and performance of an image reconstruction algorithm and detector array in such an environment. The detector array is composed of quartz Cherenkov radiators and lutetium–yttrium oxyorthosilicate inorganic scintillators detector elements with light collection via silicon photomultipliers (SiPM). The reconstruction algorithm was evaluated using ANSI testing standard N42.46-2008 for imaging performance of active interrogation systems for national security applications; this included spatial resolution, wire detection, and penetration studies. The array was tested using a 6-MVp pulsed photon beam where test objects were translated through the detector field of view demonstrating a capability to resolve a 2.05-mm wire at a source standoff of 2.2 m, a horizontal spatial resolution of 3 mm, and a contrast sensitivity of 1.5%.

Alpine glacier surface velocity measurement from UAV imagery – examining the effect of image resolution on the accuracy of results

The reliability and validity of the Glacier Surface Velocity (GSV) measurement results based on remote sensing datasets depends on the quality and spatial resolution of the image used. The typical pixel size of space-borne satellite imagery is often larger than the annual and inter-annual displacement of small alpine debris-covered glaciers. In addition, the pixel size of medium resolution satellite images (10–30 m), limits the size of a feature that can be matched. This is even more of an issue for glaciers located in arid and semi-arid environments (e.g. glaciers in Iran and high mountains of Asia) where flow velocities are not exceeding a few metres per year. Consequently, high-resolution data such as Unmanned Aerial Vehicle (UAV) images are required to calculate the surface velocity of such glaciers. However, the optimal resolution of UAV images is one of the most important challenges in GSV measurement. This paper explores the influence of UAV image resolution on the quality of GSV results. Analysis was carried out at 19 different resolutions of UAV images from 10 to 100 cm with a 5 cm bin over a debris-covered glacier, in Iran. COSI-Corr algorithm was used to perform the image correlation. To evaluate the accuracy of obtained results, manual digitisation was performed and the differences between manual GSV and frequency cross-correlator were assessed for all data sets. Moreover, displacement calculated for a stagnant off-glacier area was evaluated. While the quality of the results of the images between 10 and 30 cm is substantially the same, the obtained results indicate that the best result of GSV was not obtained using the finest image resolution. Results revealed that the highest correspondence between the measured GSV and manual digitisation was obtained in a 30 cm spatial resolution image. In addition, the 30 cm image resolution shows the minimum uncertainty over the off-glacier static area. Obtained results revealed that using too fine resolution images will lead to computational redundancy, while no improvement is observed in the accuracy of GSV results.

The spike-and-slab elastic net as a classification tool in Alzheimer's disease.

Alzheimer’s disease (AD) is the leading cause of dementia and has received considerable research attention, including using neuroimaging biomarkers to classify patients and/or predict disease progression. Generalized linear models, e.g., logistic regression, can be used as classifiers, but since the spatial measurements are correlated and often outnumber subjects, penalized and/or Bayesian models will be identifiable, while classical models often will not. Many useful models, e.g., the elastic net and spike-and-slab lasso, perform automatic variable selection, which removes extraneous predictors and reduces model variance, but neither model exploits spatial information in selecting variables. Spatial information can be incorporated into variable selection by placing intrinsic autoregressive priors on the logit probabilities of inclusion within a spike-and-slab elastic net framework. We demonstrate the ability of this framework to improve classification performance by using cortical thickness and tau-PET images from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) to classify subjects as cognitively normal or having dementia, and by using a simulation study to examine model performance using finer resolution images.

FCAU-Net for the Semantic Segmentation of Fine-Resolution Remotely Sensed Images

The semantic segmentation of fine-resolution remotely sensed images is an urgent issue in satellite image processing. Solving this problem can help overcome various obstacles in urban planning, land cover classification, and environmental protection, paving the way for scene-level landscape pattern analysis and decision making. Encoder-decoder structures based on attention mechanisms have been frequently used for fine-resolution image segmentation. In this paper, we incorporate a coordinate attention (CA) mechanism, adopt an asymmetric convolution block (ACB), and design a refinement fusion block (RFB), forming a network named the fusion coordinate and asymmetry-based U-Net (FCAU-Net). Furthermore, we propose novel convolutional neural network (CNN) architecture to fully capture long-term dependencies and fine-grained details in fine-resolution remotely sensed imagery. This approach has the following advantages: (1) the CA mechanism embeds position information into a channel attention mechanism to enhance the feature representations produced by the network while effectively capturing position information and channel relationships; (2) the ACB enhances the feature representation ability of the standard convolution layer and captures and refines the feature information in each layer of the encoder; and (3) the RFB effectively integrates low-level spatial information and high-level abstract features to eliminate background noise when extracting feature information, reduces the fitting residuals of the fused features, and improves the ability of the network to capture information flows. Extensive experiments conducted on two public datasets (ZY-3 and DeepGlobe) demonstrate the effectiveness of the FCAU-Net. The proposed FCAU-Net transcends U-Net, Attention U-Net, the pyramid scene parsing network (PSPNet), DeepLab v3+, the multistage attention residual U-Net (MAResU-Net), MACU-Net, and the Transformer U-Net (TransUNet). Specifically, the FCAU-Net achieves a 97.97% (95.05%) pixel accuracy (PA), a 98.53% (91.27%) mean PA (mPA), a 95.17% (85.54%) mean intersection over union (mIoU), and a 96.07% (90.74%) frequency-weighted IoU (FWIoU) on the ZY-3 (DeepGlobe) dataset.

A Flexible Reference-Insensitive Spatiotemporal Fusion Model for Remote Sensing Images Using Conditional Generative Adversarial Network

Due to the tradeoff between spatial and temporal resolutions of remote sensing images, spatiotemporal fusion models were proposed to synthesize the high spatiotemporal image series. Currently, spatiotemporal fusion models usually employ one coarse-resolution image acquired on a prediction date and at least another pair of coarse–fine resolution images close to the prediction time as references to derive the fine-resolution image on the prediction date. After years of development, the model accuracy has gained a certain improvement, but nearly, all the models require at least three image inputs and rigid time constraints must be applied to the references to guarantee the fusion accuracy. However, it is not always that easy to collect adequate data pairs for fine-resolution image series simulation in practice because of the bad weather condition or the time inconsistency between the coarse–fine resolution data sources, which causes some difficulties in the actual application. This article introduces the conditional generative adversarial network (CGAN) and switchable normalization technique into the spatiotemporal fusion problem and proposes a flexible deep network named the GAN-based SpatioTemporal Fusion Model (GAN-STFM) to reduce the number of model inputs and broke the time restriction on reference image selection. The GAN-STFM just needs a coarse-resolution image on the prediction date and another fine-resolution reference image at an arbitrary time in the same area for model inputs. As far as we know, this is the first spatiotemporal fusion model that requires only two images as model inputs and puts no restriction on the acquisition time of references. Even so, the GAN-STFM performs on par or better than other classical fusion models in the experiments. With this improvement, the data preparation for spatiotemporal fusion tends to be much easier than before, showing a promising perspective for practical applications.

Subpixel Change Detection Based on Improved Abundance Values for Remote Sensing Images

To achieve land cover change detection (LCCD) with both fine spatial and temporal resolutions from remote sensing images, subpixel mapping-based approaches have been widely studied in recent years. The fine spatial but coarse temporal resolution image and the coarse spatial but fine temporal image are used to accomplish LCCD by combining their advantages. However, the performance of subpixel mapping is easily affected by the accuracy of spectral unmixing, thereby reducing the reliability of LCCD. In this paper, a novel subpixel change detection scheme based on improved abundance values is proposed to tackle the aforementioned problem, in which the spatial distribution of fine spatial resolution image is borrowed to promote the accuracy of spectral unmixing. First, the coarse spatial resolution image is used to generate the original abundance image by the spectral unmixing method. Second, the spatial distribution information of the fine spatial resolution image is incorporated into the original abundance image to obtain improved abundance values. Third, the fine spatial resolution subpixel map can be generated by the subpixel mapping method using the improved abundance values. At last, the fine resolution change map can be obtained by comparing the subpixel map with the fine spatial resolution image. Experiments are conducted on a simulated dataset based on Landsat-7 images and two real datasets based on Landsat-8 and MODIS images. The results of the real datasets showed that the proposed method can effectively improve the performance of LCCD with an overall accuracy of approximately 1.26% and 0.79% to the existing methods.

Optimization-Based Hyperspectral Spatiotemporal Super-Resolution

Due to hardware limitations and financial considerations, it is challenging to acquire fine spatial and temporal resolution (FSFT) images, which leads to the interest in spatiotemporal fusion problems. Instead of directly obtaining costly FSFT images, an alternative is to fuse fine spatial, coarse temporal resolution (FSCT) images with coarse spatial, fine temporal resolution (CSFT) images. Unlike existing spatiotemporal methods which are only designed for multispectral images, this paper first proposes a new fusion framework for hyperspectral spatiotemporal super-resolution, termed HSTSR. In this paper, we first deal with the coarse temporal resolution issue by adopting the fast iterative shrinkage-thresholding algorithm (FISTA) to estimate the missing images at the intermediate time series. Then, we fuse the hyperspectral image and the multispectral image in each time series via coupled nonnegative matrix factorization (CNMF) to get FSFT hyperspectral images. Importantly, we can automatically estimate the associated spatial blurring and spectral downsampling matrices without prior satellite hardware information. Compared with other extended multispectral spatiotemporal methods, our method not only attains satisfying qualities significantly faster, but also requires much less input data.

Comparative Study of Fractional Vegetation Cover Estimation Methods Based on Fine Spatial Resolution Images for Three Vegetation Types

Comparative Study of Fractional Vegetation Cover Estimation Methods Based on Fine Spatial Resolution Images for Three Vegetation Types

Geographically Weighted Spatial Unmixing for Spatiotemporal Fusion

Spatiotemporal fusion is a technique applied to create images with both fine spatial and temporal resolutions by blending images with different spatial and temporal resolutions. Spatial unmixing (SU) is a widely used approach for spatiotemporal fusion, which requires only the minimum number of input images. However, ignorance of spatial variation in land cover between pixels is a common issue in existing SU methods. For example, all coarse neighbors in a local window are treated equally in the unmixing model, which is inappropriate. Moreover, the determination of the appropriate number of clusters in the known fine spatial resolution image remains a challenge. In this article, a geographically weighted SU (SU-GW) method was proposed to address the spatial variation in land cover and increase the accuracy of spatiotemporal fusion. SU-GW is a general model suitable for any SU method. Specifically, the existing regularized version and soft classification-based version were extended with the proposed geographically weighted scheme, producing 24 versions (i.e., 12 existing versions were extended to 12 corresponding geographically weighted versions) for SU. Furthermore, the cluster validity index of Xie and Beni (XB) was introduced to determine automatically the number of clusters. A systematic comparison between the experimental results of the 24 versions indicated that SU-GW was effective in increasing the prediction accuracy. Importantly, all 12 existing methods were enhanced by integrating the SU-GW scheme. Moreover, the identified most accurate SU-GW enhanced version was demonstrated to outperform two prevailing spatiotemporal fusion approaches in a benchmark comparison. Therefore, it can be concluded that SU-GW provides a general solution for enhancing spatiotemporal fusion, which can be used to update existing methods and future potential versions.

Scale–Relation Joint Decoupling Network for Remote Sensing Image Semantic Segmentation

As we all know, remote sensing (RS) images contain multi-scale and numerous RS objects, along with massive and complex spatial topological relationships, such as the adjacency, proximity relations of same-scale objects, and inclusion relations of cross-scale objects. However, the existing semantic segmentation methods have never explored the cross-scale relations, which are especially important when comes to the situation that the RS objects cannot be accurately identified, they could be supplemented by the surrounding contents. To address the above concern, we propose a scale-relation joint decoupling network (SRJDN) for the semantic segmentation of RS images by simultaneously considering decoupling scales and decoupling relations to excavate more complete relationships of multi-scale RS objects. The SRJDN is performed by following three steps, namely scale decoupling (SD), relation decoupling (RD), and fine-granularity guided fusion (FGF). The SD module uses dilated convolution with different rates to decouple RS objects into different scale feature groups, from small to large scales. Afterward, the RD considers all the spatial topological relationships and decouples these relationships according to the scale, which is divided into two parts, including same-scale relation extraction (SSRE) and cross-scale relation extraction (CSRE). The SSRE establishes the graph structures at each scale independently to mine the relationships of same-scale RS objects and the CSRE constructs the graph in a unified pattern between cross-scales to explore cross-scale target relationships. Third, the FGF module regards small-scale features as fine-granularity representation and applies its attention map to guide the learning of other scale features, which could mine more reliable and comprehensive saliency information and improve the feature consistency. Numerical experiments conducted on two large-scale fine-resolution RS image datasets empirically demonstrate the robustness of the proposed joint decoupling strategy and the effectiveness of the fine-granularity guided fusion in RS image semantic segmentation tasks.