Desired High Spatial Resolution Research Articles

Multiscale Spatial–Spectral Interaction Transformer for Pan-Sharpening

Pan-sharpening methods based on deep neural network (DNN) have produced state-of-the-art fusion performance. However, DNN-based methods mainly focus on the modeling of the local properties in low spatial resolution multispectral (LR MS) and panchromatic (PAN) images by convolution neural networks. The global dependencies in the images are ignored. To capture the local and global properties of the images concurrently, we propose a multiscale spatial–spectral interaction transformer (MSIT) for pan-sharpening. Specifically, we construct the multiscale sub-networks containing convolution–transformer encoder to extract the local and global features at different scales from LR MS and PAN images, respectively. Then, a spatial–spectral interaction attention module (SIAM) is designed to merge the features at each scale. In SIAM, the interaction attention is used to decouple the spatial and spectral information efficiently for the enhancement of complementarity and the reduction of redundancy in the extracted features. The features from different scales are further integrated into a multiscale reconstruction module (MRM) to generate the desired high spatial resolution multispectral image, in which the spatial and spectral information is fused scale by scale. The experiments on reduced- and full-scale datasets demonstrate that the proposed MSIT can produce better results in terms of visual and numerical analysis when compared with state-of-the-art methods.

Spatial and Spectral Extraction Network With Adaptive Feature Fusion for Pansharpening

Pansharpening methods based on deep neural networks (DNNs) have been attracting great attention due to their powerful representation capabilities. In this article, to combine the feature maps from different subnetworks efficiently, we propose a novel pansharpening method based on a spatial and spectral extraction network (SSE-Net). Different from the other methods based on DNNs that directly concatenate the features from different subnetworks, we design adaptive feature fusion modules (AFFMs) to merge these features according to their information content. First, the spatial and spectral features are extracted by the subnetworks from low spatial resolution multispectral (LR MS) and panchromatic (PAN) images. Then, by fusing the features at different levels, the desired high spatial resolution MS (HR MS) images are generated by the fusion network consisting of AFFMs. In the fusion network, the features from different subnetworks are integrated adaptively, and the redundancy among them is reduced. Moreover, the spectral ratio loss and the gradient loss are defined to ensure the effective learning of spatial and spectral features. The spectral ratio loss captures the nonlinear relationships among the bands in the MS image to reduce the spectral distortions in the fusion result. Extensive experiments were conducted on QuickBird and GeoEye-1 satellite datasets. Visual and numerical results demonstrate that the proposed method produces better fusion results compared with literature techniques. The source code is available at <uri>https://github.com/RSMagneto/SSE-Net</uri>.

Integration of Large-Scale Variable Renewable Energy Sources into the Future European Power System: On the Curtailment Challenge

The future European power system is projected to rely heavily on variable renewable energy sources (VRES), primarily wind and solar generation. However, the difficulties inherent to storing the primary energy of these sources is expected to pose significant challenges in terms of their integration into the system. To account for the high variability of renewable energy sources VRES, a novel pan-European dispatch model with high spatio-temporal resolution including load shifting is introduced here, providing highly detailed information regarding renewable energy curtailments for all Europe, typically underestimated in studies of future systems. which also includes modeling of load shifting. The model consists of four separate levels with different approaches for modeling thermal generation flexibility, storage units and demand as well as with spatial resolutions and generation dispatch formulations. Applying the developed model for the future European power system follows the results of corresponding transmission expansion planning studies, which are translated into the desired high spatial resolution. The analysis of the “large scale-RES” scenario for 2050 shows considerable congestion between northern and central Europe, which constitutes the primary cause of VRES curtailments of renewables. In addition, load shifting is shown to mostly improve the integration of solar energy into the system and not wind, which constitutes the dominant energy source for this scenario. Finally, the analysis of the curtailments time series using ideal converters shows that the best locations for their exploitation can be found in western Ireland and western Denmark.

A New Variational Approach Based on Proximal Deep Injection and Gradient Intensity Similarity for Spatio-Spectral Image Fusion

Pansharpening is a very debated spatio-spectral fusion problem. It refers to the fusion of a high spatial resolution panchromatic image with a lower spatial but higher spectral resolution multispectral image in order to obtain an image with high resolution in both the domains. In this article, we propose a novel variational optimization-based (VO) approach to address this issue incorporating the outcome of a deep convolutional neural network (DCNN). This solution can take advantages of both the paradigms. On one hand, higher performance can be expected introducing machine learning (ML) methods based on the training by examples philosophy into VO approaches. On other hand, the combination of VO techniques with DCNNs can aid the generalization ability of these latter. In particular, we formulate a $\ell _2$-based proximal deep injection term to evaluate the distance between the DCNN outcome, and the desired high spatial resolution multispectral image. This represents the regularization term for our VO model. Furthermore, a new data fitting term measuring the spatial fidelity is proposed. Finally, the proposed convex VO problem is efficiently solved by exploiting the framework of the alternating direction method of multipliers (ADMM), thus guaranteeing the convergence of the algorithm. Extensive experiments both on simulated, and real datasets demonstrate that the proposed approach can outperform state-of-the-art spatio-spectral fusion methods, even showing a significant generalization ability. Please find the project page at https://liangjiandeng.github.io/Projects_Res/DMPIF_2020jstars.html.

Remote sensing image fusion based on two-stream fusion network

Remote sensing image fusion (also known as pan-sharpening) aims at generating a high resolution multi-spectral (MS) image from inputs of a high spatial resolution single band panchromatic (PAN) image and a low spatial resolution multi-spectral image. Inspired by the astounding achievements of convolutional neural networks (CNNs) in a variety of computer vision tasks, in this paper we propose a Two-stream Fusion Network (TFNet) to address the problem of pan-sharpening. Unlike many previous CNN based methods that consider pan-sharpening as a super-resolution problem and perform pan-sharpening through mapping the stacked PAN and MS to the target high resolution MS image, the proposed TFNet aims to fuse PAN and MS images in feature domain and reconstruct the pan-sharpened image from the fused features. The TFNet mainly consists of three parts. The first part is comprised of two networks extracting features from PAN and MS images, respectively. The subsequent network fuses them together to form compact features that represent both spatial and spectral information of PAN and MS images, simultaneously. Finally, the desired high spatial resolution MS image is recovered from the fused features through an image reconstruction network. Experiments on Quickbird and GaoFen-1 images demonstrate that the proposed TFNet can fuse PAN and MS images effectively, and produce pan-sharpened images competitive with even superior to state of the arts.

Spatial and Spectral Image Fusion Using Sparse Matrix Factorization

In this paper, we present a novel spatial and spectral fusion model (SASFM) that uses sparse matrix factorization to fuse remote sensing imagery with different spatial and spectral properties. By combining the spectral information from sensors with low spatial resolution (LSaR) but high spectral resolution (HSeR) (hereafter called HSeR sensors), with the spatial information from sensors with high spatial resolution (HSaR) but low spectral resolution (LSeR) (hereafter called HSaR sensors), the SASFM can generate synthetic remote sensing data with both HSaR and HSeR. Given two reasonable assumptions, the proposed model can integrate the LSaR and HSaR data via two stages. In the first stage, the model learns from the LSaR data a spectral dictionary containing pure signatures, and in the second stage, the desired HSaR and HSeR data are predicted using the learned spectral dictionary and the known HSaR data. The SASFM is tested with both simulated data and actual Landsat 7 Enhanced Thematic Mapper Plus (ETM+) and Terra Moderate Resolution Imaging Spectroradiometer (MODIS) acquisitions, and it is also compared to other representative algorithms. The experimental results demonstrate that the SASFM outperforms other algorithms in generating fused imagery with both the well-preserved spectral properties of MODIS and the spatial properties of ETM+. Generated imagery with simultaneous HSaR and HSeR opens new avenues for applications of MODIS and ETM+.

Adjustable Model-Based Fusion Method for Multispectral and Panchromatic Images

In this paper, an adjustable model-based image fusion method for multispectral (MS) and panchromatic (PAN) images is developed. The relationships of the desired high spatial resolution (HR) MS images to the observed low-spatial-resolution MS images and HR PAN image are formulated with image observation models. The maximum a posteriori framework is employed to describe the inverse problem of image fusion. By choosing particular probability density functions, the fused HR MS images are solved using a gradient descent algorithm. In particular, two functions are defined to adaptively determine most regularization parameters using the partially fused results at each iteration, retaining one parameter to adjust the tradeoff between the enhancement of spatial information and the maintenance of spectral information. The proposed method has been tested using QuickBird and IKONOS images and compared to several known fusion methods using quantitative evaluation indices. The experimental results verify the efficacy of this method.

Improving Crop Model Inference Through Bayesian Melding With Spatially Varying Parameters

An objective for applying a Crop Simulation Model (CSM) in precision agriculture is to explain the spatial variability of crop performance and to help guide decisions related to the site-specific management of crop inputs. CSMs require inputs related to soil, climate, management, and crop genetic information to simulate crop yield. In practice, however, measuring these inputs at the desired high spatial resolution is prohibitively expensive. We propose a Bayesian modeling framework that melds a CSM with sparse data from a yield monitoring system to deliver location specific posterior predicted distributions of yield and associated unobserved spatially varying CSM parameter inputs. These products facilitate exploration of process-based explanations for yield variability. The proposed Bayesian melding model consists of a systemic component representing output from the physical model and a residual spatial process that compensates for the bias in the physical model. The spatially varying inputs to the systemic component arise from a multivariate Gaussian process, while the residual component is modeled using a univariate Gaussian process. Due to the large number of observed locations in the motivating dataset, we seek dimension reduction using low-rank predictive processes to ease the computational burden. The proposed model is illustrated using the Crop Environment Resources Synthesis (CERES)-Wheat CSM and wheat yield data collected in Foggia, Italy.

Signal enhancement techniques for rf SQUID based magnetic imaging systems

We have investigated the rf SQUID (radio-frequency superconducting quantuminterference device) and its coupling to tank circuit configurations to achieve an optimalfront-end assembly for sensitive and high spatial resolution magnetic imagingsystems. The investigation of the YBCO rf SQUID coupling to the conventionalLC tank circuits revealed that coupling from the back of the SQUID substrate enhances theSQUID signal while facilitating the front-end assembly configuration. The optimalthickness of the substrate material between the SQUID and the tank circuit is 0.4 mm forLaAlO3 resulting in an increase of the SQUID flux–voltage transfer function signal,Vspp, of 1.5 times,and 0.5 mm for SrTiO3 with an increase of Vspp of 1.62 times compared to that for direct face to face couplings. For rf coupling with acoplanar resonator, it has been found that the best configuration, in which a resonator issandwiched between the SQUID substrate and the resonator substrate, provides aVspp about 3.4 times higher than that for the worse case where the resonator and the SQUID arecoupled back to back. The use of a resonator leads to a limitation of the achievable spatialresolution due to its flux focusing characteristics. This resulted in a favouring of the use of theconventional tank circuits when considering the desired high spatial resolution. The effectof the YBCO flip chip magnetic shielding of the SQUIDs in the back-coupling with theLC tank circuit configuration has also been investigated, with a view to reducing the SQUIDeffective area to increase the spatial resolution and also for studying the effect of thecoupling of various kinds of transformers to the SQUIDs. It is revealed that there is no veryconsiderable change in the flux–voltage transfer function signal level with respect tothe effective shield area, while the lowest working temperature of the SQUIDswas slightly shifted higher by a couple of degrees, depending on the shield area.

Front-end Assembly Optimization for High-Tcrf-SQUID based Magnetic Field Imaging Systems

We have investigated the rf-SQUID and its coupling to the tank circuit configurations to achieve an optimal front-end assembly for sensitive and high spatial resolution magnetic imaging systems. The investigation on the YBCO rf-SQUID coupling to the conventional LC tank circuits revealed that the coupling from the back of the SQUID substrate enhances the SQUID signal while facilitating the front-end assembly configuration. The optimal thickness of the substrate material between the SQUID and the tank circuit is 0.4mm for LaAlO3 resulting in an increase of SQUID flux-voltage transfer function signal, Vspp, of 1.5 times, and 0.5 mm for SrTiO3 with an increase of Vspp of 1.62 times compared to that of direct face to face couplings. For the rf-coupling with co-planar resonator, CPR, it has been found that the best configuration, in which a resonator is sandwiched between the SQUID substrate and resonator substrate, provides a Vspp about 3.4 times higher than the worse case where the resonator and the SQUID are coupled back to back. It has also been observed that the noise level does not depend considerably on whether a conventional LC tank circuit or a CPR is used. Though the use of resonator leads to a limitation of the achievable spatial resolution due to its flux-focusing characteristics. This resulted in favouring the use of the conventional tank circuits when considering the desired high spatial resolution. Effect of the YBCO flip-chip magnetic shielding of the SQUIDs in the back coupling with the LC-tank circuit configuration has also been investigated, in order to reduce the SQUID effective area to increase the spatial resolution and also to study the effect of the coupling of various types of the transformers to the SQUIDs. It is revealed that there is no considerable change in the flux-voltage transfer function signal level with respect to the effective shield area, while the lowest working temperature of the SQUIDs was slightly shifted higher by a couple of degrees depending on the shield area.