Panchromatic Research Articles

Learning Spectral Cues for Multispectral and Panchromatic Image Fusion.

Recently, deep learning based multispectral (MS) and panchromatic (PAN) image fusion methods have been proposed, which extracted features automatically and hierarchically by a series of non-linear transformations to model the complicated imaging discrepancy. But they always pay more attention to the extraction and compensation of spatial details and use the mean squared error or mean absolute error as a loss function, regardless of the preservation of spectral information contained in multispectral images. For the sake of the improvements in both spatial and spectral resolution, this paper presents a novel fusion model that takes the spectral preservation into consideration, and learns the spectral cues from the process of generating a spectrally refined multispectral image, which is constrained by a spectral loss between the generated image and the reference image. Then these spectral cues are used to modulate the PAN features to obtain final fusion result. Experimental results on reduced-resolution and full-resolution datasets demonstrate that the proposed method can obtain a better fusion result in terms of visual inspection and evaluation indices when compared with current state-of-the-art methods.

A Deep-Shallow Fusion Network With Multidetail Extractor and Spectral Attention for Hyperspectral Pansharpening

Hyperspectral (HS) pansharpening aims at fusing a low-resolution HS (LRHS) image with a high-resolution panchromatic (PAN) image to obtain a hyperspectral image with both higher spectral and spatial resolutions. However, existing HS pansharpening algorithms are mainly based on multispectral (MS) pansharpening approaches, which cannot perfectly restore much spectral information and more high-frequency spatial details in the continuous spectral bands and much broader spectral range, leading to spectral distortion and spatial blur. In this paper, we develop a new hyperspectral pansharpening network architecture (called Hyper-DSNet) to fully preserve latent spatial details and spectral fidelity via a deep-shallow fusion structure with multi-detail extractor and spectral attention. Specifically, the proposed architecture mainly consists of three parts. First, to solve the problem of spatial ambiguity and exploit the potential information, five types of high-pass filter templates are used to fully extract the spatial details of the PAN image, constructing a so-called multi-detail extractor (MDE). Then, after passing a multi-scale convolution module, a deep-shallow fusion (DSF) structure, which reduces parameters by decreasing the number of output channels as the network goes deeper, is utilized sequentially. In final, a spectral attention (SA) module is conducted to preserve the spectrum for a wealth of spectral information of HS images. Visual and quantitative experiments on three commonly used simulated datasets and one full-resolution dataset demonstrate the effectiveness and robustness of the proposed Hyper-DSNet against the recent state-of-the-art hyperspectral pansharpening techniques. Ablation studies and discussions further verify our contributions, <i>e</i>.<i>g</i>., better spectral preservation and spatial detail recovery. The source code is available at <uri>https://github.com/liangjiandeng/Hyper-DSNet</uri>, and the related datasets can be found from <uri>https://github.com/liangjiandeng/HyperPanCollection</uri>.

MC-JAFN: Multilevel Contexts-Based Joint Attentive Fusion Network for Pansharpening

Pansharpening refers to a spatial–spectral contexts fusion procedure to produce high-quality multispectral (MS) images by retaining the fine spatial resolution of the panchromatic (PAN) images and the high spectral content of the MS images. This letter presents a novel end-to-end dual-branch deep learning-based fusion framework, exploiting the network to extract spatial and spectral contexts progressively in two separate branches level by level. For each level contexts extraction layer, a dual-branch weighted attentive fusion module is integrated to boost the important contexts aggregation and details injection while suppressing unimportant ones. Experimental results on two real datasets show that our method outperforms state-of-the-art methods in both objective metrics and image quality by visual appearance.

Pan-Sharpening via Deep Locally Linear Embedding Residual Network

The goal of pan-sharpening tasks is to fuse panchromatic (PAN) images and low-spatial-resolution (LR) multispectral (MS) images for the purpose of aggregating texture and spectral information. Although traditional embedding-based pan-sharpening methods achieve competitive results, they are limited by the shallow network and not suitable for large-scale datasets. In this study, we design a novel multiscale locally linear embedding residual network (LLERN) that consists of two phases: the spectral preservation phase and the structural preservation phase. As the pretreatment of the structural preservation network, the spectral preservation network aims to upscale the LR MS image while retaining spectral information. The proposed locally linear embedding residual block (LLERB) in the structural preservation phase can search for similar sparse patches from the PAN image space and embed the corresponding local geometric relationship into the residual space to enhance the MS image. Extensive experiments suggest that the proposed LLERN outperforms state-of-the-art methods from visual and quantitative perspectives, and confirm the assumption that LR image patches and residual image patches in a local region share a similar manifold structure, which can be used to guide deep-learning modeling with improved interpretability. The source code is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/jiaming-wang/LLERN</uri> .

Pansharpening Based on Spectral-Spatial Dependence for Multibands Remote Sensing Images

Pansharpening based on detail injection (DI) explores panchromatic (PAN) images and injects their spatial geometrical information into multispectral (MS) images. Traditional DI algorithms fuse remote sensing images without considering spectral and spatial dependence, which causes spectral distortion. To overcome this problem, this article proposes an advanced version of the DI model that not only achieves adaptive DI but also addresses spectral-spatial dependence. This proposed method is called pansharpening based on spectral-spatial dependence for multi-bands remote sensing images. In the proposed method, three parameters including an adaptive DI based on the correlation between MS and PAN images, the spectral dependence based on the MS band and pixel dependence, and the spatial dependence based on the detail offset amplitude are designed. An improved GDI model based on DI and the combination of the spectral and spatial dependence (SSD) is constructed. In the model, SSD enables the fused image to retain the spectral fidelity, and DI ensures the sharpness of the fused image. A performance test is conducted on various satellite datasets and the results are compared with those of several state-of-the-art fusion methods. The results show that the proposed method benefits pansharpening.

PMACNet: Parallel Multiscale Attention Constraint Network for Pan-Sharpening

Pan-sharpening, a task involving information fusion, entails merging panchromatic (PAN) images with high spatial resolution and low-resolution multispectral (LRMS) images in order to obtain high-resolution multispectral (HRMS) images. Due to deep learning&#x2019;s excellent regression capabilities, it has recently become the dominating technique for this assignment. Meanwhile, the development of the transformer, a novel deep learning architecture for natural language processing, has provided researchers with new insights. In this letter, we seek to extend transformer&#x2019;s excellent mechanisms to pixel-level fusion challenges. We designed a parallel convolutional neural network structure for learning both the regions of interest from the LRMS images and the residuals required for regression to HRMS images. Then, in our proposed pixelwise attention constraint (PAC) module, the residuals will be changed utilizing the learned region of interest. In addition, we presented a novel multireceptive-field attention block (MRFAB) to frame our network. Experiments on two datasets also show that our work is better than the mainstream algorithms at both indicators and visualization.

MMDN: Multi-Scale and Multi-Distillation Dilated Network for Pansharpening

Pansharpening is a technology involving information integration and processing in remote sensing imagery. It is applied to generate a high-resolution multispectral (HRMS) image through an effective fusion of a low spatial resolution multispectral image and a panchromatic (PAN) image. In this paper, we propose an end-to-end multi-scale and multi-distillation dilated network (MMDN) for pansharpening. In MMDN, to extract more abundant spatial details from source images, a clique structure-based multi-scale dilated block (CSMDB) is presented. The clique structure in CSMDB can fully transfer the information between feature maps obtained by the multi-scale dilated convolutional filters. Then, a multi-distillation residual information block (MRIB) is constructed to help the network capture the spatial structure of different scales in MS and PAN images. Finally, to reuse and supplement the feature information, a feature embedding strategy is designed by feeding the sum result of the output of cascaded CSMDBs and the shallow features to each MRIB. Experimental results verify that the proposed MMDN outperforms other compared state-of-the-art approaches in terms of objective and subjective evaluations.

A Two-Stage Mutual Fusion Network for Multispectral and Panchromatic Image Classification

With the rapid development of remote sensing technology, satellites can easily obtain multispectral (MS) and panchromatic (PAN) images. How to mine the essence and peculiarity of the MS and PAN images and utilize their complementary to improve classification performance is still a challenge. This paper designs a two-stage mutual fusion network (TSMF-Net) for MS and PAN image classification. The network can be divided into two stages: data fusion and feature fusion. In the data fusion stage, we propose an adaptive twin intensity-hue-saturation (ATIHS) strategy. It not only aligns the size and channels of the MS and PAN images by a novel q-Split operation, but also introduces an adaptive soft-average mask to reduce the differences between replacement components, effectively mitigating spectral distortion and paving the way for the next stage. In the feature fusion stage, we propose a feature graft block (FG-Block) in which we introduce triplet loss and design an interlaced channel addition (ICA) module. Under the supervision of triplet loss, the FG-Block separates and hauls each branch’s essential and peculiar features. With the help of the ICA module, it can effectively graft the essential feature between branches and retain the peculiar feature of each branch, improving the utilization and discrimination of features. Finally, composed of the ATIHS, FG-Blocks, and output layers, our TSMF-Net is proven to improve the accuracy of the remote sensing classification task. The experimental results on multiple datasets verify the effectiveness of our proposed algorithms. Our code is available at: https://github.com/liaoyinuo/TSMF-Net.

MetaPan: Unsupervised Adaptation With Meta-Learning for Multispectral Pansharpening

Multispectral (MS) pansharpening aims to improve the spatial resolution of MS images (MSI) using the spatial details of panchromatic (PAN) images. Due to the gap of prior knowledge between the simulated data and real-world cases, unsupervised learning-based approaches have grown increasing interest. However, some key hyper-parameters, such as the initial weights of the networks, are set manually, which significantly impacts the fusion performance. To tackle this problem, we propose a novel unsupervised adaptation method with meta-learning for MS pansharpening (MetaPan), in which the meta-learning aims to automatically learn the initial parameters of a three-stream fusion network (TSFNet) for unsupervised adaptation learning (UAL). Specifically, the TSFNet consists of a PAN stream, an MS stream, and a fusion stream, where the fusion stream implicitly leverages domain-specific knowledge of input image pairs while the other two streams explicitly inject spatial details and spectral information into the fusion stream. The MetaPan consists of a pre-training stage, a meta-learning stage, and a UAL stage. At the pre-training stage, the TSFNet is trained with the supervision of simulated ground truth such that it is universal for all image pairs. Then, the process of meta-learning optimizes for an internal representation of network parameters that can adapt to a specific image pair with UAL through only a few steps. Finally, the learned internal representation is fine-tuned to a real-world image pair (a test image pair) with UAL. Experiments on two datasets show that our method performs better than state-of-the-art methods in both quantitative metrics and visual appearance.

Hyperspectral Pansharpening Based on Improved Deep Image Prior and Residual Reconstruction

Hyperspectral pansharpening aims to synthesize a low-resolution hyperspectral image (LR-HSI) with a registered panchromatic (PAN) image to generate an enhanced HSI with high spectral and spatial resolution. Recently, the proposed HS pansharpening methods have obtained remarkable results using deep convolutional networks (ConvNets), which typically consist of three steps: 1) upsampling the LR-HSI; 2) predicting the residual image via a ConvNet; and 3) obtaining the final fused HSI by adding the outputs from first and second steps. Recent methods have leveraged deep image prior (DIP) to upsample the LR-HSI due to its excellent ability to preserve both spatial and spectral information, without learning from large datasets. However, we observed that the quality of upsampled HSIs can be further improved by introducing an additional spatial-domain constraint to the conventional spectral-domain energy function. We define our spatial-domain constraint as the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$L_{1}$ </tex-math></inline-formula> distance between the predicted PAN image and the actual PAN image. To estimate the PAN image of the upsampled HSI, we also propose a learnable spectral response function (SRF). Moreover, we noticed that the residual image between the upsampled HSI and the reference HSI mainly consists of edge information and very fine structures. In order to accurately estimate fine information, we propose a novel overcomplete network, called HyperKite, which focuses on learning high-level features by constraining the receptive from increasing in the deep layers. We perform experiments on three semisynthetic and one real HSI datasets to demonstrate the superiority of our DIP-HyperKite over the state-of-the-art pansharpening methods. The deployment codes, pretrained models, and final fusion outputs of our DIP-HyperKite and the methods used for the comparisons will be publicly made available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/wgcban/DIP-HyperKite.git</uri> .

GCFnet: Global Collaborative Fusion Network for Multispectral and Panchromatic Image Classification

Among various multimodal remote sensing data, the pairing of multispectral (MS) and panchromatic (PAN) images is widely used in remote sensing applications. This article proposes a novel global collaborative fusion network (GCFnet) for joint classification of MS and PAN images. In particular, a global patch-free classification scheme based on an encoder-decoder deep learning (DL) network is developed to exploit context dependencies in the image. The proposed GCFnet is designed based on a novel collaborative fusion architecture, which mainly contains three parts: 1) two shallow-to-deep feature fusion branches related to individual MS and PAN images; 2) a multiscale cross-modal feature fusion branch of the two images, where an adaptive loss weighted fusion strategy is designed to calculate the total loss of two individual and the cross-modal branches; 3) a probability weighted decision fusion strategy for the fusion of the classification results of three branches to further improve the classification performance. Experimental results obtained on three real datasets covering complex urban scenarios confirm the effectiveness of the proposed GCFnet in terms of higher accuracy and robustness compared to existing methods. By utilizing both sampled and non-sampled position data in the feature extraction process, the proposed GCFnet can achieve excellent performance even in a small sample-size case. The codes will be available from the website: https://github.com/SicongLiuRS/GCFnet.

Enhanced Resolution of FY4 Remote Sensing Visible Spectrum Images Utilizing Super-Resolution and Transfer Learning Techniques

Remote sensing images acquired by the FY4 satellite are crucial for regional cloud monitoring and meteorological services. Inspired by the success of deep learning networks in image super-resolution, we applied image super-resolution to FY4 visible spectrum (VIS) images. However, training a robust network directly for FY4 VIS image super-resolution remains challenging due to the limited provision of high resolution FY4 sample data. Here, we propose a super-resolution and transfer learning model, FY4-SR-Net. It is composed of pre-training and fine-tuning models. The pre-training model was developed using a deep residual network and a large number of FY4 A 4km and 1km resolution VIS images as the training data. The knowledge derived from 4 km to 1 km resolution images was incorporated into FY4 B 1 km to 0.25 km resolution VIS images. The FY4-SR-Net is fine-tuned by incorporating limited 1km and 0.25km resolution panchromatic (PAN) images, and then producing 1km super-resolution VIS images of the FY4 satellite. Using the one-day FY4 test dataset for qualitative and quantitative evaluations, the FY4-SR-Net outperformed the classic bicubic interpolation approach with a 16.12&#x0025; reduction in root mean square error (RMSE) and a 2.97&#x0025; rise in peak signal-to-noise ratio (PSNR) averages. The structural similarity (SSIM) value average increased by 0.0026. This work provides a new precedent for improving the spatial resolution of FY4 series meteorological satellites, which has important scientific significance and application properties.

A Pansharpening Based on the Non-Subsampled Contourlet Transform and Convolutional Autoencoder: Application to QuickBird Imagery

This paper presents a pansharpening technique based on the non-subsampled contourlet transform (NSCT) and convolutional autoencoder (CAE). NSCT is exceptionally proficient at presenting orientation information and capturing the internal geometry of objects. First, it’s used to decompose the multispectral (MS) and panchromatic (PAN) images into high-frequency and low-frequency components using the same number of decomposition levels. Second, a CAE network is trained to generate original low-frequency PAN images from their spatially degraded versions. Low-resolution multispectral images are then fed into the trained convolutional autoencoder network to generate estimated high-resolution multispectral images. Third, another CAE network is trained to generate original high-frequency PAN images from their spatially degraded versions. The result of low-pass CAE is fed to the trained high-pass CAE to generate estimated high-resolution multispectral images. The final pan-sharpened image is accomplished by injecting the detailed map of the spectral bands into the corresponding estimated high-resolution multispectral bands. The proposed method is tested on QuickBird datasets and compared with some existing pan-sharpening techniques. Objective and subjective results demonstrate the efficiency of the proposed method.

Transfer Representation Learning Meets Multimodal Fusion Classification for Remote Sensing Images

To maximize the complementary advantages of synergistic multimodal, a transfer representation learning fusion network (TRLF-Net) is proposed for multisource remote sensing images collaborative classification in this article. First, with respect to the feature encoding, we design a dual-branch attention sparse transfer module (DAST-Module), which combines the spatial and channel attention (CA) masks to migrate the advantage attributes of the panchromatic (PAN) and the MS images mutually. This not only enhances their respective image advantages but also facilitates the sparse fusion of low-level features. Second, for the separation of multiscale information, a deep dual-scale decomposition module (DDSD-Module) is designed, which allows the decompose of high-frequency and low-frequency components. Then it uses the decomposed information to make the essential difference as small as possible, and the surrounding contour difference is as large as possible of the complementary multimodal image through the design of the loss function. Finally, to address the problem of large intraclass and small interclass differences, we develop a representation fusion of the global and local features’ module (RFGAL-Module). It mainly adopts global features to sort local features within classes, and then outputs them in a cascade. Thus, the characterization ability of features is improved, and the global and local features are used in a coordinated manner to accomplish the sample classification tasks. In particular, the experimental results demonstrate that TRLF-Net can obtain much improved accuracy and efficiency. The code is accessible in: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/ru-willow/SRLF-Net</uri> .

Self-Supervised Pansharpening Based on a Cycle-Consistent Generative Adversarial Network

In the field of remote sensing image pansharpening, deep learning-based methods have shown impressive performances recently. However, most deep learning-based pansharpening methods are based on supervised learning, which requires a large number of training images. In addition, obtaining large amounts of images with a high spatial and spectral resolution for training may be difficult in practice. In this letter, a novel self-supervised learning method based on a cycle-consistent generative adversarial network (CycleGAN) is proposed for remote sensing image pansharpening, without requiring large volumes of data for training. The framework contains two generators and two discriminators, and applies a residual neural network to the first generator. The panchromatic (PAN) image and multispectral (MS) image are input into the first generator to obtain the fused image, and then the fused image is input into the second generator to obtain a PAN image, which should be consistent with the input PAN image. The experimental results show that the proposed method performs better than the state-of-the-art unsupervised pansharpening method, and also achieves a competitive performance when compared with a supervised method.

A Unified Pansharpening Model Based on Band-Adaptive Gradient and Detail Correction.

Pansharpening is used to fuse a panchromatic (PAN) image with a multispectral (MS) image to obtain a high-spatial-resolution multispectral (HRMS) image. Traditional pansharpening methods face difficulties in obtaining accurate details and have low computational efficiency. In this study, a unified pansharpening model based on the band-adaptive gradient and detail correction is proposed. First, a spectral fidelity constraint is designed by keeping each band of the HRMS image consistent with that of the MS image. Then, a band-adaptive gradient correction model is constructed by exploring the gradient relationship between a PAN image and each band of the MS image, so as to adaptively obtain an accurate spatial structure for the estimated HRMS image. To refine the spatial details, a detail correction constraint is defined based on the parameter transfer by designing a reduced-scale parameter acquisition model. Finally, a unified model is constructed based on the gradient and detail corrections, which is then solved by an alternating direction multiplier method. Both reduced-scale and full-scale experiments are conducted on several datasets. Compared with state-of-the-art pansharpening methods, the proposed method can achieve the best results in terms of fusion quality and has high efficiency. Specifically, our method improves the SAM and ERGAS metrics by 17.6% and 21.2% respectively compared to the traditional approach with the best average values, and improves these two metrics by 4.3% and 10.3% respectively compared to the learning-based approach with the best average values.

Spatial and Spectral Extraction Network With Adaptive Feature Fusion for Pansharpening

Pansharpening methods based on deep neural networks (DNNs) have been attracting great attention due to their powerful representation capabilities. In this article, to combine the feature maps from different subnetworks efficiently, we propose a novel pansharpening method based on a spatial and spectral extraction network (SSE-Net). Different from the other methods based on DNNs that directly concatenate the features from different subnetworks, we design adaptive feature fusion modules (AFFMs) to merge these features according to their information content. First, the spatial and spectral features are extracted by the subnetworks from low spatial resolution multispectral (LR MS) and panchromatic (PAN) images. Then, by fusing the features at different levels, the desired high spatial resolution MS (HR MS) images are generated by the fusion network consisting of AFFMs. In the fusion network, the features from different subnetworks are integrated adaptively, and the redundancy among them is reduced. Moreover, the spectral ratio loss and the gradient loss are defined to ensure the effective learning of spatial and spectral features. The spectral ratio loss captures the nonlinear relationships among the bands in the MS image to reduce the spectral distortions in the fusion result. Extensive experiments were conducted on QuickBird and GeoEye-1 satellite datasets. Visual and numerical results demonstrate that the proposed method produces better fusion results compared with literature techniques. The source code is available at <uri>https://github.com/RSMagneto/SSE-Net</uri>.

Hyperspectral Intrinsic Image Decomposition With Enhanced Spatial Information

Hyperspectral intrinsic image decomposition (HyperIID) has been proven to be a very useful approach to reduce the spectral uncertainty in the remote sensing imaging process and improve the classification. In this article, a new HyperIID with enhanced spatial information, called ESI-IID, is proposed to overcome the deficiency of low spatial resolution in the existing HyperIID methods. With the aid of high-resolution (HR) panchromatic (PAN) image, the proposed method embeds the HR spatial information into the intrinsic decomposition model and enhances spatial details of the intrinsic component. The proposed ESI-IID introduces three constraints: 1) we make the constraint on spectral information to protect it from distortion during the spatial resolution enhancement process; 2) we add the constraint on spatial information to make sure that the details of edges will be well kept; and 3) based on the assumption that the reflectance component has a strong correlation in the local neighborhood, we add the self-constraint on reflectance component, in which the similarity matrix consists of two parts extracted from hyperspectral images and PAN image, respectively. Finally, we build a matrix energy function according to the aforementioned constraints and solve it by finding the minimum Frobenius norm iteratively. Both visual and quantitative experiments on simulated and real datasets demonstrate that the proposed method outperforms other alternative methods with high reliability.

CNN-Based Hyperspectral Pansharpening With Arbitrary Resolution

Traditional hyperspectral (HS) pansharpening aims at fusing a HS image with its panchromatic (PAN) counterpart, to bring the spatial resolution of the HS image to that of the PAN image. However, in many practical applications, arbitrary resolution HS (ARHS) pansharpening is required, where the HS and PAN images need to be integrated to generate a pansharpened HS image with arbitrary resolution (usually higher than that of the PAN image). Such an innovative task brings forth new challenges for the pansharpening technique, mainly including how to reconstruct HS images beyond the training scale and how to guarantee spectral fidelity at any spatial resolutions. To tackle the challenges, we present a novel convolutional neural network (CNN)-based method for ARHS pansharpening called ARHS-CNN. It is based on a two-step relay optimization process, which is associated with a multilevel enhancement subnetwork and a rescaling subnetwork. With a careful design following the thread, our ARHS-CNN is able to pansharpen HS images to any spatial resolutions using just a single CNN model trained on a limited number of scales while meantime to keep spectral fidelity at those resolutions, which wins an obvious advantage over traditional pansharpening methods. Experimental results obtained on several datasets verify the excellent performance of our ARHS-CNN method.

A Blind Full-Resolution Quality Evaluation Method for Pansharpening

Pansharpening methods have been developed for nearly 40 years; however, how to quantitatively evaluate the quality of pansharpened images at full resolution (FR) is probably the most debated topic in this field due to the inherent unavailable of the real HR MS reference image. In this article, a novel blind FR quality evaluation method for pansharpening is proposed. In the proposed method, spatial and spectral features that are sensitive to spatial and spectral distortions of fused images are comprehensively considered and jointly learned based on online multivariate Gaussian (MVG) to construct the evaluation model. It directly outputs the quality of fused images, rather than the stepwise evaluation of spectral score, spatial score, and final overall quality score by the weighted combination of them, which may introduce contradictory results. First, a pristine benchmark evaluation model is established on the spatial features from the original high-spatial-resolution (HR) panchromatic (PAN) image and the spectral invariant assumption between ideal fused and original multispectral (MS) images. Second, a testing evaluation model for the fused image is founded. Finally, the quality of the fused image is measured based on the distance between the testing and benchmark models. The experimental results demonstrated the superior performance of the proposed method. Furthermore, the proposed method can be generalized to other interesting tasks, such as the nonreference evaluation for pansharpening with missing information and the nonreference evaluation for hyperspectral image fusion. The source code is available on <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/yyxhpkq/MQNR</uri> .