No Reference Pansharpened Image Quality Assessment Through Deep Feature Similarity

Abstract

Pansharpening refers to the process of enhancing the spatial resolution of a multispectral image with the help of a high spatial resolution panchromatic image. Quality assessment (QA) of pansharpened images helps provide a formal framework for the analysis and design of pansharpening methods and is thus extremely important. However, lack of availability of a reference multispectral image makes QA of pansharpening algorithms a challenging task. Given the popular use of QA algorithms that use a reference, this paper focuses on predicting the quality under a &#x2018;no-reference&#x0027; (NR) setting. Specifically, a learning based NR pansharpened image quality assessment (IQA) approach is adopted to predict state of the art reference based measures such as <inline-formula><tex-math notation="LaTeX">$Q2^{n}$</tex-math></inline-formula> and SAM without the need of a reference. We design an end-to-end deep pansharpening IQA network to compute the similarity of deep features fused from the panchromatic and input low resolution multispectral with similar features extracted from the given pansharpened image. To train and test our learning based approach, we create a large corpus of pansharpened images belonging to different satellites and thematic scenes by applying different pansharpening algorithms. Our experiments demonstrate that our NR pansharpened IQA algorithm achieves excellent performance and generalizes well across different satellites and resolutions. The source code and the dataset will be made available at <uri>https://github.com/neerajbadal/DPIQA</uri>.