Pansharpening of images acquired with color filter arrays

Abstract

In remote sensing, a common scenario involves the simultaneous acquisition of a panchromatic (PAN), a broad-band high spatial resolution image, and a multispectral (MS) image, which is composed of several spectral bands but at lower spatial resolution. The two sensors mounted on the same platform can be found in several very high spatial resolution optical remote sensing satellites for Earth observation (e.g., Quickbird, WorldView and SPOT) In this work we investigate an alternative acquisition strategy, which combines the information from both images into a single band image with the same number of pixels of the PAN. This operation allows to significantly reduce the burden of data downlink by achieving a fixed compression ratio of 1/(1+b/p2) compared to the conventional acquisition modes. Here, b and p denote the amount of distinct bands in the MS image and the scale ratio between the PAN and MS, respectively (e.g.: b = p = 4 as in many commercial high spatial resolution satellites). Many strategies can be conceived to generate such a compressed image from a given set of PAN and MS sources. A simple option, which will be presented here, is based on an application of the color filter array (CFA) theory. Specifically, the value of each pixel in the spatial support of the synthetic image is taken from the corresponding sample either in the PAN or in a given band of the MS up-sampled to the size of the PAN. The choice is deterministic and done according to a custom mask. There are several works in the literature proposing various methods to construct masks which are able to preserve as much spectral content as possible for conventional RGB images. However, those results are not directly applicable to the case at hand, since it deals with i) images with different spatial resolution, ii) potentially more than three spectral bands and, iii) in general, different radiometric dynamics across bands. A tentative approach to address these issues is presented in this work. The compressed image resulting from the proposed acquisition strategy will be processed to generate an image featuring both the spatial resolution of the PAN and the spectral bands of the MS. This final product allows a direct comparison with the result of any standard pan-sharpening algorithm; the latter refers to a specific instance of data fusion (i.e., fusion of a PAN and MS image), which differs from our scenario since both sources are separately taken as input. In our setting, the fusion step performed at the ground segment will jointly involve a fusion and reconstruction problem (also known as demosaicing in the CFA literature). We propose to address this problem with a variational approach. We present in this work preliminary results related to the proposed scheme on real remote sensed images, tested over two different datasets acquired by the Quickbird and Geoeye-1 platforms, which show superior performances compared to applying a basic radiometric compression algorithm to both sources before performing a pan-sharpening protocol. The validation of the final products in both scenarios allows to visually and numerically appreciate the tradeoff between the compression of the source data and the quality loss suffered.