Low Resolution Multispectral Image Research Articles

Optical SAR Images Fusion: Comparative Analysis of Resulting Images Data

Image fusion is a useful tool for integrating low spatial resolution multispectral (MS) images with a high spatial resolution panchromatic (PAN) image, thus producing a high resolution multispectral image for better understanding of the observed earth surface. A main proposed the research were the effectiveness of different image fusion methods while filtering methods added to speckle suppression in synthetic aperture radar (SAR) images. The quality assessment of the filtering fused image implemented by statistical parameter namely mean, standard deviation, bias, universal index quality image (UIQI) and root mean squared error (RMSE). In order to test the robustness of the image quality, either speckle noise (Gamma map filter) is intentionally added to the fused image. When comparing and testing result, Gram Scmidth (GS) methods have shown better results for good colour reproduction, as compared with high pass filtering (HPF). And the other hands, GS, and wavelet intensity hue saturation (W-IHS) have shown the preserving good colour with original image for Landsat TM data.

Comparison of Three Image Fusion Techniques Employed on High Resolution QuickBird Images

Image fusion involves integration of the geometric details of a high-resolution panchromatic (PAN) image and the spectral information of a low resolution multispectral (XS) image which is useful for regional planning and large scale urban mapping. Present study compares the effectiveness of three image fusion techniques namely, Principal Component Analysis (PCA), Wavelet Transform (WT) and Intensity Hue Saturation (IHS) to merge the XS information and PAN data of QuickBird satellite imagery. Comparison between the fused images obtained from the three fusion techniques is carried out on the basis of qualitative and quantitative evaluations implying, visual interpretation, inter-band correlation, correlation coefficient, standard deviation and mean. Results indicate that all three fusion techniques improves spatial resolution as well as spectral details, however, IHS technique provides the best spectral fidelity by preserving the XS integrity between all the bands under consideration.

A novel variational model for pan-sharpening based on L1 regularization

ABSTRACTThe Pan-sharpening process synthesizes a multispectral image of high spatial and high spectral quality by injecting the spatial details extracted from the high spatial resolution panchromatic image into the low-resolution multispectral image. In this letter, we propose a novel variational model for pan-sharpening based on L1 regularization, which consists of three energy terms. The first energy term extracts the geometric information from panchromatic image with exponential enhancement function, which can improve the contrast of the fused image by nonlinear expanding of gradient magnitude. The second energy term develops the low-pass filter based on the modulation transfer function (MTF) of the different MS sensor, which can preserve spectral information by determining the injected spatial details adaptively. Compared with the total variation regularization, the L1 regularization encourages the feature of sparse representation, which can improve the spatial fidelity of pan-sharpened images. In addition, introducing L1 regularization term into the variational framework can smooth the image noise and guarantee the stability of the numerical solution. Experimental results demonstrate that the proposed method outperforms several state-of-the-art pan-sharpening methods, including intensity-hue-saturation combined with Brovey transform (IHS-BT), additive wavelet luminance propotional (AWLP), total variation regularization (TVR) and MTF Contrast-based (MTF-CON) method.

Pansharpening via sparse regression

Pansharpening is an effective way to enhance the spatial resolution of a multispectral (MS) image by fusing it with a provided panchromatic image. Instead of restricting the coding coefficients of low-resolution (LR) and high-resolution (HR) images to be equal, we propose a pansharpening approach via sparse regression in which the relationship between sparse coefficients of HR and LR MS images is modeled by ridge regression and elastic-net regression simultaneously learning the corresponding dictionaries. The compact dictionaries are learned based on the sampled patch pairs from the high- and low-resolution images, which can greatly characterize the structural information of the LR MS and HR MS images. Later, taking the complex relationship between the coding coefficients of LR MS and HR MS images into account, the ridge regression is used to characterize the relationship of intrapatches. The elastic-net regression is employed to describe the relationship of interpatches. Thus, the HR MS image can be almost identically reconstructed by multiplying the HR dictionary and the calculated sparse coefficient vector with the learned regression relationship. The simulated and real experimental results illustrate that the proposed method outperforms several well-known methods, both quantitatively and perceptually.

Learning Low-Rank Decomposition for Pan-Sharpening With Spatial-Spectral Offsets.

Finding accurate injection components is the key issue in pan-sharpening methods. In this paper, a low-rank pan-sharpening (LRP) model is developed from a new perspective of offset learning. Two offsets are defined to represent the spatial and spectral differences between low-resolution multispectral and high-resolution multispectral (HRMS) images, respectively. In order to reduce spatial and spectral distortions, spatial equalization and spectral proportion constraints are designed and cast on the offsets, to develop a spatial and spectral constrained stable low-rank decomposition algorithm via augmented Lagrange multiplier. By fine modeling and heuristic learning, our method can simultaneously reduce spatial and spectral distortions in the fused HRMS images. Moreover, our method can efficiently deal with noises and outliers in source images, for exploring low-rank and sparse characteristics of data. Extensive experiments are taken on several image data sets, and the results demonstrate the efficiency of the proposed LRP.

Assessment of Multi-Temporal Image Fusion for Remote Sensing Application

Image fusion and subsequent scene analysis are important for studying Earth surface conditions from remotely sensed imagery. The fusion of the same scene using satellite data taken with different sensors or acquisition times is known as multi-sensor or multi-temporal fusion, respectively. The purpose of this study is to investigate the effects of misalignments the multi-sensor, multi-temporal fusion process when a pan-sharpened scene is produced from low spatial resolution multispectral (MS) images and a high spatial resolution panchromatic (PAN) image. It is found that the component substitution (CS) fusion method provides better performance than the multi-resolution analysis (MRA) scheme. Quantitative analysis shows that the CS-based method gives a better result in terms of spatial quality (sharpness), whereas the MRA-based method yields better spectral quality, i.e., better color fidelity to the original MS images .

A Joint Sparse and Low-Rank Decomposition for Pansharpening of Multispectral Images

Pansharpening aims to fuse a high-resolution panchromatic (PAN) image and a low-resolution multispectral (MS) image. Several synthesis techniques have been reported to solve the problem of pansharpening. Details injection (DI) consists of the cascaded processes of details extraction and injection. The former is crucial for performance. By exploiting the relationship among multiple data acquired on the same scene through different sensors, this paper first develops a joint sparse and low-rank (JSLR) decomposition with an assumption that multiple data have a common low-rank component. Then, a novel DI-type pansharpening method is proposed based on JSLR decomposition, named as JSLR-based pansharpening (JSLRP). In JSLRP, the injected spatial details are calculated as a linear combination of JSLR decomposed components. To ensure the low-rank condition, the JSLR is implemented on the PAN and MS images in the nonlocal similar patches form by adopting the nonlocal self-similarity. Finally, the superiority of JSLRP is demonstrated by comparing with several well-known methods on the reduced-scale data and full-scale data.

A Novel Pan-Sharpening Framework Based on Matting Model and Multiscale Transform

Pan-sharpening aims to sharpen a low spatial resolution multispectral (MS) image by combining the spatial detail information extracted from a panchromatic (PAN) image. An effective pan-sharpening method should produce a high spatial resolution MS image while preserving more spectral information. Unlike traditional intensity-hue-saturation (IHS)- and principal component analysis (PCA)-based multiscale transform methods, a novel pan-sharpening framework based on the matting model (MM) and multiscale transform is presented in this paper. First, we use the intensity component (I) of the MS image as the alpha channel to generate the spectral foreground and background. Then, an appropriate multiscale transform is utilized to fuse the PAN image and the upsampled I component to obtain the fused high-resolution gray image. In the fusion, two preeminent fusion rules are proposed to fuse the low- and high-frequency coefficients in the transform domain. Finally, the high-resolution sharpened MS image is obtained by linearly compositing the fused gray image with the upsampled foreground and background images. The proposed framework is the first work in the pan-sharpening field. A large number of experiments were tested on various satellite datasets; the subjective visual and objective evaluation results indicate that the proposed method performs better than the IHS- and PCA-based frameworks, as well as other state-of-the-art pan-sharpening methods both in terms of spatial quality and spectral maintenance.

Multispectral and Hyperspectral Image Fusion Based on Group Spectral Embedding and Low-Rank Factorization

Fusing low spatial resolution hyperspectral (LRHS) images and high spatial resolution multispectral (HRMS) images to obtain high spatial resolution hyperspectral images (HRHS) has received increasing interests in recent years. In this paper, a new group spectral embedding (GSE)-based LRHS and HRMS image fusion method is proposed by exploring the multiple manifold structures of spectral bands and the low-rank structure of HRHS data. First, a low-rank factorization fusion (LRFF)-based robust recovery model is developed for HRHS images, by regarding HRMS images as the spectral degradation of HRHS images and exploring the group sparse prior of difference images. Then, an assumption that grouped spectral bands share the similar local geometry is cast on LRHS and HRHS images, to formulate a GSE regularizer in the LRFF model. Finally, an iterative optimization algorithm based on augmented Lagrangian multiplier is advanced to recover HRHS images. Experimental results on several data sets show the effectiveness of the proposed method on visual and numerical comparison.

A regularized pan-sharpening approach based on self-similarity and Gabor prior

ABSTRACTIn this article, we propose a new regularization-based approach for pan-sharpening based on the concepts of self-similarity and Gabor prior. The given low spatial resolution (LR) and high spectral resolution multi-spectral (MS) image is modelled as degraded and noisy version of the unknown high spatial resolution (HR) version. Since this problem is ill-posed, we use regularization to obtain the final solution. In the proposed method, we first obtain an initial HR approximation of the unknown pan-sharpened image using self-similarity and sparse representation (SR) theory. Using self-similarity, we obtain the HR patches from the given LR observation by searching for matching patches in its coarser resolution, thereby obtaining LR–HR pairs. An SR framework is used to obtain the patch pairs for which no matches are available for the patches in LR observation. The entire set of matched HR patches constitutes initial HR approximation (initial estimate) to the final pan-sharpened image which is used to estimate the degradation matrix as used in our model. A regularization framework is then used to obtain the final solution in which we propose to use a new prior which we refer as Gabor prior that extracts the bandpass details from the registered panchromatic (Pan) image. In addition, we also include Markov random field (MRF) smoothness prior that preserves the smoothness in the final pan-sharpened image. MRF parameter is derived using the initial estimate image. The final cost function consists of data fitting term and two prior terms corresponding to Gabor and MRF. Since the derived cost function is convex, simple gradient-based method is used to obtain the final solution. The efficacy of the proposed method is evaluated by conducting the experiments on degraded as well as on un-degraded datasets of three different satellites, i.e., Ikonos-2, Quickbird, and Worldview-2. The results are compared on the basis of traditional measures as well as recently proposed quality with no reference (QNR) measure, which does not require the reference image.

Locally Linear Detail Injection for Pansharpening

This paper presents a novel method, referred to as locally linear detail injection (LLDI), for the pansharpening problem, which is based on the assumption that the spatial details of each multispectral (MS) band can be locally and linearly represented by the spatial details of panchromatic images. LLDI first exploits such assumption through scales using the modulation transfer function (MTF) of the MS instrument and then performs detail injections into the available low-resolution MS images. Visual analysis and quantitative evaluation performed on QuickBird and WorldView-2 data sets at both reduced and full scales show that the proposed LLDI achieves superior improvements over its baselines.

A Pan-Sharpening Method Based on Evolutionary Optimization and IHS Transformation

In many remote sensing applications, users usually prefer a multispectral image with both high spectral and high spatial information. This high quality image could be obtained by pan-sharpening techniques which fuse a high resolution panchromatic (PAN) image and a low resolution multispectral (MS) image. In this paper, we propose a new technique to do so based on the adaptive intensity-hue-saturation (IHS) transformation model and evolutionary optimization. The basic idea is to reconstruct the target image through a parameterized adaptive IHS transformation. An optimization objective is thus introduced by considering the relations between the fused image and the original PAN and MS images. The control parameters are optimized by an evolutionary algorithm. Experimental results show that our new approach is practical and performs much better than some state-of-the-art techniques according to the performance metrics.

A New Geometry Enforcing Variational Model for Pan-Sharpening

In this paper, a new variational method for pan-sharpening is proposed to obtain a high-resolution multispectral (MS) image from a low-resolution MS image and a high-resolution panchromatic (PAN) image. In addition to using the data generative fidelity term and wavelet-based spectral information preserving term, we also associate the Hessian structural information of the PAN image with the desired pan-sharpened MS image to enforce geometry correspondence in the fusion process. More specifically, we introduce a new geometry enforcing term called “vectorial Hessian feature consistence” and combine it with the data generative fidelity term and wavelet-based spectral information preserving term to form an unified variational model for pan-sharpening. Then, the optimal solution of the proposed variational pan-sharpening model is effectively obtained using the fast iterative shrinkage thresholding algorithm (FISTA) method. In addition to well preserving spectral information, our algorithm is also able to eliminate some undesired blocky or blurry artifacts by incorporating the curvature information. Experimental results demonstrate that the proposed method outperforms various well-known pan-sharpening methods in terms of both excellent spatial and spectral qualities.

Context-Adaptive Pansharpening Based on Image Segmentation

Pansharpened images are widely used synthetic representations of the Earth surface characterized by both a high spatial resolution and a high spectral diversity. They are usually generated by extracting spatial details from a high-resolution PANchromatic image and by injecting them into a low spatial resolution multispectral image. The details injection is performed through injection coefficients, whose values can be either uniform for the whole image (global methods) or spatially variant (context-adaptive (CA) approaches). In this paper, we propose a CA approach in which the injection coefficients are estimated over image segments achieved through a binary partition tree segmentation algorithm. The approach is applied to two credited pansharpening algorithms based on the Gram–Schmidt orthogonalization procedure and the generalized Laplacian pyramid technique. The performance assessment is performed using two different data sets acquired by the QuickBird and the WorldView-3 satellites. The validation procedure, both at full and at reduced resolution, shows the suitability of the proposed approach, which reaches a good tradeoff between accuracy and computational burden.

Pan-sharpening with structural consistency and ℓ1/2 gradient prior

ABSTRACTIn this letter, a variational method is proposed for panchromatic (Pan)-sharpening by fusing a low-resolution multispectral image with a high-resolution Pan image to generate a high-resolution multispectral image. Most methods focus on spatial intensity preservation while the proposed method tends to preserve the consistency of the spatial structure information. In addition, an metric is used to estimate gradient prior. The alternating direction method of multipliers and the difference of convex algorithm are used to guarantee the convergence of the algorithm. A large number of experiments show that the proposed method has obvious advantages compared with other advanced methods both subjectively and objectively.

Pixel-level image fusion techniques in remote sensing: a review

In the recent years remote sensing image fusion of satellite images has become a popular tool for analyzing different features presented on satellite images. We have analyzed only pixel-level image fusion techniques in this paper, which integrates a low-resolution multispectral (MS) image and high-resolution panchromatic (PAN) image to produce a more informative image than any of the single image. Generally, a PAN image is having a better spatial resolution, while the MS image is having a better spectral resolution than PAN image, due to this trade-off between MS and PAN image resolutions, it could be difficult to preserve spectral and spatial resolution in a single image without a fusion technique. In this paper, we have reviewed most popular and recent image fusion techniques and implemented them on Cartosat-1, RESOURCESAT-2, LANDSAT-8 data set. Results obtained from each method by visual analysis and quantitative measures indicated that UNB fusion algorithm outperforms other techniques used in this paper.

Pansharpening with a Guided Filter Based on Three-Layer Decomposition.

State-of-the-art pansharpening methods generally inject the spatial structures of a high spatial resolution (HR) panchromatic (PAN) image into the corresponding low spatial resolution (LR) multispectral (MS) image by an injection model. In this paper, a novel pansharpening method with an edge-preserving guided filter based on three-layer decomposition is proposed. In the proposed method, the PAN image is decomposed into three layers: A strong edge layer, a detail layer, and a low-frequency layer. The edge layer and detail layer are then injected into the MS image by a proportional injection model. In addition, two new quantitative evaluation indices, including the modified correlation coefficient (MCC) and the modified universal image quality index (MUIQI) are developed. The proposed method was tested and verified by IKONOS, QuickBird, and Gaofen (GF)-1 satellite images, and it was compared with several of state-of-the-art pansharpening methods from both qualitative and quantitative aspects. The experimental results confirm the superiority of the proposed method.

Remote Sensing Image Fusion with Convolutional Neural Network

Remote sensing image fusion (RSIF) is referenced as restoring the high-resolution multispectral image from its corresponding low-resolution multispectral (LMS) image aided by the panchromatic (PAN) image. Most RSIF methods assume that the missing spatial details of the LMS image can be obtained from the high resolution PAN image. However, the distortions would be produced due to the much difference between the structural component of LMS image and that of PAN image. Actually, the LMS image can fully utilize its spatial details to improve the resolution. In this paper, a novel two-stage RSIF algorithm is proposed, which makes full use of both spatial details and spectral information of the LMS image itself. In the first stage, the convolutional neural network based super-resolution is used to increase the spatial resolution of the LMS image. In the second stage, Gram–Schmidt transform is employed to fuse the enhanced MS and the PAN images for further improvement the resolution of MS image. Since the spatial resolution enhancement in the first stage, the spectral distortions in the fused image would be decreased in evidence. Moreover, the spatial details can be preserved to construct the fused images. The QuickBird satellite source images are used to test the performances of the proposed method. The experimental results demonstrate that the proposed method can achieve better spatial details and spectral information simultaneously compared with other well-known methods.

MTF-Based Deblurring Using a Wiener Filter for CS and MRA Pansharpening Methods

Pansharpening is the fusion of low-resolution multispectral (MS) images and high-resolution panchromatic (PAN) images to yield a high-resolution MS image. The component substitution (CS) and multiresolution analysis (MRA) methods are usually computationally efficient, making them able to handle large datasets. However, these methods often produce images that suffer from spectral and spatial distortions. The CS and MRA methods can be described using general injection schemes where details extracted from the PAN image, modulated by a band-dependent gain constant, are added to the MS image, which has been interpolated to the size of the PAN image. In this paper, we propose a simple modification of these schemes where the interpolated MS image is replaced by its deblurred version, where the deblurring kernel is matched to the modulation transfer function (MTF) of the MS sensor. This can significantly enhance the quality of the fused image. Using two real datasets and one simulated dataset, our experimental results show that using the proposed preprocessing method can significantly increase both the spectral and spatial quality of the fused image according to quantitative quality metrics.

High spectral quality pansharpening approach based on MTF-matched filter banks

Pansharpening consists in merging a low-resolution multispectral image (MS) with a high spatial resolution panchromatic image (PAN) to produce a high resolution pansharpened MS image. It consists in enhancing spatially the low-resolution MS image by injecting the missing details provided by the high-resolution PAN image. In this paper, we propose a novel pansharpening approach based on decomposition/reconstruction processing using low-pass and high-pass filter banks. On the one hand, the low-pass approximation (taking into account the imaging system modulation transfer function MTF) of the pansharpened MS image is assumed to be equal to the original MS image in order to preserve the spectral quality. On the other hand, the high-pass filter allowing us to extract the high-frequency PAN details is designed as complementary filter to the low-pass one in order to provide perfect reconstruction in the ideal case. Quantitative assessment performed on reduced and full-resolution images are used to validate the proposed technique and compare it to state-of-art. Experimental results using Pleaides and GeoEye-1 data show that our proposed fusion schema outperforms the pre-existing methods visually as well as quantitatively.