High Spatial Resolution Panchromatic Image Research Articles

An optimal weighted averaging fusion strategy for remotely sensed images

Image fusion plays a vital role in providing better visualization of remotely sensed image data. Most earth observation satellites have sensors that provide both high spatial resolution panchromatic (PAN) images and low resolution multispectral (MS) images. In this paper, we propose a new fusion algorithm that optimally combines spectral information from MS image and spatial information from the PAN image of the same scene to create a single comprehensive fused image. As the performance of the fusion scheme relies on the choice of fusion rule, the proposed algorithm is based on a weighted averaging fusion rule that uses optimal weights obtained from brain storm optimization (BSO) algorithm for the fusion of high frequency and low frequency coefficients obtained by applying Curvelet transform to the source images. The objective function in BSO is formulated with twin objectives of maximizing the entropy and minimizing the root mean square error. The fusion results are compared with the existing fusion techniques, such as Brovey, principal component analysis, discrete wavelet transform, non sub-sampled contourlet transform, and intensity hue saturation. From the experimental results and analysis, the proposed fusion algorithm gives a better fusion performance in terms of subjective and objective measures than the traditional algorithms. As a benefit, the proposed fusion scheme preserves spectral information of the MS image with increased spatial resolution and edge information.

A Variational Pansharpening Approach Based on Reproducible Kernel Hilbert Space and Heaviside Function.

Pansharpening is an important application in remote sensing image processing. It can increase the spatial-resolution of a multispectral image by fusing it with a high spatial-resolution panchromatic image in the same scene, which brings great favor for subsequent processing such as recognition, detection, etc. In this paper, we propose a continuous modeling and sparse optimization based method for the fusion of a panchromatic image and a multispectral image. The proposed model is mainly based on reproducing kernel Hilbert space (RKHS) and approximated Heaviside function (AHF). In addition, we also propose a Toeplitz sparse term for representing the correlation of adjacent bands. The model is convex and solved by the alternating direction method of multipliers which guarantees the convergence of the proposed method. Extensive experiments on many real datasets collected by different sensors demonstrate the effectiveness of the proposed technique as compared with several state-of-the-art pansharpening approaches.

A Bayesian Procedure for Full-Resolution Quality Assessment of Pansharpened Products

Pansharpening regards the fusion of a high-spatial resolution panchromatic image with a low-spatial resolution multispectral image. One of the most debated topics about pansharpening is related to the quality assessment of fused products. Two main assessment procedures are usually exploited in the literature: the reduced resolution validation and the full-resolution (FR) validation. The former has the advantage to be accurate, but the hypothesis of invariance among scales has to be assumed. The latter overcomes this limitation but paying it with a lower accuracy. In this paper, we will focus on the FR assessment proposing an approach for estimating an overall quality index at FR by using multiscale FR measurements. The problem is recast into the sequential Bayesian framework exploiting a Kalman filter to find its solution. The proposed procedure for quality evaluation has been tested on four real data sets acquired by the Pleiades, the GeoEye-1, the WorldView-3, and the WorldView-4 sensors assessing the quality of 19 pansharpened methods. The proposed approach has demonstrated its superiority with respect to the benchmark consisting of state-of-the-art quality assessment procedures.

MULTISPECTRAL AND PANCHROMATIC REGISTRATION OF ALSAT-2 IMAGES USING DENSE VECTOR MATCHING FOR PAN-SHARPENING PROCESS

Abstract. The pan-sharpening is a widely used operation in remote sensing image processing, this operation aims at combining an observable high spatial resolution panchromatic image with a multispectral one, to generate an unobservable image with the high spatial resolution of the former and a high spectral resolution of the latter. Generally, papers dealing with this problem omit the geometric part and suppose that these images are perfectly aligned, which is not necessarily the case for the raw imagery, where even the different bands in the multispectral imagery are misaligned. In this paper, new method for multispectral and panchromatic image registration is proposed to deal with the misalignment problem that reduces the pansharpening quality. This method called Dense Vector Matching (DVM) is based on the matching of a whole line-vector or column-vector from a reference band with the corresponding vector in a target band. DVM is applied on real data and has given acceptable results, where the QNR index of the pan-sharpening is better for images after band registration, also the registration error is reduced to sub-pixel using the proposed approach.

A Regression-Based High-Pass Modulation Pansharpening Approach

Pansharpening usually refers to the fusion of a high spatial resolution panchromatic (PAN) image with a higher spectral resolution but coarser spatial resolution multispectral (MS) image. Owing to the wide applicability of related products, the literature has been populated by many papers proposing several approaches and studies about this issue. Many solutions require a preliminary spectral matching phase wherein the PAN image is matched with the MS bands. In this paper, we propose and properly justify a new approach for performing this step, demonstrating that it yields state-of-the-art performance. The comparison with existing spectral matching procedures is performed by employing four data sets, concerning different kinds of landscapes, acquired by the Pleiades, WorldView-2, and GeoEye-1 sensors.

Optical SAR Images Fusion: Comparative Analysis of Resulting Images Data

Image fusion is a useful tool for integrating low spatial resolution multispectral (MS) images with a high spatial resolution panchromatic (PAN) image, thus producing a high resolution multispectral image for better understanding of the observed earth surface. A main proposed the research were the effectiveness of different image fusion methods while filtering methods added to speckle suppression in synthetic aperture radar (SAR) images. The quality assessment of the filtering fused image implemented by statistical parameter namely mean, standard deviation, bias, universal index quality image (UIQI) and root mean squared error (RMSE). In order to test the robustness of the image quality, either speckle noise (Gamma map filter) is intentionally added to the fused image. When comparing and testing result, Gram Scmidth (GS) methods have shown better results for good colour reproduction, as compared with high pass filtering (HPF). And the other hands, GS, and wavelet intensity hue saturation (W-IHS) have shown the preserving good colour with original image for Landsat TM data.

A novel variational model for pan-sharpening based on L1 regularization

ABSTRACTThe Pan-sharpening process synthesizes a multispectral image of high spatial and high spectral quality by injecting the spatial details extracted from the high spatial resolution panchromatic image into the low-resolution multispectral image. In this letter, we propose a novel variational model for pan-sharpening based on L1 regularization, which consists of three energy terms. The first energy term extracts the geometric information from panchromatic image with exponential enhancement function, which can improve the contrast of the fused image by nonlinear expanding of gradient magnitude. The second energy term develops the low-pass filter based on the modulation transfer function (MTF) of the different MS sensor, which can preserve spectral information by determining the injected spatial details adaptively. Compared with the total variation regularization, the L1 regularization encourages the feature of sparse representation, which can improve the spatial fidelity of pan-sharpened images. In addition, introducing L1 regularization term into the variational framework can smooth the image noise and guarantee the stability of the numerical solution. Experimental results demonstrate that the proposed method outperforms several state-of-the-art pan-sharpening methods, including intensity-hue-saturation combined with Brovey transform (IHS-BT), additive wavelet luminance propotional (AWLP), total variation regularization (TVR) and MTF Contrast-based (MTF-CON) method.

Assessment of Multi-Temporal Image Fusion for Remote Sensing Application

Image fusion and subsequent scene analysis are important for studying Earth surface conditions from remotely sensed imagery. The fusion of the same scene using satellite data taken with different sensors or acquisition times is known as multi-sensor or multi-temporal fusion, respectively. The purpose of this study is to investigate the effects of misalignments the multi-sensor, multi-temporal fusion process when a pan-sharpened scene is produced from low spatial resolution multispectral (MS) images and a high spatial resolution panchromatic (PAN) image. It is found that the component substitution (CS) fusion method provides better performance than the multi-resolution analysis (MRA) scheme. Quantitative analysis shows that the CS-based method gives a better result in terms of spatial quality (sharpness), whereas the MRA-based method yields better spectral quality, i.e., better color fidelity to the original MS images .

Building Outline Extraction Using a Heuristic Approach Based on Generalization of Line Segments

Efficient and fully automatic building outline extraction and simplification methods are highly demanded for three-dimensional model reconstruction tasks. In spite of the efforts put into developing such methods, the results of the recently proposed methods are still not satisfactory, especially for satellite images, due to object complexities and the presence of noise. Dealing with this problem, in this article, we propose a new approach that detects rough building boundaries (building mask) from Digital Surface Model data and then refines the resulting mask by classifying the geometrical features of the high spatial resolution panchromatic satellite image. The refined mask represents finer details of the building outlines, which are close to the original building edges. These outlines are then simplified through a parameterization phase wherein a tracing algorithm detects the building boundary points from the refined masks and a set of line segments is fitted to them. After that, for each building, the existing main orientations are determined based on the length and arc lengths of the building's line segments. Our method is able to determine the multiple main orientations of complex buildings. Through a regularization process, the line segments are then aligned and adjusted according to the building's main orientations. Finally, the adjusted line segments are intersected and connected to each other in order to form a polygon representing the building's outlines. Experimental results demonstrate that the computed building outlines are highly accurate and simple, even for large and complex buildings with inner yards.

Estimating babassu palm density using automatic palm tree detection with very high spatial resolution satellite images

High spatial resolution images as well as image processing and object detection algorithms are recent technologies that aid the study of biodiversity and commercial plantations of forest species. This paper seeks to contribute knowledge regarding the use of these technologies by studying randomly dispersed native palm tree. Here, we analyze the automatic detection of large circular crown (LCC) palm tree using a high spatial resolution panchromatic GeoEye image (0.50 m) taken on the area of a community of small agricultural farms in the Brazilian Amazon. We also propose auxiliary methods to estimate the density of the LCC palm tree Attalea speciosa (babassu) based on the detection results. We used the “Compt-palm” algorithm based on the detection of palm tree shadows in open areas via mathematical morphology techniques and the spatial information was validated using field methods (i.e. structural census and georeferencing). The algorithm recognized individuals in life stages 5 and 6, and the extraction percentage, branching factor and quality percentage factors were used to evaluate its performance. A principal components analysis showed that the structure of the studied species differs from other species. Approximately 96% of the babassu individuals in stage 6 were detected. These individuals had significantly smaller stipes than the undetected ones. In turn, 60% of the stage 5 babassu individuals were detected, showing significantly a different total height and a different number of leaves from the undetected ones. Our calculations regarding resource availability indicate that 6870 ha contained 25,015 adult babassu palm tree, with an annual potential productivity of 27.4 t of almond oil. The detection of LCC palm tree and the implementation of auxiliary field methods to estimate babassu density is an important first step to monitor this industry resource that is extremely important to the Brazilian economy and thousands of families over a large scale.

High resolution snapshot imaging spectrometer using a fusion algorithm based on grouping principal component analysis.

We reported a high resolution snapshot imaging spectrometer (HR-SIS) and a fusion algorithm based on the properties of the HR-SIS. The system consists of an imaging branch and a spectral branch. The imaging branch captures a high spatial resolution panchromatic image with 680 × 680 pixels, while the spectral branch acquires a low spatial resolution spectral image with spectral resolution of 250 cm-1. By using a fusion algorithm base on grouping principal component analysis, the spectral image is highly improved in spatial resolution. Experimental results demonstrated that the performance of the proposed algorithm is competitive with other state-of-the-art algorithms. The computing time for a single frame is less than 1 min with an Intel Core i5-4200H CPU, which can be further reduced by utilizing a graphics processing unit (GPU).

Performance Evaluation of Pansharpening Algorithms for WorldView-3 Satellite Imagery

Worldview-3 satellite sensor provides panchromatic image with high-spatial resolution and 8-band multispectral images. Therefore, an image-sharpening technique, which sharpens the spatial resolution of multispectral images by using high-spatial resolution panchromatic images, is essential for various applications of Worldview-3 images based on image interpretation and processing. The existing pansharpening algorithms tend to tradeoff between spectral distortion and spatial enhancement. In this study, we applied six pansharpening algorithms to Worldview-3 satellite imagery and assessed the quality of pansharpened images qualitatively and quantitatively. We also analyzed the effects of time lag for each multispectral band during the pansharpening process. Quantitative assessment of pansharpened images was performed by comparing ERGAS (Erreur Relative Globale Adimensionnelle de Synthese), SAM (Spectral Angle Mapper), Q-index and sCC (spatial Correlation Coefficient) based on real data set. In experiment, quantitative results obtained by MRA (Multi-Resolution Analysis)-based algorithm were better than those by the CS (Component Substitution)-based algorithm. Nevertheless, qualitative quality of spectral information was similar to each other. In addition, images obtained by the CS-based algorithm and by division of two multispectral sensors were shaper in terms of spatial quality than those obtained by the other pansharpening algorithm. Therefore, there is a need to determine a pansharpening method for Worldview-3 images for application to remote sensing data, such as spectral and spatial information-based applications.

Pansharpening with a Guided Filter Based on Three-Layer Decomposition.

State-of-the-art pansharpening methods generally inject the spatial structures of a high spatial resolution (HR) panchromatic (PAN) image into the corresponding low spatial resolution (LR) multispectral (MS) image by an injection model. In this paper, a novel pansharpening method with an edge-preserving guided filter based on three-layer decomposition is proposed. In the proposed method, the PAN image is decomposed into three layers: A strong edge layer, a detail layer, and a low-frequency layer. The edge layer and detail layer are then injected into the MS image by a proportional injection model. In addition, two new quantitative evaluation indices, including the modified correlation coefficient (MCC) and the modified universal image quality index (MUIQI) are developed. The proposed method was tested and verified by IKONOS, QuickBird, and Gaofen (GF)-1 satellite images, and it was compared with several of state-of-the-art pansharpening methods from both qualitative and quantitative aspects. The experimental results confirm the superiority of the proposed method.

High spectral quality pansharpening approach based on MTF-matched filter banks

Pansharpening consists in merging a low-resolution multispectral image (MS) with a high spatial resolution panchromatic image (PAN) to produce a high resolution pansharpened MS image. It consists in enhancing spatially the low-resolution MS image by injecting the missing details provided by the high-resolution PAN image. In this paper, we propose a novel pansharpening approach based on decomposition/reconstruction processing using low-pass and high-pass filter banks. On the one hand, the low-pass approximation (taking into account the imaging system modulation transfer function MTF) of the pansharpened MS image is assumed to be equal to the original MS image in order to preserve the spectral quality. On the other hand, the high-pass filter allowing us to extract the high-frequency PAN details is designed as complementary filter to the low-pass one in order to provide perfect reconstruction in the ideal case. Quantitative assessment performed on reduced and full-resolution images are used to validate the proposed technique and compare it to state-of-art. Experimental results using Pleaides and GeoEye-1 data show that our proposed fusion schema outperforms the pre-existing methods visually as well as quantitatively.

SIFT-FANN: An efficient framework for spatio-spectral fusion of satellite images

Image fusion techniques are widely used for remote sensing data. A special application is for using low resolution multi-spectral image with high resolution panchromatic image to obtain an image having both spectral and spatial information. Alignment of images to be fused is a step prior to image fusion. This is achieved by registering the images. This paper proposes the methods involving Fast Approximate Nearest Neighbor (FANN) for automatic registration of satellite image (reference image) prior to fusion of low spatial resolution multi-spectral QuickBird satellite image (sensed image) with high spatial resolution panchromatic QuickBird satellite image. In the registration steps, Scale Invariant Feature Transform (SIFT) is used to extract key points from both images. The keypoints are then matched using the automatic tuning algorithm, namely, FANN. This algorithm automatically selects the most appropriate indexing algorithm for the dataset. The indexed features are then matched using approximate nearest neighbor. Further, Random Sample Consensus (RanSAC) is used for further filtering to obtain only the inliers and co-register the images. The images are then fused using Intensity Hue Saturation (IHS) transform based technique to obtain a high spatial resolution multi-spectral image. The results show that the quality of fused images obtained using this algorithm is computationally efficient.

Multi-sensor Satellite Pan-sharpening based on IHS and Window Pseudo Wigner Distribution Integrated Approach: Application to WorldView-2 Imagery

ABSTRACTThis paper proposes a new approach to fuse high spatial resolution panchromatic (PAN) image with a low spatial resolution multispectral (MS) image, based on the joint use of intensity-hue-saturation (IHS) and Window Pseudo Wigner Distribution (WPWD), with a motive to reduce the spectral distortion problem of IHS technique. This approach utilises the IHS transform to fuse high-resolution spatial information into the low-resolution MS images, and uses the WPWD to reduce the colour distortion, in the way of generating a new high-resolution PAN image that highly correlates with the intensity image of the IHS transform. The new PAN image is, then, used to replace the intensity image for an inverse IHS transform. The proposed hybrid technique has been compared with the existing hybrid techniques in terms of different fusion rules, computational time, qualitative and different quantitative parameters. The very high resolution satellite dataset, especially, WorldView-2 sensor, which provides data at submeter level has been taken for the pan-sharpening. Analysis of results proved that the concept of proposed IHS and WPWD integration using different fusion rules works well and gives a good trade-off between the enhancement of spatial resolution and preservation of spectral information, along with balanced computational time, when compared to IHS, IHS-DWT, IHS-SWT and IHS-NSCT fusion techniques.

A Quality of Images Fusion for Remote Sensing Applications

Image fusion is a useful tool for integrating low spatial resolution multispectral (MS) images with a high spatial resolution panchromatic (PAN) image, thus producing a high resolution multispectral image for better understanding of the observed earth surface. It has has become an important issue for various remotes sensing (RS) problems such as land classification, change detection, object identification, image segmentation, map updating, hazard monitoring, and visualization purposes. When applied to remote sensing images, a common problem associated with existing fusion methods has been the color distortion, or degradation in the spectral quality. The main proposed of this research is to evaluate the quality of fused images for object identification. We examine the effectiveness of the following techniques multi-resolution analysis (MRA) and component substitution (CS) fusion. In order to improve this situation, the second purpose of this work is to establish an automatic and reliable way for the evaluation of the fused images, on the basis of both qualitative and quantitative metrics. In this result, It is found that the CS fusion method provides better performance than the MRA scheme. Quantitative analysis shows that the CS-based method gives a better result in terms of spatial quality (sharpness), whereas the MRA-based method yields better spectral quality, i.e., better color fidelity to the original MS images.

Improved Pansharpening with Un-Mixing of Mixed MS Sub-Pixels near Boundaries between Vegetation and Non-Vegetation Objects

Pansharpening is an important technique that produces high spatial resolution multispectral (MS) images by fusing low spatial resolution MS images and high spatial resolution panchromatic (PAN) images of the same area. Although numerous successful image fusion algorithms have been proposed in the last few decades to reduce the spectral distortions in fused images, few of these take into account the spectral distortions caused by mixed MS sub-pixels (MSPs). Typically, the fused versions of MSPs remain mixed, although some of the MSPs correspond to pure PAN pixels. Due to the significant spectral differences between vegetation and non-vegetation (VNV) objects, the fused versions of MSPs near VNV boundaries cause blurred VNV boundaries and significant spectral distortions in the fused images. In order to reduce the spectral distortions, an improved version of the haze- and ratio-based fusion method is proposed to realize the spectral un-mixing of MSPs near VNV boundaries. In this method, the MSPs near VNV boundaries are identified first. The identified MSPs are then defined as either pure vegetation or non-vegetation pixels according to the categories of the corresponding PAN pixels. Experiments on WorldView-2 and IKONOS images of urban areas using the proposed method yielded fused images with significantly clearer VNV boundaries and smaller spectral distortions than several other currently-used image fusion methods.

Multi-resolution analysis techniques and nonlinear PCA for hybrid pansharpening applications

Hyperspectral images have a higher spectral resolution (i.e., a larger number of bands covering the electromagnetic spectrum), but a lower spatial resolution with respect to multispectral or panchromatic acquisitions. For increasing the capabilities of the data in terms of utilization and interpretation, hyperspectral images having both high spectral and spatial resolution are desired. This can be achieved by combining the hyperspectral image with a high spatial resolution panchromatic image. These techniques are generally known as pansharpening and can be divided into component substitution (CS) and multi-resolution analysis (MRA) based methods. In general, the CS methods result in fused images having high spatial quality but the fused images suffer from spectral distortions. On the other hand, images obtained using MRA techniques are not as sharp as CS methods but they are spectrally consistent. Both substitution and filtering approaches are considered adequate when applied to multispectral and PAN images, but have many drawbacks when the low-resolution image is a hyperspectral image. Thus, one of the main challenges in hyperspectral pansharpening is to improve the spatial resolution while preserving as much as possible of the original spectral information. An effective solution to these problems has been found in the use of hybrid approaches, combining the better spatial information of CS and the more accurate spectral information of MRA techniques. In general, in a hybrid approach a CS technique is used to project the original data into a low dimensionality space. Thus, the PAN image is fused with one or more features by means of MRA approach. Finally the inverse projection is used to obtain the enhanced image in the original data space. These methods, permit to effectively enhance the spatial resolution of the hyperspectral image without relevant spectral distortions and on the same time to reduce the computational load of the entire process. In particular, in this paper we focus our attention on the use of Nonlinear Principal Component Analysis (NLPCA) for the projection of the image into a low dimensionality feature space. However, if on one hand the NLPCA has been proved to better represent the intrinsic information of hyperspectral images in the feature space, on the other hand an analysis of the impact of different fusion techniques applied to the nonlinear principal components in order to define the optimal framework for the hybrid pansharpening has not been carried out yet. More in particular, in this paper we analyze the overall impact of several widely used MRA pansharpening algorithms applied in the nonlinear feature space. The results obtained on both synthetic and real data demonstrate that an accurate selection of the pansharpening method can lead to an effective improvement of the enhanced hyperspectral image in terms of spectral quality and spatial consistency, as well as a strong reduction in the computational time.

Parameter selection for variational pan-sharpening by using evolutionary algorithm

Pan-sharpening is a technique that generates a high spatial resolution multi-spectral image making use of both the spectral information contained in a low spatial resolution multi-spectral image and the spatial information contained in a high spatial resolution panchromatic image. The pan-sharpening method usually contains some parameters. They are usually problem dependent and need to be set properly. In this article, we propose a variational method for pan-sharpening and use an evolutionary algorithm (EA) to choose the optimal parameters automatically. In our method, two quality measurements are combined to form an optimization objective function of the EA, and the parameters are encoded as an individual vector in the EA. The optimal parameters are generated by optimizing the objective function of the EA. The new method is compared with some other variational methods using QuickBird data. We also applied the selected parameters to different images to discuss the applicable scope. The experimental results show that our method can generate a high-quality fused image, and the same parameters’ values can be used for similar images.