Generalized Intensity-hue-saturation Research Articles

Panchromatic and Multispectral Image Fusion Combining GIHS, NSST, and PCA

Spatial and spectral information are essential sources of information in remote sensing applications, and the fusion of panchromatic and multispectral images effectively combines the advantages of both. Due to the existence of two main classes of fusion methods—component substitution (CS) and multi-resolution analysis (MRA), which have different advantages—mixed approaches are possible. This paper proposes a fusion algorithm that combines the advantages of generalized intensity–hue–saturation (GIHS) and non-subsampled shearlet transform (NSST) with principal component analysis (PCA) technology to extract more spatial information. Therefore, compared with the traditional algorithms, the algorithm in this paper uses PCA transformation to obtain spatial structure components from PAN and MS, which can effectively inject spatial information while maintaining spectral information with high fidelity. First, PCA is applied to each band of low-resolution multispectral (MS) images and panchromatic (PAN) images to obtain the first principal component and to calculate the intensity of MS. Then, the PAN image is fused with the first principal component using NSST, and the fused image is used to replace the original intensity component. Finally, a fused image is obtained using the GIHS algorithm. Using the urban, plants and water, farmland, and desert images from GeoEye-1, WorldView-4, GaoFen-7 (GF-7), and Gaofen Multi-Mode (GFDM) as experimental data, this fusion method was tested using the evaluation mode with references and the evaluation mode without references and was compared with five other classic fusion algorithms. The results showed that the algorithms in this paper had better fusion performances in both spectral preservation and spatial information incorporation.

A spatial-channel progressive fusion ResNet for remote sensing classification

In recent years, the panchromatic (PAN) and the multispectral (MS) remote sensing images classification has become a research hotspot. In this paper, we propose a spatial-channel progressive fusion residual network (SCPF-ResNet) for multi-resolution remote sensing classification. Firstly, for the inputs of the proposed network, the interactive data fusion strategy (IDFS) combines generalized-intensity-hue-saturation (GIHS), and discrete wavelet transform (DWT) to interfuse patch pairs of the PAN and the MS images, so as to increase the similarity between them, thus reduce the difference in information between them. Secondly, for the branches of feature extraction, we design an adaptive spatial attention module (ASA-Module) and an adaptive channel attention module (ACA-Module) to strengthen spatial features from both larger-sized with smaller-sized targets and spectral features among channels. Finally, we insert the ASA-Module and ACA-Module into the residual modules to form a triple-branch network and use the common spatial-channel features extracted by the Fusion_Branch to gradually enhance the pure independent features extracted by the PAN_Branch and the MS_Branch, respectively. The experimental results indicate that SCPF-ResNet can achieve competitive performance.

Remote sensing data fusion using fruit fly optimization

The image fusion (pan-sharpening) aims to generate a multispectral image with the maximum of spatial details of the panchromatic image and the spectral characteristics of the multispectral images. The generalized intensity-hue-saturation (GIHS) can produce fused images with high spatial details, but the spectral characteristics of the fused images need improvement. This article introduces an adaptive fusion method based on GIHS using the Fruit Fly Optimization Algorithm (FOA) in two stages. First, we suggest to use it to compute the optimal band weights which reduces the large difference between the intensity component and the panchromatic image. Second, we propose to apply the FOA to compute the modulation parameters that estimate the amount of spatial details to be added to the multispectral images. In this regard, an objective function that combines the coefficient of the correlation (CC) with the spatial coefficient of the correlation (SCC) is suggested. This method is tested on Pleiades, IKONOS, and ALSAT-2A images and we compared it with some existing fusion methods. The CC, the Structural Similarity Index (SSIM), The Root Mean Square Error (RMSE), the Quality with No Reference (QNR), the Relative Global Synthesis Error Metric (ERGAS) and the Relative Average Spectral Error (RASE) are used for quantitative analysis. The best values of CC, RMSE, ERGAS, RASE and QNR on all used datasets are given by the proposed method. The quantitative results and the visual analysis demonstrated that the proposed approach outperforms the four comparison methods in terms of spatial and spectral quality.

A Novel Multispectral, Panchromatic and SAR Data Fusion for Land Classification

Multisensor data fusion is addressed in this article for land classification purposes in a semiarid environment. A novel algorithm based on multispectral, panchromatic and synthetic aperture radar (SAR) data is here presented. The proposed multisensory data fusion approach relies on the generalized intensity-hue-saturation (G-IHS) transform and the A trous wavelet transform (ATWT). The fusion product is obtained by modulating the high features details of the panchromatic ATWT with the SAR texture and by replacing the high-pass details of the G-IHS Intensity component with this panchromatic-SAR modulation. After the fusion product is derived, a classification is performed by using a standard maximum likelihood classifier. The proposed algorithm is tested over a meaningful case study acquired over the Maspalomas Special Natural Reserve (Spain) and processing data from WorldView-2 (for both multispectral and panchromatic channels) and TerraSAR-X (for the SAR channel) missions. Results show a fine preservation of the spectral information contained in each multispectral band. Sharpened details are observed over built-up areas and a smoothing texture is perceived over homogeneous areas (lakes, sea, bare soil, and roads) due to the SAR-panchromatic modulation. This leads to a better overall classification accuracy of the fused image compared to outcomes obtained with a single sensor, resulting 7% and 2% more accurate than multispectral and pan-sharpening classification, respectively.

Multi-Modal Sensor Medical Image Fusion Based on Multiple Salient Features With Guided Image Filter

In this paper, we propose an efficient image fusion algorithm using multiple salient features with guided image filter to prevent the problem of low contrast detail. First, we employ the guided image filter to decompose the input images into a series of smoothed and detailed images at different scales. Second, the salient features are extracted from the decomposed smoothed images and detailed images using two different algorithms: the spectral residual (SR) algorithm for extracting mainframe information and the graph-based visual saliency (GBVS) model for extracting gradient saliency information to construct the fusion rules. In addition, generalized intensity-hue-saturation (GIHS) is adopted to combine the decomposition coefficients. Finally, the fused image is reconstructed by the fused smoothed and fused detailed images. The experimental results demonstrate that the proposed algorithm can achieve better performance than other fusion methods in the domains of MRI-PET and MRI-SPECT fusion.

Generalized IHS-BT framework for the pansharpening of high-resolution satellite imagery

Image fusion is an important technique for enhancing the spatial resolution of multispectral imagery by merging with a corresponding high-resolution panchromatic image. Many algorithms have been developed to obtain a better trade-off between spectral fidelity and spatial resolution enhancement. The recently developed techniques use sophisticated algorithms; however, three issues are often ignored. The first involves the progressively increasing amount of high-resolution satellite image data. Second, the designed method needs to be versatile to process a variety of complex terrain and surface features. The third is that the proposed method must satisfy the requirements of both visual inspection and classification. Based on the previous studies, this work proposes a generalized intensity–hue–saturation and Brovey transform framework (GF) to tackle these three problems. Eight modules are integrated into this GF: four modules are designed for classification with a new intensity matching technique to improve the spectral quality; the other four modules are the spatial-enhancement version for visual inspection by adding more native spatial details. To verify the proposed technique, the experiments are conducted for QuickBird and WorldView-2 images. The results show that the GF modules can offer the maximal spatial information content while preserving good spectral information quality.

A Novel Generalized Intensity-Hue-Saturation (GIHS) Based Pan-Sharpening Method With Variational Hessian Transferring

The popular intensity-hue-saturation (IHS) pan-sharpening method can provide high spatial quality while suffering from some spectral distortion, mainly due to the fact that it cannot estimate an accurate intensity image to substitute the original intensity image in the IHS space. To overcome this drawback, in this paper, we particularly use the modeling idea of variational complementary data fusion, and propose a novel yet highly effective pan-sharpening method with variational Hessian transferring in the generalized IHS (GIHS) transform domain, which aims to estimate a more accurate intensity image. More specifically, the novelty of proposed method consists in building a variational Hessian transferring model in the GIHS transform domain to transfer the Hessian-based spatial geometric information of panchromatic (Pan) image to the new intensity image and meanwhile take the spectral information preserving into consideration. Finally, the experimental results demonstrate the effectiveness of the proposed method which can perform higher spectral and spatial qualities, and higher efficiency than some state-of-the-art methods.

New edge adaptive GIHS-BT-SFIM fusion method and class-based approach investigation

A unified image fusion approach suitable for all employed imaging sources becomes necessary, with the development of space-borne imaging sensors. In this work, two issues were tested to improve the spectral quality of fused images and also investigating class-based fusion results. First, a Generalized Intensity-Hue-Saturation (GIHS), Brovey Transform (BT) and SFIM approach was implemented. This kind of algorithm has shown computational advantages among other fusion methods like wavelet and can be extended to different number of bands as in literature discussed. The used IHS-BT-SFIM algorithm incorporates IHS, IHS-BT, BT, BT-SFIM and SFIM methods by two adjustable parameters. Second, a method was proposed to add edge information in previous GIHS_BT_SFIM and edge enhancement by panchromatic image. Fused image was not improved significantly by panchromatic data addition. Third, an Edge Adaptive GIHS_BT_SFIM was proposed to enforce fidelity away from the edges. Using MS image off edges has shown spectral improvement in some fusion methods. Fourth, a class-based fusion was tested, which tests different coefficients for each method due to its class. The best parameters for vegetated areas were k1 = 0.6, k2 = 0.8, and for urban region, they were k1 = 0.4, k2 = 0.4. Results might be applicable in generalisation of various fusion methods.

A texture-based fusion scheme to integrate high-resolution satellite SAR and optical images

Image fusion of different types of satellite images is of widespread interest due to the increasing availability of various spaceborne imaging sensors. In this letter, a new multi-sensor image fusion method is presented, where a texture-based fusion rule is used to address the fusion of synthetic aperture radar (SAR) and optical images. At first, the integration of panchromatic (PAN) and SAR image is achieved in wavelet domain by taking weighted average of the corresponding input pixels. The weights are adaptively decided according to the local spatial autocorrelation characteristics of SAR texture image. This integrated image is fused into the multispectral (MS) image through generalized intensity-hue-saturation (GIHS) image fusion method to obtain the final fused image. To quantitatively evaluate and test the performance of the proposed method, comparisons with some existing fusion methods are carried out in this letter. Experimental results demonstrate that the proposed method shows better performance than other fusion methods and can get satisfactory fusion results.

A GIHS-based spectral preservation fusion method for remote sensing images using edge restored spectral modulation

High spatial resolution and spectral fidelity are basic standards for evaluating an image fusion algorithm. Numerous fusion methods for remote sensing images have been developed. Some of these methods are based on the intensity–hue–saturation (IHS) transform and the generalized IHS (GIHS), which may cause serious spectral distortion. Spectral distortion in the GIHS is proven to result from changes in saturation during fusion. Therefore, reducing such changes can achieve high spectralfidelity. A GIHS-based spectral preservation fusion method that cantheoretically reduce spectraldistortion is proposed in this study. The proposed algorithm consists of two steps. The first step is spectral modulation (SM), which uses the Gaussian function to extract spatial details and conduct SM of multispectral (MS) images. This method yields a desirable visual effect without requiring histogram matching between the panchromatic image and the intensity of the MS image. The second step uses the Gaussian convolution function to restore lost edge details during SM. The proposed method is proven effective and shown to provide better results compared with other GIHS-based methods.

A New Developed GIHS-BT-SFIM Fusion Method Based On Edge and Class Data

Abstract. The objective of image fusion (or sometimes pan sharpening) is to produce a single image containing the best aspects of the source images. Some desirable aspects are high spatial resolution and high spectral resolution. With the development of space borne imaging sensors, a unified image fusion approach suitable for all employed imaging sources becomes necessary. Among various image fusion methods, intensity-hue-saturation (IHS) and Brovey Transforms (BT) can quickly merge huge amounts of imagery. However they often face color distortion problems with fused images. The SFIM fusion is one of the most frequently employed approaches in practice to control the tradeoff between the spatial and spectral information. In addition it preserves more spectral information but suffer more spatial information loss. Its effectiveness is heavily depends on the filter design. In this work, two modifications were tested to improve the spectral quality of the images and also investigating class-based fusion results. First, a Generalized Intensity-Hue-Saturation (GIHS), Brovey Transform (BT) and smoothing-filter based intensity modulation (SFIM) approach was implemented. This kind of algorithm has shown computational advantages among other fusion methods like wavelet, and can be extended to different number of bands as in literature discussed. The used IHS-BT-SFIM algorithm incorporates IHS, IHS-BT, BT, BT-SFIM and SFIM methods by two adjustable parameters. Second, a method was proposed to plus edge information in previous GIHS_BT_SFIM and edge enhancement by panchromatic image. Adding panchromatic data to images had no much improvement. Third, an edge adaptive GIHS_BT_SFIM was proposed to enforce fidelity away from the edges. Using MS image off edges has shown spectral improvement in some fusion methods. Fourth, a class based fusion was tested, which tests different coefficients for each method due to its class. The best parameters for vegetated areas was k1 = 0.6, k2 = 0.8; and for urban region it was k1 = 0.4, k2 = 0.4. Results might be useful for future studies on fusion methods and their generalization.

THE INFLUENCE OF SPECTRAL WAVELENGTH ON THE QUALITY OF PANSHARPENED IMAGE SIMULATED USING HYPERSPECTRAL DATA

Abstract. Preservation of the spectral characteristics in multispectral images is important in the development of pansharpening methods because it affects the accuracy of subsequent applications, such as visual interpretation, land cover classification, and change detection. The combinations of the spectral properties (observation wavelength and width of spectral bands) of multispectral and panchromatic images affect both the spatial and spectral quality of pansharpened images. Therefore, the clarification of the relations between spectral bands and quality of pansharpened image is important for improving our understanding of pansharpening methods, and for developing better schemes for image fusion. This study investigated the influence of the spectral waveband of panchromatic images on the image quality of multispectral (MS) images using simulated images produced from hyperspectral data. Panchromatic images with different spectral band position and multispectral images with degraded spatial resolution were generated from airborne visible/infrared imaging spectrometer (AVIRIS) images and pansharpened using seven methods: additive wavelet intensity, additive wavelet principal component, generalized Laplacian pyramid with spectral distortion minimization, generalized intensity-huesaturation (GIHS) transform, GIHS adaptive, Gram–Schmidt spectral sharpening, and block-based synthetic variable ratio. The pansharpened near-infrared band was visually and statistically compared with the non-degraded image. Wide variation in quality was identified visually within and between methods depending on the spectral wavelengths of the panchromatic images. Quantitative evaluations using three frequently used indices, the correlation coefficient, erreur relative globale adimensionnelle de synthèse (ERGAS), and the Q index, showed the individual behaviors of the pansharpening methods in terms of the spectral similarity in panchromatic and near-infrared, though all methods had similar qualities in the case with the lowest similarity. These findings are discussed in terms of the fundamentals and structures of the methods.

COMPARISON OF THE SPECTRAL PROPERTIES OF PANSHARPENED IMAGES GENERATED FROM AVNIR-2 AND PRISM ONBOARD ALOS

Abstract. A considerable number of methods for pansharpening remote-sensing images have been developed to generate higher spatial resolution multispectral images by the fusion of lower resolution multispectral images and higher resolution panchromatic images. Because pansharpening alters the spectral properties of multispectral images, method selection is one of the key factors influencing the accuracy of subsequent analyses such as land-cover classification or change detection. In this study, seven pixel-based pansharpening methods (additive wavelet intensity, additive wavelet principal component, generalized Laplacian pyramid with spectral distortion minimization, generalized intensity-hue-saturation (GIHS) transform, GIHS adaptive, Gram-Schmidt spectral sharpening, and block-based synthetic variable ratio) were compared using AVNIR-2 and PRISM onboard ALOS from the viewpoint of the preservation of spectral properties of AVNIR-2. A visual comparison was made between pansharpened images generated from spatially degraded AVNIR-2 and original images over urban, agricultural, and forest areas. The similarity of the images was evaluated in terms of the image contrast, the color distinction, and the brightness of the ground objects. In the quantitative assessment, three kinds of statistical indices, correlation coefficient, ERGAS, and Q index, were calculated by band and land-cover type. These scores were relatively superior in bands 2 and 3 compared with the other two bands, especially over urban and agricultural areas. Band 4 showed a strong dependency on the land-cover type. This was attributable to the differences in the observing spectral wavelengths of the sensors and local scene variances.

An Improved Pca Fusion Method Based on Generalized Intensity–Hue–Saturation Fusion Technique

Abstract Among various image fusion methods, principal component analysis (PCA) technique is capable of quickly merging the massive volumes of data. For IKONOS imagery, PCA can yield satisfactory “spatial” enhancement but may introduce spectral distortion, appearing as a change in colors between compositions of resembled and fused multi-spectral bands. To solve this problem, a fast improved PCA fusion method based on Intensity–Hue–Saturation Fusion Technique with Spectral Adjustment is presented. The experimental results demonstrate that the proposed approach can provide better performance than the original PCA method both in processing speed and image quality.

Multiscaled combination of MR and SPECT images in neuroimaging: A simplex method based variable-weight fusion

Single-photon emission computed tomography (SPECT) images alone are difficult to understand in diagnosis, since anatomical structures are absent from the data. Studies on combination attempt to locate functional changes of the SPECT image by the magnetic resonance (MR) image. Due to the low similarity between original images, fused results are always darkened, obscured or loss some crucial anatomical structures. This paper has solved these problems by the variable-weight matrix which is estimated by minimizing the cost function using the simplex method. Under the generalized intensity-hue-saturation (GIHS) framework, the multiscaled analysis is presented for a better detail preservation. Besides, interactive approaches are discussed for the gradual variation between original images and the control of detail performance. The similarity assessment evaluates several different methods on a normal brain atlas. Two clips show the interactive property of the proposed method, while two medical cases demonstrate its clinical values. We conclude that the proposed method is superior to traditional methods, when considering the definition and the information capacity of fused results.

Biological image fusion using a NSCT based variable-weight method

Biological image fusion, as a powerful tool for the protein study, has developed with the advent of various imaging modalities in molecular biology. Combining the fluorescent image and its corresponding phase contrast image will benefit the localization of the protein. However, resulting images of traditional methods are always difficult to compromise between multimodalities. This paper has solved this problem by a variable-weight fusion rule based on the nonsubsampled contourlet transform (NSCT). The intensity components of original images are combined in the multiscaled space and the fused image is obtained in the generalized intensity-hue-saturation (GIHS) frame. Validation experiments on 117 sets of Arabidopsis images are for two purposes: the comparison among different fusion rules and the impact of the multiscaled analysis in biological image fusion. Region-based quantified indexes reveal the similarity between fused images and original ones, and therefore demonstrate the superiority of the proposed method over traditional methods.