Spectral Fusion Research Articles

Spatial–Spectral Fusion Based on Conditional Random Fields for the Fine Classification of Crops in UAV-Borne Hyperspectral Remote Sensing Imagery

The fine classification of crops is critical for food security and agricultural management. There are many different species of crops, some of which have similar spectral curves. As a result, the precise classification of crops is a difficult task. Although the classification methods that incorporate spatial information can reduce the noise and improve the classification accuracy, to a certain extent, the problem is far from solved. Therefore, in this paper, the method of spatial–spectral fusion based on conditional random fields (SSF-CRF) for the fine classification of crops in UAV-borne hyperspectral remote sensing imagery is presented. The proposed method designs suitable potential functions in a pairwise conditional random field model, fusing the spectral and spatial features to reduce the spectral variation within the homogenous regions and accurately identify the crops. The experiments on hyperspectral datasets of the cities of Hanchuan and Honghu in China showed that, compared with the traditional methods, the proposed classification method can effectively improve the classification accuracy, protect the edges and shapes of the features, and relieve excessive smoothing, while retaining detailed information. This method has important significance for the fine classification of crops in hyperspectral remote sensing imagery.

Dual-Wavelength Raman Fusion Spectroscopy.

Spatially compressed dual-wavelength Raman spectroscopy allows recording the full Raman spectrum using a detection system with limited spectral range. The common approach is to record the spectra with the two excitation lasers consecutively and then concatenate the full spectrum. However, with this approach, quantitative analysis for process monitoring is not possible as the investigated object may change between the two acquisitions. In this Note, spectral fusion is proposed as a concept to overcome this problem. The sample is illuminated by the two lasers simultaneously, hence leading to an on-chip fusion of the different parts of the Raman spectrum. It is shown that the resulting data are suitable for quantitative evaluation using univariate and multivariate methods. Dual-wavelength Raman fusion spectroscopy offers new opportunities for building highly compact devices for analytical chemistry.

Hyperspectral Image Classification Based on Belief Propagation with Multi-features and Small Sample Learning

In order to solve the “massive information but low accuracy” problem of hyperspectral image (HSI) classification, a novel HSI classification method MFSSL-BPMRF based on belief propagation (BP) Markov random field (MRF) using multi-features and small sample learning (MFSSL) is proposed in this paper. Firstly, an extended morphological multi-attributes profiles algorithm is used to extract spatial information of HSI, and a spatial–spectral multi-features fusion model is established to improve classification results. Then, BPMRF is used for image segmentation and classification because of its superiority in the spatial–spectral combination classification. MRF can describe the spatial distribution features of ground objects based on neighborhood model, and the spectral information of pixels can be integrated into the calculation of conditional probability. BP is used to learn the marginal probability distributions from the multi-features fusion information. Finally, the small sample training set is selected to enhance the computational efficiency. In the experiments of several hyperspectral images, the proposed method provides higher classification accuracy than other methods, and it is efficient for the classification with limited labeled training samples.

A semi-empirical approach for the estimation of land-surface emissivity from satellite data based on spectral index fusion using ensemble regression

ABSTRACTLand Surface Emissivity (LSE) is a key parameter in the thermal remote sensing, with several important applications, most notably in Land Surface Temperature (LST) estimation. This paper presents a semi-empirical method of LSE estimation from remote sensing data based on a fusion of spectral indices using the ensemble regression methods. The performance of the proposed method for Moderate Resolution Imaging Spectroradiometer (MODIS), Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER), Operational Land Imager (OLI) and Thermal Infrared Sensor (TIRS) data was evaluated and compared with other semi-empirical methods developed for these sensors. The proposed method was designed in four stages. In the first stage, the reflectance of non-thermal bands and emissivity of thermal bands were simulated for different classes using the ASTER spectral library and the spectral response function of each sensor. In the second stage, the dataset to be used for the training of ensemble regression was arranged by calculating a number of spectral indices, which constitute the feature space along with non-thermal bands. In the third stage, the regression between emissivity of thermal bands of each sensor and the features extracted in the second stage was derived by the use of bagging, boosting and Random Forest (RF) regression methods. In the final stage Using Normalized Difference Vegetation Index (NDVI) values, the image was categorized into three classes including vegetation, non-vegetation and mixture areas using conditions NDVI > 0.5, NDVI < 0.2 and 0.2 ≤ NDVI ≤ 0.5, respectively. The non-vegetation class was then categorized to soil, rock, and man-made classes using land use map. The spectral indices of these classes were then calculated, and the corresponding model trained in the third stage was used to estimate the LSE for that band. The results of LSE estimations were compared with the standard product of each sensor. Due to the lack of standard product for Landsat-8, the ASTER product was used as a substitute. For better analysis, the proposed method was also evaluated with other semi-empirical methods developed for MODIS, ASTER and OLI/TIRS sensors. This evaluation showed that the lowest Root Mean Square Error (RMSE) values for OLI/TIRS bands 10 and 11 are 0.0070 and 0.0075 obtained, respectively, by bagging and RF regression methods. For ASTER bands 13 and 14, the lowest RMSE values of 0.0078 and 0.0077 are both obtained by RF regression. For MODIS bands 31 and 32, the lowest RMSE values are 0.0053 and 0.0049 and obtained by boosting method. A comparison between the proposed method and other semi-empirical methods provided for these sensors demonstrated the ability of the method to improve the RMSE by up to 0.5%. Regarding the higher accuracy and applicability of the proposed method, it can serve as an effective and efficient means of estimating LSE using remote sensing data.

UAV Traffic Patrolling via Road Detection and Tracking in Anonymous Aerial Video Frames

Unmanned Aerial Vehicles (UAV) have gained great importance for patrolling, exploration, and surveillance. In this study, we have estimated a route UAV to follow, using aerial road images. In the experimental setup, for estimation, test, and validation stages, anonymous aerial road videos have been exploited, meaning a special image database was not produced for this simulation approach. In the proposed study, road portion is initially detected. Two methods are utilized to help road detection, which are k-Nearest Neighbor and Hough transformation. To form a decision loop, both results are matched. If they match each other, they are fused using spatial and spectral schemes for the comparison purpose. Once road area is detected, the road type classification is realized by Fuzzy approach. The resultant image is utilized to estimate route, over which the UAV have to fly towards that direction. In the simulation stage, an anonymous video stream previously captured by UAV is experimented to assess the performance of the underlying system for different roads. According to the implementation results, the proposed algorithm has succeeded in finding all the trial roads in the given aerial images, and the proportion of all the estimated road-portion to actual road pixels for all the images is averagely calculated as %95.40. Eventually, it is shown that UAV has followed the correct route, which is estimated by proposed approach, over the specified road using assigned video frames, and also performances of spatial and spectral fusion results are compared.

Integration of Satellite Data with High Resolution Ratio: Improvement of Spectral Quality with Preserving Spatial Details.

Commonly used image fusion techniques generally produce good results for images obtained from the same sensor, with a standard ratio of spatial resolution (1:4). However, an atypical high ratio of resolution reduces the effectiveness of fusion methods resulting in a decrease in the spectral or spatial quality of the sharpened image. An important issue is the development of a method that allows for maintaining simultaneous high spatial and spectral quality. The authors propose to strengthen the pan-sharpening methods through prior modification of the panchromatic image. Local statistics of the differences between the original panchromatic image and the intensity of the multispectral image are used to detect spatial details. The Euler’s number and the distance of each pixel from the nearest pixel classified as a spatial detail determine the weight of the information collected from each integrated image. The research was carried out for several pan-sharpening methods and for data sets with different levels of spectral matching. The proposed solution allows for a greater improvement in the quality of spectral fusion, while being able to identify the same spatial details for most pan-sharpening methods and is mainly dedicated to Intensity-Hue-Saturation based methods for which the following improvements in spectral quality were achieved: about 30% for the urbanized area and about 15% for the non-urbanized area.

Identification of Edible-vegetable-oil Types Based on Multi-kernel Learning and Multi-spectral Fusion

Aiming at the identification of edible oil quality, this study proposed a multi-spectral data fusion combined with multi-kernel learning support vector machine (SVM) method. This method used serial and wavelet fusion approaches to fuse Raman and near infrared spectral data, and established an identification model for edible-oil types, with the aid of the multi-kernel learning support vector machine (MKL-SVM). The performances of the single spectral model and spectral fusion model were compared, demonstrating that the spectral fusion could effectively improve the prediction accuracy and generalization ability of the model.

Pan-GGF: A probabilistic method for pan-sharpening with gradient domain guided image filtering

Abstract In this paper, we develop a novel probabilistic pan-sharpening method with gradient domain guided image filtering (named Pan-GGF). We employ gradient domain guided image filtering (GGIF) to enforce effective spatial fusion of panchromatic and multispectral images, and the proposed scheme shows better spatial and spectral fusion than other fusion methods, such as projection substitution, detail injection and weighted combination models. A maximum a posterior (MAP) formulation is then presented with imposing l1 norm priors on the errors between panchromatic and multispectral images in both image and gradient domains, and the proposed objective function is addressed through an efficient optimization scheme that alternates between auxiliary variables approximation and high resolution multispectral images reconstruction. Numerous experiments are performed to demonstrate the satisfactory performance of the proposed method in spatial and spectral fusion, and the proposed method outperforms several classical and state-of-the-art pan-sharpening methods in both subjective results and objective assessments.

Spectral preservation fusion for remote sensing images using focus measure operators based on fast discrete curvelet transform and hyperspherical color space

How to preserve the spectral information when enhancing the spatial details is a key issue of remote sensing image fusion. The component substitution (CS)-based fusion methods can effectively enhance the spatial details while suffering spectral distortion, and multiresolution analysis (MRA)-based methods have advantages in preserving spectral information but are not satisfactory in terms of spatial details. This paper proposes a hybrid method to integrate the advantages of CS- and MRA-based approaches. The intensity image is first obtained from an original multispectral (MS) image by hyperspherical color space (HCS) transform; then, the intensity image and original panchromatic (PAN) image are decomposed by fast discrete curvelet transform (FDCT). The focus measure operators are introduced to fused low-frequency, middle-high frequency, and fine scale subband coefficients in curvelet domain with specific fusion strategies. The final fusion image is achieved by inverse FDCT and inverse HCS transform. From the aspects of subjective and objective quality assessments, the experimental results on various types of remote sensing images including IKONOS, QuickBird, and WorldView-2 indicate that compared with existing well-known algorithms and commercial softwares, the proposed method shows obvious advantages in preserving spectral information and maintaining the spatial details.

Spectral fusion by Outer Product Analysis (OPA) to improve predictions of soil organic C

Soil organic carbon (C) is an important indicator of agricultural and environmental quality. It improves soil fertility and helps to mitigate greenhouse gas emissions. Soil spectroscopy with either vis–NIR (350–2500 nm) or mid-IR (4000–400 cm−1) spectra have been used successfully to predict organic C concentrations in soil. However, research to improve predictions of soil organic C by simply combining vis–NIR and mid-IR spectra to model them together has been unsuccessful. Here we use the Outer Product Analysis (OPA) to fuse vis–NIR and mid-IR spectra by bringing them into a common spectral domain. Using the fused data, we derived models to predict soil organic C and compared its predictions to those derived with vis–NIR and mid-IR models separately. We analyzed 1259 tropical soil samples from surface and subsurface layers across agricultural areas in Central Brazil. Soil organic C contents were determined by a modified Walkley-Black method, and vis–NIR and mid-IR reflectance spectra were obtained with a FieldSpec Pro and a Nicolet 6700 Fourier Transformed Infrared (FT-IR), respectively. Reflectances were log-transformed into absorbances. The mean content of soil organic C was 9.14 g kg−1 (SD = 5.64 g kg−1). The OPA algorithm was used to emphasize co-evolutions of each spectral domain into the same one by multiplying the absorbances from both sets of spectra to produce a matrix with all possible products between them. Support Vector Machine with linear kernel function was used for the spectroscopic modeling. Predictions of soil organic C using vis–NIR, mid-IR, and fused spectra were statistically compared by the Tukey's test using the coefficient of determination (R2), root mean squared error (RMSE), and ratio of performance to interquartile distance (RPIQ). Absorbances in vis–NIR and mid-IR were emphasized in the common spectral domain presenting stronger correlations with soil organic C than individual ranges. Soil organic C predictions with the OPA fused spectra were significantly better (R2 = 0.81, RMSE = 2.42 g kg−1, and RPIQ = 2.87) than those with vis–NIR (R2 = 0.69, RMSE = 3.38 g kg−1, and RPIQ = 2.08) or mid-IR spectra (R2 = 0.77, RMSE = 2.90 g kg−1, and RPIQ = 2.43). Fusing vis–NIR and mid-IR spectra by OPA improves predictions of soil organic C.

A Portable Personality Recognizer Based on Affective State Classification Using Spectral Fusion of Features

In this paper, we introduce a system named Portable Personality Recognizer (PPR), which classifies the personality of an individual using his/her transitions of affective states. This work attempts to reveal the latent relationship between emotions and personality of a person. Here, we train a hidden Markov model (HMM) with observable emotional states viz. Happiness (H), Anger (A), Surprise (S) and Disgust (D) and the hidden traits viz. Psychoticism (P), Extraversion (E) and Neuroticism (N). Based on the model, the system estimates the personality as Psychotic, Extravert or Neurotic. It does so by capturing the facial images of an individual using a visible and a thermal camera to decide the present affective state of the person. The emotion classification is carried out using fused eigenfeatures from the visible and blood perfused thermal images. The emotional state changes are observed using the trained HMM to estimate the personality. The proposed hardware prototype consists of a Banana Pi board with a seven inch LCD screen having a thermal and a visible camera add-ons. The system achieves an emotion classification accuracy of 87.145 percent, while an accuracy of 87.87 percent is achieved for personality recognition.

Multi spectral image fusion by deep convolutional neural network and new spectral loss function

ABSTRACTIn this paper we propose a new multispectral image fusion architecture. The proposed method includes two steps related to two neural networks. First the extracted spatial information, from panchromatic (Pan) image, is injected to upsampled multi-spectral (MS) image. In this step, the method employed a deep convolution neural network (DCNN) to estimate the spatial information of the MS image, according to multi-resolution analysis (MRA) scheme. This DCNN is trained by the low-spatial resolution version of Pan as an input, and by the spatial information as the target. This trained DCNN is called ‘Fusion network (FN)’. The FN, adaptively, estimates the spatial information of the MS images, and operates as an injection gain in the MRA scheme. In the second step, the spectral compensation is performed on the fused MS image. For this purpose, we used a novel loss function for this DCNN, to reduce the spectral distortion in the fused images, and simultaneously maintain the spatial information. This network is called ‘Spectral compensation network (SCN)’. Finally, the proposed method is compared to the several state-of-the-art methods on three datasets, using both full-reference and reduced reference criterion. The experimental results show that the proposed method can achieve competitive performance in both spatial and spectral information.

Feasibility Assessment of Synchronous Fluorescence Spectral Fusion by Application to Argan Oil for Adulteration Analysis.

Synchronous fluorescence spectroscopy (SFS) is used for quantitative analysis as well as for qualitative analysis, such as with classification methods. With SFS, determination of a useful wavelength interval between the excitation and emission wavelengths (Δλ) is required. There are a multitude of Δλ intervals that can be evaluated and optimization of the best one is complex. Presented here is a fusion approach for combining Δλ intervals, thereby negating the need to perform the selection by a skilled operator. To demonstrate the feasibility of omitting selection of the best Δλ interval, adulterated argan oil samples are studied. Argan oil is made from the argan tree, endemic to southwestern Morocco, and is well-known for its cosmetic, pharmaceutical, and nutritional applications. It is considered a luxury product and exported from Morocco around the world. Consequently, detection of argan oil adulteration followed by quantitative analysis of the adulterant concentration is important. This study uses fusion of SFS spectra obtained at ten Δλ intervals to first detect adulteration of argan oil by corn oil and then determination of the corn oil content. For detection of adulteration, 15 one-class classification methods were used simultaneously over the ten Δλ sets of SFS spectra. For tuning parameter dependent classifiers such as Mahalanobis distance, non-optimized classifiers are used. Raw classification values are used, removing the need to set classifier-dependent threshold values, albeit, ultimately, a fusion decision rule is needed for classification. For quantitative analysis, two calibration approaches are evaluated with fusion of these ten Δλ SFS spectral data sets. One is multivariate calibration by partial least squares (PLS). The second approach is a univariate calibration process where the SFS spectra are summed over respective SFS spectral ranges, also known as the area under the curve (AUC). For adulteration detection and quantitation of the corn oil, prediction errors decrease with fusion compared to individually using the ten Δλ interval SFS specific data sets. For this argan oil data set, the AUC method generally provides equivalent prediction errors to PLS.

Geographic Characterization of Leccinum rugosiceps by Ultraviolet and Infrared Spectral Fusion

ABSTRACTLeccinum rugosiceps is an edible mushroom belonging to genus Leccinum of Boletaceae. Its fruiting bodies are richer in nutrients than many vegetables and fruit. The model of support vector machine was established for the discrimination of L. rugosiceps from regions based on rapid and low-cost ultraviolet and infrared spectroscopies. The mid-level data fusion was performed by support vector machine. Compared to a single spectroscopic technique, mid-level data fusion provided higher accuracy by selecting the most significant variance from data matrixes based on partial least squares discriminant analysis. The accuracy of the classification of samples in the calibration and test sets were 85.00 and 94.74%, higher than separate measurements by ultraviolet or infrared spectroscopy. This approach has applications for authentication and quality assessment of L. rugosiceps.

表面疵病动态彩色编码融合成像检测技术

表面疵病动态彩色编码融合成像检测技术

An Integrated Framework for the Spatio–Temporal–Spectral Fusion of Remote Sensing Images

Remote sensing satellite sensors feature a tradeoff between the spatial, temporal, and spectral resolutions. In this paper, we propose an integrated framework for the spatio–temporal–spectral fusion of remote sensing images. There are two main advantages of the proposed integrated fusion framework: it can accomplish different kinds of fusion tasks, such as multiview spatial fusion, spatio–spectral fusion, and spatio–temporal fusion, based on a single unified model, and it can achieve the integrated fusion of multisource observations to obtain high spatio–temporal–spectral resolution images, without limitations on the number of remote sensing sensors. The proposed integrated fusion framework was comprehensively tested and verified in a variety of image fusion experiments. In the experiments, a number of different remote sensing satellites were utilized, including IKONOS, the Enhanced Thematic Mapper Plus (ETM+), the Moderate Resolution Imaging Spectroradiometer (MODIS), the Hyperspectral Digital Imagery Collection Experiment (HYDICE), and Systeme Pour l' Observation de la Terre-5 (SPOT-5). The experimental results confirm the effectiveness of the proposed method.

Optical multiple-image encryption in diffractive-imaging-based scheme using spectral fusion and nonlinear operation.

We report a new method for multiple-image encryption in diffractive-imaging-based encryption (DIBE) scheme. The discrete cosine transformation (DCT) spectra of the primary images are extracted, compacted and then nonlinear-transformed before being sent to the DIBE, where they are encoded into a single intensity pattern. With the help of a suggested phase retrieval algorithm, the original images can be recovered with high quality. Furthermore, due to the introduction of the nonlinear operation, the proposal is demonstrated to be robust to the currently available cryptographic attacks. The proposal probes a new way for multiple-image encryption in DIBE, and its effectiveness and feasibility have been supported by numerical simulations.

Two Ears Are Not Always Better than One: Mandatory Vowel Fusion Across Spectrally Mismatched Ears in Hearing-Impaired Listeners.

Hearing loss and auditory prostheses can alter auditory processing by inducing large pitch mismatches and broad pitch fusion between the two ears. Similar to integration of incongruent inputs in other sensory modalities, the mismatched, fused pitches are often averaged across ears for simple stimuli. Here, we measured parallel effects on complex stimulus integration using a new technique based on vowel classification in five bilateral hearing aid users and eight bimodal cochlear implant users. Continua between five pairs of synthetic vowels were created by varying the first formant spectral peak while keeping the second formant constant. Comparison of binaural and monaural vowel classification functions for each vowel pair continuum enabled visualization of the following frequency-dependent integration trends: (1) similar monaural and binaural functions, (2) ear dominance, (3) binaural averaging, and (4) binaural interference. Hearing aid users showed all trends, while bimodal cochlear implant users showed mostly ear dominance or interference. Interaural pitch mismatches, frequency ranges of binaural pitch fusion, and the relative weightings of pitch averaging across ears were also measured using tone and/or electrode stimulation. The presence of both large interaural pitch mismatches and broad pitch fusion was not sufficient to predict vowel integration trends such as binaural averaging or interference. The way that pitch averaging was weighted between ears also appears to be important for determining binaural vowel integration trends. Abnormally broad spectral fusion and the associated phoneme fusion across mismatched ears may underlie binaural speech perception interference observed in hearing aid and cochlear implant users.

高光谱干涉图像动态追踪补偿方法研究

Combining the compensation corrections based on target dynamic tracking with the spectra extraction based on non;uniform fast Fourier transform, a method was proposed to realize the high precision correction of airborne remote sensing. Through the way of real;time attitude measuring, active correction, feedback compensation from position and orientation system under poor imaging condition of non;stable platform, the interferogram was obtained. The fused image and the spectral curves were obtained from the corrected interferogram by the spatial transformation and data extraction of non;uniformity. The results of airborne flight experiment show that the multi;spectral fusion image has a good quality, and the precision of spectrum recovery has been improved greatly compared with the traditional methods. The attitude measuing system based on the proposed method has a good environmental adaptability of airborne, which can be used not only for the poor stability platform but also for the other spectral imaging detection system besides the principle of interference spectroscopy imaging. The proposed method provides a way for the movement error correction of airborne integration hyperspectral image processing platform. © 2016, Science Press. All right reserved.

High Spatial and Spectral Details Retention Fusion and Evaluation

To retain spatial and spectral details simultaneously from source images is a trade-off in image sharpening. Fourier and wavelet transform based image fusion methods retain better spectral quality but represent less spatial details as in source images. Wavelets and Laplacian pyramid perform well only at linear discontinuities because they do not consider the geometric properties of structures and do not exploit the regularity of edges. In this paper we proposed a novel image fusion method to preserve high spatial and spectral details based on the curvelet transforms. Curvelet transforms overcome the difficulty to identify the critical transient features. To retain high spatial details, finer and detailed scale coefficients of High Resolution (HR) PAN image are substituted in Low Resolution (LR) multispectral (MS) bands in the Fast Discrete Curvelet Transforms (FDCT) domain. Spectral details in the fused image are preserved by following the shape of the spectral reflectance curve of each pixel in the resample MS image. Spectral profile of the each pixel in spatially fused image is parallel to the spectral profile of the corresponding pixel in the resampled MS image. For experimental study of this method, Indian Remote Sensing (IRS) Resourcesat-1 LISS IV images are used as LR MS image and Cartosat-1 images are used as HR PAN Image. Proposed fusion method is evaluated against state of the art fusion techniques and quality measures.