Spectral Data Reduction Research Articles

Applications of Raman Spectroscopy in Mineralogy and Geochemistry

The application of Raman spectroscopy for the identification and characterization of minerals and related materials has increased appreciably during recent years. Raman spectroscopy has proven to be a most valuable and versatile analytical tool. Successful applications cover virtually all the mineralogical sub-disciplines, and have become more numerous in geochemistry. We present a general summary of present applications, illustrated by selected examples. In addition, we briefly point out several aspects of spectral acquisition, data reduction, and interpretation of Raman results that are important for the application of Raman spectroscopy as a reliable analytical tool.

A comparative study on dimensionality reduction of dielectric spectral data for the classification of basal stem rot (BSR) disease in oil palm

Basal stem rot (BSR) disease in oil palm is caused by Ganoderma boninense fungus. This plant disease is deemed highly destructive and would cause substantial economic loss. The use of spectroscopy technique with the capacity to deal with a large amount of spectral data has gained growing attention as a robust method, particularly to identify the symptoms of plant disease in its initial stage. The dimensionality reduction is pivotal in the use of spectroscopy technique due to its improved prediction performance and optimum processing. Considering that, this study assessed the feasibility of utilising dielectric spectral properties to classify the severity levels of BSR disease in oil palm across a frequency range of 100 kHz–30 MHz. The support vector machine-feature selection (SVM-FS) and principal component analysis (PCA) were applied as data reduction methods. After selecting the optimum number of significant frequencies, this study proceeded to assess the effectiveness of the support vector machine (SVM) and quadratic discriminant analysis (QDA) classifiers in identifying the four different levels of BSR disease. The performance of both classifiers with and without data reduction methods was subsequently compared in terms of the classification accuracy, while the whole spectrum data served as part of the control method. The resultant outcomes revealed that the use of QDA classifier with PCA recorded the highest classification accuracy (up to 96.36%). As for the case of without using data reduction methods, the SVM classifier recorded the highest classification accuracy at only 79.55%. Conclusively, this study proved the significance of dimensionality reduction of dielectric spectral data for the classification of BSR disease in oil palm.

Comparative analysis of mineral mapping for hyperspectral and multispectral imagery

The traditional approaches of mineral-mapping are time consuming and expensive process. Remote sensing is a tool to map the minerals precisely using their physical, chemical and optical properties. In the present study, Tirunelveli district in Tamil Nadu is selected to extract the abundant mineral such as Limestone using Hyperion and Landsat-8 OLI imageries. The chemical composition of the mineral is identified using scanning electron microscope (SEM) and energy dispersive X-ray spectroscopy (EDS) analysis. The spectral reflectance of minerals is characterized using analytical spectral device (ASD) field spectroradiometer. The minerals showed deep absorption in short wave infrared region from 1800 to 2500 nm. The mineral mapping in hyperspectral data is performed using various preliminary processing such as bad band removal, vertical strip removal, radiance and reflectance generation and postprocessing steps such as data dimensional reduction, endmember extraction and classification. To improve the classification accuracy, the vertical strip removal process is performed using a local destriping algorithm. Absolute reflectance of Hyperion and Landsat-8 OLI (Operational Land Imager) imageries is carried out using the FLAASH (fast line-of-sight atmospheric analysis of hypercubes) module. Spectral data reduction techniques in reflectance bands performed using minimum noise fraction method. The noiseless reflectance bands spatial data reduced by the Pixel Purity Index method in the threshold limit of 2.5 under 10,000 repetitions. The obtained reflectance imagery spectra compared with the spectral libraries such as USGS (United States Geological Survey), JPL (Jet Propulsion Laboratory) and field spectra. Endmembers of minerals are carried out using high probability score obtained from the various methods such as SAM (spectral angle mapper), SFF (spectral feature fitting) and BE (binary encoding). The mineral mapping of both imageries is carried out using a supervised classification approach. The results showed that hyperspectral remote sensing performed good results as compared to multispectral data.

Feasibility study on prediction of gasoline octane number using NIR spectroscopy combined with manifold learning and neural network.

Octane number is an anti-knock index of fuel gasoline, which has an important impact on the service life of engine components and the safety of vehicles. Therefore, it is a basic work involving safety to predict the gasoline octane number accurately. This work was aimed to predict the octane number of near infrared (NIR) spectroscopy by combining dimension reduction algorithm with neural network. Covariance matrix estimation (CME), known as a mathematical statistic tool, was applied to estimating the intrinsic dimensions of octane spectrum dataset. Landmark-Isometric feature mapping (L-Isomap), as a novel manifold learning algorithm, was used for dimensionality reduction of spectral data. A new method, beetle antennae search optimization BP neural network (BAS-BP), was proposed to realize the prediction of octane number. In order to verify the performance of CME-L-Isomap-BAS-BP model presented in this paper, it is compared with other models. The results showed that when CME-L-Isomap was combined with BAS-BP, the average recovery rate (AR), mean square error (MSE), mean absolute percentage error (MAPE), correlation coefficient (R) and running time were superior than other models. The satisfying results demonstrated that the CME-L-Isomap-BAS-BP model is more suitable for prediction of gasoline octane number.

Enhancement of limestone mineral identification using Hyperion imagery: a case study from Tirunelveli District, Tamil Nadu, South India

Hyperspectral remote sensing consolidates imaging and spectroscopy in a solitary system which frequently comprises big datasets and necessitates the novel processing methods. In the present study, Cheranmadevi Block of Tirunelveli District in Tamil Nadu is selected to extract the abundant limestone mineral. Hyperion is one of the freely available hyperspectral imagery containing 242 spectral bands with 10-nm intervals in the wavelength between 400 and 2500 nm. The main objectives of the present research work are to enhance the imagery visualization, end member extraction, and classification, and estimate the abundant limestone quantity by removing the striping error in Hyperion imagery. The scanning electron microscope with energy-dispersive X-ray spectroscopy analysis is performed to identify the chemical composition of limestone mineral. The spectral reflectance of limestone is characterized using analytical spectral devices like a field spectroradiometer. Limestone has deep absorption in the short-wave infrared region (1900–2500 nm) around 2320–2340 nm due to their calcite composition (CaCO3). The feature extraction in Hyperion data is performed using various preprocessing steps like bad bands removal, vertical strip removal, and radiance and reflectance creation. To improve the classification accuracy, vertical strip removal process is performed using a local destriping algorithm. The absolute reflectance is achieved by the atmospheric correction module using Fast Line-of-sight Atmospheric Analysis of Hypercubes. The acquired reflectance image spectra are compared with the spectral libraries of USGS, JPL, and field spectra. Destriping enhances qualities of Hyperion data interims of the spectral profile, radiance, reflectance, and data reduction methods. The present research work focused on the local destriping algorithm to increase the quality and quantity of limestone deposit extraction.

Surface-Enhanced Raman Scattering (SERS) With Silver Nano Substrates Synthesized by Microwave for Rapid Detection of Foodborne Pathogens.

Rapid and sensitive methods have been developed to detect foodborne pathogens, a development that is important for food safety. The aim of this study is to explore Surface-enhanced Raman scattering (SERS) with silver nano substrates to detect and identify the following three foodborne pathogens: Escherichia coli O157: H7, Staphylococcus aureus and Salmonella. All the cells were resuspended with 10 mL silver colloidal nanoparticles, making a concentration of 107 CFU/mL, and were then exposed to 785 nm laser excitation. In this study, the results showed that all the bacteria can be sensitively and reproducibly detected directly by SERS. The distinctive differences can be observed in the SERS spectral data of the three food-borne pathogens, and the silver colloidal nanoparticles can be used as highly sensitive SERS-active substrates. In addition, the assay time required only a few minutes, which indicated that SERS coupled with the silver colloidal nanoparticles is a promising method for the detection and characterization of food-borne pathogens. At the same time, principle component analysis (PCA) and hierarchical cluster analysis (HCA) made the different bacterial strains clearly differentiated based on the barcode spectral data reduction. Therefore, the SERS methods hold great promise for the detection and identification of food-borne pathogens and even for applications in food safety.

Effectiveness of spectral data reduction in detection of salt-affected soils in a small study area

Data reduction is used to aggregate or amalgamate the large data sets into smaller and manageable information pieces in order to fast and accurate classification of different attributes. However, excessive spatial or spectral data reduction may result in losing or masking important radiometric information. Therefore, we conducted this research to evaluate the effectiveness of the different spectral data reduction algorithms including Principle Component Analysis (PCA) and Minimum Noise Fraction (MNF) transformation, Pixel Purity Index (PPI), and n Dimensional Visualizer (n-DV) algorithms on accuracy of the supervised classification of the salt-affected soils applying ETM+ data beside 188 ground control points. Results revealed that data reduction caused around 20 to 30 % decreases in classification results compared to none reduced data. It seems that applying spectral data reduction algorithm in small study areas is not only supportive, but also has negative effects on classification results. Therefore, it may better to not to use the algorithms in small areas.

Comparison of models and visualization of total volatile basic nitrogen content in mutton using hyperspectral imaging and variable selection methods

ABSTRACTHyperspectral imaging combined with variable selection methods was used to perform the rapid and accurate detection and visualization of total volatile basic nitrogen content in mutton. For each sample, several spectra were extracted from the region of muscle pixels for modeling, and the model performance was improved and better than the model established with average spectrum extracting from each sample. By two steps of variable screening with competitive adaptive reweighted sampling and stepwise regression methods, the efficient dimensionality reduction of spectral data was achieved, and the important characteristic variables were selected. The nonlinear model significantly improved the model predictive capability, and the visualization distribution map based on the model was consistent with the actual change of meat spoilage.

Classification and Identification of Pesticide Residues in Cucumber Based on Near Infrared Spectroscopy

To explore the feasibility of Near Infrared spectroscopy (NIR) for rapid and nondestructive qualitative identification of pesticide residues in vegetables, this paper presents a qualitative classification and discrimination method based on Support Vector Machine (SVM) with Gaussian Radial Basis Function (RBF). The cucumber with three kinds of pesticide residue levels (no pesticide, standard pesticide residue, severe excess) was used as the research object. The total reflectance NIR spectra of two groups of cucumber were collected after spraying 24 hours and 48 hours of pesticide respectively. First, Principal Component Analysis (PCA) was applied to dimensionality reduction of spectral data. Then normalize the spectrum. Next, the SVM training model based on RBF was established, and then SVM prediction was carried out. The correct recognition rate of the training set was 100%, and the correct prediction rate of the test set was 94.4444% and 95.5556%, respectively. Experiments show that this method is accurate and fast for qualitative classification and identification of pesticide residues in cucumbers with different concentrations, and provides a new idea for rapid non-destructive detection of vegetable pesticide residues.

Nontronite mineral identification in nilgiri hills of tamil nadu using hyperspectral remote sensing

Hyperspectral Remote sensing is a tool to identify the minerals along with field investigation. Tamil Nadu has abundant minerals like 30% titanium, 52% molybdenum, 59% garnet, 69% dunite, 75% vermiculite and 81% lignite. To enhance the user and industry requirements, mineral extraction is required. To identify the minerals properly, sophisticated tools are required. Hyperspectral remote sensing provides continuous extraction of earth surface information in an accurate manner. Nontronite is an iron-rich mineral mainly available in Nilgiri hills, Tamil Nadu, India. Due to the large number of bands, hyperspectral data require various preprocessing steps such as bad bands removal, destriping, radiance conversion and atmospheric correction. The atmospheric correction is performed using FLAASH method. The spectral data reduction is carried out with minimum noise fraction (MNF) method. The spatial information is reduced using pixel purity index (PPI) with 10000 iterations. The selected end members are compared with spectral libraries like USGS, JPL, and JHU. In the Nontronite mineral gives the probability of 0.85. Finally the classification is accomplished using spectral angle mapper (SAM) method.

First Galactic Maser Observations on Ventspils Radio Telescopes – Instrumentation and Data Reduction

AbstractVentspils International Radio Astronomy Centre (VIRAC) has two fully steerable Cassegrean System 32 and 16 m radio telescopes. After renovation and modernization program the Galactic masers, particularly CH3OH research and monitoring program became one of the most important realized on these telescopes. Both telescopes are equipped with broadband cryogenic receivers covering 4.5-8.8 GHz frequency band. Digital backend consisting from DBBC-2 (Digital Base Band Convertor developed by HAT-LAB, Italy) and FLEXBUFF (data storage system based on commercially available server system) is used for data digitalization and registration. A special program complex for spectral line data reduction and correction was developed and implemented.

Development of Fatigue Loading Spectra from Flight Test Data

Aircraft structural fatigue life prediction has important military significance and the conspicuous economic value. In development phase fatigue life of flight vehicles is generally predicted based on some historic / assumed data which may differ significantly from the actual operational usage of the vehicle. Consequently, the fatigue life of an aircraft may reduce / increase. The service fatigue life of a structure depends on both the irregular amplitude load conditions and the assumptions during fatigue test. To find the actual fatigue life of a structure we need to subject the structure in laboratory with constant / variable amplitude loadings that should be representative of actual flight loads and at the same time in a form that can be applied conveniently in the laboratory. In case of uniform loading it is easy to find the number of cycles but the real service loads are normally irregular, so defining spectrum from flight data is quite difficult which corresponds to the actual random loads applying on the aircraft.In this research study, a simple methodology is proposed to extract fatigue loads spectrum from a typical flight tests data. It involves statistical analysis of flight data, data filtering, power spectral density (PSD), averaging and data reduction for generating a representative fatigue spectrum from experimental as well as actual flight data. The derived spectrum following this approach contains information of not only the number of occurrences of each load factors but also the fluctuating loads and corresponding exposure time for each occurrence. This methodology is validated by developing a spectrum from known raw data and then same procedure is applied on original flight data to generate the flight fatigue spectrum. Statistical processing of good sample size (number of flights) provides a reasonably good representative fatigue spectrum of that particular flight vehicle type. Every flight fatigue spectrum can also be used for structural health monitoring / residual life estimation on flight to flight basis.

1H NMR Metabolomics Study of Spleen from C57BL/6 Mice Exposed to Gamma Radiation.

Due to the potential risk of accidental exposure to gamma radiation, it’s critical to identify the biomarkers of radiation exposed creatures. In the present study, NMR based metabolomics combined with multivariate data analysis to evaluate the metabolites changed in the C57BL/6 mouse spleen after 4 days whole body exposure to 3.0 Gy and 7.8 Gy gamma radiations. Principal component analysis (PCA) and orthogonal projection to latent structures analysis (OPLS) are employed for classification and identification potential biomarkers associated with gamma irradiation. Two different strategies for NMR spectral data reduction (i.e., spectral binning and spectral deconvolution) are combined with normalize to constant sum and unit weight before multivariate data analysis, respectively. The combination of spectral deconvolution and normalization to unit weight is the best way for identifying discriminatory metabolites between the irradiation and control groups. Normalized to the constant sum may achieve some pseudo biomarkers. PCA and OPLS results shown that the exposed groups can be well separated from the control group. Leucine, 2-aminobutyrate, valine, lactate, arginine, glutathione, 2-oxoglutarate, creatine, tyrosine, phenylalanine, π-methylhistidine, taurine, myoinositol, glycerol and uracil are significantly elevated while ADP is decreased significantly. These significantly changed metabolites are associated with multiple metabolic pathways and may be potential biomarkers in the spleen exposed to gamma irradiation.

Importance of spatial and spectral data reduction in the detection of internal defects in food products.

Despite the importance of data reduction as part of the processing of reflection-based classifications, this study represents one of the first in which the effects of both spatial and spectral data reductions on classification accuracies are quantified. Furthermore, the effects of approaches to data reduction were quantified for two separate classification methods, linear discriminant analysis (LDA) and support vector machine (SVM). As the model dataset, reflection data were acquired using a hyperspectral camera in 230 spectral channels from 401 to 879 nm (spectral resolution of 2.1 nm) from field pea (Pisum sativum) samples with and without internal pea weevil (Bruchus pisorum) infestation. We deployed five levels of spatial data reduction (binning) and eight levels of spectral data reduction (40 datasets). Forward stepwise LDA was used to select and include only spectral channels contributing the most to the separation of pixels from non-infested and infested field peas. Classification accuracies obtained with LDA and SVM were based on the classification of independent validation datasets. Overall, SVMs had significantly higher classification accuracies than LDAs (P < 0.01). There was a negative association between pixel resolution and classification accuracy, while spectral binning equivalent to up to 98% data reduction had negligible effect on classification accuracies. This study supports the potential use of reflection-based technologies in the quality control of food products with internal defects, and it highlights that spatial and spectral data reductions can (1) improve classification accuracies, (2) vastly decrease computer constraints, and (3) reduce analytical concerns associated with classifications of large and high-dimensional datasets.

Comparative Analysis of Haar and Daubechies Wavelet for Hyper Spectral Image Classification

Abstract. With the number of channels in the hundreds instead of in the tens Hyper spectral imagery possesses much richer spectral information than multispectral imagery. The increased dimensionality of such Hyper spectral data provides a challenge to the current technique for analyzing data. Conventional classification methods may not be useful without dimension reduction pre-processing. So dimension reduction has become a significant part of Hyper spectral image processing. This paper presents a comparative analysis of the efficacy of Haar and Daubechies wavelets for dimensionality reduction in achieving image classification. Spectral data reduction using Wavelet Decomposition could be useful because it preserves the distinction among spectral signatures. Daubechies wavelets optimally capture the polynomial trends while Haar wavelet is discontinuous and resembles a step function. The performance of these wavelets are compared in terms of classification accuracy and time complexity. This paper shows that wavelet reduction has more separate classes and yields better or comparable classification accuracy. In the context of the dimensionality reduction algorithm, it is found that the performance of classification of Daubechies wavelets is better as compared to Haar wavelet while Daubechies takes more time compare to Haar wavelet. The experimental results demonstrate the classification system consistently provides over 84% classification accuracy.

Wavelet Based Identification of Substances in Terahertz Tomography Measurements

In comparison to the X-ray computed tomography Terahertz technique significantly enhances the amount of the information acquired during the sample measurement. Not only amplitude, but also phase, time and spectral characteristics can be determined in THz time-domain spectroscopy. Thus, Terahertz tomography allows localization and identification of substances within the objects due to the characteristic fingerprints in this frequency range. Certainly, an appropriate data processing and comparison algorithms are crucial for the accurate identification of the substances in the measured sample. Therefore, we present a new wavelet-based identification method which is suitable even for the substances with broad absorption curves and small or no absorption peaks. The performance of this algorithm was evaluated with the help of a tomographic sample filled with four substances, which were previously characterized for the external database. The continuous wavelet transform was applied to every data cell of the tomographic measurement and compared to the database. Received sinograms were reconstructed into images which depict estimated similarity between the measured and database substances. Furthermore, we suggest a method for the reduction of spectral data after the continuous wavelet transform. This method is based on the extraction of the distinctive features in the form of ridge lines.

Improvement of SAM Land Cover Classification of airborne hyperspectral dataUsing expert system

Hyperspectral imaging has many applications such as Land Use / Land Cover mapping, urban planning, mineral exploration, environmental monitoring, and military surveillance. This research is concerned with the investigation of the improvement of urban land cover mapping using airborne hyperspectral data. Geometric correction and atmospheric correction have been conducted. Quality of rectification has been assessed using DGPS check points. It was found that the total RMS of airborne hyperspectral data was 2.7372m. Minimum Noise Fraction (MNF) transformation and Pixel Purity Index (PPI) have been used for spectral and spatial data reduction. After that feature selection has been carried out. In this research a supervised classification method has been applied using the Spectral Angle Mapper,then, the discrimination of urban feature has been improved using expert system. An expert system classification was developed using the knowledge engineer of ERDAS Imagine 2010 for post classification refinement of initially classified output. The primary motivation behind using the expert system was to reclassify the initial Spectral Angle Mapper classification and reduce errors of omission and commission. Results revealed that the overall accuracy of SAM was 90% and kappa index was 0.89 whereas the expert system gives slightly higher overall accuracy 94.5 percent and a kappa index was 0.93.

RETRACTED: Neural network based identification of meat spoilage using Fourier-transform infrared spectra

To address the rapid and non-destructive detection of meat spoilage microorganisms during aerobic storage at chill and abuse temperatures, Fourier transform infrared spectroscopy (FTIR) with the help of an intelligent-based identification system was attempted in this work. The objective of this study is to associate simultaneously spectral data with microbiological data (log counts), for Total Viable Counts, Pseudomonas spp., Brochothrix thermosphacta, Lactic Acid Bacteria and Enterobacteriaceae. The dual purpose of the proposed modelling scheme is not only to classify meat samples in the respective quality class (i.e. fresh, semi-fresh and spoiled), but also to predict their associated microbiological population directly from FTIR spectra. An Extended Normalised Radial Basis Function neural network has been implemented, and the Bayesian Ying-Yang Expectation Maximisation algorithm has been utilised together with novel splitting operations to determine network’s size and parameter set. The dimensionality reduction of spectral data has been addressed by the implementation of a fuzzy principal component algorithm. Results confirmed the superiority of the adopted methodology compared to other schemes such as multilayer perceptron and the partial least squares techniques and indicated that spectral information obtained by FTIR spectroscopy during beef spoilage, in combination with an efficient choice of a learning-based modelling scheme could be considered as an alternative methodology for the accurate evaluation of meat spoilage.

Outlier modeling for spectral data reduction.

The spectra in spectral reflectance datasets tend to be quite correlated and therefore they can be represented more compactly using standard techniques such as principal components analysis (PCA) as part of a lossy compression strategy. However, the presence of outlier spectra can often increase the overall error of the reconstructed spectra. This paper introduces a new outlier modeling (OM) method that detects, clusters, and separately models outliers with their own set of basis vectors. Outliers are defined in terms of the robust Mahalanobis distance using the fast minimum covariance determinant algorithm as a robust estimator of the multivariate mean and covariance from which it is computed. After removing the outliers from the main dataset, the performance of PCA on the remaining data improves significantly; however, since outlier spectra are a part of the image, they cannot simply be ignored. The solution is to cluster the outliers into a small number of clusters and then model each cluster separately using its own cluster-specific PCA-derived bases. Tests show that OM leads to lower spectral reconstruction errors of reflectance spectra in terms of both normalized RMS and goodness of fit.

Optimizing Back-Propagation Gradient for Classification by an Artificial Neural Network

In a complex and changing a remote sensing system, which requires taking quick and informed decisions environment, connectionist methods have shown their great contribution in particular the reduction and classification of spectral data. In this context, this paper proposes to study the parameters that optimize the results of an artificial neural network ANN multilayer perceptron based, for classification of chemical agents on multi-spectral images. The mean squared error cost function remains one of the major parameters of the network convergence at its learning phase and a challenge that will face our approach to improve the gradient descent by the conjugate gradient method that seems fast and efficient.