Savitzky-Golay Algorithm Research Articles

Evaluating the utility of mid-infrared spectral subspaces for predicting soil properties

We propose four methods for finding local subspaces in large spectral libraries. The proposed four methods include (a) cosine angle spectral matching; (b) hit quality index spectral matching; (c) self-organizing maps and (d) archetypal analysis methods. Then evaluate prediction accuracies for global and subspaces calibration models. These methods were tested on a mid-infrared spectral library containing 1907 soil samples collected from 19 different countries under the Africa Soil Information Service project. Calibration models for pH, Mehlich-3 Ca, Mehlich-3 Al, total carbon and clay soil properties were developed for the whole library and for the subspace. Root mean square error of prediction was used to evaluate predictive performance of subspace and global models. The root mean square error of prediction was computed using a one-third-holdout validation set. Effect of pretreating spectra with different methods was tested for 1st and 2nd derivative Savitzky–Golay algorithm, multiplicative scatter correction, standard normal variate and standard normal variate followed by detrending methods. In summary, the results show that global models outperformed the subspace models. We, therefore, conclude that global models are more accurate than the local models except in few cases. For instance, sand and clay root mean square error values from local models from archetypal analysis method were 50% poorer than the global models except for subspace models obtained using multiplicative scatter corrected spectra with which were 12% better. However, the subspace approach provides novel methods for discovering data pattern that may exist in large spectral libraries.

Non-Destructive Sorting Techniques for Viable Pepper (Capsicum annuum L.) Seeds Using Fourier Transform Near-Infrared and Raman Spectroscopy

Purpose: This study examined the performance of two spectroscopy methods and multivariate classification methods to discriminate viable pepper seeds from their non-viable counterparts. Methods: A classification model for viable seeds was developed using partial least square discrimination analysis (PLS-DA) with Fourier transform near-infrared (FT-NIR) and Raman spectroscopic data in the range of (1400-2400 nm) and , respectively. The datasets were divided into 70% to calibration and 30% to validation. To reduce noise from the spectra and compare the classification results, preprocessing methods, such as mean, maximum, and range normalization, multivariate scattering correction, standard normal variate, and and derivatives with the Savitzky-Golay algorithm were used. Results: The classification accuracies for calibration using FT-NIR and Raman spectroscopy were both 99% with first derivative, whereas the validation accuracies were 90.5% with both multivariate scattering correction and standard normal variate, and 96.4% with the raw data (non-preprocessed data). Conclusions: These results indicate that FT-NIR and Raman spectroscopy are valuable tools for a feasible classification and evaluation of viable pepper seeds by providing useful information based on PLS-DA and the threshold value.

P.3.024 Translational neuroimaging reveals differential effects of chronic ketamine on brain macrostructure in humans and mice

Drug crime is a prominent issue of concern from pole to pole. In order to seek higher profits, drug gangs often add diluents and adulterants to the drugs to disperse drug products Analysis of these additives would be greatly conducive to determine the origin of drug products for law enforcement departments. A method using attenuated total reflectance-Fourier transform infrared spectroscopy and chemometrics methods to classify the heroin hydrochloride, methamphetamine hydrochloride, ketamine hydrochloride and their five additives (caffeine, phenacetin, starch, glucose, and sucrose), was developed. The Baseline correction, multivariate scatter correction, standard normal variate and Savitzky-Golay algorithm were adopted to pre-process the spectral data. Several supervised pattern recognition methods including decision tree, Bayes discriminant analysis, and support vector machine were considered as algorithms of constructing classifiers. The results reveal that, repetitive and interfering data in original spectrum data could be eliminated by principal component analysis and factor analysis. F-measure, as a comprehensive evaluation index of precision rate and recall rate, was more objective than precision rate and recall rate to reflect the ability of model to distinguish samples. It should be used as one of the indicators to evaluate the model. The CHAID classification tree could be identified as priorities in the decision tree model, while the linear kernel could be considered as the optimal kernel in the support vector machine model. The classification ability of three hydrochloride mixtures based on Bayes discriminant analysis was better than that of another models. Bayes discriminant analysis model was the more useful and practical method for classifying the target drugs of abuse than that of decision trees and support vector machine. The designed approach represents a potentially simple, non-destructive, and rapid method of classifying hydrochloride mixtures.

Inter-laboratory verification of European pharmacopoeia monograph on derivative spectrophotometry method and its application for chitosan hydrochloride

Inter-laboratory verification of European pharmacopoeia (EP) monograph on derivative spectrophotometry (DS) method and its application for chitosan hydrochloride was carried out on two generation of instruments (earlier GBC Cintra 20 and current technology TS Evolution 300). Instruments operate with different versions of Savitzky–Golay algorithm and modes of generating digital derivative spectra. For resolution power parameter, defined as the amplitude ratio A/B in DS method EP monograph, comparable results were obtained only with algorithm’s parameters smoothing points (SP) 7 and the 2nd degree polynomial and those provided corresponding data with other two modes on TS Evolution 300 Medium digital indirect and Medium digital direct. Using quoted algorithm’s parameters, the differences in percentages between the amplitude ratio A/B averages, were within accepted criteria (±3%) for assay of drug product for method transfer. The deviation of 1.76% for the degree of deacetylation assessment of chitosan hydrochloride, determined on two instruments, (amplitude 1D202; the 2nd degree polynomial and SP 9 in Savitzky–Golay algorithm), was acceptable, since it was within allowed criteria (±2%) for assay deviation of drug substance, for method transfer in pharmaceutical analyses.

Improved quantitative analysis of spectra using a new method of obtaining derivative spectra based on a singular perturbation technique.

Spectroscopy is often applied when a rapid quantitative analysis is required, but one challenge is the translation of raw spectra into a final analysis. Derivative spectra are often used as a preliminary preprocessing step to resolve overlapping signals, enhance signal properties, and suppress unwanted spectral features that arise due to non-ideal instrument and sample properties. In this study, to improve quantitative analysis of near-infrared spectra, derivatives of noisy raw spectral data need to be estimated with high accuracy. A new spectral estimator based on singular perturbation technique, called the singular perturbation spectra estimator (SPSE), is presented, and the stability analysis of the estimator is given. Theoretical analysis and simulation experimental results confirm that the derivatives can be estimated with high accuracy using this estimator. Furthermore, the effectiveness of the estimator for processing noisy infrared spectra is evaluated using the analysis of beer spectra. The derivative spectra of the beer and the marzipan are used to build the calibration model using partial least squares (PLS) modeling. The results show that the PLS based on the new estimator can achieve better performance compared with the Savitzky-Golay algorithm and can serve as an alternative choice for quantitative analytical applications.

Real-time infrared gas detection based on an adaptive Savitzky–Golay algorithm

Based on the Savitzky–Golay filter, we have developed in the present study a simple but robust method for real-time processing of tunable diode laser absorption spectroscopy (TDLAS) signals. Our method was developed to resolve the blindness of selecting the input filter parameters and to mitigate potential signal distortion induced in digital signal processing. Application of the developed adaptive Savitzky–Golay filter algorithm to the simulated and experimentally observed signals and comparison with the wavelet-based de-noising technique indicate that the newly developed method is effective in obtaining high-quality TDLAS data for a wide variety of applications including atmospheric environmental monitoring and industrial processing control.

Models for Estimating the Physical Properties of Paddy Soil Using Visible and Near Infrared Reflectance Spectroscopy

A fast and convenient soil analytical technique is needed for soil quality assessment and precision soil management. The main objective of this study was to evaluate the ability of visible (Vis) and near-infrared (NIR) refl ectance spectroscopy to predict paddy soil properties in a typical Malaysian paddy fi eld. To assess the utility of spectroscopy for soil physical characteristics (bulk density, moisture content, clay, silt and sand) prediction, 118 soil samples were used for laboratory analysis and optical measurement in the Vis-NIR region using an analytical spectral device (ASD) FieldSpec spectroradiometer (350–2500 nm). The Savitzky–Golay algorithm and stepwise multiple linear regression (SMLR) were then applied to preprocess, model, and predict the properties on the basis of their spectral refl ectance within the Vis-NIR range. One-third of the samples (40 samples) were withheld for validation purposes. The study revealed that Vis and NIR spectroscopy calibration models for all the measured soil physical characteristics provided a good fi t (R2 > 0.78); hence Vis and NIR (specifi cally NIR refl ectance) can be considered to be a reliable tool to assess soil physical properties of Malaysian paddy fi elds. The results of this study could contribute signifi cantly to developing site-specifi c management.

Recovery of Raman spectra with low signal-to-noise ratio using Wiener estimation.

Raman spectroscopy is a powerful non-destructive technique for qualitatively and quantitatively characterizing materials. However, noise often obscures interesting Raman peaks due to the inherently weak Raman signal, especially in biological samples. In this study, we develop a method based on spectral reconstruction to recover Raman spectra with low signal-to-noise ratio (SNR). The synthesis of narrow-band measurements from low-SNR Raman spectra eliminates the effect of noise by integrating the Raman signal along the wavenumber dimension, which is followed by spectral reconstruction based on Wiener estimation to recover the Raman spectrum with high spectral resolution. Non-negative principal components based filters are used in the synthesis to ensure that most variance contained in the original Raman measurements are retained. A total of 25 agar phantoms and 20 bacteria samples were measured and data were used to validate our method. Four commonly used de-noising methods in Raman spectroscopy, i.e. Savitzky-Golay (SG) algorithm, finite impulse response (FIR) filtration, wavelet transform and factor analysis, were also evaluated on the same set of data in addition to the proposed method for comparison. The proposed method showed the superior accuracy in the recovery of Raman spectra from measurements with extremely low SNR, compared with the four commonly used de-noising methods.

Smoothing of X-ray diffraction data andKα2elimination using penalized likelihood and the composite link model

X-ray diffraction scans consist of series of counts; these numbers obey Poisson distributions with varying expected values. These scans are often smoothed and theKα2component is removed. This article proposes a framework in which both issues are treated. Penalized likelihood estimation is used to smooth the data. The penalty combines the Poisson log-likelihood and a measure for roughness based on ideas from generalized linear models. To remove theKα doublet the model is extended using the composite link model. As a result the data are decomposed into two smooth components: aKα1and aKα2part. For both smoothing andKα2removal, the weight of the applied penalty is optimized automatically. The proposed methods are applied to experimental data and compared with the Savitzky–Golay algorithm for smoothing and the Rachinger method forKα2stripping. The new method shows better results with less local distortion. Freely available software in MATLAB and R has been developed.

Classification of rapeseed colors using Fourier transform mid-infrared photoacoustic spectroscopy

Fourier transform mid-infrared photoacoustic spectroscopy (FTIR-PAS) combined with multivariate discriminant analysis was employed to classify colors of rapeseeds. A total of 129 rapeseed varieties representing three colors (black, reddish and mottled-yellow) were scanned in the range of 500–4000 cm−1. A Savitzky–Golay algorithm was used for the spectral pretreatment. Principal components analysis (PCA) gave an overview of sample distribution in the score space of principal components. The whole sample set was divided into calibration and prediction sets, according to the Kennard–Stone algorithm. Classification models were developed using linear discriminant analysis combined with principal components analysis (PCA-LDA), partial least square discriminant analysis (PLS-DA), and support vector machine (SVM). Results showed that the best accuracy was achieved by the SVM model, with the overall error rates (ERs) of 1.1% and 2.5%, in calibration and prediction sets, respectively. Besides, the PLS-DA model performed slightly better than the PCA-LDA model. This work had demonstrated the good potential of FTIR-PAS to classify rapeseed colors.

Optimizing Savitzky–Golay Parameters for Improving Spectral Resolution and Quantification in Infrared Spectroscopy

Calculating derivatives of spectral data by the Savitzky-Golay (SG) numerical algorithm is often used as a preliminary preprocessing step to resolve overlapping signals, enhance signal properties, and suppress unwanted spectral features that arise due to nonideal instrument and sample properties. Addressing these issues, a study of the simulated and measured infrared data by partial least-squares regression has been conducted. The simulated data sets were modeled by considering a range of undesired chemical and physical spectral anomalies and variations that can occur in a measured spectrum, such as baseline variations, noise, and scattering effects. The study has demonstrated the importance of the optimization of the SG parameters during the conversion of spectra into derivative form, specifically window size and polynomial order of the fitting curve. A specific optimal window size is associated with an exact component of the system being estimated, and this window size does not necessarily apply for some other component present in the system. Since the optimization procedure can be time-consuming, as a rough guideline spectral noise level can be used for assessment of window size. Moreover, it has been demonstrated that, when the extended multiplicative signal correction (EMSC) is used alongside the SG procedure, the derivative treatment of data by the SG algorithm must precede the EMSC normalization.

Classification of Tablets containing Dipyrone, Caffeine and Orphenadrine by Near Infrared Spectroscopy and Chemometric Tools

The objective of this study was to classify samples of tablets containing dipyrone, caffeine and orphenadrine using near infrared (NIR) spectroscopy and chemometric techniques. The data set had 300 spectra of samples from three tablets per batch and four different manufacturers. The pre-processing was accomplished by Savitzky-Golay algorithm with the first derivative, window with 17 points and second-order polynomial. The tablet classification was performed using chemometric models based on principal component analysis (PCA), soft independent modeling of class analogies (SIMCA), genetic algorithm- (GA-LDA) and successive projection algorithm-linear discriminant analysis (SPA-LDA). For PCA analysis, clusters were observed for each group of tablets. The SIMCA model was built using 15 and 30 spectral measures for the training set of similar drugs and reference drugs, respectively. The GA-LDA model used 12 variables, whereas SPA-LDA selected only two wavelengths, 1572 and 1933 nm. The methodology allowed a quick and non-destructive classification of the tablets and without the need for conventional analytical determinations.

A discriminant method for classification of Moroccan olive varieties by using direct FT-IR analysis of the mesocarp section

Four cultivars of olives picked up in the Moroccan region of Beni Mellal were subjected to a characterization and classification study. Analytical data were collected by Fourier transform infrared spectroscopy (FTIR), applied on the mesocarp of the fresh olives without any preliminary treatment. The spectral data were pre-treated by derivative elaboration based on the Savitzky–Golay algorithm to reduce noise and increase analytical information. Partial least squares discriminant analysis (PLS-DA) was performed to elaborate the measurement data and assess the discriminant features of the four cultivars. The PLS model was optimized by applying the Martens’ uncertainty test which provided to select the vibrational frequencies giving the most useful information. The optimized model resulted able to separate the four classes and classify new objects into the appropriate defined classes with a percentage prediction of 97%. The proposed method represents a real novelty to classify olives of different varieties by means of a rapid, inexpensive and reliable procedure.

Derivative FTIR spectroscopy for cluster analysis and classification of morocco olive oils

Fourier transform infrared (FTIR) spectra were employed for differentiation and classification of olive oils from several producing regions of Morocco. A preliminary treatment of the FTIR data was done by a derivative elaboration based on the Savitzky–Golay algorithm to reduce the noise and extract a largest number of analytical information from the spectra. A multivariate statistical procedure based on cluster analysis (CA) coupled to partial least squares-discriminant analysis (PLS-DA), was elaborated, providing an effective classification method. On the basis of a hierarchical agglomerative CA and principal component analysis (PCA), four distinctive clusters were recognised. The PLS-DA procedure was then applied to classify samples from the same regions, picked in different times, or unknown olive oil samples. The model was optimised by applying the Martens’ Uncertainty Test that provided to select the wavelength zones giving the most useful analytical information. The proposed method furnished results reliable in classifying olive oils from different lands with the advantages of being rapid, inexpensive and requiring no prior separation procedure.

Pretreatment of near Infrared Spectral Data in Fast Biomass Analysis

The ability to rapidly evaluate the chemical composition of biomass feedstock for purposes of process monitoring and optimisation is useful for gauging the potential applications and value of such feedstocks. Near infrared (NIR) spectroscopy, coupled with multivariate analysis and data pretreatment, was evaluated to remove interference from physical heterogeneity that could mask chemical property responses. Pretreatment methods included standard normal variate (SNV), multiplicative scattering correction (MSC), 1st derivative with the Savitzky-Golay algorithm (1st derivative), 2nd derivative with the Savitzky-Golay algorithm (2nd derivative), extended multiplicative signal correction (EMSC) and combinations of 1st derivative/2nd derivative with SNV. Results indicated that, of these methods, EMSC was most effective for diffuse reflectance NIR analysis of lignocellulosic biomass. The EMSC-pretreated data not only best accessed the chemical similarity of the probed feedstocks in our hierarchical cluster analysis but also consistently led to the overall best prediction of the chemical composition of the biomass.

Determination of degree of polymerization of insulating paper using near infrared spectroscopy and multivariate calibration

A simple alternative method to determine the degree of polymerization (DP) in transformer insulating papers, using near infrared spectroscopy (NIR), was developed to reduce the costs and the analysis time of the conventional method. Seventy-five samples of Kraft, crepe and cardboard types of paper, in different stages of degradation (DP varying between 200 and 900 units) were collected from transformers over a period of 3 years. The sample set was analyzed according to the conventional method to compare with the proposed alternative spectroscopic method. The spectra were obtained by diffuse reflectance using the spectral range between 1260 and 2500 nm. Some pre-processing strategies of the spectral data were evaluated and the best results were obtained when the derivatives spectra were used (Savitzky–Golay algorithm employing a 21 point window). NIR spectroscopy combined with partial least squares regression method revealed a simple and fast technique to determine the DP of the papers. The prediction error of 83, obtained after external validation, is acceptable for evaluation of the remaining useful life of insulating paper.

Classification of vegetation in North Tibet Plateau based on MODIS time-series data

Based on the 16d-composite MODIS (moderate resolution imaging spectroradiometer)-NDVI(normalized difference vegetation index) time-series data in 2004, vegetation in North Tibet Plateau was classified and seasonal variations on the pixels selected from different vegetation type were analyzed. The Savitzky-Golay filtering algorithm was applied to perform a filtration processing for MODIS-NDVI time-series data. The processed time-series curves can reflect a real variation trend of vegetation growth. The NDVI time-series curves of coniferous forest, high-cold meadow, high-cold meadow steppe and high-cold steppe all appear a mono-peak model during vegetation growth with the maximum peak occurring in August. A decision-tree classification model was established according to either NDVI time-series data or land surface temperature data. And then, both classifying and processing for vegetations were carried out through the model based on NDVI time-series curves. An accuracy test illustrates that classification results are of high accuracy and credibility and the model is conducive for studying a climate variation and estimating a vegetation production at regional even global scale.

The pH Dependence of Heme Pocket Hydration and Ligand Rebinding Kinetics in Photodissociated Carbonmonoxymyoglobin

We monitored the occupancy of a functionally important non-coordinated water molecule in the distal heme pocket of sperm whale myoglobin over the pH range 4.3-9.4. Water occupancy was assessed by using time-resolved spectroscopy to detect the perturbation of the heme visible band absorption spectrum caused by water entry after CO photodissociation ( Goldbeck, R. A., Bhaskaran, S., Ortega, C., Mendoza, J. L., Olson, J. S., Soman, J., Kliger, D. S., and Esquerra, R. M. (2006) Proc. Natl. Acad. Sci. U. S. A. 103, 1254-1259 ). We found that the water occupancy observed during the time interval between ligand photolysis and diffusive recombination decreased by nearly 20% as the pH was lowered below 6. This decrease accounted for most of the concomitant increase in the observed CO bimolecular recombination rate constant, as the lower water occupancy presented a smaller kinetic barrier to CO entry into the pocket at lower pH. These results were consistent with a model in which the distal histidine, which stabilizes the water molecule within the distal pocket by accepting a hydrogen bond, tends to swing out of the pocket upon protonation and destabilize the water occupancy at low pH. Extrapolation of this model to lower pH suggests that the additional increase in ligand association rate constant observed previously in stopped-flow studies at pH 3 may also be due in part to reduced distal water occupancy concomitant with further His64 protonation and coupled protein conformational change.

Exploring the Cooperativity of the Fast Folding Reaction of a Small Protein Using Pulsed Thiol Labeling and Mass Spectrometry

It has been difficult to obtain directly residue-specific information on side chain packing during a fast (ms) protein folding reaction. Such information is necessary to determine the extent to which structural changes in different parts of the protein molecule are coupled together in defining the cooperativity of the overall folding transition. In this study, structural changes occurring during the major fast folding reaction of the small protein barstar have been characterized at the level of individual residue side chains. A pulsed cysteine labeling methodology has been employed in conjunction with mass spectrometry. This provides, with ms temporal resolution, direct information on structure formation at 10 different locations in barstar during its folding. Cysteine residues located on the surface of native barstar, at four different positions, remain fully solvent-accessible throughout the folding process, indicating the absence of any ephemeral nonnative structure in which these four cysteine residues get transiently buried. For buried cysteine residues, the rates of the change in cysteine-thiol accessibility to rapid chemical labeling by the thiol reagent methyl methanethiosulfonate appear to be dependent upon the location of the cysteine residue in the protein and are different from the rate measured by the change in tryptophan fluorescence. But the rates vary over only a 3-fold range. Nevertheless, a comparison of the kinetics of the change in accessibility of the cysteine 3 thiol with those of the change in the fluorescence of tryptophan 53, as well as of their denaturant dependences, indicates that the major folding reaction comprises more than one step.

Allosteric Effectors Influence the Tetramer Stability of Both R- and T-states of Hemoglobin A

The contribution of heterotropic effectors to hemoglobin allostery is still not completely understood. With the recently proposed global allostery model, this question acquires crucial significance, because it relates tertiary conformational changes to effector binding in both the R- and T-states. In this context, an important question is how far the induced conformational changes propagate from the binding site(s) of the allosteric effectors. We present a study in which we monitored the interdimeric interface when the effectors such as Cl-, 2,3-diphosphoglycerate, inositol hexaphosphate, and bezafibrate were bound. We studied oxy-Hb and a hybrid form (alphaFeO2)2-(betaZn)2 as the T-state analogue by monitoring heme absorption and Trp intrinsic fluorescence under hydrostatic pressure. We observed a pressure-dependent change in the intrinsic fluorescence, which we attribute to a pressure-induced tetramer to dimer transition with characteristic pressures in the 70-200-megapascal range. The transition is sensitive to the binding of allosteric effectors. We fitted the data with a simple model for the tetramer-dimer transition and determined the dissociation constants at atmospheric pressure. In the R-state, we observed a stabilizing effect by the allosteric effectors, although in the T-analogue a stronger destabilizing effect was seen. The order of efficiency was the same in both states, but with the opposite trend as inositol hexaphosphate > 2,3-diphosphoglycerate > Cl-. We detected intrinsic fluorescence from bound bezafibrate that introduced uncertainty in the comparison with other effectors. The results support the global allostery model by showing that conformational changes propagate from the effector binding site to the interdimeric interfaces in both quaternary states.