Large Spectral Data Sets Research Articles

Tests of MATISSE on large spectral datasets from the ESO archive

The automated stellar classification algorithm, MATISSE, has been developed at the Observatoire de la Côte d’Azur (OCA) as a means of determining stellar effective temperatures, surface gravities and chemical abundances for large samples of stellar spectra. MATISSE is one of the key programmes to be used in the analysis of the spectra that will be obtained by the Radial Velocity Spectrometer (RVS) on the European Space Agency’s (ESA) Gaia satellite. In preparation for the Gaia mission, rigorous testing of MATISSE is currently being carried out using large datasets of spectra from the European Southern Observatory (ESO) archive (The AMBRE Project, a collaboration between ESO and OCA). The derived stellar parameters will be delivered to ESO in order to make them available to the astronomical community via the Virtual Observatory. This paper presents the preliminary results of the analysis of the archived FEROS spectra.

Characterization of the lateral distribution of fluorescent lipid in binary-constituent lipid monolayers by principal component analysis.

Lipid lateral organization in binary-constituent monolayers consisting of fluorescent and nonfluorescent lipids has been investigated by acquiring multiple emission spectra during measurement of each force-area isotherm. The emission spectra reflect BODIPY-labeled lipid surface concentration and lateral mixing with different nonfluorescent lipid species. Using principal component analysis (PCA) each spectrum could be approximated as the linear combination of only two principal vectors. One point on a plane could be associated with each spectrum, where the coordinates of the point are the coefficients of the linear combination. Points belonging to the same lipid constituents and experimental conditions form a curve on the plane, where each point belongs to a different mole fraction. The location and shape of the curve reflects the lateral organization of the fluorescent lipid mixed with a specific nonfluorescent lipid. The method provides massive data compression that preserves and emphasizes key information pertaining to lipid distribution in different lipid monolayer phases. Collectively, the capacity of PCA for handling large spectral data sets, the nanoscale resolution afforded by the fluorescence signal, and the inherent versatility of monolayers for characterization of lipid lateral interactions enable significantly enhanced resolution of lipid lateral organizational changes induced by different lipid compositions.

Nonparametric ground-motion evaluation of high frequency shear-wave Fourier spectra in Korea

A nonparametric inversion method has been employed to obtain the detailed attenuation characteristics of horizontal shear-wave Fourier spectra at high frequencies based on the large spectral dataset mostly from small earthquakes occurring at shallow depths in the southern part of the Korean Peninsula. Nonparametric expressions of the spectral attenuation were evaluated for 59 frequency bands in the range of 1 Hz ≤ f ≤ 30 Hz and hypocentral distance ranging from 10 to 400 km by applying the nonparametric inversion method to the observed spectral dataset. Based on the nonparametric expressions, three linear regions roughly separated at 60 and 110 km in distance could be identified and the use of the conventional functional form of Q 0 f η assuming the geometrical spreading of R −0.5 could be validated for 3 Hz ≤ f ≤ 30 Hz and R ≥ 190 km with the best fitting result of 292f 0.53. In the near-source region of R < 70 km, the spectral attenuation due to the geometrical spreading ranged from R −1.3 to R −1.5 irrespective of the Q-values. The distance-attenuation trend of the recent observed dataset from a moderate inland earthquake in Korea was also well predicted by the inverted nonparametric path term.

Phosphorylation-Specific MS/MS Scoring for Rapid and Accurate Phosphoproteome Analysis

The promise of mass spectrometry as a tool for probing signal-transduction is predicated on reliable identification of post-translational modifications. Phosphorylations are key mediators of cellular signaling, yet are hard to detect, partly because of unusual fragmentation patterns of phosphopeptides. In addition to being accurate, MS/MS identification software must be robust and efficient to deal with increasingly large spectral data sets. Here, we present a new scoring function for the Inspect software for phosphorylated peptide tandem mass spectra for ion-trap instruments, without the need for manual validation. The scoring function was modeled by learning fragmentation patterns from 7677 validated phosphopeptide spectra. We compare our algorithm against SEQUEST and X!Tandem on testing and training data sets. At a 1% false positive rate, Inspect identified the greatest total number of phosphorylated spectra, 13% more than SEQUEST and 39% more than X!Tandem. Spectra identified by Inspect tended to score better in several spectral quality measures. Furthermore, Inspect runs much faster than either SEQUEST or X!Tandem, making desktop phosphoproteomics feasible. Finally, we used our new models to reanalyze a corpus of 423,000 LTQ spectra acquired for a phosphoproteome analysis of Saccharomyces cerevisiae DNA damage and repair pathways and discovered 43% more phosphopeptides than the previous study.

Wavelength band optimization in spectral near-infrared optical tomography improves accuracy while reducing data acquisition and computational burden

Multispectral near-infrared (NIR) tomographic imaging has the potential to provide information about molecules absorbing light in tissue, as well as subcellular structures scattering light, based on transmission measurements. However, the choice of possible wavelengths used is crucial for the accurate separation of these parameters, as well as for diminishing crosstalk between the contributing chromophores. While multispectral systems are often restricted by the wavelengths of laser diodes available, continuous-wave broadband systems exist that have the advantage of providing broadband NIR spectroscopy data, albeit without the benefit of the temporal data. In this work, the use of large spectral NIR datasets is analyzed, and an objective function to find optimal spectral ranges (windows) is examined. The optimally identified wavelength bands derived from this method are tested using both simulations and experimental data. It is found that the proposed method achieves images as qualitatively accurate as using the full spectrum, but improves crosstalk between parameters. Additionally, the judicious use of these spectral windows reduces the amount of data needed for full spectral tomographic imaging by 50%, therefore increasing computation time dramatically.

Retrieval of chlorophyll concentration from leaf reflectance spectra using wavelet analysis

The dynamics of foliar chlorophyll concentrations have considerable significance for plant–environment interactions, ecosystem functioning and crop growth. Hyperspectral remote sensing has a valuable role in the monitoring of such dynamics. This study focussed upon improving the accuracy of chlorophyll quantification by applying wavelet analysis to reflectance spectra. Leaf-scale radiative transfer models were used to generate very large spectral data sets with which to develop and rigorously test refinements to the approach and compare it with existing spectral indices. The results demonstrated that by decomposing leaf spectra, the resultant wavelet coefficients can be used to generate accurate predictions of chlorophyll concentration, despite wide variations in the range of other biochemical and biophysical factors that influence leaf reflectance. Wavelet analysis outperformed predictive models based on untransformed spectra and a range of spectral indices. The paper discusses the possibilities for further refining the wavelet approach and for extending the technique to the sensing of a variety of vegetation properties at a range of spatial scales.

Parallel Processing of Large Datasets from NanoLC-FTICR-MS Measurements

A new approach for automatic parallel processing of large mass spectral datasets in a distributed computing environment is demonstrated to significantly decrease the total processing time. The implementation of this novel approach is described and evaluated for large nanoLC-FTICR-MS datasets. The speed benefits are determined by the network speed and file transfer protocols only and allow almost real-time analysis of complex data (e.g., a 3-gigabyte raw dataset is fully processed within 5 min). Key advantages of this approach are not limited to the improved analysis speed, but also include the improved flexibility, reproducibility, and the possibility to share and reuse the pre- and postprocessing strategies. The storage of all raw data combined with the massively parallel processing approach described here allows the scientist to reprocess data with a different set of parameters (e.g., apodization, calibration, noise reduction), as is recommended by the proteomics community. This approach of parallel processing was developed in the Virtual Laboratory for e-Science (VL-e), a science portal that aims at allowing access to users outside the computer research community. As such, this strategy can be applied to all types of serially acquired large mass spectral datasets such as LC-MS, LC-MS/MS, and high-resolution imaging MS results.

Automated derivation of stellar atmospheric parameters and chemical abundances: the MATISSE algorithm

We present an automated procedure for the derivation of atmospheric parameters (Teff, log g, [M/H]) and individual chemical abundances from stellar spectra. The MATrix Inversion for Spectral SythEsis (MATISSE) algorithm determines a basis, B_\theta(\lambda), allowing to derive a particular stellar parameter \theta by projection of an observed spectrum. The B_\theta(\lambda) function is determined from an optimal linear combination of theoretical spectra and it relates, in a quantitative way, the variations in the spectrum flux with variations in \theta. An application of this method to the GAIA/RVS spectral range is described, together with its performances for different types of stars of various metallicities. Blind tests with synthetic spectra of randomly selected parameters and observed input spectra are also presented. The method gives rapid, accurate and stable results and it can be efficiently applied to the study of stellar populations through the analysis of large spectral data sets, including moderate to low signal to noise spectra.

Space weathering of near-Earth and main belt silicate-rich asteroids: observations and ion irradiation experiments

In this paper we report the results of a comparison between ion irradiation experiments (N + ,A r + ,A r ++ ) on silicates, a large spectral data set of silicate-rich (S-type) asteroids, and ordinary chondrite meteorites (OCs). Ion irradiation experiments – conducted on Fe-poor olivine, Fe-poor orthopyroxene, bulk silicate-rich rocks and one OC – have been monitored by means of reflectance spectroscopy (0.3–2.5 µm). All these experiments produce reddening and darkening of reflectance spectra. The observational data consist of a set of visible and near-infrared (0.4–2.4 µm) spectra of S-type asteroids, that belong to main belt (MBAs) and near-Earth (NEOs) populations. By analyzing the spectra of OCs, MBAs, and NEOs, we find a similar mineralogy between most asteroids and meteorites, but different distributions of spectral slopes. We interpret these findings in the frame of space weathering induced by solar wind ion irradiation.

Application of Fast Fourier Transform Cross-Correlation for the Alignment of Large Chromatographic and Spectral Datasets

Preprocessing of chromatographic and spectral data is an important aspect of analytical sciences. In particular, recent advances in proteomics have resulted in the generation of large data sets that require analysis. To assist accurate comparison of chemical signals, we propose two methods for the alignment of multiple spectral data sets. Based on methods previously described, each chromatograph or spectrum to be aligned is divided and aligned as individual segments to a reference. However, our methods make use of fast Fourier transform for the rapid computation of a cross-correlation function that enables alignments between samples to be optimized. The proposed methods are demonstrated in comparison with an existing method on a chromatographic and a mass spectral data set. It is shown that our methods provide an advantage of speed and a reduction of the number of input parameters required. The software implementations for the proposed alignment methods are available under the downloads section at http://ptcl.chem.ox.ac.uk/~jwong/specalign.

Control charts for chemometric evaluation of Raman spectra

Principal component analysis (PCA) is a method used for evaluation of vibration spectra that allows separating important information from the minor effects. From the point of view of data structure the most valuable information is depicted in score graphs of several Principal Components (PCs), the contribution of various spectral regions can be viewed in the x-loadings charts. Questionable problems of PCA are the identification of outliers for various spectral variables and various PCs and mainly how to compare mutually multiple large spectral data sets. To solve these issues we attempt to introduce control charts in multivariate spectral analysis integrating them with PCA. A utility was developed to provide comparison of new samples (spectral data sets) according to previous set (calibration) for data in individual PCs. Similar spectral data sets are then inside the control limits while the different spectral data exceed the limits. The correlated data in individual PCs can be easily distinguished from dissimilar spectral data.

Automatic alignment of individual peaks in large high-resolution spectral data sets

Pattern recognition techniques are effective tools for reducing the information contained in large spectral data sets to a much smaller number of significant features which can then be used to make interpretations about the chemical or biochemical system under study. Often the effectiveness of such approaches is impeded by experimental and instrument induced variations in the position, phase, and line width of the spectral peaks. Although characterizing the cause and magnitude of these fluctuations could be important in its own right (pH-induced NMR chemical shift changes, for example) in general they obscure the process of pattern discovery. One major area of application is the use of large databases of 1H NMR spectra of biofluids such as urine for investigating perturbations in metabolic profiles caused by drugs or disease, a process now termed metabonomics. Frequency shifts of individual peaks are the dominant source of such unwanted variations in this type of data. In this paper, an automatic procedure for aligning the individual peaks in the data set is described and evaluated. The proposed method will be vital for the efficient and automatic analysis of large metabonomic data sets and should also be applicable to other types of data.

Multivariate statistical analysis of time-of-flight secondary ion mass spectrometry images using AXSIA

Time-of-flight secondary ion mass spectrometry (TOF-SIMS) by its parallel nature, generates complex and very large datasets quickly and easily. An example of such a large dataset is a spectral image where a complete spectrum is collected for each pixel. Unfortunately, the large size of the data matrix involved makes it difficult to extract the chemical information from the data using traditional techniques. Because time constraints prevent an analysis of every peak, prior knowledge is used to select the most probable and significant peaks for evaluation. However, this approach may lead to a misinterpretation of the system under analysis. Ideally, the complete spectral image would be used to provide a comprehensive, unbiased materials characterization based on full spectral signatures. Automated eXpert spectral image analysis (AXSIA) software developed at Sandia National Laboratories implements a multivariate curve resolution technique that was originally developed for energy dispersive X-ray spectroscopy (EDS) [Microsci. Microanal. 9 (2003) 1]. This paper will demonstrate the application of the method to TOF-SIMS. AXSIA distills complex and very large spectral image datasets into a limited number of physically realizable and easily interpretable chemical components, including both spectra and concentrations. The number of components derived during the analysis represents the minimum number of components needed to completely describe the chemical information in the original dataset. Since full spectral signatures are used to determine each component, an enhanced signal-to-noise is realized. The efficient statistical aggregation of chemical information enables small and unexpected features to be automatically found without user intervention.

Variations of Large Spectral Sets; Two-Dimensional Correlation Analysis of Loadings Spectra of Principal Component Analysis

This work attempted to interpret the principal component loadings spectra of principal component analysis on large spectral data sets with multi-variables using two-dimensional (2D) correlation analysis. Three examples of visible/near infrared (NIR) spectra of chicken muscles under different conditions were given and discussed. 2D analysis indicated that characteristic bands from loadings spectra are in good agreement with those from a small number of spectra induced by simple external perturbations. Although some advantages of 2D correlation analysis (such as sequential changes in intensity) were not available, it might still be useful for the understanding of large and complex spectral data sets with multi component variations.

Calibration transfer of near-infrared spectra based on compression of wavelet coefficients

Near-infrared (NIR) spectrometers are widely used as online analyzers. We propose a transfer algorithm of a calibration model for the near-infrared spectrometer. The method, which is based on direct standardization in the compressed wavelet domain, is called WTDS. Calibration transfer in the compressed wavelet domain permits a reduction in processing time for the analysis of large spectral data sets. A calibration transfer based on a compression of wavelet coefficients offers increased robustness against testing errors, with no significant loss of performance. We select the proper set of wavelet coefficients based on the hard thresholding approach. The WTDS shows an improvement in calibration transfer performance, although wavelet coefficients are compressed.

Automatic correction of peak shifts in Raman spectra before PLS regression

A simple algorithm is presented for the fast and fully automatic removal of peak shifts in large spectral data sets. It is able to remove peak shifts exceeding the discrete spectral resolution. The algorithm has been applied to Raman spectra of three solution copolymerizations of styrene (Sty) and butyl acrylate (BA) performed on three separate days. Small Raman peak shifts, smaller than the spectral resolution, and induced by the on/off switching of the laser and the repositioning of the grating, could successfully be removed prior to partial least squares (PLS) regression. It is demonstrated that after shift correction, the ability of PLS to predict Sty and BA concentrations is improved. Due to its speed, the algorithm is suitable to eliminate real time (on-line) spectral shifts from the spectra. This makes re-calibration or model adaptation forced by spectral shifts superfluous. The proposed algorithm may likewise be used to remove peak/wavelength shifts in mid-IR, near-IR, and NMR spectra, as well as to remove retention time shifts in chromatograms.

Automatic correction for phase shifts, frequency shifts, and lineshape distortions across a series of single resonance lines in large spectral data sets.

A new model-free method is presented that automatically corrects for phase shifts, frequency shifts, and additional lineshape distortions of one single resonance peak across a series of in vivo NMR spectra. All separate phase and frequency variations are quickly and directly derived from the common lineshape in the data set using principal component analysis and linear regression. First, the new approach is evaluated on simulated data in order to quantitatively assess the phase and frequency shifts which can be removed by the proposed correction procedure. Subsequently, the value of the method is demonstrated on in vivo (31)P NMR spectra from skeletal muscle of the hind leg of the mouse focusing on the phosphocreatine resonance which is distorted by the experimental procedure. Phase shifts, frequency shifts, and lineshape distortions with respect to the common lineshape in the spectral data set could successfully be removed.

Hyperspectral identification of coral reef features

Remote sensing is regarded as an efficient and accurate tool for mapping and monitoring changes in coral reef extent and well being consistently over large geographic areas. However, several operational restrictions limit the accuracy with which coral reefs can be monitored remotely. A primary restriction is that the spectral responses of numerous features in the coral reef environment are optically similar, which has the potential of contributing to misclassification errors. In 1996, spectral reflectance data were collected in Fiji using an in situ underwater cosine receptor and a 10m underwater fibre optic cable, which permitted sampling at depth while scuba diving. In 1997, spectral reflectance in situ measurements of exposed coral reef features with little or no water cover were collected in Indonesia using the same radiometer, but a nonwaterproof remote cosine receptor. These spectral datasets were compared and analysed to test the following hypotheses. First, geographic location does not affect the spectral reflectance characteristics; second, the morphology of reef features does affect the spectral reflectance characteristics; third, bleached coral and healthy coral have distinct spectral reflectance characteristics; and finally, a spectral reflectance index will aid image classification. Results indicate that the spectra measured in Fiji and those measured in Indonesia are statistically similar, so all spectra were merged into one large spectral dataset and principal components analysis was used to determine the most representative spectra. Derivative spectroscopy was then used to conclude that spectral discrimination is indeed possible between 654 and 674nm, between 582 and 686nm and between 506 and 566nm. The proportion of correctly identified spectra using the three-step procedure of first derivatives is 75% with the main source of error resulting from spectral variability of algae reflectance. The results of this feasibility study indicate that hyperspectral remote sensing of a coral reef environment will lead to accurate identification and subsequent monitoring of changes in coral health and overall well being.

Clipped representations of fourier-transform ion-cyclotron resonance mass spectra

Because of its high mass resolution over a wide mass range, a Fourier transform/mass spectrometry (F.t./m.s.) experiment may generate very large spectral data sets (⩾ 64 K each), creating severe data storage problems for g.c./F.t./m.s. and for mass spectral library storage, search and retrieval. Fortunately, the stored experimental data need not be highly precise. In this paper, it is shown that useful mass spectra can be produced from time-domain F.t./m.s. data that have been clipped to only 1 bit/word, thereby offering a potential reduction by a factor of ⩾ 20 in data storage requirement. In many cases, mass spectral resolution can actually be enhanced by the clipping process. Finally, it is shown tha a compound can be identified quickly and accurately by comparison of its clipped representation with similarly clipped data for library compounds.