PLS Algorithm Research Articles

Variable reduction algorithm for atomic emission spectra: application to multivariate calibration and quantitative analysis of industrial samples

A variable selection procedure has been developed and used to reduce the number of wavelength data points necessary to formulate a predictive multivariate model for Pt, Pd and Rh using full atomic emission spectra (5684 wavelength data points per spectrum) obtained using a Segmented-Array Charge-Coupled Device Detector (SCCD) for inductively coupled plasma atomic emission spectrometry (ICP-AES). The first stage was the application of an Uninformative Variable Elimination Partial Least Squares (UVE-PLS) algorithm which identified the PLS regression coefficients that are equal to zero at a specified significance level, and removed the associated variables. The second stage was the application of an Informative Variable Degradation PLS (IVD) algorithm, which ranked variables using the ratio of their PLS mean regression coefficients and regression coefficient estimated standard errors (estimated using the jackknife). Variable selection was then achieved by examination of the cumulative sum of these ratios. The algorithms were applied to emission spectra for the determination of Pt, Pd and Rh in a synthetic matrix (Al, Mg, Ce, Zr, Ba, In, Sc and Y at various concentrations). The variable selection routines reduced 5684 wavelength data points to 47, 110 and 334 and gave relative root mean square error (RRMSE) values of 4.60, 3.20 and 1.65% for Pt, Pd and Rh, respectively, compared with 12.6, 8.32 and 27.2% obtained using all 5684 data points; and 5.19, 7.06 and 3.18% obtained when using 166 pre-selected, integrated atomic emission lines. There was no requirement for wavelength or background correction point selection or for wavelength alignment because the entire available segmented spectrum was used in the calibration model. This approach was extended by using real industrial fusion samples to build a calibration model in order to predict the concentration of Au, Ag and Pd in similar matrices. RRMSEs of 23, 29 and 9.1% were obtained for the respective elements in test samples.

Convergence rate analysis of fast predictor-based least squares algorithm

The numerical stability and convergence rate properties of least mean square (LMS) and recursive least squares (RLS) algorithms have attracted much interest in the literature, however, very few investigations have been reported concerning the fast Newton transversal filter (FNTF) algorithm. The FNTF algorithm spans the range of adaptive algorithms from LMS to RLS and is, therefore, more flexible in implementation. One of the reasons for the lack of analysis of FNTF seems that its derivation is much more complicated since it is based on the min-max principle under the assumption that the input signal is autoregressive of smaller order than that of the filter. Recently, FNTF has been shown to be almost equivalent to the fast predictor-based LS (FPLS) algorithm which is a complexity-reduced version of the PLS algorithm. In this paper, we evaluate the convergence rate of FPLS by considering the eigenvalues of the transition matrix of the state vector and show that the convergence gets slower as the order of the predictor decreases.

A Recursive Multi-Block PLS Algorithm for Monitoring Industrial Processes

Industrial processes often present a large number of highly correlated variables that are frequently measured and consist of several operating units. For monitoring such processes multi-block partial least squares (MBPLS) has shown to be capable of dividing the variables according to operating units and can reduce the number variables that describe significant process variation with sufficient accuracy. As shown in this paper, MBPLS may run into difficulties when the process exhibits nonstationary behaviour. To overcome this deficiency, the development of recursive MBPLS is discussed including an application study that relates to a realistic simulation of a fluid catalytic cracking unit.

Exploring the phenotypic expression of a regulatory proteome-altering gene by spectroscopy and chemometrics

Evaluating gene effects on proteomes and the resulting indirect pleiotropic effects through the cell machinery on the chemical phenotype constitutes a formidable challenge to the analytical chemist. This paper demonstrates that near-infrared (NIR) spectroscopy and chemometrics on the level of the barley seed phenotype is able to differentiate between genetic and environmental effects in a PCA model involving normal barley lines and the gene regulator lys3a in different genetic backgrounds. The gene drastically changes the proteome quantitatively and qualitatively, as displayed in two-dimensional electrophoresis, resulting in a radically changed amino acid and chemical composition. A synergy interval partial least squares regression model (si-PLSR) is tested to select combinations of spectral segments which have a high correlation to defined chemical components indicative of the lys3a gene, such as direct effects of the changed proteome, for example, the amide content, or indirect effects due to changes in carbohydrate and fat composition. It is concluded that the redundancy of biological information on the DNA sequence level is also represented at the phenotypic level in the dataset read by the NIR spectroscopic sensor from the chemical physical fingerprint. The PLS algorithm chooses spectral intervals which combine both direct and indirect proteome effects. This explains the robustness of NIR spectral predictions by PLSR for a wide range of chemical components. The new option of using spectroscopy, analytical chemistry and chemometrics in modeling the genetically based covariance of physical/chemical fingerprints of the intact phenotype in plant breeding and biotechnology is discussed.

Some theoretical aspects of partial least squares regression

We give a survey of partial least squares regression with one y variable from a theoretical point of view. Some general comments are made on the motivation as seen by a statistician to study particular chemometric methods, and the concept of soft modelling is criticized from the same angle. Various aspects of the PLS algorithm are considered and the population PLS model is defined. Asymptotic properties of the prediction error are briefly discussed and the relation to other regression methods are commented upon. Results indicating positive and negative properties of PLSR are mentioned, in particular the recent result of Butler, Denham and others which seem to show that PLSR can not be an optimal regression method in any reasonable way. The only possible path left towards some kind of optimality, it seems, is by first trying to find a good motivation for the population model and then possibly finding an optimal estimator under this model. Some results on this are sketched.

Multivariate analysis and optimization of process variable trajectories for batch processes

A new methodology for analyzing batch and semi-batch process variable trajectories is proposed in this paper for process development and optimization. It is aimed at identifying trajectory features such as cumulative effects and time-specific effects of process variables on final product quality. A new pathway multi-block PLS algorithm, valid under the assumption of linear and additive effects, is proposed to efficiently incorporate information provided by intermediate quality measurements, which help in identifying time-specific effects. Extraction of trajectory features is illustrated using designed experiments on a fundamental simulation model for SBR emulsion copolymerization. The methodology is shown to provide information useful for improving final product quality through trajectory modifications. It is also shown that intermediate quality measurements can significantly reduce the number of batch runs necessary for feature extraction (than when only final product quality is available).

Choice of latent explanatory variables: a multiobjective optimization approach

A multiobjective, optimization-based approach for finding latent explanatory variables for linear models is presented. The best choice of a set of latent explanatory variables is made by minimizing a user-specified combination of criteria. In this paper, three criteria are used: (i) the data matrix-related residue, (ii) the observation- or measurement-related residue and (iii) the condition number of the new data matrix of the latent explanatory variables. Successful application of the proposed technique toward identification of a multivariable pilot-scale plant is presented. The proposed algorithm is compared with the well-known PLS algorithm, and the result shows that the proposed algorithm is better than the PLS algorithm. Copyright © 2000 John Wiley & Sons, Ltd.

Multivariate image regression (MIR): implementation of image PLSR?first forays

In the effort of analysing multivariate images, image PLS has been considered interesting. In this paper, image PLS (MIR) is compared with image PCA (MIA) by studying a comparison data set. While MIA has been commercially available for some time, image PLS has not. The kernel PLS algorithm of Lindgren has been implemented in a development environment which is a combination of G (LabVIEW) and MATLAB. In this presentation the power of this environment, as well as an early example in image regression, will be demonstrated. With kernel PLS, all PLS vectors (eigenvectors and eigenvalues) can be calculated from the joint variance–covariance (X′Y and Y′X) and association (Y′Y and X′X) matrices. The dimensions of the kernel matrices X′YY′X and Y′XX′Y are K × K (K is the number of X-variables) and M × M (M is the number of Y-variables) respectively. Hence their size is dependent only on the number of X and Y-variables and not on the number of observations (pixels), which is crucial in image analysis. The choice of LabVIEW as development platform has been based on our experience of a very short implementation time combined with user-friendly interface possibilities. Integrating LabVIEW with MATLAB has speeded up the decomposition calculations, which otherwise are slow. Also, algorithms for matrix calculations are easier to formulate in MATLAB than in LabVIEW. Applying this algorithm on a representative test image which shows many of the typical features found in technical imagery, we have shown that image PLS (MIR) decomposes the data differently than image PCA (MIA), in accordance with chemometric experience from ordinary two-way matrices. In the present example the Y-reference texture-related image used turned out to be able to force a rather significant ‘tilting’ compared with an ‘ordinary MIA’ of the primary structures in the original, spectral R/G image. Copyright © 2000 John Wiley & Sons, Ltd.

A serial extension of multiblock PLS

A novel multiblock PLS algorithm called S-PLS (serial PLS) is presented. S-PLS models the separate predictor blocks serially, making it a supplement to hierarchical PLS. In the S-PLS algorithm the predictor blocks are connected only via the response Y. The block models are calculated using the Y residuals from the previous block model. This allows for an independent interpretation of the separate block models. In each block model the classical PLS algorithm is used. The principles of S-PLS are demonstrated on two chemical applications. In the first example, which is non-linear, S-PLS makes it possible to separate the linear and non-linear parts in the model. The second example illustrates how a model with two predictor blocks can be analysed with S-PLS. Copyright © 1999 John Wiley & Sons, Ltd.

A serial extension of multiblock PLS

A novel multiblock PLS algorithm called S-PLS (serial PLS) is presented. S-PLS models the separate predictor blocks serially, making it a supplement to hierarchical PLS. In the S-PLS algorithm the predictor blocks are connected only via the response Y. The block models are calculated using the Y residuals from the previous block model. This allows for an independent interpretation of the separate block models. In each block model the classical PLS algorithm is used. The principles of S-PLS are demonstrated on two chemical applications. In the first example, which is non-linear, S-PLS makes it possible to separate the linear and non-linear parts in the model. The second example illustrates how a model with two predictor blocks can be analysed with S-PLS. Copyright © 1999 John Wiley & Sons, Ltd.

3D-QSAR study of insecticidal neonicotinoid compounds based on 3-way partial least squares model

A choice of an active conformer and the corresponding alignment rule is an important problem for determining the success of 3D-QSAR study. For flexible molecules, this problem is the most difficult one and construction of the method with appropriate chemometric tools has been required. Recently, Bro [J. Chemom., 10, 1996, 47–61] has proposed a trilinear PLS algorithm as the trilinear extension of standard bilinear PLS in the field of analytical chemistry. Bro's 3-way PLS method seems to be suitable to the 3D-QSAR problem but only few attempts have so far been made at the subject. The object of this study is to investigate the ability of Bro's 3-way PLS method for solving the conformer/alignment problem in 3D-QSAR study. The structure-activity data of insecticidal neonicotinoid compounds were used as a test example. The 3-way arrays were constructed from eight sample vectors and eight electrostatic similarity matrices derived from eight combinations of conformers and alignment rules. The correlation between the 3-way arrays and the insecticidal activity vector was investigated by Bro's 3-way PLS method. The 3-way PLS model with three significant components was obtained, and from its PLS loading the best combination of conformer and alignment rule could be selected.

Simultaneous Determination of Multicomponents in Air Toxic Organic Compounds with Partial Least-Squares Method Using FTIR Spectroscopy*

The multicomponent analysis with Partial Least-Squares Method (PLS) using FTIR spectroscopy has been studied in this paper. In order to ensure the reliability of the prediction results, a criterion SO and SA is created in the PLS algorithm to conclude the similarity between the prediction samples and the calibration ones. An example for the simultaneous determination of Ethylbenzene, Styrene, o-Xylene, m-Xylene and p-Xylene is supplied. The experimental designs of the both the calibration and prediction samples are discussed as well as the significant factor number. With proper multivariate calibration conditions, the average mean relative error (MRE %) is 0.25 % and the average relative standard deviations (RSD%) for Ethylbenzene, Styrene, o-Xylene, m-Xylene and p-Xylene are 0.19, 0.15, 0.06, 0.44 and 0.13 %, respectively. *The project was supported by National Natural Science Foundation of China and Ph.D Foundation of National Education Committee.

The partial least squares algorithm: a truncated Cayley-Hamilton series approximation used to solve the regression problem

In this paper it is shown that the PLS algorithm for univariate data is equivalent to using a truncated Cayley-Hamilton polynomial expression of degree 1 ≤ a≤ r for the matrix inverse (X T X) -1 ∈ R rxr used to compute the LS solution. Furthermore, the a coefficients in this polynomial are computed as the LS optimal solution (minimizing parameters) to the prediction error. The resulting solution is non-iterative. The solution can be expressed in terms of one matrix inverse and is given by B PLS = K a (K a T X T XK a ) -1 K a T X T Y where K a ∈ R rxr is the controllability (Krylov) matrix for the pair (X T X,X T Y). The iterative PLS algorithm for computing the orthogonal weighting matrix W a which is presented in the literature is in this paper shown to be equivalent to computing an orthonormal basis (using, e.g., the QR algorithm) for the column space of K a . The PLS solution can then equivalently be computed as B PLS = W a (W a T X T XW a ) - 1 W a T X T Y, where W a is the Q-orthogonal matrix from the QR decomposition K a = W a R. Furthermore, we have presented an optimal and non-iterative truncated Cayley-Hamilton polynomial LS solution for multivariate data. This solution is found as the minimizing solution of a prediction error criterion.

A multiway 3D QSAR analysis of a series of (S)-N-[(1-ethyl-2-pyrrolidinyl)methyl]-6-methoxybenzamides.

Recently, the multilinear PLS algorithm was presented by Bro and later implemented as a regression method in 3D QSAR by Nilsson et al. In the present article a well-known set of (S)-N-[(1-ethyl-2-pyrrolidinyl)methyl]-6-methoxybenzamides, with affinity towards the dopamine D2 receptor subtype, was utilised for the validation of the multilinear PLS method. After exhaustive conformational analyses on the ligands, the active analogue approach was employed to align them in their presumed pharmacologically active conformations, using (-)-piquindone as a template. Descriptors were then generated in the GRID program, and 40 calibration compounds and 18 test compounds were selected by means of a principal component analysis in the descriptor space. The final model was validated with different types of cross-validation experiments, e.g. leave-one-out, leave-three-out and leave-five-out. The cross-validated Q2 was 62% for all experiments, confirming the stability of the model. The prediction of the test set with a predicted Q2 of 62% also established the predictive ability. Finally, the conformations and the alignment of the ligands in combination with multilinear PLS, obviously, played an important role for the success of our model.

Correction of non-linearities in spectroscopic multivariate calibration by using transformed original variables and PLS regression

The use of transformed original variables is proposed to correct non-linearities in the original data. The additional information that these new variables introduce can be linearly modelled by the PLS algorithm, giving better models than the ones obtained by using only the original variables. Non-informative absorbances can be eliminated later giving more stable models. Several artificial data sets have been studied to check the validity of the method in function of parameters such as degree of non-linearity, noise in X variables and noise in y variable. Three real data sets, with different degrees and sources of non-linearity, have been tested and the methodology proposed always yielded better results (lower rmsecv values or the same rmsecv values with lower complexity) than the PLS regression with original variables.

Recursive exponentially weighted PLS and its applications to adaptive control and prediction

A new and fast recursive, exponentially weighted PLS algorithm which provides greatly improved parameter estimates in most process situations is presented. The potential of this algorithm is illustrated with two process examples: (i) adaptive control of a two by two simulated multivariable continuous stirred tank reactor; and (ii) updating of a prediction model for an industrial flotation circuit. The performance of the recursive PLS algorithm is shown to be much better than that of the recursive least squares algorithm. The main advantage of the recursive PLS algorithm is that it does not suffer from the problems associated with correlated variables and short data windows. During adaptive control, it provided satisfactory control when the recursive least squares algorithm experienced difficulties (i.e., ‘blew’ up) due to the ill-conditioned covariance matrix, (XTX)t. For the industrial soft sensor application, the new algorithm provided much improved estimates of all ten response variables.

Stability improvement of an optochemical heavy metal ion sensor by covalent receptor binding

An optochemical sensor for the detection of mercury(II) and cadmium(II) is presented. The sensing layer consists of a thin film of a polymeric matrix (hydrogel) which contains a metal complexing porphyrin derivative as optical transducer. The spectral responses that occur when metal ions are complexed by the porphyrin receptor are used as sensor signal. In order to enhance the stability of the sensor which is terminated by leaching of the receptor dye a monofunctionalized porphyrin derivative (5-(4′-aminophenyl)-10,15-20-tris(4′-sulfonatophenyl) porphyrin, ATPPS) was covalently bound to poly(2-hydroxyethylmethacrylate) (PolyHEMA). The sensor was prepared by layering PolyHEMA coupled prophyrin as a thin film (thickness 2.5 μm) onto a poly(methycrylate) (PMMA) substrate. The procedure developed for sensor fabrication results in sensitive layers with a good chemical and mechanical stability. After an initial decrease a constant sensor signal is obtained at ca. 85% of the starting value over a period of 4 weeks of continuous operation. Within a measuring time of 10 min detection limits of 1 × 10 −7 mol l −1 (20 μg l −1) and 6 × 10 −7 mol l −1 (63 μg l −1 for Hg(II) and Cd(II), respectively, are obtained. The spectral differences of porphyrin-Hg(II)- and -Cd(II) complexes would allow simultaneous detection using appropriate procedures for spectral deconvolution (e.g. PLS algorithm).

Spectrophotometric determination of Allura Red (R40) in soft drink powders using the universal calibration matrix for partial least squares multivariate method

The Kool-Aid powders (Kraft General Foods) contain one or two of the following dyes: Allura Red (FD and C Red-40, R40), Sunset Yellow (FD and C Yellow-6, Y6) Tartrazine (FD and C Yellow-5, Y5), Erythrosine B (FD and C Red-3, R3), Amaranth (FD and C Red-2, R2) and Brilliant Blue FCF (FD and C Blue 1, B1), depending on the taste of the drink. In this work the “universal” calibration matrix is proposed for the determination of R40 in soft drink powders by partial least squares method using Spectrophotometric data. The training set of samples consists of 23 solutions: three of them contain only R40 at different concentrations and the rest of them are the binary mixtures of R40 with one of the following dyes: R3, R2, Y5 or Y6; all at the concentrations varying in the range 2–22 mg l −1 (exception: R3 2–12 mg l −1). The good analytical performance of R40 calibration was obtained ( R 2 = 0.9993, RMSD = 0.2125, REP = 2.20%) and this calibration matrix was applied to the analysis of the real samples containing only one dye (R40) and to the samples containing also other dyes, commonly used as the color additives in drink powders. The results obtained were compared with the results of the official spectrophotometric method and with the results of PLS algorithm for different binary dye calibration matrices. A good statistical agreement was obtained in each case, which confirms that some interferences occurring in Spectrophotometric determinations can be eliminated using PLS algorithm and including some possibly interfering compounds in calibration samples.

Quantitative Gas Analysis with FT-IR: A Method for CO Calibration Using Partial Least-Squares with Linearized Data

Calibration spectra of CO in the 2.38–5100 ppm concentration range (22 spectra) have been measured with a spectral resolution of 4 cm−1, in the mid-IR (2186–2001 cm−1) region, with a Fourier transform infrared (FT-IR) instrument. The multivariate calibration method partial least-squares (PLS1) was used to model the CO calibration spectra in order to improve the sensitivity and to flag possible outliers in the prediction step. The relation between the absorbance values and concentrations was strongly nonlinear. This result was caused mainly by the low spectral resolution of the instrument. To improve the model predictions, we have linearized the data prior to making the model calculations. The linearization scheme presented here simplified the data pretreatment, because the function needed to linearize the data might be approximated by co-absorbance peak areas representing the concentrations. The integrated absorbance areas, rather than the concentration values, were used as input to the PLS algorithm. A fifth-order polynomial was used to calculate the concentrations from the predicted absorbance areas. The PLS algorithm used on the linearized data reduced the number of factors in the calibration model. Our results reveal that the calibration model based on the linearized data had a high concentration prediction accuracy throughout the entire concentration range.

Multicomponent analysis of phenols in waste waters of the coal conversion industry by means of UV-spectrometry

A procedure for the determination of phenols has been developed as an alternative to the method DIN 38 409 H16. Based on multicomponent analysis using the PLS algorithm it is possible to analyze phenols selectively within three groups. Moreover, two methods of back-ground correction have been explored, the so called UVMA-method [1] and the method of turbid standard solutions. The procedure described provides advantages in the determination of poly-hydric and/or para-substituted phenols. It can be preferably used for the analysis of phenolic waste waters of the brown coal conversion industry. Furthermore, sample preparation is not required because turbidity does not interfere the analysis.