Nonlinear Iterative Partial Least Squares Research Articles

NipalsMCIA: Flexible Multi-Block Dimensionality Reduction in R via Non-linear Iterative Partial Least Squares.

With the increased reliance on multi-omics data for bulk and single cell analyses, the availability of robust approaches to perform unsupervised analysis for clustering, visualization, and feature selection is imperative. Joint dimensionality reduction methods can be applied to multi-omics datasets to derive a global sample embedding analogous to single-omic techniques such as Principal Components Analysis (PCA). Multiple co-inertia analysis (MCIA) is a method for joint dimensionality reduction that maximizes the covariance between block- and global-level embeddings. Current implementations for MCIA are not optimized for large datasets such such as those arising from single cell studies, and lack capabilities with respect to embedding new data. We introduce nipalsMCIA , an MCIA implementation that solves the objective function using an extension to Non-linear Iterative Partial Least Squares (NIPALS), and shows significant speed-up over earlier implementations that rely on eigendecompositions for single cell multi-omics data. It also removes the dependence on an eigendecomposition for calculating the variance explained, and allows users to perform out-of-sample embedding for new data. nipalsMCIA provides users with a variety of pre-processing and parameter options, as well as ease of functionality for down-stream analysis of single-omic and global-embedding factors. nipalsMCIA is available as a BioConductor package at https://bioconductor.org/packages/release/bioc/html/nipalsMCIA.html , and includes detailed documentation and application vignettes. Supplementary Materials are available online.

Solving the missing value problem in PCA by Orthogonalized-Alternating Least Squares (O-ALS)

Dealing with missing data poses a challenge in Principal Component Analysis (PCA) since the most common algorithms are not designed to handle them. Several approaches have been proposed to solve the missing value problem in PCA, such as Imputation based on SVD (I-SVD), where missing entries are filled by imputation and updated in every iteration until convergence of the PCA model, and the adaptation of the Nonlinear Iterative Partial Least Squares (NIPALS) algorithm, able to work skipping the missing entries during the least-squares estimation of scores and loadings. However, some limitations have been reported for both approaches. On the one hand, convergence of the I-SVD algorithm can be very slow for data sets with a high percentage of missing data. On the other hand, the orthogonality properties among scores and loadings might be lost when using NIPALS.To solve these issues and perform PCA of data sets with missing values without the need of imputation steps, a novel algorithm called Orthogonalized-Alternating Least Squares (O-ALS) is proposed. The O-ALS algorithm is an alternating least-squares algorithm that estimates the scores and loadings subject to the Gram-Schmidt orthogonalization constraint. The way to estimate scores and loadings is adapted to work only with the available information.In this study, the performance of O-ALS is tested and compared with NIPALS and I-SVD in simulated data sets and in a real case study. The results show that O-ALS is an accurate and fast algorithm to analyze data with any percentage and distribution pattern of missing entries, being able to provide correct scores and loadings in cases where I-SVD and NIPALS do not perform satisfactorily.

Analytical solution to partial least squares

Partial least squares (PLS) is a widely used multivariate statistical technique, which can be used in neuroimaging, process monitoring, economics, etc. Because the standard PLS is trained by nonlinear iterative partial least squares (NIPALS) algorithm, which can only obtain the numerical solution rather than the analytical solution. Therefore, it is hard for the subsequent theoretical analysis in PLS and the iterative optimization process is computationally intensive. This paper proposes an analytical solution to take the place of NIPALS. This determination meaningfully contributes to future theoretical analysis of PLS. In addition, the analytical solution demonstrates that the selection of initial vector in NIPALS does not affect the PLS training results. Moreover, the analytical solution avoids the multiple iterative calculations, so the divergence of the calculation process will be avoided and the computation burden will be reduced significantly.

Imputation of missing daily rainfall data; A comparison between artificial intelligence and statistical techniques

Handling missing values is a critical component of the data processing in hydrological modeling. The key objective of this research is to assess statistical techniques (STs) and artificial intelligence-based techniques (AITs) for imputing missing daily rainfall values and recommend a methodology applicable to the mountainous terrain of northern Thailand. In this study, 30 years of daily rainfall data was collected from 20 rainfall stations in northern Thailand and randomly 25–35 % of data was deleted from four target stations based on Spearman correlation coefficient between the target and neighboring stations. Imputation models were developed on training and testing datasets and statistically evaluated by mean absolute error (MAE), root mean square error (RMSE), coefficient of determination (R2), and correlation coefficient (r). This study used STs, including arithmetic averaging (AA), multiple linear regression (MLR), normal-ratio (NR), nonlinear iterative partial least squares (NIPALS) algorithm, and linear interpolation was used.•STs results were compared with AITs, including long-short-term-memory recurrent neural network (LSTM-RNN), M5 model tree (M5-MT), multilayer perceptron neural networks (MLPNN), support vector regression with polynomial and radial basis function SVR-poly and SVR-RBF.•The findings revealed that MLR imputation model achieved an average MAE of 0.98, RMSE of 4.52, and R2 was about 79.6 % at all target stations. On the other hand, for the M5-MT model, the average MAE was 0.91, RMSE was about 4.52, and R2 was around 79.8 % compared to other STs and AITs. M5-MT was most prominent among AITs. Notably, the MLR technique stood out as a recommended approach due to its ability to deliver good estimation results while offering a transparent mechanism and not necessitating prior knowledge for model creation.

Multiple Factor Analysis Based on NIPALS Algorithm to Solve Missing Data Problems

Missing or unavailable data (NA) in multivariate data analysis is often treated with imputation methods and, in some cases, records containing NA are eliminated, leading to the loss of information. This paper addresses the problem of NA in multiple factor analysis (MFA) without resorting to eliminating records or using imputation techniques. For this purpose, the nonlinear iterative partial least squares (NIPALS) algorithm is proposed based on the principle of available data. NIPALS presents a good alternative when data imputation is not feasible. Our proposed method is called MFA-NIPALS and, based on simulation scenarios, we recommend its use until 15% of NAs of total observations. A case of groups of quantitative variables is studied and the proposed NIPALS algorithm is compared with the regularized iterative MFA algorithm for several percentages of NA.

Support vector machine-based rapid detection and quantification of butter yellow adulteration in mustard oil using NIR spectra

Butter yellow (BY; 3′-methyl-4-dimethyl-aminoazobenzene) is a carcinogenic adulterant of edible mustard oil. In this study, Fourier-transform near-infrared (FT-NIR) spectral data was combined with multivariate data analysis to both detect and quantify BY adulteration in mustard oil. Fifteen pure mustard oil and one hundred fifty mustard oil samples with BY (0.1–1% w/v) as the adulterant were evaluated. It was observed that sonication- based processing of adulterated samples provides sensitive fingerprints during FT-NIR analysis when compared to heat- based processing. Principal component analysis (PCA) of spectral regions 6000–6064 cm−1 showed well-defined discrimination of pure and adulterated mustard oil with 100% variability. For quantification of BY both NIPALS (non-linear iterative partial least squares) based partial least square (PLS) and LS-SVM (least squares support vector machine) were carried for full-spectrum as well as selected fingerprints using RCGA (real coded genetic algorithm), a variable selection method. Overall, RCGA-LS-SVM model showed best calibration and prediction model with high precision and accuracy based on low root mean square error of cross validation (RMSECV = 0.0073) and prediction (RMSEP = 0.0060). RCGA has indeed lowered the number of variables to mere 30 variables to predict BY in the range of (0.1–1% w/v) with a correlation coefficient of calibration (RC2 = 0.9997) and prediction (RP2 = 0.9996). The key fingerprints detect CH3 and CH functional groups (1st overtone) of BY. This study demonstrated that FT-NIR spectroscopy, combined with selected 30 variables using RCGA-PLS procedure could be a robust technique for the rapid quantification of butter yellow in mustard oil samples. The study would certainly help in quality control and quality assurance of mustard oil and could be emulated for other edible oils.

Application of ED-XRF spectra for determination of macroelements in edible seaweeds

The application of an increased number of edible seaweeds in gastronomy requires accurate, rapid, and non-destructive mineral evaluation. In this work, energy dispersive X-ray fluorescence (ED-XRF) spectra were used to determine Ca, K, and P in edible green, red, and brown seaweeds from the Portuguese coasts. ED-XRF raw spectra of seaweed samples were correlated against results of conventional inductively coupled plasma-optical emission spectrometry (ICP-OES), by the full cross-validation partial least-square (CVPLS) regression method, with nonlinear iterative partial least squares (NIPALS) algorithm. The standard reference material (Thallus laminariae) was used for the validation of CVPLS. Statistical parameters, correlation coefficients (R2), root mean square errors (RMSE), and bias, were used for the assessment of the CVPLS method. For selected minerals, R2 was high (0.88–0.95), while RMSE values (< 1.02) and bias (0) were low, thus indicating a viable potential for their routine analysis by ED-XRF. These findings highlighted the quality of ED-XRF spectral data, which can be used for an efficient non-destructive determination of macroelements in edible seaweeds on a large food industrial scale.

Principal Component Analysis of 1D 1H Diffusion Edited NMR Spectra of Protein Therapeutics.

The one-dimensional (1D) diffusion edited proton NMR method, Protein Fingerprint by Lineshape Enhancement (PROFILE) has been demonstrated to be suitable for higher order structure (HOS) characterization of protein therapeutics including monoclonal antibodies. Recent reports in the literature have demonstrated its advantages for HOS characterization over traditional methods such as circular dichroism and Fourier-transform infrared spectroscopy. Previously, we have demonstrated that the PROFILE method is complementary with high resolution 2D methyl correlated NMR methods and how both may be deployed as a multi-modal platform to further the utility of NMR for HOS characterization. A major limitation of the PROFILE method remains its need for high signal to noise data due to its reliance on convolution difference processing and linear correlation metrics to assess spectral similarity. Here we present an alternative method for analyzing 1D diffusion edited spectra, which overcomes this limitation by using nonlinear iterative partial least squares (NIPALS) principal component analysis, and which we dub PROtein Fingerprint Observed Using NIPALS Decomposition (PROFOUND). We demonstrate that results from the PROFOUND method are robust with respect to instrument, operator and in the presence of high experimental noise and how it may be employed to provide quantitative assessment of spectral similarity.

Polymethyl Methacrylate Quality Modeling with Missing Data Using Subspace Based Model Identification

This paper addresses the problem of quality modeling in polymethyl methacrylate (PMMA) production. The key challenge is handling the large amounts of missing quality measurements in each batch due to the time and cost sensitive nature of the measurements. To this end, a missing data subspace algorithm that adapts nonlinear iterative partial least squares (NIPALS) algorithms from both partial least squares (PLS) and principal component analysis (PCA) is utilized to build a data driven dynamic model. The use of NIPALS algorithms allows for the correlation structure of the input–output data to minimize the impact of the large amounts of missing quality measurements. These techniques are utilized in a simulated case study to successfully model the PMMA process in particular, and demonstrate the efficacy of the algorithm to handle the quality prediction problem in general.

Key-performance-indicators-related fault subspace extraction for the reconstruction-based fault diagnosis

As a typical data-driven technology, projection to latent structure (PLS) has been successfully applied in the quality-related fault diagnosis. However, the oblique decomposition induced by PLS results in redundant component in fault subspace, which imposes a negative influence on the reconstruction-based fault diagnosis. Thus, two fault subspace methods are proposed, including nonlinear iterative partial least squares (NIPALS) fault subspace (N-FS) and improved PLS (IPLS) fault subspace (I-FS) extraction methods. For N-FS, the fault subspace is extracted by the nonlinear iteration, which captures variations of the output. For I-FS, through orthogonal decomposition by IPLS, the useless information is largely eliminated and a purer fault subspace is extracted by the novel iteration mode. A quality-related fault diagnosis strategy is designed, where the fault can be reconstructed by a lower dimensional fault subspace. Two case studies including simulation example and Tennessee Eastman process are conducted to validate the effectiveness of the proposed methods.

Simultaneous detection of glucose, triglycerides, and total cholesterol in whole blood by Fourier-Transform Raman spectroscopy

In this paper, a reagent-free simultaneous and direct detection method of three analytes in human blood based on Fourier-transform Raman (FT-Raman) spectroscopy with 1064 nm laser radiation was proposed for the first time. A total of 161 human blood samples were characterized by FT-Raman spectroscopy under the excitation laser source of 1064 nm. In order to achieve a robust regression model, the Nonlinear Iterative Partial Least Squares (NIPALS) with orthogonal signal correction (OSC) algorithm and sample set partition based on a joint x-y distance (SPXY) is used to establish multivariate calibration models. The root means square error of cross-validation (RMSECV), root mean square error of prediction (RMSEP), correlation coefficients (R2) and ratio of performance to deviation (RPD) were 0.34255 mg/dL, 0.3662 mg/dL, 0.99982 and 56.3524 for glucose, 0.33656 mg/dL, 0.75736 mg/dL, 0.99967 and 34.9169 for total cholesterol (TC), and 0.29956 mg/dL, 0.27469 mg/dL, 0.99998 and 173.5098 for triglycerides (TG), respectively. The analysis results showed that the proposed method could be able to accurately predict the concentration of glucose, TC and TG in blood. This method can instantaneous multi-component detection on whole blood.

Subspace Based Model Identification for an Industrial Bioreactor: Handling Infrequent Sampling Using Missing Data Algorithms

This manuscript addresses the problem of modeling an industrial (Sartorius) bioreactor using process data. In the context of the Sartorius Bioreactor, it is important to appropriately address the problem of dealing with a large number of variables, which are not always measured or are measured at different sampling rates, without taking recourse to simpler interpolation- or imputation-based approaches. To this end, a dynamic model for the Sartorius Bioreactor is developed via appropriately adapting a recently presented subspace model identification technique, which in turn uses nonlinear iterative partial least squares (NIPALS) algorithms to gracefully handle the missing data. The other key contribution is evaluating the ability of the identification approach to provide insight into the process by computing interpretable variables such as metabolite rates. The results demonstrate the ability of the proposed approach to model data from the Sartorius Bioreactor.

Visible near-infrared spectroscopy as an aid for archaeological interpretation

We present a pilot study using visible near-infrared spectroscopy (vis-NIRS) to investigate ancient geochemical activity signals from an excavation of the Viking-Age Ribe Emporia, Denmark. Our aim is to study whether vis-NIRS is applicable to archaeological soil analysis and if it can provide additional insight into the use of space inside houses. Using 1059 bulk soil samples sampled in a 0.25 cm grid, we test if vis-NIRS is useful for predicting magnetic susceptibility and anthropogenic elements such as Ca, Cu, P and Sr. Furthermore, we test both portable X-ray fluorescence (pXRF), magnetic susceptibility and bulk chemistry (by inductively coupled plasma mass spectrometry, ICP-MS) as reference data for training the vis-NIRS models. Predictions were performed using a partial least squares (PLS) regression with a non-linear iterative partial least squares (NIPALS) algorithm, and the best cross-validation predictions for each element based on both portable pXRF and ICP-MS as well as predictions for magnetic susceptibility were analysed spatially using ordinary kriging. The spatial analysis showed reduced precision for all elements and magnetic susceptibility compared to reference maps. Additionally, the spectral predictions had difficulties in predicting the entire elemental and magnetic susceptibility range and especially struggled with the prediction of hotspot areas. Elemental predictions based on the two different reference datasets, collected with pXRF and ICP-MS, performed equally well. The more cost-efficient approach was pXRF for calibration data. Further development is needed for the application of vis-NIRS to be fully implemented together with geochemical tools as a regular part of archaeological soil analysis on site.

Domain adaptation for regression under Beer–Lambert’s law

We consider the problem of unsupervised domain adaptation (DA) in regression under the assumption of linear hypotheses (e.g. Beer–Lambert’s law) – a task recurrently encountered in analytical chemistry. Following the ideas from the non-linear iterative partial least squares (NIPALS) method, we propose a novel algorithm that identifies a low-dimensional subspace aiming at the following two objectives: (i) the projections of the source domain samples are informative w.r.t. the output variable and (ii) the projected domain-specific input samples have a small covariance difference. In particular, the latent variable vectors that span this subspace are derived in closed-form by solving a constrained optimization problem for each subspace dimension adding flexibility for balancing the two objectives. We demonstrate the superiority of our approach over several state-of-the-art (SoA) methods on different DA scenarios involving unsupervised adaptation of multivariate calibration models between different process lines in Melamine production and equality to SoA on two well-known benchmark datasets from analytical chemistry involving (unsupervised) model adaptation between different spectrometers. The former dataset is published with this work11https://github.com/RNL1/Melamine-Dataset.

Subspace based model identification for missing data

AbstractThis article addresses the problem of missing process data in data‐driven dynamic modeling approaches. The key motivation is to avoid using imputation methods or deletion of key process information when identifying the model, and utilizing the rest of the information appropriately at the model building stage. To this end, a novel approach is developed that adapts nonlinear iterative partial least squares (NIPALS) algorithms from both partial least squares (PLS) and principle component analysis (PCA) for use in subspace identification. Note that the existing subspace identification approaches often utilize singular value decomposition (SVD) as part of the identification algorithm which is generally not robust to missing data. In contrast, the NIPALS algorithms used in this work leverage the inherent correlation structure of the identification matrices to minimize the impact of missing data values while generating an accurate system model. Furthermore, in computing the system matrices, the calculated scores from the latent variable methods are utilized as the states of the system. The efficacy of the proposed approach is shown via simulation of a nonlinear batch process example.

Comparison of locally weighted PLS strategies for regression and discrimination on agronomic NIR data

AbstractIn multivariate calibrations, locally weighted partial least squared regression (LWPLSR) is an efficient prediction method when heterogeneity of data generates nonlinear relations (curvatures and clustering) between the response and the explicative variables. This is frequent in agronomic data sets that gather materials of different natures or origins. LWPLSR is a particular case of weighted PLSR (WPLSR; ie, a statistical weight different from the standard 1/n is given to each of the n calibration observations for calculating the PLS scores/loadings and the predictions). In LWPLSR, the weights depend from the dissimilarity (which has to be defined and calculated) to the new observation to predict. This article compares two strategies of LWPLSR: (a) “LW”: the usual strategy where, for each new observation to predict, a WPLSR is applied to the n calibration observations (ie, entire calibration set) vs (b) “KNN‐LW”: a number of k nearest neighbors to the observation to predict are preliminary selected in the training set and WPLSR is applied only to this selected KNN set. On three illustrating agronomic data sets (quantitative and discrimination predictions), both strategies overpassed the standard PLSR. LW and KNN‐LW had close prediction performances, but KNN‐LW was much faster in computation time. KNN‐LW strategy is therefore recommended for large data sets. The article also presents a new algorithm for WPLSR, on the basis of the “improved kernel #1” algorithm, which is competitor and in general faster to the already published weighted PLS nonlinear iterative partial least squares (NIPALS).

Electrocardiogram compression using the nonlinear iterative partial least squares algorithm: a comparison between adaptive and non-adaptive approach

Data compression is applicable in reducing amount of data to be stored and it can be applied in several data collecting processes, being generated by lossy or lossless compression algorithms. Due to its large amount of data, the use of compression is desirable in ECG signals. In this work, we present the accepted nonlinear iterative partial least squares (NIPALS) method as an option to ECG compression method, as recommended by Nicolosi (1999). In addition, we compare the results based in an adaptive and non-adaptive version of this method, by using the MIT arrhythmia database. As a help to obtain a better comparison, we have developed an abnormality indicator related to possible abnormalities in the waveform and a decision method that helps to choose between adaptive or non-adaptive approach. Results showed that the adaptive approach is better than the non-adaptive approach, for the NIPALS compression algorithm.

Electrocardiogram compression using the nonlinear iterative partial least squares algorithm: a comparison between adaptive and non-adaptive approach

Data compression is applicable in reducing amount of data to be stored and it can be applied in several data collecting processes, being generated by lossy or lossless compression algorithms. Due to its large amount of data, the use of compression is desirable in ECG signals. In this work, we present the accepted nonlinear iterative partial least squares (NIPALS) method as an option to ECG compression method, as recommended by Nicolosi (1999). In addition, we compare the results based in an adaptive and non-adaptive version of this method, by using the MIT arrhythmia database. As a help to obtain a better comparison, we have developed an abnormality indicator related to possible abnormalities in the waveform and a decision method that helps to choose between adaptive or non-adaptive approach. Results showed that the adaptive approach is better than the non-adaptive approach, for the NIPALS compression algorithm.

Rapid Assessment of Monovarietal Portuguese Extra Virgin Olive Oil's (EVOO's) Fatty Acids by Fourier‐Transform Near‐Infrared Spectroscopy (FT‐NIRS)

A rapid method for the evaluation of fatty acids (FA) of monovarietal Portuguese extra virgin olive oils (EVOOs) (n = 82) from Alentejo, south‐central region of Portugal, is developed based on Fourier‐transform near‐infrared spectroscopy (FT‐NIRS). The contents of FA‐components (previously determined per conventional gas chromatography) are correlated with FT‐NIRS data (independent variable), by means of full cross‐validation partial least squares regression (CVPLSR). CVPLSR is run using standard normal variate (SNV) data pre‐processing and the nonlinear iterative partial least squares (NIPALS) algorithm. Obtained correlation models are characterized with good statistics: high correlation coefficients (R &gt; 0.85) and low root mean square errors (RMSE &lt; 2.04). Obtained values of the residual predictive deviation (RPD) of CVPLSR‐model, higher than 5.0 (C18:1, C18:2, MUFA, PUFA) confirm the FT‐NIRS as a potential method for rapid quality control of EVOO's FA.Practical Applications: In this work for the first time, fatty acid (FA) profile of monovarietal extra virgin olive oil (EVOO) samples of the greatest olive region in Portugal—Alentejo—is analyzed by Fourier‐transform near‐infrared spectroscopy (FT‐NIRS) and chemometrics. Obtained high‐quality results allow one to recommend this technique from monitoring to quality control analysis of FA (in particular, of representative unsaturated FA), in a great number of olive oil samples, in near‐real time.Quality of olive oil can be assessed by target or non‐target analysis and it is of mandatory importance. Fourier‐transform near‐infrared spectroscopy (FT‐NIRS), combined with chemometrics, for the analysis of the fatty acid profile of varietal olive oils from Alentejo (Portugal) can be a useful technique for quality control. Furthermore, FT‐NIRS is rapid, non‐destructive, and a more environmental friendly technique.

HDDA] sparse subspace constrained partial least squares

ABSTRACTIn this paper, we investigate the objective function and deflation process for sparse Partial Least Squares (PLS) regression with multiple components. While many have considered variations on the objective for sparse PLS, the deflation process for sparse PLS has not received as much attention. Our work highlights a flaw in the Statistically Inspired Modification of Partial Least Squares (SIMPLS) deflation method when applied in sparse PLS regression. We also consider the Nonlinear Iterative Partial Least Squares (NIPALS) deflation in sparse PLS regression. To remedy the flaw in the SIMPLS method, we propose a new sparse PLS method wherein the direction vectors are constrained to be sparse and lie in a chosen subspace. We give insight into this new PLS procedure and show through examples and simulation studies that the proposed technique can outperform alternative sparse PLS techniques in coefficient estimation. Moreover, our analysis reveals a simple renormalization step that can be used to improve the estimation of sparse PLS direction vectors generated using any convex relaxation method.