Latent Variable Regression Methods Research Articles

Improved Semi-Supervised Data-Mining-Based Schemes for Fault Detection in a Grid-Connected Photovoltaic System

Fault detection is a necessary component to perform ongoing monitoring of photovoltaic plants and helps in their safety, maintainability, and productivity with the desired performance. In this study, an innovative technique is introduced by amalgamating Latent Variable Regression (LVR) methods, namely Principal Component Regression (PCR) and Partial Least Square (PLS), and the Triple Exponentially Weighted Moving Average (TEWMA) statistical monitoring scheme. The TEWMA scheme is known for its sensitivity to uncovering changes of small magnitude. Nevertheless, TEWMA can only be utilized for monitoring single variables and ignoring the correlation among monitored variables. To alleviate this difficulty, the LVR methods (i.e., PCR and PLS) are used as residual generators. Then, the TEWMA is applied to the obtained residuals for fault detection purposes, where the detection threshold is computed via kernel density estimation to improve its performance and widen its applicability in practice. Real data with different fault scenarios from a 9.54 kW photovoltaic plant has been used to verify the efficiency of the proposed schemes. Results revealed the superior performance of the PLS-TEWMA chart compared to the PLS-TEWMA chart, particularly in detecting anomalies with small changes. Moreover, they have almost comparable performance for large anomalies.

Variable Selection and Redundancy in Multivariate Regression Models

Variable selection is a topic of interest in many scientific communities. Within chemometrics, where the number of variables for multi-channel instruments like NIR spectroscopy and metabolomics in many situations is larger than the number of samples, the strategy has been to use latent variable regression methods to overcome the challenges with multiple linear regression. Thereby, there is no need to remove variables as such, as the low-rank models handle collinearity and redundancy. In most studies on variable selection, the main objective was to compare the prediction performance (RMSE or accuracy in classification) between various methods. Nevertheless, different methods with the same objective will, in most cases, give results that are not significantly different. In this study, we present three other main objectives: i) to eliminate variables that are not relevant; ii) to return a small subset of variables that has the same or better prediction performance as a model with all original variables; and iii) to investigate the consistency of these small subsets.

Separating common from distinctive variation

BackgroundJoint and individual variation explained (JIVE), distinct and common simultaneous component analysis (DISCO) and O2-PLS, a two-block (X-Y) latent variable regression method with an integral OSC filter can all be used for the integrated analysis of multiple data sets and decompose them in three terms: a low(er)-rank approximation capturing common variation across data sets, low(er)-rank approximations for structured variation distinctive for each data set, and residual noise. In this paper these three methods are compared with respect to their mathematical properties and their respective ways of defining common and distinctive variation.ResultsThe methods are all applied on simulated data and mRNA and miRNA data-sets from GlioBlastoma Multiform (GBM) brain tumors to examine their overlap and differences. When the common variation is abundant, all methods are able to find the correct solution. With real data however, complexities in the data are treated differently by the three methods.ConclusionsAll three methods have their own approach to estimate common and distinctive variation with their specific strength and weaknesses. Due to their orthogonality properties and their used algorithms their view on the data is slightly different. By assuming orthogonality between common and distinctive, true natural or biological phenomena that may not be orthogonal at all might be misinterpreted.Electronic supplementary materialThe online version of this article (doi:10.1186/s12859-016-1037-2) contains supplementary material, which is available to authorized users.

An optimization‐based undeflated PLS (OUPLS) method to handle missing data in the training set

Advances in sensory systems have led to many industrial applications with large amounts of highly correlated data, particularly in chemical and pharmaceutical processes. With these correlated data sets, it becomes important to consider advanced modeling approaches built to deal with correlated inputs in order to understand the underlying sources of variability and how this variability will affect the final quality of the product.Additional to the correlated nature of the data sets, it is also common to find missing elements and noise in these data matrices. Latent variable regression methods such as partial least squares or projection to latent structures (PLS) have gained much attention in industry for their ability to handle ill‐conditioned matrices with missing elements. This feature of the PLS method is accomplished through the nonlinear iterative PLS (NIPALS) algorithm, with a simple modification to consider the missing data. Moreover, in expectation maximization PLS (EM‐PLS), imputed values are provided for missing data elements as initial estimates, conventional PLS is then applied to update these elements, and the process iterates to convergence.This study is the extension of previous work for principal component analysis (PCA), where we introduced nonlinear programming (NLP) as a means to estimate the parameters of the PCA model. Here, we focus on the parameters of a PLS model.As an alternative to modified NIPALS and EM‐PLS, this paper presents an efficient NLP‐based technique to find model parameters for PLS, where the desired properties of the parameters can be explicitly posed as constraints in the optimization problem of the proposed algorithm. We also present a number of simulation studies, where we compare effectiveness of the proposed algorithm with competing algorithms. Copyright © 2014 John Wiley & Sons, Ltd.

Models and Methods for Quantitative Analysis of Surface-Enhanced Raman Spectra

The quantitative analysis of surface-enhanced Raman spectra using scattering nanoparticles has shown the potential and promising applications in in vivo molecular imaging. The diverse approaches have been used for quantitative analysis of Raman pectra information, which can be categorized as direct classical least squares models, full spectrum multivariate calibration models, selected multivariate calibration models, and latent variable regression (LVR) models. However, the working principle of these methods in the Raman spectra application remains poorly understood and a clear picture of the overall performance of each model is missing. Based on the characteristics of the Raman spectra, in this paper, we first provide the theoretical foundation of the aforementioned commonly used models and show why the LVR models are more suitable for quantitative analysis of the Raman spectra. Then, we demonstrate the fundamental connections and differences between different LVR methods, such as principal component regression, reduced-rank regression, partial least square regression (PLSR), canonical correlation regression, and robust canonical analysis, by comparing their objective functions and constraints.We further prove that PLSR is literally a blend of multivariate calibration and feature extraction model that relates concentrations of nanotags to spectrum intensity. These features (a.k.a. latent variables) satisfy two purposes: the best representation of the predictor matrix and correlation with the response matrix. These illustrations give a new understanding of the traditional PLSR and explain why PLSR exceeds other methods in quantitative analysis of the Raman spectra problem. In the end, all the methods are tested on the Raman spectra datasets with different evaluation criteria to evaluate their performance.

Sparse integrative clustering of multiple omics data sets

High resolution microarrays and second-generation sequencing platforms are powerful tools to investigate genome-wide alterations in DNA copy number, methylation, and gene expression associated with a disease. An integrated genomic profiling approach measuring multiple omics data types simultaneously in the same set of biological samples would render an integrated data resolution that would not be available with any single data type. In this study, we use penalized latent variable regression methods for joint modeling of multiple omics data types to identify common latent variables that can be used to cluster patient samples into biologically and clinically relevant disease subtypes. We consider lasso (Tibshirani, 1996), elastic net (Zou and Hastie, 2005), and fused lasso (Tibshirani et al., 2005) methods to induce sparsity in the coefficient vectors, revealing important genomic features that have significant contributions to the latent variables. An iterative ridge regression is used to compute the sparse coefficient vectors. In model selection, a uniform design (Fang and Wang, 1994) is used to seek "experimental" points that scattered uniformly across the search domain for efficient sampling of tuning parameter combinations. We compared our method to sparse singular value decomposition (SVD) and penalized Gaussian mixture model (GMM) using both real and simulated data sets. The proposed method is applied to integrate genomic, epigenomic, and transcriptomic data for subtype analysis in breast and lung cancer data sets.

Product transfer between sites using Joint-Y PLS

The product transfer problem refers to that of estimating the operating conditions in a target plant in order for it to yield a final product with the same set of final characteristics as is yielded in another source plant. The estimation of such new operating conditions in the target plant often involves a combination of expertise, rules of thumb and in some cases some fundamental modeling. The use of latent variable modeling using historical data from both sites has been proposed in the past as an alternative approach (C.M. Jaeckle and J.F. MacGregor, Product transfer between plants using historical process data, AICHE J., 46 (2000) 1989–1997). This paper presents a new latent variable regression method: Joint-Y PLS (JYPLS) that is ideally suited for modeling the common latent variable structure in multiple plants. The mathematical foundations for the JYPLS model are presented and several parameter estimation approaches are given. A technique is proposed to solve the product transfer problem using the JYPLS model and two examples are given including an industrial scale-up case for a batch pulp digester from the pulp and paper industry.

Jackknife and bootstrap methods in the identification of dynamic models

A new criterion based on a Jackknife or a Bootstrap statistic is proposed for identifying non-parsimonious dynamic models (FIR, ARX). It is applicable for selecting the number of components in latent variable regression methods or the constraining parameter in regularized least squares regression methods. These meta parameters are used to overcome ill-conditioning caused by model over-parameterization, when fitted using prediction error or least squares methods. In all cases studied, using PLS for parameter estimation, the proposed criterion led to the selection of better models, in the mean square error sense, than when selected via cross-validation. The methodology also provides approximate confidence intervals for the model parameters and the step and impulse response of the system.

A statistical framework for multivariate latent variable regression methods based on maximum likelihood

A statistical framework is developed to contrast methods used for parameter estimation for a latent variable multivariate regression (LVMR) model. This model involves two sets of variables, X and Y, both with multiple variables and sharing a common latent structure with additive random errors. The methods contrasted are partial least squares (PLS) regression, principal component regression (PCR), reduced rank regression (RRR) and canonical co-ordinate regression (CCR). The framework is based on a constrained maximum likelihood analysis of the model under assumptions of multivariate normality. The constraint is that the estimates of the latent variables are restricted to be linear functions of the X variables, which is the form of the estimates for the methods being contrasted. The resulting framework is a continuum regression that goes from RRR to PCR depending on the ratio of error variances in the X and Y spaces. PLS does not arise as a member of the continuum; however, the method does offer some insight into why PLS would work well in practice. The constrained maximum likelihood result is also compared with the unconstrained maximum likelihood analysis to investigate the impact of the constraint. The results are illustrated on a simulated example. Copyright © 1999 John Wiley & Sons, Ltd.

A Calibration Tutorial for Spectral Data. Part 2. Partial Least Squares Regression Using Matlab and Some Neural Network Results

Part 1 explained multiplicative scatter correction (MSC), the building of a principal component regression (PCR) model and how the test data can be used in prediction. Emphasis was on data pretreatment for linearistion and on spectral/chemical interpretation of the results. Part 2 discusses partial least squares (PLS or PLSR) regression. The data set prepared in Part 1 is also used here. Details on data pretreatment are, therefore, not repeated. Some details of PLS modeling are explained using the calculations of the example. Also, the interpretation of the PLS model gets some attention. Neural network calculation results are included for comparison. Artifical neural networks (ANN) are non-linear, so linearisation is not considered necessary. Latent variable regression methods such as PLS and PCR and ANNs are all successive approximations to the unknown function y = f(x) that forms the basis of all calibration methods. In latent variable regression, the rank of the model determines the degree of approximation. In ANNs, the number of hidden nodes and the number of iterations determine the degree of approximation.

Multi-component principal component regression and partial least-squares analyses of overlapped chromatographic peaks

Principal component and partial least-squares in latent variable regression methods were applied to the multivariate calibration of overlapping chromatographic peaks for toluene, isooctane and ethanol mixtures. The degree of peak overlap was varied using column temperatures of 105, 120 and 130°C. Even using the most severely overlapped peaks (130°C), the analysis errors obtained for validation set samples using both regression techniques were of the same size as those encountered using simple linear regression for individual determination of the three constituents. Truncation of the overlapped peak chromatograms appeared to lower the noise level without a significant loss of statistical information about the constituent concentrations.