Multiway Partial Least Squares Research Articles

Analyzing the occurrence of foaming in batch fermentation processes using multiway partial least square approaches

AbstractThis article presents an application of multiway partial least squares (MPLS) methods to develop interpretative correlation models to monitor the foaming occurrence and improve batch fermentation. We choose the exhaust differential pressure as a quality variable to quantify the foaming occurrence and consider three‐dimensional datasets of different batches, process variables, and measurements. We integrate batch‐wise unfolding (BWU) and observation‐wise unfolding (OWU) of plant datasets with standard, dynamic, and kernel PLS methods. We find that dynamic PLS (DPLS) with OWU and time‐lagged quality variables to be the most efficient, accurate, and easy to implement. The BWU approach is useful for analyzing the differences between batches and identifying abnormalities and outliers, while the OWU quantifies the variation within a given batch. With OWU, the DPLS method with one unit of time lag in the quality variable is the most effective, accurate, and easy to implement. With both BWU and OWU, we identify the quantitative effects of process variables on the quality variable and providence guidance to improve fermentation performance.

Chemometric Tools in the Analysis of Pharmaceutics Samples: a Comparison Among Several Multivariate Calibration Methods

Bivariate calibration algorithm is compared with the results obtained by the usage of high-dimensional calibration methods such as partial least squares (PLS) and multi-way partial least-squares (N-PLS) by using UV-Vis spectrophotometric data of first and second-order. The algorithms were applied to the determination of a mixture of an analgesic and a stimulant compound and their actual concentrations of them were calculated by using spectroscopic data. The direct reading of absorbance values at 227 nm and 271 nm were employed for quantification of the compounds in the case of the bivariate method. The approaches of first-order and multi-way methods were applied with a previous optimization of the calibration matrix by constructing sets of calibration and validation with 20 and 10 samples (mixtures) respectively according to a central composite design and their UV absorption spectra were recorded at 200-350 nm. All algorithms were satisfactorily applied to the simultaneous determination of these compounds in pharmaceutical formulations with mean percentage recovery of 100.5 ± 3.67, 98.7 ± 3.42, and 100.5 ± 3.74 for bivariate, PLS-1, and N-PLS, respectively. The statistical evaluation of the bivariate method showed that this procedure is comparable with those algorithms that employ high-dimensional structured information. The aim of the work is to compare the methods under study and it can be seen that there are no significant differences, so a simple spectrophotometer can be used up to a very specialized one. However, the advantage of bivariate calibration is its simplicity, due to the minimal experimental manipulation.

Comparative study on wavelet functional partial least squares soft sensor for complex batch processes

Conventional data-driven models for batch processes conduct unfolding operations and neglect the continuous property. A novel soft sensor method is proposed based on the wavelet functional partial least squares (WFPLS). The three-dimensional (3D) batch data is transferred into an even two-dimensional function matrix no matter whether it has even-length or uneven-length. The orthogonal wavelet basis functions are selected by the proposed active algorithm to approximate the variables' trajectories. Two versions of WFPLS are developed to predict quality based on different assumptions of the loading function. By comparing one of the proposed methods and the classical multi-way partial least squares (MPLS), the root means square errors are relatively descended by 20.6566% and 63.9925%, R2 are improved by 2.5813% and 83.2518%, and the corresponding computation time of the proposed method is only 5.7% and 41% of the time used by MPLS in the numerical case and in the industrial case, respectively.

A novel unsupervised approach for batch process monitoring using deep learning

Process monitoring is an important tool used to ensure safe operation of a process plant and to maintain high quality of end products. The focus of this work is on unsupervised Statistical Process Control (SPC) of batch processes using Deep Learning (DL). A DL architecture referred as Multiway Partial Least Squares Autoencoder (MPLS-AE) is proposed and trained using a genetic optimization algorithm with a novel objective function that directly maximizes the average fault detection rate (FDR¯). The efficacy of the proposed method is demonstrated on an industrial scale Penicillin process. Comparisons of the proposed algorithm with linear Multiway Principal Component Analysis (MPCA) and Multiway Partial Least Squares (MPLS) based fault detection (FD) algorithm, trained with the same objective as used by the DL model, demonstrates the superiority of the deep learning based approach. The use of dynamic control limits significantly improves the detection rates for both the linear and DL models.

Simple and fast spectrophotometric method based on chemometrics for the measurement of multicomponent adsorption kinetics

AbstractUV–Vis spectrophotometry has been widely applied in the quantitative analysis of different analytes in many fields, but it still faces challenges in the analysis of multicomponent solutions due to strong spectral overlap and low selectivity. In this paper, UV–Vis spectrophotometric method combined with chemometrics algorithms was established for the simultaneous determination of liquid‐phase adsorption kinetics of multicomponent for the first time. Partial least squares (PLS), stepwise regression, and multiway PLS (N‐PLS) models were established by using the original spectrum and processed spectrum pretreated by two different methods. The prediction results were validated by high‐performance liquid chromatography (HPLC). N‐PLS model showed the best prediction ability and the prediction relative error was within 9%. Adsorption kinetics studies indicated that the pseudo‐second‐order model can describe the adsorption behavior of rutin and quercetin on AB‐8 resin more accurately. This method was fast, low cost, accurate and reliable, simple, and efficient for the determination of multicomponent liquid‐phase adsorption kinetics.

Second-order calibration in combination with fluorescence fibre-optic data modelling as a novel approach for monitoring the maturation stage of plums

In this work, non-destructive autofluorescence of plums was employed to study the chlorophylls’ concentration evolution along the maturation process. For that, excitation-emission matrices (EEMs), containing full fluorescence information, were collected with a fibre-optic, assembled to a spectrofluorometer. Data analysis was performed with several second-order multi-way algorithms, such as parallel factor analysis (PARAFAC), multi-way partial least-squares (N-PLS), unfolded partial least-squares (U-PLS), and multivariate curve resolution-alternating least-squares (MCR-ALS). Firstly, the EEMs of each plum, collected each week along the maturation process, were processed with PARAFAC. Two components were used to model the data and the excitation and emission loadings were obtained. Score values for the first PARAFAC component showed a clear evolution with time, increasing during the first five weeks, and decreasing for the last weeks. Also, the chlorophyll concentrations obtained by HPLC analysis, in the skin and the whole fruit, were compared with those obtained with different algorithms mentioned before. Best results were obtained in the case of skin for all algorithms. Similar correlation coefficients (r) were obtained in all cases (0.899 (PARAFAC); 0.940 (U-PLS); 0.936 (N-PLS) and 0.958 (MCR-ALS)). When the elliptical joint confidence region (EJCR), for the slope and intercept, were calculated, the theoretically expected values of 1 and 0, for the slope and intercept, respectively, were included in all ellipses. However, it was observed that for the skin data and U-PLS and N-PLS algorithms, the EJCR confidence region was smaller than in the other cases.

Non-destructive fluorescence spectroscopy combined with second-order calibration as a new strategy for the analysis of the illegal Sudan I dye in paprika powder

This paper presents a novel strategy for determination of the illegal dye Sudan I in paprika powder. The method is based on fluorescence spectroscopy combined with second-order calibration, which was employed for the first time for this purpose. The method is non-destructive and requires no sample preparation. It was probed that Sudan I exhibited fluorescence; however, the color of paprika samples affected the signal and it was not possible to quantify this adulterant by means of univariate and first-order calibration. To model the effect of variability of color in samples, a central composite experimental design was performed with varying ASTA (American Spices Trade Association) color values and Sudan I concentrations. Different second-order algorithms were tried for quantification. The best results for calibration and validation were obtained from Unfolded-Partial Least-Squares (U-PLS) and Multi-way Partial Least-Squares (N-PLS). The level of detection ranges were 0.4 – 3 mg/g and 0.5 – 3 mg/g for U-PLS and N-PLS, respectively. This was lower than other methods found in the literature.

Online prediction of quality‐related variables for batch processes using a sequential phase partition method

Batch processes inherently have multiple operation phases; different phases exhibit different characteristics. Hence, it is reasonable to partition the process into phases and build sub‐phase models for online quality prediction. To this end, a sequential phase partition method based on the information increment is proposed. To address the multiphase behaviours in batch processes, this work utilizes a new information increment index to capture the dynamic characteristics of batch processes along a time direction and divides the process into sub‐phases. Next, phase‐based multiway partial least squares (MPLS) models are built to model within‐phase characteristics and predict the quality‐related variables online. Information increment is able to exploit the process evolution by focusing on the changing variable correlations derived from two adjacent extend time slice. It directly utilizes the available process measurements of successful history batch processes without data transformation or dimensionality reduction. The method is sequential and can overcome the limits of some phase partition methods that may divide the samples with discontinuous time sequence but similar characteristics into the same phase. In addition, the information increment is capable of reflecting the change of the process intuitively with high computation efficiency. Advantages of the proposed method are illustrated by two case studies, a penicillin simulation platform and an industrial application of Escherichia coli (E. coli) fermentation, respectively.

Optimal selection of time resolution for batch data analysis. Part I: Predictive modeling

Soft sensors based on multiway partial least squares (MW‐PLS) are often used to estimate, in useful time, the end quality of batch processes, due to their ability to deal with high dimensional and noisy data. However, PLS and its variants only bring parsimony to the variables' mode. The time mode, which is the main source of complexity in MW‐PLS, remains unchanged. Parsimony on the time dimension can be achieved by manipulating the variables' resolution or granularity. In this article, we address the optimal selection of resolution for each individual batch variable, as an additional degree of freedom for maximizing the predictive performance of industrial soft sensors. The proposed methodology will conduct, simultaneously, the optimal selection of (1) variables, (2) resolutions, and (3) stages. At the end, a multiresolution PLS model (MR‐PLS) will be obtained, that optimally predicts the batch‐end quality within the class of all MW‐PLS approaches. © 2018 American Institute of Chemical Engineers AIChE J, 64: 3923–3933, 2018

Integrated B2B‐NMPC control strategy for batch/semibatch crystallization processes

The uncertainty in crystallization kinetics is of major concern in manufacturing processes, which can result in deterioration of most model‐based control strategies. In this study, uncertainties in crystallization kinetic parameters were characterized by Bayesian probability distributions. An integrated B2B‐NMPC control strategy was proposed to first update the kinetic parameters from batch to batch using a multiway partial least‐squares (MPLS) model, which described the variances of kinetic parameters from that of process variables and batch‐end product qualities. The process model with updated kinetic parameters was then incorporated into an NMPC design, the extended prediction self‐adaptive control (EPSAC), for online control of the final product qualities. Promising performance of the proposed integrated strategy was demonstrated in a simulated semibatch pH‐shift reactive crystallization process to handle major crystallization kinetic uncertainties of L‐glutamic acid, wherein smoother and faster convergences than the conventional B2B control were observed when process dynamics were shifted among three scenarios of kinetic uncertainties. © 2017 American Institute of Chemical Engineers AIChE J, 2017

Multivariate batch to batch optimisation of fermentation processes incorporating validity constraints

This paper presents an innovative optimisation technique, which utilises an adaptive Multiway Partial Least Squares (MPLS) model to track the dynamics of a batch process from one batch to the next. Utilising this model, an optimisation algorithm solves a quadratic cost function that identifies operating conditions for the subsequent batch that should increase yield. Hard constraints are shown to be required when solving the cost function to ensure that batch conditions do not vary too greatly from one batch to the next. Furthermore, validity constraints are imposed to prevent the PLS model from extrapolating significantly when determining new operating conditions. The capabilities of the proposed technique are illustrated through its application to two benchmark fermentation simulations, where its performance is shown to compare favourably with alternative batch-to-batch optimisation techniques.

Multivariate Approaches for Simultaneous Determination of Avanafil and Dapoxetine by UV Chemometrics and HPLC-QbD in Binary Mixtures and Pharmaceutical Product.

Multivariate UV-spectrophotometric methods and Quality by Design (QbD) HPLC are described for concurrent estimation of avanafil (AV) and dapoxetine (DP) in the binary mixture and in the dosage form. Chemometric methods have been developed, including classical least-squares, principal component regression, partial least-squares, and multiway partial least-squares. Analytical figures of merit, such as sensitivity, selectivity, analytical sensitivity, LOD, and LOQ were determined. QbD consists of three steps, starting with the screening approach to determine the critical process parameter and response variables. This is followed by understanding of factors and levels, and lastly the application of a Box-Behnken design containing four critical factors that affect the method. From an Ishikawa diagram and a risk assessment tool, four main factors were selected for optimization. Design optimization, statistical calculation, and final-condition optimization of all the reactions were Carried out. Twenty-five experiments were done, and a quadratic model was used for all response variables. Desirability plot, surface plot, design space, and three-dimensional plots were calculated. In the optimized condition, HPLC separation was achieved on Phenomenex Gemini C18 column (250 × 4.6 mm, 5 μm) using acetonitrile-buffer (ammonium acetate buffer at pH 3.7 with acetic acid) as a mobile phase at flow rate of 0.7 mL/min. Quantification was done at 239 nm, and temperature was set at 20°C. The developed methods were validated and successfully applied for simultaneous determination of AV and DP in the dosage form.

Quantitative Analysis of Active Pharmaceutical Ingredients (APIs) Using a Potentiometric Electronic Tongue in a SIA Flow System

AbstractAn advanced potentiometric electronic tongue and Sequential Injection Analysis (SIA) measurement system was applied for the quantitative analysis of mixtures containing three active pharmaceutical ingredients (APIs): acetaminophen, ascorbic acid and acetylsalicylic acid, in the presence of various amounts of caffeine as interferent. The flow‐through sensor array was composed of miniaturized classical ion‐selective electrodes based on plasticized PVC membranes containing only ion exchangers. Partial Least Squares (PLS) analysis of the steady‐state sensor array responses, measured in API mixtures prepared by the SIA system permitted a correct quantitative analysis of acetylsalicylic acid and ascorbic acid. Further optimization using multiway PLS fed by dynamic responses without additional feature extraction did not improve significantly the resolution of acetaminophen. Lastly, the chemometric treatment, involving the extraction of dynamic components of the transient response employing the Wavelet transform, the removal of less‐significant coefficients by means of Causal Index pruning and training of an Artificial Neural Network (ANN) with the selected coefficients, allowed the simultaneous determination of all the three studied APIs, while counterbalancing any interference due to caffeine.

Direct analysis in real time mass spectrometry, a process analytical technology tool for real-time process monitoring in botanical drug manufacturing

A promising process analytical technology (PAT) tool has been introduced for batch processes monitoring. Direct analysis in real time mass spectrometry (DART-MS), a means of rapid fingerprint analysis, was applied to a percolation process with multi-constituent substances for an anti-cancer botanical preparation. Fifteen batches were carried out, including ten normal operations and five abnormal batches with artificial variations. The obtained multivariate data were analyzed by a multi-way partial least squares (MPLS) model. Control trajectories were derived from eight normal batches, and the qualification was tested by R2 and Q2. Accuracy and diagnosis capability of the batch model were then validated by the remaining batches. Assisted with high performance liquid chromatography (HPLC) determination, process faults were explained by corresponding variable contributions. Furthermore, a batch level model was developed to compare and assess the model performance. The present study has demonstrated that DART-MS is very promising in process monitoring in botanical manufacturing. Compared with general PAT tools, DART-MS offers a particular account on effective compositions and can be potentially used to improve batch quality and process consistency of samples in complex matrices.

A LASSO-based batch process modeling and end-product quality prediction method

A least absolute shrinkage and selection operator (LASSO) based batch process modeling and end-product quality prediction method is developed in this paper, which overcomes the shortcomings of both multiway partial least squares (MPLS) and the phase-based methods. The proposed phase-LASSO approach models not only the phase characteristics but also the within-phase and between-phase time-dependent information. In the meantime, it automatically selects the critical-to-quality phases via LASSO-type regularization. As a result, phase-LASSO has good prediction accuracy and model interpretation. The effectiveness of the proposed method is illustrated by the case study on injection molding.

Quality relevant nonlinear batch process performance monitoring using a kernel based multiway non-Gaussian latent subspace projection approach

Multiway kernel partial least squares method (MKPLS) has recently been developed for monitoring the operational performance of nonlinear batch or semi-batch processes. It has strong capability to handle batch trajectories and nonlinear process dynamics, which cannot be effectively dealt with by traditional multiway partial least squares (MPLS) technique. However, MKPLS method may not be effective in capturing significant non-Gaussian features of batch processes because only the second-order statistics instead of higher-order statistics are taken into account in the underlying model. On the other hand, multiway kernel independent component analysis (MKICA) has been proposed for nonlinear batch process monitoring and fault detection. Different from MKPLS, MKICA can extract not only nonlinear but also non-Gaussian features through maximizing the higher-order statistic of negentropy instead of second-order statistic of covariance within the high-dimensional kernel space. Nevertheless, MKICA based process monitoring approaches may not be well suited in many batch processes because only process measurement variables are utilized while quality variables are not considered in the multivariate models. In this paper, a novel multiway kernel based quality relevant non-Gaussian latent subspace projection (MKQNGLSP) approach is proposed in order to monitor the operational performance of batch processes with nonlinear and non-Gaussian dynamics by combining measurement and quality variables. First, both process measurement and quality variables are projected onto high-dimensional nonlinear kernel feature spaces, respectively. Then, the multidimensional latent directions within kernel feature subspaces corresponding to measurement and quality variables are concurrently searched for so that the maximized mutual information between the measurement and quality spaces is obtained. The I2 and SPE monitoring indices within the extracted latent subspaces are further defined to capture batch process faults resulting in abnormal product quality. The proposed MKQNGLSP method is applied to a fed-batch penicillin fermentation process and the operational performance monitoring results demonstrate the superiority of the developed method as apposed to the MKPLS based process monitoring approach.

The Application of Nonlinear Partial Least Square to Batch Processes

Multivariate statistical process control (MSPC) techniques play an important role in industrial batch process monitoring and control. One particularly popular approach to MSPC is partial least squares (PLS), which has been successfully applied many times in the modelling, estimation and control of batch processes. However, the nonlinear nature of many real, complex chemical systems means that traditional linear PLS is not always suitable. In this paper, the use of a nonlinear multi-way PLS is proposed to address the issues of non-linearity in batch processes. By analysing and comparing linear multi-way PLS, Neural network multi-way PLS, Type I and Type II nonlinear multi-way PLS models, the advantages and limitations of these methods are identified and summarised.

Multiway Interval Partial Least Squares for Batch Process Performance Monitoring

The method of interval Partial Least Squares (iPLS) is combined with multiway partial least-squares (MPLS) to allow the building of enhanced statistical process performance monitoring models. A novel algorithm is proposed for segmenting batch duration, or spectral data in applications employing spectroscopy data into several subintervals for which independent PLS models can be constructed. The approach deviates from the method of using subintervals of equal length and the practice of choosing only a subset of these subintervals for prediction and/or monitoring. The proposed approach provides dramatic reduction in the number of subintervals required and subsequently the number of PLS models required to give improved prediction and monitoring performance. The proposed method, MiPLS, is applied to the well-known benchmark fed-batch penicillin production simulator, Pensim, for quality variable prediction and fault detection.

Four-way multivariate calibration using ultra-fast high-performance liquid chromatography with fluorescence excitation–emission detection. Application to the direct analysis of chlorophylls a and b and pheophytins a and b in olive oils

A four-way multivariate calibration approach, based on the combination of ultra-fasthigh-performance liquid chromatographic data and four-way algorithms, is described for the first time. To achieve this goal, several emission wavelength–elution time matrices (ETMs) were recorded as a function of the excitation wavelength. Each sample was injected into the chromatograph eight times, in sequential mode, each time exciting at a different wavelength across the excitation spectra of the compounds of interest, and the emission spectra were recorded along the full chromatogram using a fast scanning fluorescence detector. The data were obtained in a very short time with an ultrafast chromatographic system operating in gradient mode. The three-wayETM data thus obtained for the calibration sample set and for each of the test samples were joined into a single four-way array, which was subsequently analyzed with parallel factor analysis (PARAFAC), unfolded partial least-squares with residual trilinearization (U-PLS/RTL) and multi-way partial least-squares with residual trilinearization (N-PLS/RTL) multivariate calibration algorithms. Best results were found when either U-PLS/RTL or N-PLS/RTL algorithms were used to perform the multivariate calibration. The method was applied to the direct determination of chlorophylls a and b and pheophytins a and b in olive oil samples, in the presence of other interfering fluorescent compounds, and without previous sample treatment.

Evolution of PLS for Modeling SAR and omics Data.

In quantitative structure-activity relationship (QSAR), multivariate statistical methods are commonly used for data analysis. Partial least squares (PLS) is of particular interest because it can analyze data with strongly collinear, noisy and numerous X variables, and also simultaneously models several activity variables Y. PLS provides several prediction regions and diagnostic plots as statistical measures. PLS has evolved for coping with the severe demands imposed by complex data structures. In this review article, we outline the algorithms of five advanced PLS techniques and provide some representative examples of each. The selected models are Nonlinear PLS, Multiway PLS, Hierarchical PLS, Orthogonal PLS, and Bi-modal PLS. Studies of particular aspects of living cells (such as the set of genes or proteins in the cell and their interactions) are collectively known as the -omics fields. Omics integrate heterogeneous scientific disciplines and include chemogenomics, proteomics, and metabolomics. The datasets produced within the omics fields are numerous, megavariate and extremely complex. The data structures are frequently incomplete, noisy, nonlinear and collinear demanding modern and powerful multivariate data analysis methods. In particular, the omics technologies have steered biology towards the adoption of orthogonal PLS. We also describe future prospects for the use of PLS algorithms in the omics fields.