Maximum Likelihood Principal Component Analysis Research Articles

Exploring the scores: Procrustes analysis for comprehensive exploration of multivariate data

Exploratory data analysis (EDA) through the projection of multivariate data into spaces of low dimensionality using methods such as principal components analysis (PCA) are at the core of chemometric applications in many fields, including metabolomics, biomarker discovery, food authentication, and many others. In addition to revealing underlying class structures in the data, unsupervised EDA methods have become a de facto method of confirmatory analysis for object classification (e.g., by health status, provenance), especially for small sample sizes, because they are not plagued with problems of overfitting that often accompany supervised methods. However, the characteristics of the scores plots for EDA projection methods are often highly dependent on data analysis options chosen, such as the type of preprocessing used, the projection method employed, the variables selected and (in the case of multiblock data) how the data are combined. The combinations of these parameters can lead to hundreds of different scores plots that need to be manually assessed for results that are interesting to the researcher. The present work is intended to expedite this process through a relational analysis of multiple results using Procrustes analysis to compare projections and applying hierarchical clustering to summarize the results in the form of a dendrogram. The software developed, ScorXplor, allows projections to be quickly assessed for their similarity and quality, with interactive plotting of scores plots for visual evaluation. Moreover, the approach provides a better understanding of the role and relationships among the various analysis options (preprocessing, analysis tools, etc.). The method is demonstrated using multiblock spectral data (UV–visible, near-infrared, mid-infrared) for flavored olive oils from different regions of Italy, implementing different preprocessing and fusion options, and applying PCA and maximum likelihood PCA as projection methods.

Practical data-driven modeling and robust predictive control of mammalian cell fed-batch process

Even if the performances of bioprocesses can be significantly improved by model-based control, there often remains a tradeoff between model complexity and control robustness. This paper proposes an original data-driven strategy for fast design of dynamic bioprocess models with minimal complexity (i.e., minimal number of bioreactions). Maximum likelihood principal component analysis (MLPCA) is applied to infer the minimal reaction scheme from a 25-state mammalian cell culture database. Then, a systematic algorithm is used to provide a continuous kinetic model formulation assuming all rates to occur simultaneously, which may be far from true cell metabolic conditions sometimes presenting discontinuous metabolic switches. A robust model predictive formulation is therefore adopted to reduce the impact of model structural uncertainty on the process performances. Additional numerical results show that the proposed strategy presents excellent performances in presence of unexpected metabolic switches.

On the Number of Reactions and Stoichiometry of Bioprocess Macroscopic Models: an Implicit Sparse Identification Approach

This paper presents a data-driven methodology to infer a macroscopic reaction scheme with stoichiometric parameters from a bioprocess database. The data sets consist of measurements of a few extracellular species, i.e., biomass, substrates, and products of interest. The proposed original procedure is based on implicit sparse identification. The methodology is illustrated with two case studies: (i) data generated by a two-step anaerobic digestion model and (ii) an experimental data set from the production of therapeutic proteins using mammalian cell cultures. Finally, the results of the latter application are compared with a standard data-driven algorithm, e.g., maximum-likelihood principal component analysis.

A systematic study on the effect of different error structures on pseudo‐univariate and multivariate figures of merit

AbstractIn this study, the effect of different error structures on psedounivariate and multivariate analytical figures of merit in simulated data of hyphenated chromatographic systems was investigated. Different error structures (e.g., homoscedastic, heteroscedastic, and correlated) were investigated. For this purpose, five components systems at five concentration levels with three replicates were simulated. Different types of error were added to the data. Multivariate Curve Resolution‐Alternating Least Squares (MCR‐ALS) and Maximum Likelihood Principal Component Analysis (MLPCA‐MCR‐ALS) methods were used. After resolution, pseudo‐univariate and multivariate analytical figures of merit were calculated and compared. As it expected, the detection limit for noisy datasets is higher than the noise‐free datasets, whereas the slopes of the calibration curves are not significantly different. Comparing the results generally showed that the detection limit values in multivariable mode were better than the univariate mode. The LODs of data (pseudo‐univariate and multivariate) with homoscedastic and correlated error structure by MCR‐ALS and MLPCA‐MCR‐ALS were the same. The analysis of data with heteroscedastic error structure by MLPCA‐MCR‐ALS had a lower detection limit than analysis with MCR‐ALS. The figures of merit obtained from WLS and OLS regression in heteroscedastic datasets were compared and better LODs were obtained after WLS method.

Determination of ingredients in packaged pharmaceutical tablets by energy dispersive X‐ray diffraction and maximum likelihood principal component analysis multivariate curve resolution‐alternating least squares with correlation constraint

AbstractEnergy dispersive X‐ray diffraction (EDXRD) and maximum likelihood principal component analysis multivariate curve resolution‐alternating least squares (MLPCA‐MCR‐ALS) with correlation constraint were used to quantify the composition of packaged pharmaceutical formulations. Recorded EDXRD profiles from unpackaged and packaged samples of ternary mixtures were modelled together in order to recover the concentrations as well as the pure profiles of the constituent compounds. MLPCA was used as a data pretreatment step to MCR‐ALS, accounting for the high noise and nonconstant variance observed in the EDXRD profiles and was shown to improve the resolution accuracy of MCR‐ALS for the data set. Local correlation constraints were applied in the MCR‐ALS procedure in order to model unpackaged and packaged samples simultaneously while accounting for the matrix effect of the packaging materials. The composition of the formulations was estimated with root‐mean‐square error of prediction for each component, including paracetamol, being approximately 2.5 %w/w for unpackaged and packaged samples. Paracetamol concentration was resolved simultaneously for the unpackaged and packaged samples to a greater degree of accuracy than achieved by partial least squares regression (PLSR) when modelling the contexts separately. By modelling the effects of the packaging and incorporating accurate reference information of unpackaged samples into the resolution of packaged samples, the potential of EDXRD and MLPCA‐MCR‐ALS for the identification and quantification of packaged solid‐dosage medicine in nondestructive screening and counterfeit medicine detection has been raised.

Beyond principal components: a critical comparison of factor analysis methods for subspace modelling in chemistry.

Multivariate data analysis tools have become an integral part of modern analytical chemistry, and principal component analysis (PCA) is perhaps foremost among these. PCA is central in approaching many problems in data exploration, classification, calibration, modelling, and curve resolution. However, PCA is only one form of a broader group of factor analysis (FA) methods that are rarely employed by chemists. The dominance of PCA in chemistry is primarily a consequence of history and convenience, but this has obscured the potential advantages of other FA tools that are widely used in other fields. The purpose of this article, which is intended for those who are already familiar with the mathematical foundations and applications of PCA, is to develop a framework to relate PCA to other commonly used FA methods from the perspective of chemical applications. Specifically, PCA is compared to maximum likelihood factor analysis (MLFA), principal axis factorization (PAF) and maximum likelihood PCA (MLPCA). Similarities and differences are highlighted with regard to the assumptions and constraints of the models, algorithms employed, and calculation of scores and loadings. Practical aspects such as data dimensionality, preprocessing, rank estimation, improper solutions (Heywood cases), and software implementation are considered. The performance of the four methods is compared using both simulated and experimental data sets. While PCA provides the most reliable estimates when measurement error variance is uniform (homoscedastic noise) and MLPCA works best when the error covariance matrix is explicitly known, MLFA and PAF have the distinct advantage of providing information about measurement uncertainty and adapting to situations of unknown heteroscedastic errors, eliminating the need for scaling. Moreover, MLFA in particular is shown to be tolerant to deviations from model linearity. These results make a strong case for increased application of other FA methods in chemistry.

Balanced scaling as a pretreatment step in Multivariate Curve Resolution analysis of noisy data

Analysis of data sets with heteroscedastic error has been a challenging problem in the chemometrics literature. Different methods have been proposed for analyzing this type of data, in particular, using the Multivariate Curve Resolution Alternating Least Squares (MCR-ALS) method. The present paper introduces the Balanced Scaling (BS) approach as a pretreatment step combined with the Multivariate Curve Resolution Alternating Least Squares (BS-MCR-ALS) method as an adequate procedure to analyze data with heteroscedastic noise. In particular, for the analysis of environmental data, the Balanced Scaling (BS) method can be a useful approach to provide an optimal individual data scaling. The performance of the BS-MCR-ALS method is compared with the performance of the Maximum Likelihood Principal Component Analysis Multivariate Curve Resolution Alternating Least Squares (MLPCA-MCR-ALS) method, and also with the performance of the traditional Multivariate Curve Resolution Alternating Least Squares (MCR-ALS) method in the analysis of data sets with different type of error structures. The results obtained in this comparison revealed that the solutions obtained by BS-MCR-ALS and MLPCA-MCR-ALS were very similar.

Data-driven modeling and optimal control of the production of Fructo-Oligosaccharides by Aureobasidium Pullulans

The first objective of this study is to derive a macroscopic dynamic model of the production of Fructo-Oligosaccharides (FOS) by Aureobasidium pullulans based on sets of experimental data collected from batch and fed-batch cultures. The model should be of low dimension, so as to be identifiable based on the available data, and so as to be suitable for optimization and control purposes. A maximum likelihood principal component analysis is used to determine the appropriate number of reactions and the corresponding stoichiometry. Further, products of Monod factors are chosen to describe the reaction kinetics. The model parameters are estimated using a weighted least-squares method, and model simplification achieved by eliminating parameters associated to large uncertainties, are performed in a step-by-step, systematic way. In addition, the model structural identifiability is confirmed using generating series and the software GenSSI. Identification is successfully achieved, leading to satisfactory direct and cross-validation results. The second objective is to exploit the model and to maximize the FOS concentration at an a priori undetermined time using Pontryagin maximum principle. The optimal feed rate is in the form of a bang-bang control, which is easily implemented in practice.

Simulation of 1/fα noise for analytical measurements

A simple procedure is described that can be used to generate 1/fα noise, also known as power law noise, in simulated analytical measurement vectors. Certain types of power law noise, such as pink noise (α=1), dominate many types of analytical signals, so its simulation is important in optimizing data processing strategies. In this work, simulated 1/fα error sequences are created directly from white noise via the theoretical measurement error covariance matrix (ECM) by rotation and scaling. The 1/fα ECM is obtained from the coefficients of a finite impulse response filter and is easily adapted to generate multiplicative 1/fα noise that is probably more common for analytical systems exhibiting proportional noise characteristics. Simulating 1/fα noise directly from the ECM offers two main advantages. First, 1/fα noise can be easily simulated in the presence of other common analytical measurement errors by additive combination of the ECMs. Second, the theoretical ECM can be used to model real experimental measurement noise. It is shown that the power spectral density function of measurement error sequences generated by the proposed method closely approximates the theoretical behaviour of 1/fα noise. To demonstrate the utility of this method in evaluating data processing methods, simulated data exhibiting 1/f (pink) noise is analyzed by maximum likelihood principal component analysis (MLPCA) that takes measurement error structure into account, and baseline noise is simulated using brown noise to test baseline fitting by asymmetric least squares (AsLS).

Implications of measurement error structure on the visualization of multivariate chemical data: hazards and alternatives

The analysis of multivariate chemical data is commonplace in fields ranging from metabolomics to forensic classification. Many of these studies rely on exploratory visualization methods that represent the multidimensional data in spaces of lower dimensionality, such as hierarchical cluster analysis (HCA) or principal components analysis (PCA). However, such methods rely on assumptions of independent measurement errors with uniform variance and can fail to reveal important information when these assumptions are violated, as they often are for chemical data. This work demonstrates how two alternative methods, maximum likelihood principal components analysis (MLPCA) and projection pursuit analysis (PPA), can reveal chemical information hidden from more traditional techniques. Experimental data to compare different methods consists of near-infrared (NIR) reflectance spectra from 108 samples of wood that are derived from four different species of Brazilian trees. The measurement error characteristics of the spectra are examined and it is shown that, by incorporating measurement error information into the data analysis (through MLPCA) or using alternative projection criteria (i.e., PPA), samples can be separated by species. These techniques are proposed as powerful tools for multivariate data analysis in chemistry.

Identification and optimal control of fructo-oligosaccharide production

This article deals with the development of a reduced model describing the temporal evolution of the fructo-oligosaccharide production by Aureobasidium pullulans and the optimal process control. First, a reduced model is derived from a detailed model reproducing with good accuracy the dynamics of the fructo-oligosaccharide production. The reduced model is obtained using maximum likelihood principal component analysis and parameter identification based on a weighted least squares criterion. Next, the fructo-oligosaccharide concentration at an a priori undetermined time is maximized using Pontryagin maximum principle. The methodology is analyzed based on experimental data from batch and fed-batch cultures, and results are compared with those obtained with another simple model available in the literature.

Dynamic modeling of hybridoma cell cultures using maximum likelihood principal component analysis

RésuméHybridoma cells are commonly grown for the production of monoclonal antibodies (MAb). For monitoring and control purposes of the bioreactors, dynamic models of the cultures are required. However these models are difficult to infer from the usually available experimental data. This paper explores an experimental case study where hybridoma cells are grown in a sequential batch reactor. A simple macroscopic reaction scheme is first derived using a maximum likelihood principal component analysis. Subsequently, nonlinear least-squares estimation is used to identify the kinetic parameters. The resulting dynamic model reproduces quite satisfactorily the experimental data, as evidenced in direct validation tests. However, following a parametric sensitivity analysis, this model appears over-parameterized. A simplified model is therefore proposed, offering comparable direct validation results and very good cross-validation results. The procedure proves systematic and could be applied to other case studies.

Receptor modeling of environmental aerosol data using MLPCA-MCR-ALS

Receptor models apportion the measured mass of an ambient particulate matter (PM) sample at a given site, called the receptor, to its emission sources by using multivariate factor analysis. In this work, a general workflow for PM data quality assessment, measurements uncertainty calculations, receptor modeling and error in model parameters estimation is proposed. Maximum likelihood principal component analysis - multivariate curve resolution–alternating least squares (MLPCA-MCR-ALS) is proposed for general bilinear receptor modeling of noisy environmental datasets and compared with other approaches used in the field. Equations to obtain PM-source contribution estimates (PM-SCE) and contribution-to-species to identify emission sources and their attributes are proposed. Propagation of experimental uncertainties in the parameters of the receptor model is obtained using extensive computer resampling methods. Results are shown for source apportionment of a particulate matter (PM10) air monitoring data set obtained under Fairmode-WG3 project.

Calibration transfer between NIR spectrometers: New proposals and a comparative study

Calibration transfer between near‐infrared (NIR) spectrometers is a subtle issue in chemometrics and process industry. In fact, as even very similar instruments may generate strongly different spectral responses, regression models developed on a first NIR system can rarely be used with spectra collected by a second apparatus. In this work, two novel methods to perform calibration transfer between NIR spectrometers are proposed. Both of them permit to exploit the specific relationships between instruments for imputing new unmeasured spectra, which will be then resorted to for building an improved predictive model, suitable for the analysis of future incoming data. Specifically, the two approaches are based on trimmed scores regression and joint‐Y partial least squares regression, respectively. The performance of these novel strategies will be assessed and compared to that of well‐established techniques such as maximum likelihood principal component analysis and piecewise direct standardisation in two real case studies.

Macroscopic Dynamic Modeling of Sequential Batch Cultures of Hybridoma Cells: An Experimental Validation.

Hybridoma cells are commonly grown for the production of monoclonal antibodies (MAb). For monitoring and control purposes of the bioreactors, dynamic models of the cultures are required. However these models are difficult to infer from the usually limited amount of available experimental data and do not focus on target protein production optimization. This paper explores an experimental case study where hybridoma cells are grown in a sequential batch reactor. The simplest macroscopic reaction scheme translating the data is first derived using a maximum likelihood principal component analysis. Subsequently, nonlinear least-squares estimation is used to determine the kinetic laws. The resulting dynamic model reproduces quite satisfactorily the experimental data, as evidenced in direct and cross-validation tests. Furthermore, model predictions can also be used to predict optimal medium renewal time and composition.

Assessment of maximum likelihood PCA missing data imputation

Maximum likelihood principal component analysis (MLPCA) was originally proposed to incorporate measurement error variance information in principal component analysis (PCA) models. MLPCA can be used to fit PCA models in the presence of missing data, simply by assigning very large variances to the non‐measured values. An assessment of maximum likelihood missing data imputation is performed in this paper, analysing the algorithm of MLPCA and adapting several methods for PCA model building with missing data to its maximum likelihood version. In this way, known data regression (KDR), KDR with principal component regression (PCR), KDR with partial least squares regression (PLS) and trimmed scores regression (TSR) methods are implemented within the MLPCA method to work as different imputation steps. Six data sets are analysed using several percentages of missing data, comparing the performance of the original algorithm, and its adapted regression‐based methods, with other state‐of‐the‐art methods. Copyright © 2016 John Wiley & Sons, Ltd.

Contamination characteristics and source apportionment of trace metals in soils around Miyun Reservoir.

Due to their toxicity and bioaccumulation, trace metals in soils can result in a wide range of toxic effects on animals, plants, microbes, and even humans. Recognizing the contamination characteristics of soil metals and especially apportioning their potential sources are the necessary preconditions for pollution prevention and control. Over the past decades, several receptor models have been developed for source apportionment. Among them, positive matrix factorization (PMF) has gained popularity and was recommended by the US Environmental Protection Agency as a general modeling tool. In this study, an extended chemometrics model, multivariate curve resolution-alternating least squares based on maximum likelihood principal component analysis (MCR-ALS/MLPCA), was proposed for source apportionment of soil metals and applied to identify the potential sources of trace metals in soils around Miyun Reservoir. Similar to PMF, the MCR-ALS/MLPCA model can incorporate measurement error information and non-negativity constraints in its calculation procedures. Model validation with synthetic dataset suggested that the MCR-ALS/MLPCA could extract acceptable recovered source profiles even considering relatively larger error levels. When applying to identify the sources of trace metals in soils around Miyun Reservoir, the MCR-ALS/MLPCA model obtained the highly similar profiles with PMF. On the other hand, the assessment results of contamination status showed that the soils around reservoir were polluted by trace metals in slightly moderate degree but potentially posed acceptable risks to the public. Mining activities, fertilizers and agrochemicals, and atmospheric deposition were identified as the potential anthropogenic sources with contributions of 24.8, 14.6, and 13.3%, respectively. In order to protect the drinking water source of Beijing, special attention should be paid to the metal inputs to soils from mining and agricultural activities.

Performance and validation of MCR-ALS with quadrilinear constraint in the analysis of noisy datasets

This study explores the effect of noise propagation on the resolution capability of multivariate curve resolution-alternating least squares with a recently developed quadrilinearity constraint (MCR-ALSQ). To investigate the effect of application of the quadrilinearity constraint, four environmental profiles were simulated and three types of noise viz. homoscedastic, heteroscedastic, and constant-proportional noise at three different levels were added to the simulated dataset. The profiles recovered with MCR-ALSQ were compared with the ones recovered by bilinear MCR-ALS (MCR-ALSB). The effect of maximum likelihood principal component analysis (MLPCA) as a pre-processing step in MCR-ALSQ (MLPCA–MCR-ALSQ), and MCR-ALSB (MLPCA–MCR-ALSB) analysis was also studied and results were compared with ones obtained with MCR-ALSQ and MCR-ALSB models. The recovery and similarity of the resolved profiles with theoretical ones were assessed in terms of similarity coefficient (r2) and similarity angle (θ). The results of this study conclude that MCR-ALSQ is appropriate to analyze four-way quadrilinear datasets, and that the use of MLPCA as a pre-processing step before MCR-ALSQ improves the resolution profiles to a great extent even in the presence of high levels of noise (heteroscedastic, and constant-proportional).

Practical approaches to principal component analysis for simultaneously dealing with missing and censored elements in chemical data

Multivariate chemical data often contain elements that are missing completely at random and the so-called left-censored elements whose values are only known to be below a definite threshold value (reporting limit). In the last several years, attention has been paid to developing methods for dealing with data containing missing elements and those that can handle data with missing elements and outliers. However, processing data with both missing and left-censored elements is still an ongoing problem.The aim of this work was to investigate which method is most suitable for handling left-censored and missing completely at random elements that are present simultaneously in chemical data by using a comparison of the generalized nonlinear iterative partial least squares (NIPALS11NIPALS – nonlinear iterative partial least squares.) algorithm that has been recently proposed, methods that include uncertainty information like maximum likelihood principal component analysis, MLPCA22MLPCA – maximum likelihood principal component analysis., and replacement methods.The results of the Monte Carlo simulation study for artificial and real data sets showed that substitution with half of the reporting limit can be used when the percentage of left-censored elements per variable is up to 30–40%. The generalized NIPALS algorithm is generally recommended for a large percentage of left-censored elements per variable and particularly when a large number of variables are censored. The expectation-maximization approach applied to data with censored elements substituted with half of the reporting limits can be a strategy for dealing with missing and left-censored elements in data, but if the converge criterion is not fulfilled, then the generalized NIPALS algorithm can be applied.

On the derivation of simple dynamic models of anaerobic digestion using macroscopic bioreaction schemes

In this study, the problem of developing simple dynamic models of an anaerobic digestion process is tackled using an identification procedure which proceeds in several consecutive steps. Starting from sets of experimental data describing the time evolution of several key component concentrations, i.e. biomass(es), substrates and products, the minimal number of macroscopic bioreactions required to represent the data at hand, as well as the parameters of the associated stoichiometry matrix, are determined using maximum likelihood principal component analysis. Then, the structure of the kinetic laws, together with their parameters, are identified using likelihood ratio tests to navigate through the branches of decision trees made of various kinetic structures. The effectiveness of the modelling procedure is illustrated with a simulated example of anaerobic digestion. As usual sensors only provide aggregate measurements of the component concentration, a deeper investigation of practical identifiability is achieved in relation with the influence of measurement errors.