Multivariate Classification Methods Research Articles

Contaminant detection in pistachio nuts by different classification methods applied to short-wave infrared hyperspectral images

In this paper different multivariate classification methods applied to hyperspectral images acquired in the short-wave infrared range (SWIR: 1000–2500 nm) were evaluated for quality control of pistachio nuts. In more detail, the detection of contaminants in edible pistachio products was assessed. Six material classes (i.e. edible and inedible pistachio nuts, pistachio shells, pistachio husks, twigs, and stones) were investigated. Samples were divided into two groups: a training set and a validation set. The acquired hyperspectral images were first explored by Principal Component Analysis (PCA). The following multivariate classification methods were selected in order to verify and compare their efficiency and robustness: Partial Least Squares-Discriminant Analysis (PLS-DA), Principal Component Analysis with Discriminant Analysis (PCA-DA), Principal Component Analysis with k-Nearest Neighbor (PCA-kNN), and Classification And Regression Tree (CART). The classification results obtained for the four models in terms of prediction maps and values of the performance parameters ( Sensitivity , Specificity, and Efficiency ) were good in most cases. The main misclassification errors occur between ‘Edible pistachio’ and ‘Inedible pistachio’ classes and between ‘Pistachio shell’, ‘Pistachio husk’, and ‘Twig’ classes, confirming the similar spectral features of such classes observed by PCA. PCA-kNN provided the best discrimination ability in prediction, with Efficiency values from 0.92 to 0.99, followed by PLS-DA and PCA-DA. The performance of CART decreased from the calibration to the validation phase. The overall results demonstrated that SWIR-HSI technology, coupled with multivariate analysis modeling, is a promising approach to develop both ‘off-line’ and ‘on-line’ fast, reliable and robust contaminant detection in edible pistachios. • Pistachio quality control by short-wave infrared hyperspectral imaging. • Chemometric strategies applied to hyperspectral images for contaminant detection in pistachio nuts. • Detection of edible pistachio and different contaminants in automated and non-destructive way. • Comparison of different classification models for pistachios and contaminants identification. • Identification of a robust and efficient strategy for quality control of edible pistachio.

Classification of pulse flours using near-infrared hyperspectral imaging

With increasing consumer interest in healthy food formulations made from pulse flours, the research gap in characterizing them based on their pulse type and milling technique has come to limelight. To this end, the feasibility of using hyperspectral imaging in the visible near infrared (Vis-NIR) (400–1000 nm) and short wave infrared (SWIR) (1000–2500 nm) regions to classify pulse flours (viz. chickpea, yellow pea, navy bean and green lentil) based on the pulse type and milling methods was investigated. Unsupervised and supervised classification models were developed using unsupervised principal component analysis (PCA) and supervised partial least squares discriminant analysis (PLS-DA). Supervised classification for pulse flour type demonstrated 100% accuracy in the Vis-NIR wavelength range of 530–700 nm, which primarily exploited the color attributes of the flour samples. For milling method based classification, PLS-DA models developed using SWIR regions of 1370–1500 nm and 1700–2000 nm played a significant role in discriminating flour samples yielding 95% classification accuracy. These regions are associated with the O–H and N–H overtones of the proteins found in flour samples. Conclusively, hyperspectral imaging in the range of 400–2500 nm combined with multivariate data classification methods can reliably be used by the food industry for characterization of pulse flours.

Comparison of four classification statistical methods for characterising virgin olive oil quality during storage up to 18 months

This study examines the ability of fluorescence spectroscopy for monitoring the quality of 70 Moroccan virgin olive oils belonging to three varieties and originating from three regions of Morocco. By applying principal component analysis and factorial discriminant analysis to the emission spectra acquired after excitation wavelengths set at 270, 290, and 430 nm, a clear differentiation between samples according to their storage time was observed. The obtained results were confirmed following the application of four multivariate classification methods: partial least squares regression, principal component regression, support vector machine, and multiple linear regression on the emission spectra. The best prediction model of storage time was obtained by applying partial least squares regression since a coefficient of determination (R2) and a root mean square error of prediction (RMSEP) of 0.98 and 24.85 days were observed, respectively. The prediction of the chemical parameters allowed to obtain excellent validation models with R2 ranging between 0.98 and 0.99 for free acidity, peroxide value, chlorophyll level, k232, and k270.

Raman spectroscopic‐based chemometric modeling in assessment of red pepper ripening phases and carotenoids accumulation

AbstractThe main goal of the present study is validation of different chemometric models in Raman spectroscopic monitoring of different maturity phases of the red pepper fruit. Successive ripening stages commonly corresponding with different fruit coloration (green, green‐brown, brown‐red, and deep red) were assessed. The fruit maturation process in red pepper is related to alteration in composition and content of different primary and secondary metabolites, including the most represented carotenoid, the capsanthin. Raman microspectroscopy with wavelength of 532 nm was used to obtain the spectra of the pericarp of different maturation phases. Several multivariate classification methods, such as Principal Component Analysis–Linear Discriminant Analysis (PCA–LDA), Partial Least Squares‐Discriminant Analysis (PLS‐DA), and Soft Independent Modeling of Class Analogy (SIMCA) were tested to determine the model which best fits the target ripening phases related to composition and the content of the most represented carotenoids. Concerning different fruit ripening phases, several characteristic bands were obtained, including those at 1514–1517, 1151, and 1003 cm−1, all assigned to carotenoids with nine conjugated double bonds in the main polyene chain (e.g., lutein, beta carotene, beta cryptoxanthin, zeaxanthin, capsorubin, and capsanthin). All tested classification chemometric models had a high rate of prediction accuracy (between 95% and 100%). The SIMCA showed the best result, probably because of using a different algorithm compared with the other tested models.

Rapid determination and classification of zeolites based on Si/Al ratio using FTIR spectroscopy and chemometrics

The aim of this study was to develop fast and facile methods for the determination and classification of zeolite samples based on Si/Al ratio by fourier transform infrared spectroscopy (FTIR). The ZSM-5, ZSM-48 and mordenite catalysts spectral data were used for estimation of Si/Al ratio using two approaches according to PLS-R and SVM-R algorithms in whole (600–4000 cm−1) and fingerprint (3000–3800 cm−1) spectral regions. The drift-FTIR spectral data were also analyzed by classification method using the soft independent modelling of class analogies (SIMCA) for mordenite, ZSM-5 and ZSM-48 catalysts classification. Performance of the regression models was adequate with good statistical results. The correlation coefficients (R2) were about 0.99 and 0.96 for SVM and PLS regression models. The results obtained for the SVM model of the MIR spectra was very good and encouraging, since the RMSEP was low (6%) and the quantitative results were in accordance with X-ray fluorescence analysis. Then the samples were classified into three mordenite, ZSM-5 and ZSM-48 classes based on their Si/Al ratio by the SIMCA method using raw and pre-treated spectra. Using SIMCA technique, the 100% of samples were correctly classified in pre-treated spectra. The accurate and stable model was obtained with pre-treated spectra for SIMCA model (100% of sensitivity and 100% of specificity). The results indicated that drift-FTIR spectroscopy in conjunction with multivariate calibration and classification methods is a fast and reliable methodology that can be easily implemented for determination and classification of zeolite catalysts based on their Si/Al ratio.

Multivariate times series classification through an interpretable representation

Multivariate time series classification is a machine learning task with increasing importance due to the proliferation of information sources in different domains (economy, health, energy, crops, etc.). Univariate methods lack the ability to capture the relationships between the different variables that compose a multivariate time series and therefore cannot be directly extrapolated to multivariate environments. Despite the good performance and competitive results of the multivariate proposals published to date, they are hard to interpret due to their high complexity. In this paper, we propose a multivariate time series classification method based on an alternative representation of the time series, composed of a set of 41 descriptive time series features, in order to improve the interpretability of time series and results obtained. Our proposal uses traditional classifiers over the extracted features to look for relationships between the different variables that form a multivariate time series. We have selected four state-of-the-art algorithms as base classifiers to evaluate our method. We have tested our proposal on the complete University of East Anglia repository, obtaining highly interpretable results capable of explaining the relationships between the features that compose the time series and achieving performance results statistically indistinguishable from the best algorithms of the state-of-the-art.

Profiling of transcriptional biomarkers in FFPE liver samples: PLS-DA applications for detection of illicit administration of sex steroids and clenbuterol in veal calves

Administering growth promoters to meat-producing animals is strictly regulated within the European Community to preserve consumers’ safety. Nevertheless, illicit misuse of steroids and other veterinary drugs to increase animal production has still been found by National Residues Control Plans (NRCPs), mainly based on expensive targeted multi-class/multi-residue analytical methods, used for confirming quantification analysis of these residues in tissues and biological fluids. The setup of complementary novel diagnostic methods, based on indirect biomarkers of exposure to anabolic substances is, therefore, needed to update current tests available in NRCPs at the screening level, where cheaper, sensitive and untargeted methods are required. Biological effects of illicit treatment transcriptomic analysis has reached a good grade of standardization, identified by microarray and RNAseq technologies of large panels of differentially expressed gene targets (DEG) related to abuse of different classes of anabolic molecules (e.g. sex steroids, thyreostats, β2-agonists, glucocorticoids, etc.). The aim of the work was to identify a minimum set of molecular markers able to detect illicit administration of two of the most recurrent classes of growth promoters (sex steroids and B2-agonists) on formalin-fixed paraffin-embedded (FFPE) tissues, being a potentially straightforward sampling strategy at the slaughterhouse level. A PCR array to allow quantitative profiling of 48 genetic targets was, therefore, developed on FFPE liver samples from veal calves. Conventional univariate and alternative approaches based on multivariate classification methods were considered for gene expression profiling of 92 samples collected from different animal trials, where effects of nandrolone, estradiol, their combination, and clenbuterol were studied.The developed Partial Least Squares – Discriminant Analysis (PLS-DA) models show the successful setup of a novel screening strategy based on multiple transcriptional biomarkers, able to expand tested sample sizes, and consequently strengthen current NRCPs.

Multivariate classification of provinces of Vietnam according to the level of sustainable development

Abstract The research aims to classify the level of sustainability of 63 provinces in Vietnam upon 24 indicators reflecting three main dimensions of sustainable development by using multivariate classification method for the year 2014–2016. First of all, the principal component analysis (PCA) was applied to group quantitative variables that reflect important aspects of each component of sustainable development of localities in Vietnam into a number of limited dimensions (factors). The results of PCA illustrate 8 principal components in which 3 main components of economic and social pillar, and 2 main components for environmental pillar. After that, the second method was applied by using the hierarchical methods of cluster analysis for the set of 8 principal components conducted by PCA. With the candidate solution divided all of the provinces into 6 clusters, the analysis shows that the higher level of sustainable development belongs to South East, Red River Delta, and North Central, in which 4 provinces of South East (Ho Chi Minh City, Binh Duong, Dong Nai, and Ba Ria – Vung Tau) form a group with the highest level of sustainability. The regions such as South Central Coastal, Highland area, and Mekong River Delta have the medium level; the North midland and mountain areas are regions at the low level of sustainability.

Intrusion detection of cyber physical energy system based on multivariate ensemble classification

The tight coupling of information and communication technology and traditional energy system has given birth to a cyber physical energy system (CPES). CPES indeed improves the economic operation and control efficiency of the energy system, but it also brings new cyber risk issues, threatening the secure operation of the energy system. Consequently, this paper proposes a new multivariate ensemble classification (MEC) method to detect intrusions in CPES, thereby enhancing the baseline cybersecurity of CPES. MEC simultaneously takes into account the detection accuracy, stability and computing efficiency. In MEC, extreme gradient boosting, light gradient boosting machine and extreme learning machine are separately designed as individual detectors for intrusion identification. Then, ensemble learning based decision-making is developed to strategically aggregate the results of all individual detectors. Finally, the effectiveness of the proposed MEC is validated on IEEE standard 14-, 57- and 118-bus systems. The obtained results demonstrate that the MEC method has an attractive potential in real applications.

Mineral-resource prediction using advanced data analytics and machine learning of the QUEST-South stream-sediment geochemical data, southwestern British Columbia, Canada

In this study we apply multivariate statistical and predictive classification methods to interpret geochemical data from 8545 stream-sediment samples collected in southern British Columbia, Canada. Data for 35 elements were corrected for laboratory bias and adjusted for values reported below the lower limit of detection. Each sample site was attributed with the closest British Columbia MINFILE occurrence within 2.5 km. MINFILE occurrences were grouped into ‘GroupModels’ based on similarities between the British Columbia Geological Survey mineral deposit models and geochemical signatures. These data were used to create a training dataset of 474 observations, including 100 samples not attributed with a MINFILE occurrence. The training set was used to generate predictions for the mineral deposit models from which posterior probabilities were estimated for the remaining 8071 samples. The data underwent a centred log-ratio transformation and then characterization using either principal component analysis (PCA) or t-distributed stochastic neighbour embedding using 9 dimensions (t-SNE) prior to classification by random forests. The posterior probabilities generated from the t-SNE metric provide a slightly higher level of prediction accuracy compared to the posterior probabilities obtained using the PCA metric. The results are comparable to those obtained using a conventional catchment analysis approach and expert-driven model. The approach presented here provides a repeatable, consistent and defensible methodology for the identification of prospective mineralized terrains and mineral systems.

PLS-DA vs sparse PLS-DA in food traceability. A case study: Authentication of avocado samples

Conventional and sparse partial least squares-discriminant analysis (PLS-DA and sPLS-DA) have been successfully tested in order to authenticate avocado samples in terms of three different geographical origins and six kinds of cultivar. For this, lipid chromatographic fingerprints of different avocado fruits have been acquired using gas chromatography coupled with flame ionization detector (GC-FID) and employed for building classification models. In addition, classification models concatenating strategy has been applied, which has proved to be successful to resolve multiclass problems in food authentication. Finally, fine performance metrics around of 0.95 were obtained for both multivariate classification methods.

Identification of resistance in Escherichia coli and Klebsiella pneumoniae using excitation-emission matrix fluorescence spectroscopy and multivariate analysis

Klebsiella pneumoniae and Escherichia coli are part of the Enterobacteriaceae family, being common sources of community and hospital infections and having high antimicrobial resistance. This resistance profile has become the main problem of public health infections. Determining whether a bacterium has resistance is critical to the correct treatment of the patient. Currently the method for determination of bacterial resistance used in laboratory routine is the antibiogram, whose time to obtain the results can vary from 1 to 3 days. An alternative method to perform this determination faster is excitation-emission matrix (EEM) fluorescence spectroscopy combined with multivariate classification methods. In this paper, Linear Discriminant Analysis (LDA), Quadratic Discriminant Analysis (QDA) and Support Vector Machines (SVM), coupled with dimensionality reduction and variable selection algorithms: Principal Component Analysis (PCA), Genetic Algorithm (GA), and the Successive Projections Algorithm (SPA) were used. The most satisfactory models achieved sensitivity and specificity rates of 100% for all classes, both for E. coli and for K. pneumoniae. This finding demonstrates that the proposed methodology has promising potential in routine analyzes, streamlining the results and increasing the chances of treatment efficiency.

Multiple time-series convolutional neural network for fault detection and diagnosis and empirical study in semiconductor manufacturing

The development of information technology and process technology have been enhanced the rapid changes in high-tech products and smart manufacturing, specifications become more sophisticated. Large amount of sensors are installed to record equipment condition during the manufacturing process. In particular, the characteristics of sensor data are temporal. Most the existing approaches for time series classification are not applicable to adaptively extract the effective feature from a large number of sensor data, accurately detect the fault, and provide the assignable cause for fault diagnosis. This study aims to propose a multiple time-series convolutional neural network (MTS-CNN) model for fault detection and diagnosis in semiconductor manufacturing. This study incorporates data augmentation with sliding window to generate amounts of subsequences and thus to enhance the diversity and avoid over-fitting. The key features of equipment sensor can be learned automatically through stacked convolution-pooling layers. The importance of each sensor is also identified through the diagnostic layer in the proposed MTS-CNN. An empirical study from a wafer fabrication was conducted to validate the proposed MTS-CNN and compare the performance among the other multivariate time series classification methods. The experimental results demonstrate that the MTS-CNN can accurately detect the fault wafers with high accuracy, recall and precision, and outperforms than other existing multivariate time series classification methods. Through the output value of the diagnostic layer in MTS-CNN, we can identify the relationship between each fault and different sensors and provider valuable information to associate the excursion for fault diagnosis.

A perfect tandem: chemometric methods and microfluidic colorimetric twin sensors on paper. Beyond the traditional analytical approach

Chemometrics has proven to be a powerful tool for processing multivariate analytical data aimed at locating and extracting useful information relating to a particular analyte or material system in a complex sample from non-specific analytical signals that have been previously acquired and recorded by one or more analytical instruments or devices. In this paper, the basis for the application of both classification and quantitation multivariate methods is described, using a colorimetric twin sensor produced on a microfluidic paper-based analytical device (μPAD) as instructive example. The selected twin sensor provides a dual analytical signal for four anions (acetate, cyanide, fluoride and phosphate) in aqueous solution (concentration interval 0–0.1 M) from two reagent dyes (alizarin and 4-nitrophenol), which are immobilised in parallel on the same device containing four microfluidic channels designed in the form of an X. In this way, a data vector is obtained from each test whose elements are the colour coordinates obtained from the four responses, which is then used to build the chemometric models to be applied. Two multivariate classification methods (partial least squares discriminant analysis and support vector machine classification) are explored and the latter makes it possible to detect the presence or absence of each anion in an aqueous solution mixture. Single (each dye dataset separately) and fused (merging the two dye datasets) models were built and a support vector machine was shown to be the best classification method, obtaining sensitivity and precision values of 100% in almost all cases.

Introducing a multi-variate classification method: Risky driving acceptance among different heterogeneous driver sub-cultures

Introduction: Heterogeneous driving populations with many different origins are likely to have various sub-cultures that comprise of drivers with shared driver characteristics, most likely with dissimilar traffic safety cultures. An innovative methodology in traffic safety research is introduced which is beneficial for large datasets with multiple variables, making it useful for the multi-variate classification of drivers, driving attitudes and/or (risky) driving behaviours. Method: With the application of multidimensional scaling analysis (MDS), this study explores traffic safety culture in the State of Qatar using a questionnaire and investigates the similarity patterns between the questionnaire items, aiming to classify attitudes towards risky driving behaviours into themes. MDS is subsequently applied to classify drivers within a heterogeneous driving sample into sub-cultures with shared driver characteristics and different risky driving attitudes. Results: Results show that acceptance of speeding is highest among the young Arabic students and acceptance of distraction and drivers’ negligence such as phone use and not wearing a seatbelt is highest among male Arab drivers. Acceptance of extreme risk-taking like intoxicated driving and red-light running is highest among South-Asian business drivers. Conclusion: It is important and practical to understand risky behavioural habits among sub-cultures and thereby focussing on groups of drivers instead of individuals, because groups are easier to approach and drivers within sub-cultures are found to influence each other. By indicating which groups of drivers are most likely to perform specific risky driving themes, it is possible to target these groups and effectively emphasise certain subsets of risky driving behaviours during training or traffic safety education. Practical Applications: This study provides guidance for the improvement of driver education and targeted traffic safety awareness campaigns, intending to make changes to attitudes and habits within specific driver sub-cultures with the aim to improve traffic safety on the longer term.

Assessing the information content of ERP signals in schizophrenia using multivariate decoding methods.

Multivariate pattern classification (decoding) methods are commonly employed to study mechanisms of neurocognitive processing in typical individuals, where they can be used to quantify the information that is present in single-participant neural signals. These decoding methods are also potentially valuable in determining how the representation of information differs between psychiatric and non-psychiatric populations. Here, we examined ERPs from people with schizophrenia (PSZ) and healthy control subjects (HCS) in a working memory task that involved remembering 1, 3, or 5 items from one side of the display and ignoring the other side. We used the spatial pattern of ERPs to decode which side of the display was being held in working memory. One might expect that decoding accuracy would be inevitably lower in PSZ as a result of increased noise (i.e., greater trial-to-trial variability). However, we found that decoding accuracy was greater in PSZ than in HCS at memory load 1, consistent with previous research in which memory-related ERP signals were larger in PSZ than in HCS at memory load 1. We also observed that decoding accuracy was strongly related to the ratio of the memory-related ERP activity and the noise level. In addition, we found similar noise levels in PSZ and HCS, counter to the expectation that PSZ would exhibit greater trial-to-trial variability. Together, these results demonstrate that multivariate decoding methods can be validly applied at the individual-participant level to understand the nature of impaired cognitive function in a psychiatric population.

Authentication of the Geographical Origin of Margarines and Fat-Spread Products from Liquid Chromatographic UV-Absorption Fingerprints and Chemometrics.

Fat-spread products are a stabilized emulsion of water and vegetable oils. The whole fat content can vary from 10 to 90% (w/w). There are different kinds, which are differently named, and their composition depends on the country in which they are produced or marketed. Thus, having analytical solutions to determine geographical origin is required. In this study, some multivariate classification methods are developed and optimised to differentiate fat-spread-related products from different geographical origins (Spain and Morocco), using as an analytical informative signal the instrumental fingerprints, acquired by liquid chromatography coupled with a diode array detector (HPLC-DAD) in both normal and reverse phase modes. No sample treatment was applied, and, prior to chromatographic analysis, only the samples were dissolved in n‑hexane. Soft independent modelling of class analogy (SIMCA) and partial least squares-discriminant analysis (PLS-DA) were used as classification methods. In addition, several classification strategies were applied, and performance of the classifications was evaluated applying proper classification metrics. Finally, 100% of samples were correctly classified applying PLS-DA with data collected in reverse phase.

A multidimensional approach for the identification of environmental and health wellness territories

Abstract The evolution of the social and economic context in which the concept of wellbeing takes shape and the availability of new statistical information imposes constant checks on the set of indicators. The present work analyzes the relationships between environmental quality and health status indicators identified by the National Institute of Statistics (Istat) in Italian provinces. The data are analyzed through multivariate statistical methods (Totally Fuzzy and Relative) and methods of territorial classification (SaTScan) able to synthesize the various information and represent spatial distribution of environmental and health wellness.

Identification of Luminescent Markers for Gunshot Residues: Fluorescence, Raman Spectroscopy, and Chemometrics

Gunshot residue (GSR) is an evidence of major importance in firearm-related crimes. The recent introduction of nontoxic ammunition has made impossible the characterization of GSR particles by the current methods employed by forensic experts. To overcome this drawback, the introduction of luminescent markers was proposed, allowing on-site visual detection of luminescent gunshot residue (LGSR) at the crime scene. Three different luminescent markers coordinated with europium for specific and selective encoding of ammunition have been proposed. To promote a variety of versatile tools for GSR analysis, spectroscopic techniques combined with chemometric methods can be applied to achieve a reliable, fast, and nondestructive means to identify LGSR and discriminate among the different markers. Luminescence (emission and excitation), normal, and resonance Raman spectroscopies associated with principal component analysis (PCA) and partial least squares discriminant analysis (PLS-DA) were evaluated. The classification model using the complementary information on emission and excitation spectra, a.k.a. data fusion, provided a 100% correct classification for all markers. A comprehensive study has been developed to show that the insertion of luminescent markers enables not only the easy localization of GSR residues but also the possibility of ammunition encoding through the use of multivariate classification methods.

Automatic classification of major depression disorder using arterial spin labeling MRI perfusion measurements.

Neuroimaging-based multivariate pattern-recognition methods have been successfully used to develop diagnostic algorithms to distinguish patients with major depressive disorder (MDD) from healthy controls (HC). We developed and evaluated the accuracy of a multivariate classification method for the differentiation of MDD and HC using cerebral blood flow (CBF) features measured by non-invasive arterial spin labeling (ASL) MRI. Twenty-two medication-free patients with the diagnosis of MDD based on DSM-IV criteria and 22 HC underwent pseudo-continuous 3-D-ASL imaging to assess CBF. Using an atlas-based approach, regional CBF was determined in various brain regions and used together with sex and age as classification features. A linear kernel support vector machine was used for feature ranking and selection as well as for the classification of patients with MDD and HC. Permutation testing was used to test for significance of the classification results. The automatic classifier based on CBF features showed a statistically significant accuracy of 77.3% (P = 0.004) with a specificity of 80% and sensitivity of 75% for classification of MDD versus HC. The features that contributed to the classification were sex and regional CBF of the cortical, limbic, and paralimbic regions. Machine-learning models based on CBF measurements are capable of differentiating MDD from HC with high accuracy. The use of larger study cohorts and inclusion of other imaging measures may improve the performance of the classifier to achieve the accuracy required for clinical application.