Mixture Discriminant Analysis Research Articles

Tensor mixture discriminant analysis with applications to sensor array data analysis

Tensor mixture discriminant analysis with applications to sensor array data analysis

The buzzOmeter system: In situ audio recordings of pollinators in flight

Abstract The role of sounds produced by free‐flying insects is challenging to research due to technical difficulties in obtaining audio recordings suitable for playback experiments. Experimental studies using flight sounds are needed to understand if buzzes carry information and by whom it is perceived. We developed the ‘buzzOmeter system’ for recording untethered, flying insects in their habitat, followed by file processing that allows precise measurements of acoustic parameters, including those dependent on the distance of the sound source from the microphone, that is signal magnitude measurements. The system consists of commercially available elements and open source software. We provide a practical guide for the assembly and use of two alternative setups of the buzzOmeter system, followed by a video tutorial on file processing and an R script for the assignment of audio recordings to the corresponding species based on mixture discriminant analysis. Recordings of nine insect species (bees, wasps and lepidopterans) obtained with the use of our system in various habitats demonstrate its feasibility for field studies. Diverse species interactions are based on sound, and our new tool can aid researchers studying acoustical signalling in predator–prey, pollinator‐plant and mimic‐model complexes, among others.

A Flexible Probabilistic Framework With Concurrent Analysis of Continuous and Categorical Data for Industrial Fault Detection and Diagnosis

Operational data from industrial processes typically consist of continuous and categorical variables. They reveal different aspects of the operational conditions, and both are useful for comprehensive fault detection and diagnosis (FDD). However, due to the multiple production modes, the variables usually do not follow Gaussian or Bernoulli distributions. Furthermore, their correlations can be different across normal and various faulty classes. Thus, the main challenge lies in how to properly fuse the complementary information in the two types of variables and accurately characterize the complicated distribution for each class. This work proposes a flexible probabilistic framework that can concurrently analyze continuous and categorical variables for FDD. Our framework specifies a finite mixture model for each class. Thus, it can handle non-Gaussian and non-Bernoulli variables and capture their correlations under the conditional independence assumption. We then introduce the variational inference for parameter estimation, which makes our framework adaptive to the different distributions of various classes. Furthermore, a unified statistical index is designed for two types of variables, giving our method extra capability to distinguish unknown faults. Finally, the effectiveness of our method is validated on the Tennessee Eastman (TE) process and a practical industrial plant process. The average <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">F1</i> score of our method is improved by 3.4/3.9 percentage points in the single-mode/multimode situation on the TE process compared with traditional mixture discriminant analysis. In the industrial plant process, when unknown faults are added, the average <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">F1</i> score of our method is 91.1% and only 1.2 percentage points lower than that on the test set without unknown faults.

A novel approach for assessing flood risk with machine learning and multi-criteria decision-making methods

Hazardous flooding occurs across most climate zones. Owing to the lack of appropriate infrastructures and applicable predictive methods, flooding in arid and semi-arid regions may be especially damaging. Based on a study of Abarkuh County, Iran, we introduce an integrated approach for identifying high-priority flood risk areas using machine learning (ML) and multi-criteria decision-making (MCDM) methods, which is transferable to other (semi)arid regions. Results indicate that among the ML models we examined—including classification and regression tree (CART), mixture discriminant analysis (MDA), and support vector machine (SVM)—the SVM model performs best. We estimate that 75% of the study area is subject to high or very flood hazard. Our application of the Jackknife technique identifies precipitation, vegetation, and drainage density as the most important conditional factors for regional flood hazards. Our analytical network process (ANP)-decision making trial and evaluation laboratory (DEMATEL) results reveal that population density and agricultural area density have the greatest influence on flood vulnerability. Results integrating SVM and ANP-DEMATEL flood hazard and vulnerability maps indicate that 6% of the study area is at high or very high flood risk. Application of this approach can assist local authorities in identifying priority areas for flood management interventions.

A test of the (hu)MANid analytical program using a native American population.

Recently, Berg and Kenyhercz (2017) developed a free web-based software package, (hu)MANid, that classifies mandibles by ancestry and sex based on either linear or mixture discriminant analysis of 11 osteometric and six morphoscopic variables. The metric and morphoscopic variables assessed using (hu)MANid have been determined to be highly replicable, however, few external validation studies have been conducted. This article provides a test of the (hu)MANid analytical software using an independent sample (n = 52) of Native American mandibles from the Great Lakes region to investigate the accuracy of the program for identifying this important demographic group. When processed using linear discriminant analysis in (hu)MANid, 82.7% of the mandibles (43/52) were correctly classified as Native American. When processed using mixture discriminant analysis in (hu)MANid, 67.3% of the mandibles (35/52) were correctly classified as Native American. The difference in accuracy between the methods is not statistically significant. Our results suggest (hu)MANid is an accurate tool for anthropologists attempting to determine whether skeletal remains are Native American when establishing forensic significance, creating a biological profile, and conducting work associated with the federal Native American Graves Protection and Repatriation Act.

An Ensemble Approach of Feature Selection and Machine Learning Models for Regional Landslide Susceptibility Mapping in the Arid Mountainous Terrain of Southern Peru

This study evaluates the utility of the ensemble framework of feature selection and machine learning (ML) models for regional landslide susceptibility mapping (LSM) in the arid climatic condition of southern Peru. A historical landslide inventory and 24 different landslide influencing factors (LIFs) were prepared using remotely sensed and auxiliary datasets. The LIFs were evaluated using multi-collinearity statistics and their relative importance was measured to select the most discriminative LIFs using the ensemble feature selection method, which was developed using Chi-square, gain ratio, and relief-F methods. We evaluated the performance of ten different ML algorithms (linear discriminant analysis, mixture discriminant analysis, bagged cart, boosted logistic regression, k-nearest neighbors, artificial neural network, support vector machine, random forest, rotation forest, and C5.0) using different accuracy statistics (sensitivity, specificity, area under curve (AUC), and overall accuracy (OA)). We used suitable combinations of individual ML models to develop different ensemble ML models and evaluated their performance in LSM. We assessed the impact of LIFs on ML performance. Among all individual ML models, the k-nearest neighbors (sensitivity = 0.72, specificity = 0.82, AUC = 0.86, OA = 78%) and artificial neural network (sensitivity = 0.71, specificity = 0.85, AUC = 0.87, OA = 79%) algorithms showed the best performance using the top five LIFs, while random forest, rotation forest, and C5.0 (sensitivity = 0.76–0.81, specificity = 0.87, AUC = 0.90–0.93, OA = 82–84%) outperformed other models when developed using all twenty-four LIFs. Among ensemble models, the ensemble of k-nearest neighbors and rotation forest, k-nearest neighbors and artificial neural network, and artificial neural network and rotation forest outperformed other models (sensitivity = 0.72–0.73, specificity = 0.83–0.84, AUC = 0.86, OA = 79%) using the top five LIFs. The landslide susceptibility maps derived using these models indicate that ~2–3% and ~10–12% of the total study area fall within the “very high” and “high” susceptibility. The obtained susceptibility maps can be efficiently used to prioritize landslide mitigation activities.

Badland erosion susceptibility mapping using machine learning data mining techniques, Firozkuh watershed, Iran

Badlands are landforms related to runoff, with dissected V-shaped valleys, short steep slopes, and high drainage density, and results from a very important type of erosion that develops due to a complex interaction of conditioning factors, including climatic, hydrologic, geologic and soil properties, topographic characteristics, and land use. The main goals of this study were (1) create badland susceptibility maps of the Firozkuh watershed and five machine learning algorithms (models)—functional discriminant analysis (FDA), generalized linear model (GLM), mixture discriminant analysis (MDA), multivariate adaptive regression spline (MARS), and support vector machine (SVM); and (2) compare the accuracy of these models. Sixteen conditioning factors were chosen to model and classify badland susceptibility based on a literature review, data availability, and field surveys. Model accuracy was assessed using ROC curve and AUC analyses. The analyses showed that SVM was “excellent,” MARS was “very good,” MDA was “good,” and GLM and FDA were “moderate” in classification accuracy. The land area of the very high and high classes ranged from 31 to 51% of the Firozkuh watershed for the GLM and SVM models, respectively. This indicates that badland erosion is a very important problem in the study area. Climatic, hydrologic, geologic, topographic, and soil conditions as well as land use changes render the Firozkuh watershed prone to badland formation and soil erosion which results in substantial socioeconomic losses. Badland susceptibility mapping is an important tool that can be used to improve managing future badland erosion in the Firozkuh watershed and other areas affected by badland erosion.

Analytical techniques for mapping multi-hazard with geo-environmental modeling approaches and UAV images

The quantitative spatial analysis is a strong tool for the study of natural hazards and their interactions. Over the last decades, a range of techniques have been exceedingly used in spatial analysis, especially applying GIS and R software. In the present paper, the multi-hazard susceptibility maps compared in 2020 and 2021 using an array of data mining techniques, GIS tools, and Unmanned aerial vehicles. The produced maps imply the most effective morphometric parameters on collapsed pipes, gully heads, and landslides using the linear regression model. The multi-hazard maps prepared using seven classifiers of Boosted regression tree (BRT), Flexible discriminant analysis (FDA), Multivariate adaptive regression spline (MARS), Mixture discriminant analysis (MDA), Random forest (RF), Generalized linear model (GLM), and Support vector machine (SVM). The results of each model revealed that the greatest percentage of the study region was low susceptible to collapsed pipes, landslides, and gully heads, respectively. The results of the multi-hazard models represented that 52.22% and 48.18% of the study region were not susceptible to any hazards in 2020 and 2021, while 6.19% (2020) and 7.39% (2021) of the region were at the risk of all compound events. The validation results indicate the area under the receiver operating characteristic curve of all applied models was more than 0.70 for the landform susceptibility maps in 2020 and 2021. It was found where multiple events co-exist, what their potential interrelated effects are or how they interact jointly. It is the direction to take in the future to determine the combined effect of multi-hazards so that policymakers can have a better attitude toward sustainable management of environmental landscapes and support socio-economic development.

Ensemble enhanced active learning mixture discriminant analysis model and its application for semi-supervised fault classification

Ensemble enhanced active learning mixture discriminant analysis model and its application for semi-supervised fault classification

A Test of the (hu)MANid Program on a Diverse Sample of Mandibles

The (hu)MANid program is a free, web-based software using mandibular metric and morphoscopic variables to estimate sex and ancestry. The database consists of 1,745 individuals from 15 populations (modern, historic, and prehistoric). The required use of a mandibulometer presents two limitations. It is an expensive piece of equipment not available in all labs, and many users do not have experience using a mandibulometer, so error can be high. To address this, three-dimensional surface scans (3D scans) provide an alternative method for data collection. Here, the authors assess the accuracy of the (hu)MANid program on a diverse sample, compare accuracy from 3D scans and physical measurements, and evaluate whether mandibulometer measurements increase classification rates. Six metric and morphoscopic variables were collected from 3D scans (Ntotal = 555) and from bone (Ntotal = 41). Three additional mandibulometer measurements were collected from bone. Mixture discriminant analysis and non-stepwise options were applied due to reported higher correct classification and no significant difference when using stepwise options. Test samples with matching reference groups from the digital measurements had higher than chance average correct classifications for ancestry and sex/ancestry combined. Pooled sexes had correct classifications higher than chance for all comparisons except females using digital measurements. Physical measurements had a higher correct classification rate than digital measurements despite the small sample size. Including mandibulometer measurements did not significantly increase classification rates for physical measurements. Results indicate that hu(MAN)id is better than chance but suggest additional testing due to a lack of diversity in the physical measurements.

Performance Comparison of Ensemble Learning and Supervised Algorithms in Classifying Multi-label Network Traffic Flow

Network traffic classification is of significant importance. It helps identify network anomalies and assists in taking measures to avoid them. However, classifying network traffic correctly is a challenging task. This study aims to compare ensemble learning methods with normal supervised classification to come up with improved classification methods. Three types of network traffic were classified (Benign, Malicious, and Outliers). The data were collected experimentally by using Paessler Router Traffic Grapher software and online and were analyzed by R software. The datasets were used to train five supervised models (k-nearest neighbors, mixture discriminant analysis, Naïve Bayes, C5.0 classification model, and regularized discriminant analysis). The models were trained by 70% of the samples and the rest 30% were used for validation. The same samples were used separately in predicting individual accuracy. The results were compared to the ensemble learning models which were built with the use of the same datasets. Among the five supervised classifiers, k-nearest neighbors and C5.0 classification scored the highest accuracy of 0.868 and 0.761. The ensemble learning classifiers Bagging (Random Forest) and Boosting (eXtreme Gradient Boosting) had accuracy of 0.904 and 0.902 respectively. The results show that the ensemble learning method has higher accuracy compared to the normal supervised classifiers. Therefore, it can be used to detect malicious activities in network traffic as well as anomalies with improved accuracy.

Identification of hepatic steatosis in living liver donors by machine learning models.

Selecting an optimal donor for living donor liver transplantation is crucial for the safety of both the donor and recipient, and hepatic steatosis is an important consideration. We aimed to build a prediction model with noninvasive variables to evaluate macrovesicular steatosis in potential donors by using various prediction models. The study population comprised potential living donors who had undergone donation workup, including percutaneous liver biopsy, in the Republic of Korea between 2016 and 2019. Meaningful macrovesicular hepatic steatosis was defined as >5%. Whole data were divided into training (70.5%) and test (29.5%) data sets based on the date of liver biopsy. Random forest, support vector machine, regularized discriminant analysis, mixture discriminant analysis, flexible discriminant analysis, and deep neural network machine learning methods as well as traditional logistic regression were employed. The mean patient age was 31.4 years, and 66.3% of the patients were men. Of the 1652 patients, 518 (31.4%) had >5% macrovesicular steatosis on the liver biopsy specimen. The logistic model had the best prediction power and prediction performances with an accuracy of 80.0% and 80.9% in the training and test data sets, respectively. A cut‐off value of 31.1% for the predicted risk of hepatic steatosis was selected with a sensitivity of 77.7% and specificity of 81.0%. We have provided our model on the website (https://hanseungbong.shinyapps.io/shiny_app_up/) under the name DONATION Model. Our algorithm to predict macrovesicular steatosis using routine parameters is beneficial for identifying optimal potential living donors by avoiding superfluous liver biopsy results.

Unobserved classes and extra variables in high-dimensional discriminant analysis

In supervised classification problems, the test set may contain data points belonging to classes not observed in the learning phase. Moreover, the same units in the test data may be measured on a set of additional variables recorded at a subsequent stage with respect to when the learning sample was collected. In this situation, the classifier built in the learning phase needs to adapt to handle potential unknown classes and the extra dimensions. We introduce a model-based discriminant approach, Dimension-Adaptive Mixture Discriminant Analysis (D-AMDA), which can detect unobserved classes and adapt to the increasing dimensionality. Model estimation is carried out via a full inductive approach based on an EM algorithm. The method is then embedded in a more general framework for adaptive variable selection and classification suitable for data of large dimensions. A simulation study and an artificial experiment related to classification of adulterated honey samples are used to validate the ability of the proposed framework to deal with complex situations.

Rigorous assessment of data mining algorithms in gestational diabetes mellitus prediction

Gestational diabetes mellitus (GDM) is the type of diabetes that affects pregnant women due to high blood sugar levels. The women with gestational diabetes have a chance of miscarriage during pregnancy and having chance of developing type-2 diabetes in the future. It is a general practice to take proper diabetes test like OGTT to detect GDM. This test is to be done during 24 to 28 weeks of pregnancy. In addition, the use of machine learning could be exploited for predicting gestational diabetes. The main goal of this work is to propose optimal ML algorithms for effective prediction of gestational diabetes mellitus and there by avoid it’s side effects and future complications. In this work different machine algorithms are planned to be compared for their performance in predicting GDM. Before analysing the algorithms they are implemented using 10 fold cross validation technique to obtain better performance. The algorithms implemented are Linear Discriminant Analysis, Mixture Discriminant Analysis, Quadratic Discriminant Analysis, Flexible Discriminant Analysis, Regularized Discriminant Analysis and Feed Forward Neural Networks. These algorithms are compared depending on performance measures accuracy, kappa statistic, sensitivity, specificity, precision and F-measure. Then feed forward neural networks and Flexible Discriminant Analysis are obtained as optimal in this work.

Spatial mapping Zataria multiflora using different machine-learning algorithms

Understanding the relationships between environmental factors that influence the distribution of medicinal plants is crucial to identifying suitable habitats for them. Studies have shown that the responses of such species to environmental influences is unclear. A region known as one of the most important areas for Zataria multiflora is Fars Province in southwestern Iran. This study determines the importance of 13 environmental factors (elevation, distance from rivers, distance from roads, pH, electrical conductivity, mean annual temperature, mean annual rainfall, slope angle, slope aspect, plan curvature, and soil properties) to the distribution of Zataria multiflora. The suitable habitats for Zataria multiflora are distinguished by the species-variable relationships and the application of five machine-learning techniques (MLTs): generalized linear model (GLM), generalized boosting model (GBM), boosted regression tree (BRT), functional discrimination analysis (FDA), and mixture discriminant analysis (MDA). The partial least-squares algorithm was used to determine the rank of importance of each variable. The results reveal that the most important factors influencing Zataria multiflora distribution are slope, elevation, EC, and mean annual temperature. The MLTs were applied, and the predictions were classified into four classes (very high, high, moderate, and low). Results indicate that the region of low habitat suitability is in the central portion of the study area and the percentages of the study area classified as having low potential are: 62.01% for GLM, 74.51% for MDA, 63.56% for FDA, 70.14% for GBM, and 23.19% for BRT. The models accuracies as indicated by AUC are MDA (93.6%), GBM (90.3%), GLM (89.1%), FDM (88.1%), and (83.4%). Modeling of habitat suitability can improve farmers’ and managers’ decisions regarding the protection of medicinal plants.

Fast estimation for robust supervised classification with mixture models

• The Robust Mixture Discriminant Analysis model leads to a convex optimization problem. • A consensus based formulation can be solved efficiently using ADMM. • This formulation scales efficiently to thousands of samples. • The main limitation of the Robust Mixture Discriminant Analysis comes from the clustering. Label noise is known to negatively impact the performance of classification algorithms. In this paper, we develop a model robust to label noise that uses both labelled and unlabelled samples. In particular, we propose a novel algorithm to optimize the model parameters that scales efficiently w.r.t. the number of training samples. Our contribution relies on a consensus formulation of the original objective function that is highly parallelizable. The optimization is performed with the Alternating Direction Method of Multipliers framework. Experimental results on synthetic datasets show an improvement of several orders of magnitude in terms of processing time, with no loss in terms of accuracy. Our method appears also tailored to handle real data with significant label noise.

Overview of Algorithms for Using Particle Morphology in Pre-Detonation Nuclear Forensics

A major goal in pre-detonation nuclear forensics is to infer the processing conditions and/or facility type that produced radiological material. This review paper focuses on analyses of particle size, shape, texture (“morphology”) signatures that could provide information on the provenance of interdicted materials. For example, uranium ore concentrates (UOC or yellowcake) include ammonium diuranate (ADU), ammonium uranyl carbonate (AUC), sodium diuranate (SDU), magnesium diuranate (MDU), and others, each prepared using different salts to precipitate U from solution. Once precipitated, UOCs are often dried and calcined to remove adsorbed water. The products can be allowed to react further, forming uranium oxides UO3, U3O8, or UO2 powders, whose surface morphology can be indicative of precipitation and/or calcination conditions used in their production. This review paper describes statistical issues and approaches in using quantitative analyses of measurements such as particle size and shape to infer production conditions. Statistical topics include multivariate t tests (Hotelling’s T2), design of experiments, and several machine learning (ML) options including decision trees, learning vector quantization neural networks, mixture discriminant analysis, and approximate Bayesian computation (ABC). ABC is emphasized as an attractive option to include the effects of model uncertainty in the selected and fitted forward model used for inferring processing conditions.

Using machine learning classification to detect simulated increases of de facto reuse and urban stormwater surges in surface water

Water quality events such as increases in stormwater or wastewater effluent in drinking water sources pose hazards to drinking water consumers. Stormwater and wastewater effluent enter Lake Mead—an important drinking water source in the southwest USA—via the Las Vegas Wash. Previous studies have applied machine learning and online instruments to detect contamination in water distribution systems. However, alert systems at drinking water intakes would provide more time for corrective action. An array of online instruments measuring pH, conductivity, redox potential, turbidity, temperature, tryptophan-like fluorescence, UV absorbance (UVA254), TOC, and chlorophyll-a was fed raw water directly from Lake Mead. Wastewater effluent, dry weather Las Vegas Wash, and storm-impacted Las Vegas Wash samples were blended into the instrument inlets at known ratios to simulate three types of adverse water quality events. Data preprocessing was conducted to correct for diurnal patterns or instrument drift. Supervised machine learning was conducted using previously published models in R. Ninety-nine models were screened on the raw data. Eight high-performing models were evaluated in-depth and optimized. Weighted k-Nearest Neighbors, Single C5.0 Ruleset, Mixture Discriminant Analysis, and an ensemble of these three models had accuracy over 97% when assigning test set data among three classes (Normal, Event, or Maintenance). The ensemble detected all event types at the earliest timepoint and had one false positive that was not a lag error (i.e., consecutively following a true positive). Omitting Maintenance, the Adaboost model had over 99% test set accuracy and zero false positives that were not lag errors. Data preprocessing was beneficial, but the optimal methods were model-specific. All nine water quality variables were useful for most models, but UVA254 and turbidity were most important.

Landslide susceptibility assessment and mapping using state-of-the art machine learning techniques

Landslides pose a serious risk to human life and the natural environment. Here, we compare machine learning algorithms including the generalized linear model (GLM), mixture discriminant analysis (MDA), boosted regression tree (BRT), and functional discriminant analysis (FDA) to evaluate the landslide exposure regions in Fars Province, comprising an area of approximately 7% of Iran. Initially, an aggregate of 179 historical landslide occurrences was prepared and partitioned. Subsequently, ten landslide conditioning factors (LCFs) were generated. The partial least squares algorithm was utilized to assess the significance of the LCFs with the help of a training dataset which indicated that distance from road had the maximum significance in forecasting landslides, followed by altitude (Al), lithological units, and slope degree. Finally, the LSMs generated using BRT, GLM, MDA, and FDA were validated and compared using cut-off reliant and independent validation measures. The results of the validation metrics showed that GLM and BRT had an AUC of 0.908, while FDA and MDA had AUCs of 0.858 and 0.821, respectively. The results from our case study can be utilized to develop strategies and plans to minimize the loss of human lives and the natural environment.

Spatial modeling of susceptibility to subsidence using machine learning techniques

Land subsidence is a hazard that results from conditioning factors that cause environmental change and generate social and economic impacts. Some of these factors may increase dissolution of calcareous stones, change groundwater storage, or stem from mining, faulting, and seismic activity. Semnan Plain, Iran is experiencing land subsidence that, along with secondary impacts like surface fissures, is becoming increasingly troublesome. This study modeled land subsidence and created a susceptibility map using multivariate adaptive regression spline (MARS), mixture discriminant analysis (MDA), and boosted regression tree (BRT) machine-learning methods. Analysis of satellite imagery, documents reporting past subsidence, and a contemporary field survey identified 65 sinkholes on Semnan Plain. Twelve conditioning factors were selected for analysis from a review of the scholarly literature, field investigation, and data availability. The three methods were used to model subsidence from a training subset of the known sites. The models were validated with the remaining subset of subsidence locations. Finally, susceptibility maps were used to predict other sites that are highly likely to see subsidence. Receiver operating characteristic curves and the area under the curves (AUC) were applied to assess the accuracies of the maps. AUC values (0.637, 0.783, and 0.712 for the BRT, MARS, and MDA models respectively) showed that MARS generated the most accurate model, MDA generated the second most accurate, and BRT’s was the least accurate model. Subsidence susceptibility maps as produced here can be useful, meaningful, and functional tools for local and regional planners and policy makers involved in land use planning, resource management, and hazard mitigation.