Stacked Generalization Technique Research Articles

A clinical marker-based modeling framework to preoperatively predict lymph node and vascular space involvement in endometrial cancer patients

IntroductionEndometrial cancer (EC) has high mortality at advanced stages. Poor prognostic factors include grade 3 tumors, deep myometrial invasion, lymph node metastasis (LNM), and lymphovascular space invasion (LVSI). Preoperative knowledge of patients at higher risk of lymph node involvement, when such involvement is not suspected, would benefit surgery planning and patient prognosis. This study implements an ensemble machine learning approach that evaluates Cancer Antigen 125 (CA125) along with histologic type, preoperative grade, and age to predict LVSI, LNM and stage in EC patients. MethodsA retrospective chart review spanning January 2000 to January 2015 at a regional hospital was performed. Women 18 years or older with a diagnosis of EC and preoperative or within one-week CA125 measurement were included (n = 842). An ensemble machine learning approach was implemented based on a stacked generalization technique to evaluate CA125 in combination with histologic type, preoperative grade, and age as predictors, and LVSI, LNM and disease stage as outcomes. ResultsThe ensemble approach predicted LNM and LVSI in EC patients with AUROCTEST of 0.857 and 0.750, respectively, and predicted disease stage with AUROCTEST of 0.665. The approach achieved AUROCTEST for LVSI and LNM of 0.750 and 0.643 for grade 1 patients, and of 0.689 and 0.952 for grade 2 patients, respectively. ConclusionAn ensemble machine learning approach offers the potential to preoperatively predict LVSI, LNM and stage in EC patients with adequate accuracy based on CA125, histologic type, preoperative grade, and age.

Unraveling air–water two-phase flow patterns in water pipelines based on multiple signals and convolutional neural networks

Abstract Flow pattern identification (FPI) is crucial for evaluating air entrapment in water pipelines and ensuring the safety of pipeline operations. The presence of two-phase flow in water pipelines not only leads to pressure fluctuations but also induces pipeline vibration. However, current research has primarily focused on using pressure-related signals for FPI, and the analysis of vibration signals in FPI is rare. In this study, FPI in water pipelines is investigated based on convolutional neural networks (CNNs) using high-frequency vibration signals. The information fusion of vibration signals in FPI is newly proposed via the stacked generalization technique. The proposed method is compared with pressure signal-based FPI methods and the effect of signal sampling parameters on FPI accuracy is discussed. The results show that the performance of vibration signals (including axial or radial acceleration signals) outperforms pressure signals in both time and frequency domains. Moreover, the fusion of vibration signals shows the superior results compared to any univariate signals. The duration of sampling has a more significant impact on the results of FPI than the sampling frequency. This study provides a new way that FPI theory is applied to solve air entrapment evaluation in water pipelines.

Automatic classification of patients with myocardial infarction or myocarditis based only on clinical data: A quick response.

In acute cardiovascular disease management, the delay between the admission in a hospital emergency department and the assessment of the disease from a Delayed Enhancement cardiac MRI (DE-MRI) scan is one of the barriers for an immediate management of patients with suspected myocardial infarction or myocarditis. This work targets patients who arrive at the hospital with chest pain and are suspected of having a myocardial infarction or a myocarditis. The main objective is to classify these patients based solely on clinical data in order to provide an early accurate diagnosis. Machine learning (ML) and ensemble approaches have been used to construct a framework to automatically classify the patients according to their clinical conditions. 10-fold cross-validation is used during the model's training to avoid overfitting. Approaches such as Stratified, Over-sampling, Under-sampling, NearMiss, and SMOTE were tested in order to address the imbalance of the data (i.e. proportion of cases per pathology). The ground truth is provided by a DE-MRI exam (normal exam, myocarditis or myocardial infarction). The stacked generalization technique with Over-sampling seems to be the best one providing more than 97% of accuracy corresponding to 11 wrong classifications among 537 cases. Generally speaking, ensemble classifiers such as Stacking provided the best prediction. The five most important features are troponin, age, tobacco, sex and FEVG calculated from echocardiography. Our study provides a reliable approach to classify the patients in emergency department between myocarditis, myocardial infarction or other patient condition from only clinical information, considering DE-MRI as ground-truth. Among the different machine learning and ensemble techniques tested, the stacked generalization technique is the best one providing an accuracy of 97.4%. This automatic classification could provide a quick answer before imaging exam such as cardiovascular MRI depending on the patient's condition.

Prediction of ground vibration due to mine blasting in a surface lead–zinc mine using machine learning ensemble techniques

Ground vibration due to blasting is identified as a challenging issue in mining and civil activities. Peak particle velocity (PPV) is one of the blasting undesirable consequences, which is resulted during emission of vibration in blasted bench. This study focuses on the PPV prediction in the surface mines. In this regard, two ensemble systems, i.e., the ensemble of artificial neural networks and the ensemble of extreme gradient boosting (EXGBoosts) were developed for PPV prediction in one of the largest lead–zinc open-pit mines in the Middle East. For ensemble modeling, several ANN and XGBoost base models were separately designed with different architectures. Then, the validation indices such as coefficient determination (R2), root mean square error (RMSE), mean absolute error (MAE), the variance accounted for (VAF), and Accuracy were used to evaluate the performance of the base models. The five top base models with high accuracy were selected to construct an ensemble model for each of the methods, i.e., ANNs and XGBoosts. To combine the outputs of the top base models and achieve a single result stacked generalization technique, was employed. Findings showed ensemble models increase the accuracy of PPV predicting in comparison with the best individual models. The EXGBoosts was superior method for predicting of the PPV, which obtained values of R2, RMSE, MAE, VAF, and Accuracy corresponding to the EXGBoosts were (0.990, 0.391, 0.257, 99.013(%), 98.216), and (0.968, 0.295, 0.427, 96.674(%), 96.059), for training and testing datasets, respectively. However, the sensitivity analysis indicated that the spacing (r = 0.917) and number of blast-holes (r = 0.839) had the highest and lowest impact on the PPV intensity, respectively.

Predicting the potential distribution of wheatear birds using stacked generalization-based ensembles

Habitat suitability models, usually referred to as species distribution models (SDMs), are widely applied in ecology for many purposes, including species conservation, habitat discovery, and gain evolutionary insights by estimating the distribution of species. Machine learning algorithms as well as statistical models have been recently used to predict the distribution of species. However, they seemed to have some limitations due to the data and the models used. Therefore, this study proposes a novel approach for assessing habitat suitability based on ensemble learning techniques. Three heterogeneous ensembles were built using the stacked generalization method to model the distribution of four wheatear species (Oenanthe deserti, Oenanthe leucopyga, Oenanthe leucura, and Oenanthe oenanthe) located in Morocco. Initially, a set of base-learners were constructed by virtue of training for each specie's dataset six machine learning algorithms (Multi-Layer Perceptron (MLP), Support Vector Classifier (SVC), K-nearest neighbors (KNN), Decision Trees (DT), Gradient Boosting Classifier (GB), and Random Forest (RF)). Then, the predictions of these base learners were fed as training data to train three meta-learners (Logistic Regression (LR), SVC, and MLP). To evaluate and assess the performance of the proposed approaches, we used: (1) six performance criteria (accuracy, recall, precision, F1-score, AUC, and TSS), (2) Borda Count (BC) ranking method based on multiple criteria to rank the best-performing models, and (3) Scott Knott (SK) test to statistically compare the performance of the presented models. The results based on the six-evaluation metrics showed that stacked ensembles outperformed their singles in all species datasets, and the stacked model with SVC as a meta-learner outperformed the other two ensembles. The results showed the potential of using ensemble learning techniques to model species distribution and recommend the use of the stacked generalization technique as a combination strategy since it gave better results compared to single models in four wheatear species datasets. Moreover, to assess the impact of future climate changes on the distribution of the four wheatear species, the best-performing distribution model was selected and projected into the current and future climatic conditions. The distributions of the Moroccan wheatear birds were found to be slightly affected by future climate changes.

Upgrading the Performance of Machine Learning Based Chronic Disease Prediction Systems using Stacked Generalization Technique

Upgrading the Performance of Machine Learning Based Chronic Disease Prediction Systems using Stacked Generalization Technique

LieToMe: An Ensemble Approach for Deception Detection from Facial Cues.

Deception detection is a relevant ability in high stakes situations such as police interrogatories or court trials, where the outcome is highly influenced by the interviewed person behavior. With the use of specific devices, e.g. polygraph or magnetic resonance, the subject is aware of being monitored and can change his behavior, thus compromising the interrogation result. For this reason, video analysis-based methods for automatic deception detection are receiving ever increasing interest. In this paper, a deception detection approach based on RGB videos, leveraging both facial features and stacked generalization ensemble, is proposed. First, a face, which is well-known to present several meaningful cues for deception detection, is identified, aligned, and masked to build video signatures. These signatures are constructed starting from five different descriptors, which allow the system to capture both static and dynamic facial characteristics. Then, video signatures are given as input to four base-level algorithms, which are subsequently fused applying the stacked generalization technique, resulting in a more robust meta-level classifier used to predict deception. By exploiting relevant cues via specific features, the proposed system achieves improved performances on a public dataset of famous court trials, with respect to other state-of-the-art methods based on facial features, highlighting the effectiveness of the proposed method.

Text-to-speech synthesis system with Arabic diacritic recognition system

Text-to-speech synthesis system has been widely studied for many languages. However, speech synthesis for Arabic language has not sufficient progresses and it is still in its first stage. Statistical parametric synthesis based on hidden Markov models was the most commonly applied approach for Arabic language. Recently, synthesized speech quality based on deep neural networks was found as intelligible as human voice. This paper describes a Text-To-Speech (TTS) synthesis system for modern standard Arabic language based on statistical parametric approach and Mel-cepstral coefficients. Deep neural networks achieved state-of-the-art performance in a wide range of tasks, including speech synthesis. Our TTS system includes a diacritization system which is very important for Arabic TTS application. Our diacritization system is also based on deep neural networks. In addition to the use deep techniques, different methods were also proposed to model the acoustic parameters in order to address the problem of acoustic models accuracy. They are based on linguistic and acoustic characteristics (e.g. letter position based diacritization system, unit types based synthesis system, diacritic marks based synthesis system) and based on deep learning techniques (stacked generalization techniques). Experimental results show that our diacritization system can generate a diacritized text with high accuracy. As regards the speech synthesis system, the experimental results and subjective evaluation show that our proposed method for synthesis system can generate intelligible and natural speech.