Multistage Ensemble Research Articles

Multi-stage ensemble with refinement for noisy labeled data classification

Deep neural networks (DNNs) have made remarkable progress in image classification. However, since DNNs can memorize all the label information in the training dataset due to their excellent feature learning ability, the generalization performance deteriorates when they are trained on the noisy labeled dataset that can be easily found in real-world problems. In this paper, we propose a multi-stage ensemble method with label refinement to build an effective classification model under noisy labels. The proposed method iteratively refines the dataset by re-labeling the samples at the end of each stage, which enables the models trained at each stage to learn different features. By integrating these models, the proposed multi-stage ensemble method exerts powerful generalization performance. Also, we suggest a novel dataset refinement method, demonstrating the effectiveness of a robust function in distinguishing corrupted samples. Experimental results on the benchmark and real-world datasets show that the proposed method outperforms the existing methods on the noisy labeled dataset classification.

Design of a Multi-Stage Ensemble Model for Thyroid Prediction Using Learning Approaches

Design of a Multi-Stage Ensemble Model for Thyroid Prediction Using Learning Approaches

Brain cancer classification based on multistage ensemble generative adversarial network and convolutional neural network.

An advanced approach that capitalizes on the synergies between multimodal feature fusionand the dual-path networkis presented in this manuscript. Our proposed methodology harnesses a combination of potent techniques, merging the benefits of nonlinear mapping and expansive perception. The foundation of our methodology lies in leveraging well-established pretrained models, namely EfficientNet-B7, ResNet-152, and a meticulously crafted custom convolutional neural network(CNN), to effectively extract salient features from the data. These models are combined in a two-stage ensemble approach. We employ maximum variance unfolding (MVU) to select the most relevant attributes from the extracted features. In this study, we propose a hybrid approach that integrates a generative adversarial network and Neural Autoregressive Distribution Estimation (NADE-K) with a CNN. The resulting two-stage ensemble hybrid CNN model achieves an accuracy of 99.63%. The implementation of the two-stage ensemble hybrid CNN with MVU demonstrates significant improvements in brain tumor classification.

A novel multistage ensemble approach for prediction and classification of diabetes.

Diabetes mellitus is a metabolic syndrome affecting millions of people worldwide. Every year, the rate of occurrence rises drastically. Diabetes-related problems across several vital organs of the body can be fatal if left untreated. Diabetes must be detected early to receive proper treatment, preventing the condition from escalating to severe problems. Tremendous health sciences and biotechnology advancements have resulted in massive data that generated massive Electronic Health Records and clinical information. The exponential increase of electronically gathered information has resulted in more complicated, accurate prediction models that can be updated continuously using machine learning techniques. This research mainly emphasizes discovering the best ensemble model for predicting diabetes. A new multistage ensemble model is proposed for diabetes prediction. In this model, accuracy is predicated on the Pima Indian Diabetes dataset. The accuracy of the proposed ensemble model is compared with the existing machine learning model, and the experimental results demonstrate the performance of the proposed model in terms of higher Precision, f-measure, Recall, and area under the curve.

A short‐term wind power prediction method based on deep learning and multistage ensemble algorithm

AbstractWind power prediction (WPP) is extremely important in promoting the power grid's consumption of wind power. To improve the accuracy of WPP, a three‐stage multiensemble short‐term WPP method based on ensemble learning and deep learning is proposed in this paper. In the first stage, variational mode decomposition and wavelet transform were applied to decompose the original data sequence into different frequency bands. In the second stage, based on the decomposition sequences, the stacked denoising autoencoder (SDAE), long short‐term memory (LSTM), and bidirectional long short‐term memory (BLSTM) were used to predict the sequences; and 42 submodels were obtained. In the third stage, a support vector machine (SVM) was applied to give weight to each submodel to obtain the final ensemble prediction results. Based on three‐stage integration, a new multi‐integration model is proposed that repeats the third‐stage integration operation. A case study is presented to verify the effectiveness and superiority of the proposed three‐stage multi‐integration WPP method. The normalized root mean square error (NRMSE) decreased by 0.0343 compared with LSTM, decreased by 0.0336 compared with BLSTM, and decreased by 0.0323 compared with SDAE, which demonstrated the effectiveness of the proposed new multistage ensemble and deep learning WPP method.

A Multi-Stage Ensemble Neural Network for Accurate Differentiation Between False and Real Alarms in Quality Control Charts

A Multi-Stage Ensemble Neural Network for Accurate Differentiation Between False and Real Alarms in Quality Control Charts

A multi-stage ensemble network system to diagnose adolescent idiopathic scoliosis

To develop a deep learning algorithm to automatically evaluate and diagnose scoliosis on full spinal X-ray images. This retrospective study collected full spinal X-ray images (anteroposterior) from four hospital databases from January 1, 2018, to March 31, 2021. The data were divided into training and validation sets. Full spinal X-ray images for external validation were independently collected at one hospital from April 1, 2021, to June 30, 2021. Model effectiveness was validated with a public dataset. Statistical software R was used to analyze the accuracy and sensitivity of the model curvature and anatomical balance parameters and assess interrater consistency. This study included 788 and 185 training and test datasets, respectively. The accuracy and recall of the algorithm model for the Cobb angle, apical vertebrae (AV), upper vertebrae, and lower vertebrae were 89.36%, 85.71%, 77.2%, and 80.24% and 97.35%, 93.38%, 84.11%, and 87.42%, respectively. The symmetric mean absolute percentage error at the Cobb angle was 5.99%, and the automatic measurement time was 1.7 s. The mean absolute error values of the Cobb angle and the distances between the center sacral vertical line and AV and C7 plumb line were 1.07° and 1.12 and 1.38 mm, respectively. Statistical analysis confirmed that the Cobb angle results were in good agreement with the gold standard (interclass coefficients of 0.996, 0.978, and 0.825; p < 0.001). Our deep learning algorithm model had high sensitivity and accuracy for scoliosis, which could help radiologists improve their diagnostic efficiency. •Our deep learning algorithm model had high sensitivity and accuracy for scoliosis, which could help radiologistsimprove their diagnostic efficiency. • Multi-center validation data were used in this study to guarantee the reliability of the research. • Algorithmic model measures 200 times faster than radiologists.

Automated tumor proportion scoring for PD-L1 expression based on multistage ensemble strategy in non-small cell lung cancer

IntroductionProgrammed cell death ligand-1 (PD-L1) expression is a promising biomarker for identifying treatment related to non-small cell lung cancer (NSCLC). Automated image analysis served as an aided PD-L1 scoring tool for pathologists to reduce inter- and intrareader variability. We developed a novel automated tumor proportion scoring (TPS) algorithm, and evaluated the concordance of this image analysis algorithm with pathologist scores.MethodsWe included 230 NSCLC samples prepared and stained using the PD-L1(SP263) and PD-L1(22C3) antibodies separately. The scoring algorithm was based on regional segmentation and cellular detection. We used 30 PD-L1(SP263) slides for algorithm training and validation.ResultsOverall, 192 SP263 samples and 117 22C3 samples were amenable to image analysis scoring. Automated image analysis and pathologist scores were highly concordant [intraclass correlation coefficient (ICC) = 0.873 and 0.737]. Concordances at moderate and high cutoff values were better than at low cutoff values significantly. For SP263 and 22C3, the concordances in squamous cell carcinomas were better than adenocarcinomas (SP263 ICC = 0.884 vs 0.783; 22C3 ICC = 0.782 vs 0.500). In addition, our automated immune cell proportion scoring (IPS) scores achieved high positive correlation with the pathologists TPS scores.ConclusionsThe novel automated image analysis scoring algorithm permitted quantitative comparison with existing PD-L1 diagnostic assays and demonstrated effectiveness by combining cellular and regional information for image algorithm training. Meanwhile, the fact that concordances vary in different subtypes of NSCLC samples, which should be considered in algorithm development.

Multistage Ensemble Learning Model With Weighted Voting and Genetic Algorithm Optimization Strategy for Detecting Chronic Obstructive Pulmonary Disease

Chronic Obstructive Pulmonary Disease (COPD) is a life-threatening lung ailment and a significant cause of morbidity and fatality globally. The early detection of COPD can provide timely proper medication and reduce the mortality rate. To obtain proper treatment and lessen the death rate, this study proposes a novel ensemble model: the Multistage Ensemble model (MSEN) with an optimized weighted voting technique to detect COPD early and help clinicians provide proper and timely medication. In this study, there are two pools of classifiers created in which four classifiers are placed in each pool. These two pools of classifiers are employed to form two weighted ensemble models based on a weighted voting strategy. This study combines those generated ensemble models using a weighted voting technique to form an MSEN model. The genetic algorithm is utilized to optimize the hyperparameters of each classifier in each pool. The weights of two generated ensemble models and each classifier are optimized using the grid search technique. This study employs the K-Nearest Neighbors approach to fill in the missing values, isolation forest to remove the outliers, and the LightGBM with Recursive Feature Elimination for feature selection. An evaluation of the suggested MSEN model is conducted on a real-world Exasens dataset to validate the suggested model's effectiveness which exhibits that the proposed model obtains better performance to detect COPD and provides superior performance than other machine learning models and existing benchmark techniques.

A Novel Multi-Stage Ensemble Model With a Hybrid Genetic Algorithm for Credit Scoring on Imbalanced Data

Credit scoring models are the cornerstone of the modern financial industry. After years of development, artificial intelligence and machine learning have led to the transformation of traditional credit scoring models based on statistics. In this study, a novel multi-stage ensemble model with a hybrid genetic algorithm is proposed to achieve accurate and stable credit prediction. To alleviate the adverse effects of imbalanced data in credit scoring models, the Instance Hardness Threshold method is extended using a majority voting strategy to deal with data imbalance. To eliminate redundant and irrelevant features in the dataset and select well-performing base classifiers, a new hybrid genetic algorithm is proposed to obtain the optimal feature subset and base classifier subset. To aggregate the predictive power of the base classifiers, a stacking approach is adopted to integrate the optimal base classifiers into the ensemble model. The proposed model is tested on three standard imbalanced credit scoring datasets, compared with similar state-of-the-art approaches, and evaluated using four well-known evaluation indicators. The experimental results prove the effectiveness of the proposed model and demonstrate its superiority.

Fast and accurate tumor segmentation of histology images using persistent homology and deep convolutional features.

Tumor segmentation in whole-slide images of histology slides is an important step towards computer-assisted diagnosis. In this work, we propose a tumor segmentation framework based on the novel concept of persistent homology profiles (PHPs). For a given image patch, the homology profiles are derived by efficient computation of persistent homology, which is an algebraic tool from homology theory. We propose an efficient way of computing topological persistence of an image, alternative to simplicial homology. The PHPs are devised to distinguish tumor regions from their normal counterparts by modeling the atypical characteristics of tumor nuclei. We propose two variants of our method for tumor segmentation: one that targets speed without compromising accuracy and the other that targets higher accuracy. The fast version is based on a selection of exemplar image patches from a convolution neural network (CNN) and patch classification by quantifying the divergence between the PHPs of exemplars and the input image patch. Detailed comparative evaluation shows that the proposed algorithm is significantly faster than competing algorithms while achieving comparable results. The accurate version combines the PHPs and high-level CNN features and employs a multi-stage ensemble strategy for image patch labeling. Experimental results demonstrate that the combination of PHPs and CNN features outperform competing algorithms. This study is performed on two independently collected colorectal datasets containing adenoma, adenocarcinoma, signet, and healthy cases. Collectively, the accurate tumor segmentation produces the highest average patch-level F1-score, as compared with competing algorithms, on malignant and healthy cases from both the datasets. Overall the proposed framework highlights the utility of persistent homology for histopathology image analysis.

Classifying Gene Expression Data of Cancer Using Multistage Ensemble of Neural Networks

Cancer classification based on the DNA microarray data is a difficult problem because of huge numbers of genes (features) relative to the number of training samples, and out of this huge number of genes most of are irrelevant. Therefore, it is a difficult task to create accurate classifiers for gene expression analysis that handles only a few samples with overwhelming number of noisy genes. Many contributions are already done for cancer classification of DNA microarray using neural network ensemble, which include, weighted averaging, majority voting, simple averaging and ranking. However, each method has some limitations, dependent on the application areas. For the proposed ensemble classification of gene expression data of cancer, first we extract the most relevant genes then apply a new multistage ensembles combination scheme, where the classified copy of results of training samples from first stage neural networks are used as an input features for second stage neural network. In this way we have checked the recognition accuracy for two benchmark gene expression data sets and got the encouraging results.

Multistage ensemble of feedforward neural networks for prediction of heating energy consumption

Feedforward neural network models are created for prediction of heating energy consumption of a university campus. Actual measured data are used for training and testing the models. Multistage neural network ensemble is proposed for the possible improvement of prediction accuracy. Previously trained feed-forward neural networks are first separated into clusters, using k-means algorithm, and then the best network of each cluster is chosen as a member of the ensemble. Three different averaging methods (simple, weighted and median) for obtaining ensemble output are applied. Besides this conventional approach, single radial basis neural network in the second level is used to aggregate the selected ensemble members. It is shown that heating energy consumption can be predicted with better accuracy by using ensemble of neural networks than using the best trained single neural network, while the best results are achieved with multistage ensemble.