Ensemble Classification Model Research Articles

An Efficient Multi-Class Privacy-Preserving-Based Encryption Framework for Large Distributed Databases

This paper introduces a novel hybrid filter-based ensemble multi-class classification model for distributed privacy-preserving applications. The conventional privacy-preserving multi-class learning models have limited capacity to enhance the true positive rate, mainly due to computational time and memory constraints, as well as the static nature of metrics for parameter optimization and multi-class perturbation processes. In this research, we develop the proposed model on large medical and market databases with the aim of enhancing multi-party data confidentiality through a security framework during the privacy-preserving process. Moreover, we also introduce a secure multi-party data perturbation process to improve computational efficiency and privacy-preserving performance. Experimental results were evaluated on different real-time privacy-preserving datasets, such as medical and market datasets, using different statistical metrics. The evaluation results demonstrate that the proposed multi-party-based multi-class privacy-preserving model performs statistically better than conventional approaches.

Classification method for multiple power quality disturbances via label distribution enhancement and multi‐granular feature optimization

AbstractLarge‐scale integration of distributed generation and widespread use of power electronic equipment make power quality disturbances (PQDs) more complicated. There are still unsolved problems for the classification of multiple power quality disturbances (MPQDs) that consist of various kinds of single disturbances: (1) since the difference between the contribution degrees of every contained single disturbance to the composite disturbance is ignored, the logical labels cannot completely describe the composite disturbance; (2) the optimal feature in one granular space may not be optimal in another. These drawbacks lead to the degradation of MPQD classification, which should be considered a multi‐label model rather than single‐label model used in existing methods. Therefore, this paper proposes a novel method to improve the classification performance of MPQDs. The label distribution, representing the contribution degree of a single disturbance to the composite disturbance, is introduced. In addition, the high‐dimensional feature space is reduced by multigranular optimization, where the fuzziness and redundancy are removed by the modified rough‐set method. To improve the performance of the classifier, the ensemble classification model based on homogeneous classifier integration is proposed to integrate the base classifiers constructed by the feature vectors from different granularity spaces. A large number of field recordings are applied to validate the proposed method. The results show that the proposed method performs better than traditional methods, especially under noisy environments.

Machine Learning and Causal Approaches to Predict Readmissions and Its Economic Consequences Among Canadian Patients With Heart Disease: Retrospective Study.

Unplanned patient readmissions within 30 days of discharge pose a substantial challenge in Canadian health care economics. To address this issue, risk stratification, machine learning, and linear regression paradigms have been proposed as potential predictive solutions. Ensemble machine learning methods, such as stacked ensemble models with boosted tree algorithms, have shown promise for early risk identification in specific patient groups. This study aims to implement an ensemble model with submodels for structured data, compare metrics, evaluate the impact of optimized data manipulation with principal component analysis on shorter readmissions, and quantitatively verify the causal relationship between expected length of stay (ELOS) and resource intensity weight (RIW) value for a comprehensive economic perspective. This retrospective study used Python 3.9 and streamlined libraries to analyze data obtained from the Discharge Abstract Database covering 2016 to 2021. The study used 2 sub-data sets, clinical and geographical data sets, to predict patient readmission and analyze its economic implications, respectively. A stacking classifier ensemble model was used after principal component analysis to predict patient readmission. Linear regression was performed to determine the relationship between RIW and ELOS. The ensemble model achieved precision and slightly higher recall (0.49 and 0.68), indicating a higher instance of false positives. The model was able to predict cases better than other models in the literature. Per the ensemble model, readmitted women and men aged 40 to 44 and 35 to 39 years, respectively, were more likely to use resources. The regression tables verified the causality of the model and confirmed the trend that patient readmission is much more costly than continued hospital stay without discharge for both the patient and health care system. This study validates the use of hybrid ensemble models for predicting economic cost models in health care with the goal of reducing the bureaucratic and utility costs associated with hospital readmissions. The availability of robust and efficient predictive models, as demonstrated in this study, can help hospitals focus more on patient care while maintaining low economic costs. This study predicts the relationship between ELOS and RIW, which can indirectly impact patient outcomes by reducing administrative tasks and physicians' burden, thereby reducing the cost burdens placed on patients. It is recommended that changes to the general ensemble model and linear regressions be made to analyze new numerical data for predicting hospital costs. Ultimately, the proposed work hopes to emphasize the advantages of implementing hybrid ensemble models in forecasting health care economic cost models, empowering hospitals to prioritize patient care while simultaneously decreasing administrative and bureaucratic expenses.

Semi-supervised classifier ensemble model for high-dimensional data

To complete the challenging task of high-dimensional data classification with limited labeled samples, we propose two semi-supervised learning models, namely the random subspace classifier ensemble model (SSRS) and its adaptive version (ASSRS). Considering the unique characteristics of high-dimensional data, SSRS selects subspaces of sample and feature dimensions and then reduces the dimensions of each subspace. To improve SSRS performance further, we designed a sample-labeling auxiliary algorithm, adaptive sample subspace algorithm, and adaptive weight voting rule for ASSRS to increase the proportion of labeled samples, obtain a suitable sample subspace for each feature subspace, and acquire a relative optimal weight for each base classifier. Experiments revealed that the performances of SSRS and ASSRS were better than those of other competitive algorithms and that the performance of ASSRS was stronger than that of SSRS. Additionally, we can accurately label samples in datasets where the proportion of labeled samples is relatively low by using SSRS and ASSRS. Because analysts are facing large numbers of high-dimensional datasets with limited labels, it is important to make accurate predictions based on a limited proportion of labeled data.

Deep learning-based ensemble model for classification of breast cancer

Deep learning-based ensemble model for classification of breast cancer

Detecting COVID-19 from Chest X-rays Using Convolutional Neural Network Ensembles

Starting in late 2019, the coronavirus SARS-CoV-2 began spreading around the world and causing disruption in both daily life and healthcare systems. The disease is estimated to have caused more than 6 million deaths worldwide [WHO]. The pandemic and the global reaction to it severely affected the world economy, causing a significant increase in global inflation rates, unemployment, and the cost of energy commodities. To stop the spread of the virus and dampen its global effect, it is imperative to detect infected patients early on. Convolutional neural networks (CNNs) can effectively diagnose a patient’s chest X-ray (CXR) to assess whether they have been infected. Previous medical image classification studies have shown exceptional accuracies, and the trained algorithms can be shared and deployed using a computer or a mobile device. CNN-based COVID-19 detection can be employed as a supplement to reverse transcription-polymerase chain reaction (RT-PCR). In this research work, 11 ensemble networks consisting of 6 CNN architectures and a classifier layer are evaluated on their ability to differentiate the CXRs of patients with COVID-19 from those of patients that have not been infected. The performance of ensemble models is then compared to the performance of individual CNN architectures. The best ensemble model COVID-19 detection accuracy was achieved using the logistic regression ensemble model, with an accuracy of 96.29%, which is 1.13% higher than the top-performing individual model. The highest F1-score was achieved by the standard vector classifier ensemble model, with a value of 88.6%, which was 2.06% better than the score achieved by the best-performing individual model. This work demonstrates that combining a set of top-performing COVID-19 detection models could lead to better results if the models are integrated together into an ensemble. The model can be deployed in overworked or remote health centers as an accurate and rapid supplement or back-up method for detecting COVID-19.

Predicting Energy Consumption in Residential Buildings Using Advanced Machine Learning Algorithms

The share of residential building energy consumption in global energy consumption has rapidly increased after the COVID-19 crisis. The accurate prediction of energy consumption under different indoor and outdoor conditions is an essential step towards improving energy efficiency and reducing carbon footprints in the residential building sector. In this paper, a PSO-optimized random forest classification algorithm is proposed to identify the most important factors contributing to residential heating energy consumption. A self-organizing map (SOM) approach is applied for feature dimensionality reduction, and an ensemble classification model based on the stacking method is trained on the dimensionality-reduced data. The results show that the stacking model outperforms the other models with an accuracy of 95.4% in energy consumption prediction. Finally, a causal inference method is introduced in addition to Shapley Additive Explanation (SHAP) to explore and analyze the factors influencing energy consumption. A clear causal relationship between water pipe temperature changes, air temperature, and building energy consumption is found, compensating for the neglect of temperature in the SHAP analysis. The findings of this research can help residential building owners/managers make more informed decisions around the selection of efficient heating management systems to save on energy bills.

Analysis of Stacked Ensemble Classification Models in Cervical Cancer Prediction using Machine learning

Analysis of Stacked Ensemble Classification Models in Cervical Cancer Prediction using Machine learning

Underground Water Level Prediction in Remote Sensing Images Using Improved Hydro Index Value with Ensemble Classifier

The economic sustainability of aquifers across the world relies on accurate and rapid estimates of groundwater storage changes, but this becomes difficult due to the absence of in-situ groundwater surveys in most areas. By closing the water balance, hydrologic remote sensing measures offer a possible method for quantifying changes in groundwater storage. However, it is uncertain to what extent remote sensing data can provide an accurate assessment of these changes. Therefore, a new framework is implemented in this work for predicting the underground water level using remote sensing images. Generally, the water level is defined into five levels: Critical, Overexploited, Safe, Saline, and Semi-critical, based on water quantity. In this manuscript, the remote sensing images were acquired from remote sensing images. At first, Wiener filtering was employed for preprocessing. Secondly, the Vegetation Indexes (VI) (Normalized Difference Vegetation Index (NDVI), Normalized Difference Snow Index (NDSI), Infrared index (IRI), Radar Vegetation Index (RVI)), and statistical features (entropy, Root Mean Square (RMS), Skewness, and Kurtosis) were extracted from the preprocessed remote sensing images. Then, the extracted features were combined as a novel hydro index, which was fed to the Ensemble Classifier (EC): Neural Networks (NN), Support Vector Machine (SVM), and improved Deep Convolutional Neural Network (DCNN) models for underground water level prediction in the remote sensing images. The obtained results prove the efficacy of the proposed framework by using different performance measures. The results shows that the False Positive Rate (FPR) of the proposed EC model is 0.0083, which is better than that of existing methods. On the other hand, the proposed EC model has a high accuracy of 0.90, which is superior to the existing traditional models: Long Short-Term Memory (LSTM) network, Naïve Bayes (NB), Random Forest (RF), Recurrent Neural Network (RNN), and Bidirectional Gated Recurrent Unit (Bi-GRU).

Predicting medical device failure: a promise to reduce healthcare facilities cost through smart healthcare management.

The advancement of biomedical research generates myriad healthcare-relevant data, including medical records and medical device maintenance information. The COVID-19 pandemic significantly affects the global mortality rate, creating an enormous demand for medical devices. As information technology has advanced, the concept of intelligent healthcare has steadily gained prominence. Smart healthcare utilises a new generation of information technologies, such as the Internet of Things (loT), big data, cloud computing, and artificial intelligence, to completely transform the traditional medical system. With the intention of presenting the concept of smart healthcare, a predictive model is proposed to predict medical device failure for intelligent management of healthcare services. Present healthcare device management can be improved by proposing a predictive machine learning model that prognosticates the tendency of medical device failures toward smart healthcare. The predictive model is developed based on 8,294 critical medical devices from 44 different types of equipment extracted from 15 healthcare facilities in Malaysia. The model classifies the device into three classes; (i) class 1, where the device is unlikely to fail within the first 3 years of purchase, (ii) class 2, where the device is likely to fail within 3 years from purchase date, and (iii) class 3 where the device is likely to fail more than 3 years after purchase. The goal is to establish a precise maintenance schedule and reduce maintenance and resource costs based on the time to the first failure event. A machine learning and deep learning technique were compared, and the best robust model for smart healthcare was proposed. This study compares five algorithms in machine learning and three optimizers in deep learning techniques. The best optimized predictive model is based on ensemble classifier and SGDM optimizer, respectively. An ensemble classifier model produces 77.90%, 87.60%, and 75.39% for accuracy, specificity, and precision compared to 70.30%, 83.71%, and 67.15% for deep learning models. The ensemble classifier model improves to 79.50%, 88.36%, and 77.43% for accuracy, specificity, and precision after significant features are identified. The result concludes although machine learning has better accuracy than deep learning, more training time is required, which is 11.49 min instead of 1 min 5 s when deep learning is applied. The model accuracy shall be improved by introducing unstructured data from maintenance notes and is considered the author's future work because dealing with text data is time-consuming. The proposed model has proven to improve the devices' maintenance strategy with a Malaysian Ringgit (MYR) cost reduction of approximately MYR 326,330.88 per year. Therefore, the maintenance cost would drastically decrease if this smart predictive model is included in the healthcare management system.

Weighted Average Ensemble Deep Learning Model for Stratification of Brain Tumor in MRI Images.

Brain tumor diagnosis at an early stage can improve the chances of successful treatment and better patient outcomes. In the biomedical industry, non-invasive diagnostic procedures, such as magnetic resonance imaging (MRI), can be used to diagnose brain tumors. Deep learning, a type of artificial intelligence, can analyze MRI images in a matter of seconds, reducing the time it takes for diagnosis and potentially improving patient outcomes. Furthermore, an ensemble model can help increase the accuracy of classification by combining the strengths of multiple models and compensating for their individual weaknesses. Therefore, in this research, a weighted average ensemble deep learning model is proposed for the classification of brain tumors. For the weighted ensemble classification model, three different feature spaces are taken from the transfer learning VGG19 model, Convolution Neural Network (CNN) model without augmentation, and CNN model with augmentation. These three feature spaces are ensembled with the best combination of weights, i.e., weight1, weight2, and weight3 by using grid search. The dataset used for simulation is taken from The Cancer Genome Atlas (TCGA), having a lower-grade glioma collection with 3929 MRI images of 110 patients. The ensemble model helps reduce overfitting by combining multiple models that have learned different aspects of the data. The proposed ensemble model outperforms the three individual models for detecting brain tumors in terms of accuracy, precision, and F1-score. Therefore, the proposed model can act as a second opinion tool for radiologists to diagnose the tumor from MRI images of the brain.

Ensemble graph neural network model for classification of major depressive disorder using whole-brain functional connectivity.

Major depressive disorder (MDD) is characterized by impairments in mood and cognitive functioning, and it is a prominent source of global disability and stress. A functional magnetic resonance imaging (fMRI) can aid clinicians in their assessments of individuals for the identification of MDD. Herein, we employ a deep learning approach to the issue of MDD classification. Resting-state fMRI data from 821 individuals with MDD and 765 healthy controls (HCs) is employed for investigation. An ensemble model based on graph neural network (GNN) has been created with the goal of identifying patients with MDD among HCs as well as differentiation between first-episode and recurrent MDDs. The graph convolutional network (GCN), graph attention network (GAT), and GraphSAGE models serve as a base models for the ensemble model that was developed with individual whole-brain functional networks. The ensemble's performance is evaluated using upsampling and downsampling, along with 10-fold cross-validation. The ensemble model achieved an upsampling accuracy of 71.18% and a downsampling accuracy of 70.24% for MDD and HC classification. While comparing first-episode patients with recurrent patients, the upsampling accuracy is 77.78% and the downsampling accuracy is 71.96%. According to the findings of this study, the proposed GNN-based ensemble model achieves a higher level of accuracy and suggests that our model produces can assist healthcare professionals in identifying MDD.

A New Method for Determination of Optimal Borehole Drilling Location Considering Drilling Cost Minimization and Sustainable Groundwater Management.

Drilling boreholes for the exploration of groundwater incurs high cost with potential risk of failures. However, borehole drilling should only be done in regions with a high probability of faster and easier access to water-bearing strata, so that groundwater resources can be effectively managed. However, regional strati-graphic uncertainties drive the decision of the optimal drilling location search. Unfortunately, due to the unavailability of a robust solution, most contemporary solutions rely on physical testing methods that are resource intensive. In this regard, a pilot study is conducted to determine the optimal borehole drilling location using a predictive optimization technique that takes strati-graphic uncertainties into account. The study is conducted in a localized region of the Republic of Korea using a real borehole data set. In this study we proposed an enhanced Firefly optimization algorithm based on an inertia weight approach to find an optimal location. The results of the classification and prediction model serve as an input to the optimization model to implement a well-crafted objective function. For predictive modeling a deep learning based chained multioutput prediction model is developed to predict groundwater-level and drilling depth. For classification of soil color and land-layer a weighted voting ensemble classification model based on Support Vector Machines, Gaussian Naïve Bayes, Random Forest, and Gradient Boosted Machine is developed. For weighted voting, an optimal set of weights is determined using a novel hybrid optimization algorithm. Experimental results validate the effectiveness of the proposed strategy. The proposed classification model achieved an accuracy of 93.45% and 95.34% for soil-color and land-layer, respectively. While the mean absolute error achieved by proposed prediction model for groundwater level and drilling depth is 2.89% and 3.11%, respectively. It is found that the proposed predictive optimization framework can adaptively determine the optimal borehole drilling locations for high strati-graphic uncertainty regions. The findings of the proposed study provide an opportunity to the drilling industry and groundwater boards to achieve sustainable resource management and optimal drilling performance.

Deep feature fusion and optimized feature selection based ensemble classification of liver lesions

ABSTRACT Classification of liver abnormalities is crucial for the early identification of liver cancer. In clinical settings, radiological professionals typically make diagnoses manually which is subjective, time-consuming and vulnerable to error. Therefore, there is still a demand for precise classification of liver diseases. We propose an Ensemble learning-based classification model to classify liver lesions on CT images. To excerpt all the essential facts from the image, deep feature fusion is incorporated by concatenating the features from pre-trained deep CNN models densenet201 and InceptionResnetV2. To minimize the feature space and boost classification accuracy, hybrid optimization methodologies, Genetic Algorithms and Ant Colony Optimization are applied. Finally, a Heterogeneous Ensemble classifier divides the retrieved features into four groups (liver abscess, liver cirrhosis, hepatocellular carcinoma, and metastasis). It is clearly seen and observed that 98.3% accuracy is contributed by ensemble classifier with the support of concatenated deep features and this classifier excels in all other ways and means.

Ensemble Classification Model for EV Identification From Smart Meter Recordings

Electric vehicles (EVs) often consume large amounts of energy, and uncoordinated charging of many EVs may lead to grid overload, adversely impacting other customers. Electricity distributors require full visibility on the EV distribution to better manage operation planning of their distribution grid. However, they often have incomplete knowledge of EV presence in their network. Identifying EV customers (charging at home) using smart meter data is a nontrivial task for the grid network and energy scheduling. The difficulties include recognizing charging patterns, balancing the number of EV and non-EV customers during modeling, and building an efficient classification model. In this article, we propose a periodic pattern recognition method to extract useful EV charging patterns. Real world smart meter datasets are unbalanced with few EVs and majority of energy customers are those without EVs. We improve Kmedoids evaluated by dynamic time warping to obtain the representative non-EV training samples so that balanced samples over EV and non-EV customers can be obtained. We develop an ensemble classification model (ECM) by taking advantages of multiple classifiers, in which the optimization consists of obtaining the optimal subset of periodic patterns and the optimal parameters in each classifier and the optimal weights for combining classifiers. The superiority of the proposed ECM is demonstrated in comparison to several baseline models.

Ensemble of deep transfer learning models for real-time automatic detection of face mask.

The COVID-19 pandemic is causing a global health crisis. Public spaces need to be safeguarded from the adverse effects of this pandemic. Wearing a facemask has become an adequate protection solution many governments adopt. Manual real-time monitoring of face mask wearing for many people is becoming a difficult task. This paper applies three heterogeneous deep transfer learning models, viz., ResNet50, Inception-v3, and VGG-16, to prepare an ensemble classification model for detecting whether a person is wearing a mask. The ensemble classification model is underlined by the concept of the weighted average technique. The proposed framework is based on two phases. An off-line phase that aims to prepare a classification model by following training-testing steps to detect and locate facemasks. Then in the second online phase, it is deployed to detect real-time faces from live videos, which are captured by a web-camera. The prepared model is compared with several state-of-the-art models. The proposed model has achieved the highest classification accuracy of 99.97%, precision of 0.997, recall of 0.997, F1-score of 0.997 and kappa coefficient 0.994. The superiority of the model over state-of-the-art compared methods is well evident from the experimental results.

Wearable sensor-based gait analysis to discriminate early Parkinson’s disease from essential tremor

BackgroundDifferentiating early-stage Parkinson's disease (PD) from essential tremor (ET) is challenging since they have some overlapping clinical features. Since early-stage PD may present with slight gait impairment and ET generally does not, gait analysis could be used to differentiate PD from ET using machine learning.ObjectiveTo differentiate early-stage PD from ET via machine learning using gait and postural transition parameters calculated using the raw kinematic signal captured from inertial measurement unit (IMU) sensors.MethodsGait and postural transition parameters were collected from 84 early-stage PD and 80 ET subjects during the Time Up and Go (TUG) test. We randomly split our data into training and test data. Within the training data, we separated the TUG test into four components: standing, straight walk, turning, and sitting to build weighted average ensemble classification models. The four components’ weight indices were trained using logistic regression. Several ensemble models’ leave-one-out cross-validation (LOOCV) performances were compared. Independent test data were used to evaluate the model with the best LOOCV performance.ResultsThe best weighted average ensemble classification model LOOCV results included an accuracy of 84%, Kappa of 0.68, sensitivity of 85.9%, specificity of 82.1%, and AUC of 0.912. Thirty-three gait and postural transition parameters, such as Arm–Symbolic Symmetry Index and 180° Turn–Max Angular Velocity, were included in Feature Group III. The independent test data achieved a 75.8% accuracy.ConclusionsOur findings suggest that gait and postural transition parameters obtained from wearable sensors combined with machine learning had the potential to distinguish between early-stage PD and ET.

Modeling human's collision avoidance direction in an encounter situation using an ensemble classifier

Predicting and responding to a vessel's collision avoidance course is crucial for navigational safety. To predict the course of deciding collision avoidance, the results of human decisions should be modeled to avoid collision in a potentially collision-prone situation. This study derives a supervised learning-based ensemble classification model to model humans' decisions to avoid collision. To illustrate the model, ship trajectory data were collected and used to construct predictor and outcome variables. Figures obtained from the collision-avoidance algorithm were included in the prediction variables. These variables were applied to the classification model. Thus, a well-fitted model with high performance was constructed. This model provides insight to predict the relationship between an avoidance direction output by a collision avoidance algorithm and the variables used in the algorithm. The results indicate that the proposed model can be used to predict the direction of avoidance in a given situation.

Deep Feature Meta-Learners Ensemble Models for COVID-19 CT Scan Classification

The infectious nature of the COVID-19 virus demands rapid detection to quarantine the infected to isolate the spread or provide the necessary treatment if required. Analysis of COVID-19-infected chest Computed Tomography Scans (CT scans) have been shown to be successful in detecting the disease, making them essential in radiology assessment and screening of infected patients. Single-model Deep CNN models have been used to extract complex information pertaining to the CT scan images, allowing for in-depth analysis and thereby aiding in the diagnosis of the infection by automatically classifying the chest CT scan images as infected or non-infected. The feature maps obtained from the final convolution layer of the Deep CNN models contain complex and positional encoding of the images’ features. The ensemble modeling of these Deep CNN models has been proved to improve the classification performance, when compared to a single model, by lowering the generalization error, as the ensemble can meta-learn from a broader set of independent features. This paper presents Deep Ensemble Learning models to synergize Deep CNN models by combining these feature maps to create deep feature vectors or deep feature maps that are then trained on meta shallow and deep learners to improve the classification. This paper also proposes a novel Attentive Ensemble Model that utilizes an attention mechanism to focus on significant feature embeddings while learning the Ensemble feature vector. The proposed Attentive Ensemble model provided better generalization, outperforming Deep CNN models and conventional Ensemble learning techniques, as well as Shallow and Deep meta-learning Ensemble CNNs models. Radiologists can use the presented automatic Ensemble classification models to assist identify infected chest CT scans and save lives.

Weighted ensemble model for image classification.

The Deep Convolutional Neural Network (DCNN) classification models are being tremendously used across many research fields including medical science for image classification. The accuracy of the model and reliability on the results of the model are the key attributes which determine whether a particular model should be used for a specific application or not. A highly accurate model is always desirable for all applications of machine learning as well as deep learning. This paper presents a DCNN based heterogeneous ensemble approach where all DCNN models can be trained on a single dataset and each model can contribute of towards the final output of the ensemble model. The contribution of each model is weighted according to its individual accuracy on the given dataset. Models with higher accuracy has higher contribution in the final output of ensemble model, whereas the models with lower accuracy has lower contribution. This approach, when tested on two different X-ray images datasets of Covid-19, has confirmed the significant increase in 3-class accuracy as compared to the models in literature.