Ensemble Learning System Research Articles

BIoMT-ISeg: Blockchain internet of medical things for intelligent segmentation.

In the quest of training complicated medical data for Internet of Medical Things (IoMT) scenarios, this study develops an end-to-end intelligent framework that incorporates ensemble learning, genetic algorithms, blockchain technology, and various U-Net based architectures. Genetic algorithms are used to optimize the hyper-parameters of the used architectures. The training process was also protected with the help of blockchain technology. Finally, an ensemble learning system based on voting mechanism was developed to combine local outputs of various segmentation models into a global output. Our method shows that strong performance in a condensed number of epochs may be achieved with a high learning rate and a small batch size. As a result, we are able to perform better than standard solutions for well-known medical databases. In fact, the proposed solution reaches 95% of intersection over the union, compared to the baseline solutions where they are below 80%. Moreover, with the proposed blockchain strategy, the detected attacks reached 76%.

A Broad Ensemble Learning System for Drifting Stream Classification

A Broad Ensemble Learning System for Drifting Stream Classification

A Novel Ensemble Learning System for Cyberattack Classification

A Novel Ensemble Learning System for Cyberattack Classification

Group Decision Making-Based Fusion for Human Activity Recognition in Body Sensor Networks.

Ensemble learning systems (ELS) have been widely utilized for human activity recognition (HAR) with multiple homogeneous or heterogeneous sensors. However, traditional ensemble approaches for HAR cannot always work well due to insufficient accuracy and diversity of base classifiers, the absence of ensemble pruning, as well as the inefficiency of the fusion strategy. To overcome these problems, this paper proposes a novel selective ensemble approach with group decision-making (GDM) for decision-level fusion in HAR. As a result, the fusion process in the ELS is transformed into an abstract process that includes individual experts (base classifiers) making decisions with the GDM fusion strategy. Firstly, a set of diverse local base classifiers are constructed through the corresponding mechanism of the base classifier and the sensor. Secondly, the pruning methods and the number of selected base classifiers for the fusion phase are determined by considering the diversity among base classifiers and the accuracy of candidate classifiers. Two ensemble pruning methods are utilized: mixed diversity measure and complementarity measure. Thirdly, component decision information from the selected base classifiers is combined by using the GDM fusion strategy and the recognition results of the HAR approach can be obtained. Experimental results on two public activity recognition datasets (The OPPORTUNITY dataset; Daily and Sports Activity Dataset (DSAD)) suggest that the proposed GDM-based approach outperforms the well-known fusion techniques and other state-of-the-art approaches in the literature.

Ship Classification in Synthetic Aperture Radar Images Based on Multiple Classifiers Ensemble Learning and Automatic Identification System Data Transfer Learning

With the continuous development of earth observation technology, space-based synthetic aperture radar (SAR) has become an important source of information for maritime surveillance, and ship classification in SAR images has also become a hot research direction in the field of maritime ship monitoring. In recent years, the remote sensing community has proposed several solutions to the problem of ship object classification in SAR images. However, it is difficult to obtain an adequate amount of labeled SAR samples for training classifiers, which limits the application of machine learning, particularly deep learning methods, in SAR image ship object classification. In contrast, as a real-time automatic tracking system for monitoring ships at sea, a ship automatic identification system (AIS) can provide a large amount of relatively easy-to-obtain labeled ship samples. Therefore, to solve the problem of SAR image ship classification and improve the classification performance of learning models with limited samples, we proposed a SAR image ship classification method based on multiple classifiers ensemble learning (MCEL) and AIS data transfer learning. The core idea of our method is to transfer the MCEL model trained on AIS data to SAR image ship classification, which mainly includes three steps: first, we use the acquired global space-based AIS data to build a dataset for ship object classification models training; then, the ensemble learning model is constructed by combining multiple base classifiers; and finally, the trained classification model is transferred to SAR images for ship type prediction. Experiments show that the proposed method achieves a classification accuracy of 85.00% for the SAR ship classification, which is better than the performance of each base classifier. This proves that AIS data transfer learning can effectively solve the problem of SAR ship classification with limited samples, and has important application value in maritime surveillance.

A screening test proposal for congenital defects based on maternal serum metabolomics profile

Historically, noninvasive techniques are only able to identify chromosomal anomalies that accounted for <50% of all congenital defects; the other congenital defects are diagnosed via ultrasound evaluations in the later stages of pregnancy. Metabolomic analysis may provide an important improvement, potentially addressing the need for novel noninvasive and multicomprehensive early prenatal screening tools. A growing body of evidence outlines notable metabolic alterations in different biofluids derived from pregnant women carrying fetuses with malformations, suggesting that such an approach may allow the discovery of biomarkers common to most fetal malformations. In addition, metabolomic investigations are inexpensive, fast, and risk-free and often generate high performance screening tests that may allow early detection of a given pathology. This study aimed to evaluate the diagnostic accuracy of an ensemble machine learning model based on maternal serum metabolomic signatures for detecting fetal malformations, including both chromosomal anomalies and structural defects. This was a multicenter observational retrospective study that included 2 different arms. In the first arm, a total of 654 Italian pregnant women (334 cases with fetuses with malformations and 320 controls with normal developing fetuses) were enrolled and used to train an ensemble machine learning classification model based on serum metabolomics profiles. In the second arm, serum samples obtained from 1935 participants of the New Zealand Screening for Pregnancy Endpoints study were blindly analyzed and used as a validation cohort. Untargeted metabolomics analysis was performed via gas chromatography-mass spectrometry. Of note, 9 individual machine learning classification models were built and optimized via cross-validation (partial least squares-discriminant analysis, linear discriminant analysis, naïve Bayes, decision tree, random forest, k-nearest neighbor, artificial neural network, support vector machine, and logistic regression). An ensemble of the models was developed according to a voting scheme statistically weighted by the cross-validation accuracy and classification confidence of the individual models. This ensemble machine learning system was used to screen the validation cohort. Significant metabolic differences were detected in women carrying fetuses with malformations, who exhibited lower amounts of palmitic, myristic, and stearic acids; N-α-acetyllysine; glucose; L-acetylcarnitine; fructose; para-cresol; and xylose and higher levels of serine, alanine, urea, progesterone, and valine (P<.05), compared with controls. When applied to the validation cohort, the screening test showed a 99.4%±0.6% accuracy (specificity of 99.9%±0.1% [1892 of 1894 controls correctly identified] with a sensitivity of 78%±6% [32 of 41 fetal malformations correctly identified]). This study provided clinical validation of a metabolomics-based prenatal screening test to detect the presence of congenital defects. Further investigations are needed to enable the identification of the type of malformation and to confirm these findings on even larger study populations.

A machine learning approach to predicting equilibrium ripple wavelength

Sand ripples are geomorphic features on the seafloor that affect bottom boundary layer dynamics including wave attenuation and sediment transport. We present a new equilibrium ripple predictor using a machine learning approach that outputs a probability distribution of wave-generated equilibrium wavelengths and statistics including an estimate of ripple height, the most probable ripple wavelength, and sediment and flow parameterizations. The Bayesian Optimal Model System (BOMS) is an ensemble machine learning system that combines two machine learning algorithms and two deterministic empirical ripple predictors with a Bayesian meta-learner to produce probabilistic wave-generated equilibrium ripple wavelength estimates in sandy locations. A ten-fold cross validation of BOMS resulted in an adjusted R-squared value of 0.93 and an average root mean square error (RMSE) of 8.0 cm. During both cross validation and testing on three unique field datasets, BOMS provided more accurate wavelength predictions than each individual base model and other common ripple predictors.

Ensemble deep learning system for early breast cancer detection

Ensemble deep learning system for early breast cancer detection

Intelligent Ensemble Deep Learning System for Blood Glucose Prediction Using Genetic Algorithms

Forecasting blood glucose (BG) values for patients can help prevent hypoglycemia and hyperglycemia events in advance. To this end, this study proposes an intelligent ensemble deep learning system to predict BG values in 15, 30, and 60 min prediction horizons (PHs) based on historical BG values collected via continuous glucose monitoring devices as an endogenous factor and carbohydrate intake and insulin administration information (times) as exogenous factors. Although there are numerous deep learning algorithms available, this study applied five algorithms, namely, recurrent neural network (RNN), which is optimized for sequence data (e.g., time‐series), and RNN‐based algorithms (e.g., long short‐term memory (LSTM), stacked LSTM, bidirectional LSTM, and gated recurrent unit). Then, a genetic algorithm (GA) was applied to the five prediction models to optimize their weights through ensemble techniques and to yield (output) the final predicted BG values. The performance of the proposed model was compared to that of the autoregressive integrated moving average (ARIMA) model as a baseline. The results show that the proposed model significantly outperforms the baseline in terms of the root mean square error (RMSE) and continuous glucose error grid analysis. For the valid 29 diabetic patients for the multivariate models, the RMSE was 11.08 (±3.19), 19.25 (±5.28), and 31.30 (±8.81) mg/DL for 15, 30, and 60 min PH, respectively. When the same data were applied to univariate models, the RMSE was 11.28 (±3.34), 19.99 (±5.59), and 33.13 (±9.27) mg/DL for 15, 30, and 60 min PH, respectively. Both the univariate and multivariate models showed a statistically significant difference compared with the baseline at a 5% statistical significance level. Instead of using a model with a single algorithm, applying a GA based on each output of a model with multiple algorithms was found to play a significant role in improving model performance.

Prediction of Drug-target Binding Affinity by An Ensemble Lear ning System with Network Fusion Information

Background: Verifying interactions between drugs and targets is key to discover new drugs. Many computational methods have been developed to predict drug-target interactions and performed successfully, but challenges still exist in the field. Objective: We have made an attempt to develop a machine learning method to predict drug-target affinity, which can determine the strength of the binding relationship between drug and target. Methods: This paper proposes an integrated machine learning system for drug-target binding affinity prediction based on network fusion. First, multiple similarity networks representing drugs or targets are calculated. Second, multiple networks representing drugs (targets) are fused separately. Finally, the characteristic information of splicing drugs and targets was used for model construction and training. By integrating multiple similarity networks, the model fully embodies the complementarity of network information, and the most complete features of information can be obtained after the redundancy is removed. Results: Experimental results showed that our model obtained good results for DTI binding affinity. Conclusion: It is still challenging to predict drug-target affinity. This paper proposes to use an integrated system of fusion network information for addressing the issue, and the proposed method performs well, which can provide a certain data basis for the subsequent work.

Proposing an ensemble learning model based on neural network and fuzzy system for keratoconus diagnosis based on Pentacam measurements

The present study was done to evaluate efficiency of an ensemble learning structure for automatic keratoconus diagnosis and to categorize eyes into four different groups based on a combination of 19 parameters obtained from Pentacam measurements. Pentacam data from 450 eyes were enrolled in the study. Eyes were separated into training, validation, and testing sets. An ensemble system was used to analyze corneal measurements and categorize the eyes into four groups. The ensemble system was trained to consider indices from both anterior and posterior corneal surfaces. Efficiency of the ensemble system was evaluated and compared in each group. The best accuracy was achieved by the ensemble system with both multilayer perceptron and neuro-fuzzy system classifiers alongside the Naïve Bayes combination method. The accuracy achieved in KC versus N distinction task was equal to 98.2% with 99.1% of sensitivity and 96.2% of specificity for KC detection. The global accuracy was equal to 98.2% for classification of 4 groups, with an average sensitivity of 98.5% and specificity of 99.4%. In this study, authority of an ensemble learning system to work out intricate problems was presented. Despite using fewer parameters, herein, comparable or, in some cases, better results were obtained than methods reported in the literature. The proposed method demonstrated very good accuracy in discriminating between normal eyes and different stages of keratoconus eyes. In some cases, it was not possible to directly compare our results with the literature, due to differences in definitions of KC group as well as differences in selection of items and parameters.

HEAL: Heterogeneous Ensemble and Active Learning Framework

Network Intrusion Detection is a well-known, relevant problem that has gained even more interest due to the exponential growth of technologies, systems and volume of data. The constantly evolving attacks require a continuous effort towards the development of novel and robust detection solutions. In this paper we propose a Heterogeneous Ensemble and Active Learning (HEAL) system, a novel tool that incorporates the implementation of a dynamic heterogeneous ensemble model with active learning capabilities. This ensures a solution that: i) adapts to changes in data through time, ii) remains robust providing good performance, iii) handles a continuous flow of data, and iv) requires less human intervention when compared against pure active learning solutions. HEAL system uses multiple individual base models to build a heterogeneous ensemble learner that adapts to the specific data characteristics. Then, active learning is applied to the ensemble so that it is retrained and re-evaluated with respect to time and new instances. Instances where the model has a low confidence are labeled by a domain expert. A new model is retrained with these instances and its performance is evaluated. The deployed model is replaced when the new model exhibits performance advantages. Finally, an experimental comparison of the performance at different stages is carried out in a case study using the well-known NSL-KDD data set. In this study we show the advantages of using HEAL system.

Prediction of Ecosystem Service Function of Grain for Green Project Based on Ensemble Learning

The Grain for Green Project (GGP) was implemented over 20 years ago as one of six major forestry projects in China, and its scope of implementation is still expanding. However, it is still unclear how many ecosystem services (ESs) the project will produce in the future. The GGP’s large-scale ecological monitoring officially started in 2015 and there is a lack of early monitoring data, making it challenging to predict the future ecological benefits. Therefore, this paper proposes a method to predict future ESs by using ecological monitoring data. First, a new ensemble learning system, auto-XGBoost-ET-DT, is developed based on ensemble learning theory. Using the GGP’s known ESs in 2015, 2017, and 2019, the missing ESs of the past decade have been evaluated via reverse regression. Data from 2020 to 2022 within a convolution neural network and the coupling coordination degree model have been used to analyze the coupling between the prediction results and economic development. The results show that the growth distributions of ESs in three years were as follows: soil consolidation, 3.70–6.34%; forest nutrient retention, 2.72–.71%; water conservation, 2.52–6.09%; carbon fixation and oxygen release, 3.00–4.64%; and dust retention, 3.82–5.75%. The coupling coordination degree of the ESs and economic development has been improved in 97% of counties in 2020 compared with 2019. The results verify a feasible ES prediction method and provide a basis for the progressive implementation of the GGP.

Incremental Weighted Ensemble Broad Learning System for Imbalanced Data

Broad learning system (BLS) is a novel and efficient model, which facilitates representation learning and classification by concatenating feature nodes and enhancement nodes. In spite of the efficient properties, BLS is still suboptimal when facing with imbalance problem. Besides, outliers and noises in imbalanced data remain a challenge for BLS. To address the above issues, in this paper we first propose a weighted BLS, which assigns a weight to each training sample, and adopt a general weighting scheme, which augments the weight of samples from the minority class. To further explore the prior distribution of original data, we design a density based weight generation mechanism to guide the specific weight matrix generation and propose the adaptive weighted broad learning system (AWBLS). This mechanism considers the inter-class and intra-class distance simultaneously in the density calculation. Finally, we propose the incremental weighted ensemble broad learning system (IWEB) by utilizing a progressive mechanism to further improve the stability and robustness of AWBLS. Extensive comparative experiments on 38 real-world data sets verfy that IWEB outperforms most of the imbalance ensemble classification methods.

An analytics model for TelecoVAS customers\u2019 basket clustering using ensemble learning approach

Value-Added Services at a Mobile Telecommunication company provide customers with a variety of services. Value-added services generate significant revenue annually for telecommunication companies. Providing solutions that can provide customers of a telecommunication company with relevant and engaging services has become a major challenge in this field. Numerous methods have been proposed so far to analyze customer basket and provide related services. Although these methods have many applications, they still face difficulties in improving the accuracy of bids. This paper combines the X-Means algorithm, the ensemble learning system, and the N-List structure to analyze the customer portfolio of a mobile telecommunication company and provide value-added services. The X-Means algorithm is used to determine the optimal number of clusters and clustering of customers in a mobile telecommunication company. The ensemble learning algorithm is also used to assign categories to new Elder customers, and finally to the N-List structure for customer basket analysis. By simulating the proposed method and comparing it with other methods including KNN, SVM, and deep neural networks, the accuracy improved to about 7%.

Multi source retinal fundus image classification using convolution neural networks fusion and Gabor-based texture representation

Glaucoma is one of the most known irreversible chronic eye disease that leads to permanent blindness but its earlier diagnosis can be treated. Convolutional neural networks (CNNs), a branch of deep learning, have an impressive record for applications in image analysis and interpretation, including medical imaging. This necessity is justified by their capacity and adaptability to extract pertinent features automatically from the original image. In other hand, the use of ensemble learning algorithms has an important impact to improve the classification rate. In this paper, a two-stage-based image processing and ensemble learning approach is proposed for automated glaucoma diagnosis. In the first stage, the generation of different modalities from original images is adopted by the application of advanced image processing techniques especially Gabor filter-based texture image. Next, each dataset constructing from the corresponding modality will be learned by an individual CNN classifier. Aggregation techniques will be then applied to generate the final decision taking into account the outputs of all CNNs classifiers. Experiments were carried out on Rime-One dataset for glaucoma diagnosis. The obtained results proved the superiority of the proposed ensemble learning system compared to the existing studies with classification accuracy of 89.63%.

Multi source retinal fundus image classification using convolution neural networks fusion and Gabor-based texture representation

Glaucoma is one of the most known irreversible chronic eye disease that leads to permanent blindness but its earlier diagnosis can be treated. Convolutional neural networks (CNNs), a branch of deep learning, have an impressive record for applications in image analysis and interpretation, including medical imaging. This necessity is justified by their capacity and adaptability to extract pertinent features automatically from the original image. In other hand, the use of ensemble learning algorithms has an important impact to improve the classification rate. In this paper, a two-stage-based image processing and ensemble learning approach is proposed for automated glaucoma diagnosis. In the first stage, the generation of different modalities from original images is adopted by the application of advanced image processing techniques especially Gabor filter-based texture image. Next, each dataset constructing from the corresponding modality will be learned by an individual CNN classifier. Aggregation techniques will be then applied to generate the final decision taking into account the outputs of all CNNs classifiers. Experiments were carried out on Rime-One dataset for glaucoma diagnosis. The obtained results proved the superiority of the proposed ensemble learning system compared to the existing studies with classification accuracy of 89.63%.

Optimizing Sensor Deployment for Multi-Sensor-Based HAR System with Improved Glowworm Swarm Optimization Algorithm.

Human activity recognition (HAR) technology that analyzes and fuses the data acquired from various homogeneous or heterogeneous sensor sources has motivated the development of enormous human-centered applications such as healthcare, fitness, ambient assisted living and rehabilitation. The concurrent use of multiple sensor sources for HAR is a good choice because the plethora of user information provided by the various sensor sources may be useful. However, a multi-sensor system with too many sensors will bring large power consumption and some sensor sources may bring little improvements to the performance. Therefore, the multi-sensor deployment research that can gain a tradeoff among computational complexity and performance is imperative. In this paper, we propose a multi-sensor-based HAR system whose sensor deployment can be optimized by selective ensemble approaches. With respect to optimization of the sensor deployment, an improved binary glowworm swarm optimization (IBGSO) algorithm is proposed and the sensor sources that have a significant effect on the performance of HAR are selected. Furthermore, the ensemble learning system based on optimized sensor deployment is constructed for HAR. Experimental results on two datasets show that the proposed IBGSO-based multi-sensor deployment approach can select a smaller number of sensor sources while achieving better performance than the ensemble of all sensors and other optimization-based selective ensemble approaches.

Performance-weighted-voting model: An ensemble machine learning method for cancer type classification using whole-exome sequencing mutation.

With improvements in next-generation DNA sequencing technology, lower cost is needed to collect genetic data. More machine learning techniques can be used to help with cancer analysis and diagnosis. We developed an ensemble machine learning system named performance-weighted-voting model for cancer type classification in 6,249 samples across 14 cancer types. Our ensemble system consists of five weak classifiers (logistic regression, SVM, random forest, XGBoost and neural networks). We first used cross-validation to get the predicted results for the five classifiers. The weights of the five weak classifiers can be obtained based on their predictive performance by solving linear regression functions. The final predicted probability of the performance-weighted-voting model for a cancer type can be determined by the summation of each classifier's weight multiplied by its predicted probability. Using the somatic mutation count of each gene as the input feature, the overall accuracy of the performance-weighted-voting model reached 71.46%, which was significantly higher than the five weak classifiers and two other ensemble models: the hard-voting model and the soft-voting model. In addition, by analyzing the predictive pattern of the performance-weighted-voting model, we found that in most cancer types, higher tumor mutational burden can improve overall accuracy. This study has important clinical significance for identifying the origin of cancer, especially for those where the primary cannot be determined. In addition, our model presents a good strategy for using ensemble systems for cancer type classification.

A new approach to diagnosing prostate cancer through magnetic resonance imaging

This paper combines improved GrowCut and Zernik feature extraction and ensemble learning techniques such as KNN, SVM, and MLP algorithms to prostate cancer detection and lesion segmentation in MRI. We use improved GrowCut algorithm for segmentation of the suspected cancer area and the combination of machine learning algorithms such as KNN, SVM, and MLP in the ensemble learning system to detect prostate cancer. We found that the accuracy of this method, which is a combination of several methods, improved by about 20% compared to other methods. Other metrics such as precision, recall, and error of proposed method have been improved.