Ensemble Deep Learning Method Research Articles

Feasibility Study on the Influence of Data Partition Strategies on Ensemble Deep Learning: The Case of Forecasting Power Generation in South Korea

Ensemble deep learning methods have demonstrated significant improvements in forecasting the solar panel power generation using historical time-series data. Although many studies have used ensemble deep learning methods with various data partitioning strategies, most have only focused on improving the predictive methods by associating several different models or combining hyperparameters and interactions. In this study, we contend that we can enhance the precision of power generation forecasting by identifying a suitable data partition strategy and establishing the ideal number of partitions and subset sizes. Thus, we propose a feasibility study of the influence of data partition strategies on ensemble deep learning. We selected five time-series data partitioning strategies—window, shuffle, pyramid, vertical, and seasonal—that allow us to identify different characteristics and features in the time-series data. We conducted various experiments on two sources of solar panel datasets collected in Seoul and Gyeongju, South Korea. Additionally, LSTM-based bagging ensemble models were applied to combine the advantages of several single LSTM models. The experimental results reveal that the data partition strategies positively influence the forecasting of power generation. Specifically, the results demonstrate that ensemble models with data partition strategies outperform single LSTM models by approximately 4–11% in terms of the coefficient of determination (R2) score.

Detection and classification of skin diseases with ensembles of deep learning networks in medical imaging

Skin disorders are a serious worldwide public health issue that affects a large number of individuals. In recent years, with the fast advancement of technology and the use of different data mining approaches, treatment of skin predictive classification has really become highly predictive as well as accurate. As a result, the type of machine learning approaches capable of efficiently differentiating skin condition categorization is essential. So far, no one machine learning approach has outperformed the others in terms of skin disease prediction. In this research, we introduce a new method that combines two separate data mining approaches into a single unit, as well as an ensemble approach that combines both data mining techniques into a single group. We explore different data mining strategies to categorize the skin condition using an informative Dermatology publicly accessible dataset ISIC2019 images, and then apply an ensemble deep learning method. Furthermore, the presented ensemble technique, which is based on machine and deep learning, was tested on Dermatology datasets and was able to categorize skin disorders into seven categories.

Detection of Metastatic Breast Cancer from Whole-Slide Pathology Images Using an Ensemble Deep-Learning Method

Background: Metastasis is the main cause of death toll among breast cancer patients. Since current approaches for diagnosis of lymph node metastases are time-consuming, deep learning (DL) algorithms with more speed and accuracy are explored for effective alternatives. Methods: A total of 220025 whole-slide pictures from patients’ lymph nodes were classified into two cohorts: testing and training. For metastatic cancer identification, we employed hybrid convolutional network models. The performance of our diagnostic system was verified using 57458 unlabeled images that utilized criteria that included accuracy, sensitivity, specificity, and P-value. Results: The DL-based system that was automatically and exclusively capable of quantifying and identifying metastatic lymph nodes was engineered. Quantification was made with 98.84% accuracy. Moreover, the precision of VGG16 and Recall was 92.42% and 91.25%, respectively. Further experiments demonstrated that metastatic cancer differentiation levels could influence the recognition performance. Conclusion: Our engineered diagnostic complex showed an elevated level of precision and efficiency for lymph node diagnosis. Our innovative DL-based system has a potential to simplify pathological screening for metastasis in breast cancer patients.

Comprehensive Study for Breast Cancer Using Deep Learning and Traditional Machine Learning

Comprehensive Study for Breast Cancer Using Deep Learning and Traditional Machine Learning

A Voting-Based Ensemble Deep Learning Method Focused on Multi-Step Prediction of Food Safety Risk Levels: Applications in Hazard Analysis of Heavy Metals in Grain Processing Products.

Grain processing products constitute an essential component of the human diet and are among the main sources of heavy metal intake. Therefore, a systematic assessment of risk factors and early-warning systems are vital to control heavy metal hazards in grain processing products. In this study, we established a risk assessment model to systematically analyze heavy metal hazards and combined the model with the K-means++ algorithm to perform risk level classification. We then employed deep learning models to conduct a multi-step prediction of risk levels, providing an early warning of food safety risks. By introducing a voting-ensemble technique, the accuracy of the prediction model was improved. The results indicated that the proposed model was superior to other models, exhibiting the overall accuracy of 90.47% in the 7-day prediction and thus satisfying the basic requirement of the food supervision department. This study provides a novel early-warning model for the systematic assessment of the risk level and further allows the development of targeted regulatory strategies to improve supervision efficiency.

Spatiotemporal Synergistic Ensemble Deep Learning Method and Its Application to S-Wave Velocity Prediction

Spatiotemporal Synergistic Ensemble Deep Learning Method and Its Application to S-Wave Velocity Prediction

Forecasting Daily Tourism Demand for Tourist Attractions with Big Data: An Ensemble Deep Learning Method

Accurate forecasting of daily tourism demand is a meaningful and challenging task, but studies on this issue are scarce. To address this issue, multisource time series data, relating to past tourist volumes, web search information, daily weather conditions, and the dates of public holidays, are selected as the forecasting variables. To fully capture the relationship between these forecasting variables and actual tourism demand automatically, an ensemble of long short-term memory (LSTM) networks is proposed with a correlation-based predictor selection (CPS) algorithm. The effectiveness of the proposed method is verified in daily tourism demand forecasting for the Huangshan Mountain Area, benchmarked against 11 forecasting methods. This study contributes to the literature by (1) introducing the use of big data in daily tourism demand forecasting, (2) proposing an ensemble of LSTM networks for daily tourism demand forecasting, and (3) providing an effective predictor selection algorithm in ensemble learning.

Breast cancer detection using an ensemble deep learning method

In this work, the effectiveness of the deep learning model is applied for one-dimensional data when converted to images. This work is based on the effective conversion of one-dimensional data to images and designing a stacked ensemble deep learning model that can increase the performance of classification accuracy in comparison to single models. Breast cancer detection from gene expression dataset and breast histopathology images is considered using the proposed ensemble model. The gene expression data is one-dimensional. Using the t-Distributed Stochastic Neighbor embedding technique and Convex Hull algorithm the one-dimensional data is converted to an image. Existing methods are using the datasets directly for training and classification, whereas the proposed method uses the dataset as well as the decomposed forms of the same for improving the performance. It involves two-stage classification. The first stage consists of three Convolutional Neural Networks as the base classifiers. Empirical Wavelet Transform and Variational Mode Decomposition are the two methods used to decompose the dataset so that the models can be trained at the molecular level, making our model robust in comparison to state-of-the-art methods. The first stage classification outcomes are used to train the second stage classifier “Multilayer Perceptron”. The gene expression dataset collected from Mendeley and is used for the generation of two-dimensional synthetic datasets. The synthetic datasets and breast histopathology image datasets are used for the training and validation of the proposedmodel. The improved results obtained in this work show the effectiveness of our method

Improving protein fold recognition using triplet network and ensemble deep learning.

Protein fold recognition is a critical step toward protein structure and function prediction, aiming at providing the most likely fold type of the query protein. In recent years, the development of deep learning (DL) technique has led to massive advances in this important field, and accordingly, the sensitivity of protein fold recognition has been dramatically improved. Most DL-based methods take an intermediate bottleneck layer as the feature representation of proteins with new fold types. However, this strategy is indirect, inefficient and conditional on the hypothesis that the bottleneck layer's representation is assumed as a good representation of proteins with new fold types. To address the above problem, in this work, we develop a new computational framework by combining triplet network and ensemble DL. We first train a DL-based model, termed FoldNet, which employs triplet loss to train the deep convolutional network. FoldNet directly optimizes the protein fold embedding itself, making the proteins with the same fold types be closer to each other than those with different fold types in the new protein embedding space. Subsequently, using the trained FoldNet, we implement a new residue-residue contact-assisted predictor, termed FoldTR, which improves protein fold recognition. Furthermore, we propose a new ensemble DL method, termed FSD_XGBoost, which combines protein fold embedding with the other two discriminative fold-specific features extracted by two DL-based methods SSAfold and DeepFR. The Top 1 sensitivity of FSD_XGBoost increases to 74.8% at the fold level, which is ~9% higher than that of the state-of-the-art method. Together, the results suggest that fold-specific features extracted by different DL methods complement with each other, and their combination can further improve fold recognition at the fold level. The implemented web server of FoldTR and benchmark datasets are publicly available at http://csbio.njust.edu.cn/bioinf/foldtr/.

A voting-based ensemble deep learning method focusing on image augmentation and preprocessing variations for tuberculosis detection.

Tuberculosis (TB) is known as a potentially dangerous and infectious disease that affects mostly lungs worldwide. The detection and treatment of TB at an early stage are critical for preventing the disease and decreasing the risk of mortality and transmission of it to others. Nowadays, as the most common medical imaging technique, chest radiography (CXR) is useful for determining thoracic diseases. Computer-aided detection (CADe) systems are also crucial mechanisms to provide more reliable, efficient, and systematic approaches with accelerating the decision-making process of clinicians. In this study, we propose voting and preprocessing variations-based ensemble CNN model for TB detection. We utilize 40 different variations in fine-tuned CNN models based on InceptionV3 and Xception by also using CLAHE (contrast-limited adaptive histogram equalization) preprocessing technique and 10 different image transformations for data augmentation types. After analyzing all these combination schemes, three or five best classifier models are selected as base learners for voting operations. We apply the Bayesian optimization-based weighted voting and the average of probabilities as a combination rule in soft voting methods on two TB CXR image datasets to get better results in various numbers of models. The computational results indicate that the proposed method achieves 97.500% and 97.699% accuracy rates on Montgomery and Shenzhen datasets, respectively. Furthermore, our method outperforms state-of-the-art results for the two TB detection datasets in terms of accuracy rate.

An ensemble deep learning method as data fusion system for remote sensing multisensor classification

Because of the great achievements in designing remote sensing sensors, the extraction of useful information from multisource remote sensing data remains a challenging problem. Most of the recent research projects have applied single deep learning systems for data fusion and classification. The idea of using ensemble deep learning algorithms through a multisensor fusion system can improve the performance of data fusion tasks. In this research, however, a multi-sensor classification strategy, which is based on deep learning ensemble procedure and decision fusion framework, is investigated for the fusion of Light Detection and Ranging (LiDAR), Hyperspectral Images (HS), and very high-resolution Visible (Vis) images. This research proposes a basic classifier based on deep Convolutional Neural Network (CNN) in which the softmax layer is replaced by a Support Vector Machine (CNN-SVM). Then, a random feature selection is applied to generate two separate CNN-SVM ensemble systems, one for LiDAR and Vis and the other one for HS data. To overcome the similarity and overfitting between the deep features and the classifiers provided by two ensemble systems and to select the best subsets of the classifiers, two diversity measures select the most diverse combinations of the classifiers. Finally, a decision fusion method combines the obtained diverse classifiers from CNN ensembles. Results demonstrate that the proposed method achieves higher accuracy, and its performance outperforms some of the existing methods. The proposed ensemble CNN method improved single deep CNN, random forest, and Adaboost between 2% to 10% in terms of classification accuracy.

Multi-model ensemble deep learning method for intelligent fault diagnosis with high-dimensional samples

Multi-model ensemble deep learning method for intelligent fault diagnosis with high-dimensional samples

A predictive framework in healthcare: Case study on cardiac arrest prediction

Data-driven healthcare uses predictive analytics to enhance decision-making and personalized healthcare. Developing prognostic models is one of the applications of predictive analytics in medical environments. Various studies have used machine learning techniques for this purpose. However, there is no specific standard for choosing prediction models for different medical purposes. In this paper, the ISAF framework was proposed for choosing appropriate prediction models with regard to the properties of the classification methods. As one of the case study applications, a prognostic model for predicting cardiac arrests in sepsis patients was developed step by step through the ISAF framework. Finally, a new modified stacking model produced the best results. We predict 85 % of heart arrest cases one hour before the incidence (sensitivity> = 0.85) and 73 % of arrest cases 25 h before the occurrence (sensitivity> = 0.73). The results indicated that the proposed prognostic model has significantly improved the prediction results compared to the two standard systems of APACHE II and MEWS. Furthermore, compared to previous research, the proposed model has extended the prediction interval and improved the performance criteria.

Real-time visual inspection system for grading fruits using computer vision and deep learning techniques

Traditional manual visual grading of fruits has been one of the important challenges faced by the agricultural industry due to its laborious nature as well as inconsistency in inspection and classification process. Automated defects detection using computer vision and machine learning has become a promising area of research with a high and direct impact on the domain of visual inspection. In this study, we propose an efficient and effective machine vision system based on the state-of-the-art deep learning techniques and stacking ensemble methods to offer a non-destructive and cost-effective solution for automating the visual inspection of fruits’ freshness and appearance. We trained, tested and compared the performance of various deep learning models including ResNet, DenseNet, MobileNetV2, NASNet and EfficientNet to find the best model for the grading of fruits. The proposed system also provides a real time visual inspection using a low cost Raspberry Pi module with a camera and a touchscreen display for user interaction. The real time system efficiently segments multiple instances of the fruits from an image and then grades the individual objects (fruits) accurately. The system was trained and tested on two data sets (apples and bananas) and the average accuracy was found to be 99.2% and 98.6% using EfficientNet model for apples and bananas test sets, respectively. Additionally, a slight improvement in the recognition rate (0.03% for apples and 0.06% for bananas) was noted while applying the stacking ensemble deep learning methods. The performance of the developed system has been found higher than the existing methods applied to the same data sets previously. Further, during real-time testing on actual samples, the accuracy was found to be 96.7% for apples and 93.8% for bananas which indicates the efficacy of the developed system.

Credit scoring with an ensemble deep learning classification methods – comparison with tradicional methods

Credit scoring attracts special attention of financial institutions. In recent years, deep learning methods have been particularly interesting. In this paper, we compare the performance of ensemble deep learning methods based on decision trees with the best traditional method, logistic regression, and the machine learning method benchmark, support vector machines. Each method tests several different algorithms. We use different performance indicators. The research focuses on standard datasets relevant for this type of classification, the Australian and German datasets. The best method, according to the MCC indicator, proves to be the ensemble method with boosted decision trees. Also, on average, ensemble methods prove to be more successful than SVM.

Classification of colorectal tissue images from high throughput tissue microarrays by ensemble deep learning methods

Tissue microarray (TMA) core images are a treasure trove for artificial intelligence applications. However, a common problem of TMAs is multiple sectioning, which can change the content of the intended tissue core and requires re-labelling. Here, we investigate different ensemble methods for colorectal tissue classification using high-throughput TMAs. Hematoxylin and Eosin (H&E) core images of 0.6 mm or 1.0 mm diameter from three international cohorts were extracted from 54 digital slides (n = 15,150 cores). After TMA core extraction and color enhancement, five different flows of independent and ensemble deep learning were applied. Training and testing data with 2144 and 13,006 cores included three classes: tumor, normal or “other” tissue. Ground-truth data were collected from 30 ngTMA slides (n = 8689 cores). A test augmentation is applied to reduce the uncertain prediction. Predictive accuracy of the best method, namely Soft Voting Ensemble of one VGG and one CapsNet models was 0.982, 0.947 and 0.939 for normal, “other” and tumor, which outperformed to independent or ensemble learning with one base-estimator. Our high-accuracy algorithm for colorectal tissue classification in high-throughput TMAs is amenable to images from different institutions, core sizes and stain intensity. It helps to reduce error in TMA core evaluations with previously given labels.

Power load forecasting based on long short term memory-singular spectrum analysis

Electrical load forecasting, mainly short-term load forecasting (STLF), plays a vital role in efficient power system planning by making it more intelligent, sustainable, and reliable. However, due to the presence of skewness and irregularities in the observed data, it becomes a challenging task to improve the accuracy of STLF. To handle this, we propose a new model, named Singular Spectrum Analysis-Long Short Term Memory (SSA-LSTM). SSA is used to remove the high-frequency components (high volatility data) from the load series. Based on the resultant series obtained from SSA, LSTM model forecasts the final load. We have used six publicly available datasets to evaluate the performance of our proposed model. With Kaggle Global Energy Forecasting Competition (GEFCom2012) dataset, the generated analytical results show 59.83% improvement over Tao’s benchmark model and 12.46%, 12.48%, 4.17%, 5.42%, 5.68%, 3.28%, 4.73% improvement over refined multi-linear regression, Moving average and Exponential smoothing (MA & ES), Support Vector Regression (SVR) model, feed-forward neural network (FNN), deep belief network (DBN), LSTM and Singular Spectrum Analysis (SSA) respectively. Our model has outperformed MA & ES, SVR, FNN, DBN, ensemble deep learning method (EDBN), SSA and LSTM in South Australia, Tasmania, Queensland, and Victoria electrical load dataset by 11.38%, 23.54%, 3.05%, and 1.39% respectively using RMSE error metric.

An ensemble deep learning method with optimized weights for drone-based water rescue and surveillance

Today’s deep learning architectures, if trained with proper dataset, can be used for object detection in marine search and rescue operations. In this paper a dataset for maritime search and rescue purposes is proposed. It contains aerial-drone videos with 40,000 hand-annotated persons and objects floating in the water, many of small size, which makes them difficult to detect. The second contribution is our proposed object detection method. It is an ensemble composed of a number of the deep convolutional neural networks, orchestrated by the fusion module with the nonlinearly optimized voting weights. The method achieves over 82% of average precision on the new aerial-drone floating objects dataset and outperforms each of the state-of-the-art deep neural networks, such as YOLOv3, -v4, Faster R-CNN, RetinaNet, and SSD300. The dataset is publicly available from the Internet.

Least square based ensemble deep learning for inertia tensor identification of combined spacecraft

The high accurate identification of inertia tensor of combined spacecraft, which is composed of a servicing spacecraft and a target, is necessary to perform attitude control. Due to the uncertainty of the operating environments of combined spacecraft, the measurement noise of the angular rate may be very complex and will seriously influence the identification accuracy. This paper proposes a least square based weighted ensemble deep learning method to realize a highly accurate identification for the inertia tensor of combined spacecraft in complex operating environments. In this method, a single deep neural network regression model is firstly constructed as an individual model for the ensemble deep learning, and then is trained by enough training data and a designed training strategy. After obtaining a certain number of accurate and diverse single models, all the outputs of single models are combined by several linear functions. A least square based ensemble method for regression is developed to automatically determine the optimal weights of these linear functions. The identification performance of the proposed ensemble deep learning method is tested using two testing sets with different ranks of measurement noises and is compared with other identification methods. The results demonstrate that the proposed ensemble deep learning method has a significant advantage compared with other methods for identifying inertia tensor.

Ensemble deep learning in bioinformatics

The remarkable flexibility and adaptability of ensemble methods and deep learning models have led to the proliferation of their application in bioinformatics research. Traditionally, these two machine learning techniques have largely been treated as independent methodologies in bioinformatics applications. However, the recent emergence of ensemble deep learning—wherein the two machine learning techniques are combined to achieve synergistic improvements in model accuracy, stability and reproducibility—has prompted a new wave of research and application. Here, we share recent key developments in ensemble deep learning and look at how their contribution has benefited a wide range of bioinformatics research from basic sequence analysis to systems biology. While the application of ensemble deep learning in bioinformatics is diverse and multifaceted, we identify and discuss the common challenges and opportunities in the context of bioinformatics research. We hope this Review Article will bring together the broader community of machine learning researchers, bioinformaticians and biologists to foster future research and development in ensemble deep learning, and inspire novel bioinformatics applications that are unattainable by traditional methods. Recent developments in machine learning have seen the merging of ensemble and deep learning techniques. The authors review advances in ensemble deep learning methods and their applications in bioinformatics, and discuss the challenges and opportunities going forward.