Ensemble of Deep Variational Mixture Models for Unsupervised Clustering

Automated accurate fire detection system using ensemble pretrained residual network

PurposeThe purpose of this paper is to present a deep ensemble neural network model for the detection of forest fires in aerial vehicle videos.Design/methodology/approachPresented deep ensemble models include four convolutional neural networks (CNNs): a faster region-based CNN (Faster R-CNN), a simple one-stage object detector (RetinaNet) and two different versions of the you only look once (Yolo) models. The presented method generates its output by fusing the outputs of these different deep learning (DL) models.FindingsThe presented fusing approach significantly improves the detection accuracy of fire incidents in the input data.Research limitations/implicationsThe computational complexity of the proposed method which is based on combining four different DL models is relatively higher than that of using each of these models individually. On the other hand, however, the performance of the proposed approach is considerably higher than that of any of the four DL models.Practical implicationsThe simulation results show that using an ensemble model is quite useful for the precise detection of forest fires in real time through aerial vehicle videos or images.Social implicationsBy this method, forest fires can be detected more efficiently and precisely. Because forests are crucial breathing resources of the earth and a shelter for many living creatures, the social impact of the method can be considered to be very high.Originality/valueThis study fuses the outputs of different DL models into an ensemble model. Hence, the ensemble model provides more potent and beneficial results than any of the single models.

The aim of this study was to distinguish canine lymphoma from other diseases,particularly reactive lymphoid hyperplasia (RLH), based on fine needle aspiration (FNA)images. We developed four deep learning models based on Vision Transformer (ViT) andInception-v3, which were pre-trained image classification models. The two models out offour were ViT and Inception-v3, and the remained were the two types of combination, i.e.,ensemble learning models, of ViT and Inception-v3; the mean of class probabilities of ViTand Inception-v3 (Ensemble model A; MEAN) and the maximum probabilities of ViT andInception-v3 (Ensemble model B; MAX). A total of 2,290 FNA images of canine lymphoma and871 FNA images of RLH were analyzed. The FNA images were obtained from the twenty-fiveslides of fourteen lymphoma cases and eight slides of seven RLH cases in two hospitals.Three types of training and test datasets were prepared from the above image datasets forfair evaluation of the models. Three deep learning-based image classification models(Inception-v3 and the two ensemble models) attained high performance of >80% accuracy,recall and area under the curve (AUC) values for all three datasets. ViT did not archivehigh performance, except the precision (>0.85). This study is an example of showingpotentials of deep learning models through image classification problem in caninelymphoma.

Recently, the research community has shown significant interest in the continuous temporal data obtained from motion sensors in wearable devices. These data are useful for classifying and analysing different human activities in many application areas such as healthcare, sports and surveillance. The literature has presented a multitude of deep learning models that aim to derive a suitable feature representation from temporal sensory input. However, the presence of a substantial quantity of annotated training data is crucial to adequately train the deep networks. Nevertheless, the data originating from the wearable devices are vast but ineffective due to a lack of labels which hinders our ability to train the models with optimal efficiency. This phenomenon leads to the model experiencing overfitting. The contribution of the proposed research is twofold: firstly, it involves a systematic evaluation of fifteen different augmentation strategies to solve the inadequacy problem of labeled data which plays a critical role in the classification tasks. Secondly, it introduces an automatic feature-learning technique proposing a Multi-Branch Hybrid Conv-LSTM network to classify human activities of daily living using multimodal data of different wearable smart devices. The objective of this study is to introduce an ensemble deep model that effectively captures intricate patterns and interdependencies within temporal data. The term “ensemble model” pertains to fusion of distinct deep models, with the objective of leveraging their own strengths and capabilities to develop a solution that is more robust and efficient. A comprehensive assessment of ensemble models is conducted using data-augmentation techniques on two prominent benchmark datasets: CogAge and UniMiB-SHAR. The proposed network employs a range of data-augmentation methods to improve the accuracy of atomic and composite activities. This results in a 5% increase in accuracy for composite activities and a 30% increase for atomic activities.

Background Patients have the highest risk of subsequent fractures in the first few years after an initial fracture, yet models to predict short-term subsequent risk have not been developed. Purpose To develop and validate a deep learning prediction model for subsequent fracture risk using digitally reconstructed radiographs from hip CT in patients with recent hip fractures. Materials and Methods This retrospective study included adult patients who underwent three-dimensional hip CT due to a fracture from January 2004 to December 2020. Two-dimensional frontal, lateral, and axial digitally reconstructed radiographs were generated and assembled to construct an ensemble model. DenseNet modules were used to calculate risk probability based on extracted image features and fracture-free probability plots were output. Model performance was assessed using the C index and area under the receiver operating characteristic curve (AUC) and compared with other models using the paired t test. Results The training and validation set included 1012 patients (mean age, 74.5 years ± 13.3 [SD]; 706 female, 113 subsequent fracture) and the test set included 468 patients (mean age, 75.9 years ± 14.0; 335 female, 22 subsequent fractures). In the test set, the ensemble model had a higher C index (0.73) for predicting subsequent fractures than that of other image-based models (C index range, 0.59-0.70 for five of six models; P value range, < .001 to < .05). The ensemble model achieved AUCs of 0.74, 0.74, and 0.73 at the 2-, 3-, and 5-year follow-ups, respectively; higher than that of most other image-based models at 2 years (AUC range, 0.57-0.71 for five of six models; P value range, < .001 to < .05) and 3 years (AUC range, 0.55-0.72 for four of six models; P value range, < .001 to < .05). Moreover, the AUCs achieved by the ensemble model were higher than that of a clinical model that included known risk factors (2-, 3-, and 5-year AUCs of 0.58, 0.64, and 0.70, respectively; P < .001 for all). Conclusion In patients with recent hip fractures, the ensemble deep learning model using digital reconstructed radiographs from hip CT showed good performance for predicting subsequent fractures in the short term. © RSNA, 2024 Supplemental material is available for this article. See also the editorial by Li and Jaremko in this issue.

Classification of EEG signals using Transformer based deep learning and ensemble models

Reinforced concrete structural walls (RCSWs) are one of the most efficient lateral force-resisting systems used in buildings, providing sufficient strength, stiffness, and deformation capacities to withstand the forces generated during earthquake ground motions. Identifying the failure mode of the RCSWs is a critical task that can assist engineers and designers in choosing appropriate retrofitting solutions. This study evaluates the efficiency of three ensemble deep neural network models, including the model averaging ensemble, weighted average ensemble, and integrated stacking ensemble for predicting the failure mode of the RCSWs. The ensemble deep neural network models are compared against previous studies that used traditional well-known ensemble models (AdaBoost, XGBoost, LightGBM, CatBoost) and traditional machine learning methods (Naïve Bayes, K-Nearest Neighbors, Decision Tree, and Random Forest). The weighted average ensemble model is proposed as the best-suited prediction model for identifying the failure mode since it has the highest accuracy, precision, and recall among the alternative models. In addition, since complex and advanced machine learning-based models are commonly referred to as black-box, the SHapley Additive exPlanation method is also used to interpret the model workflow and illustrate the importance and contribution of the components that impact determining the failure mode of the RCSWs.

In the last 50 years, with the growth of cities and increase in the number of vehicles and mobility, traffic has become troublesome. As a result, traffic flow prediction started to attract attention as an important research area. However, despite the extensive literature, traffic flow prediction still remains as an open research problem, specifically for long-term traffic flow prediction. Compared to the models developed for short-term traffic flow prediction, the number of models developed for long-term traffic flow prediction is very few. Based on this shortcoming, in this study, we focus on long-term traffic flow prediction and propose a novel deep ensemble model (DEM). In order to build this ensemble model, first, we developed a convolutional neural network (CNN), a long short-term memory (LSTM) network and a gated recurrent unit (GRU) network as deep learning models, which formed the base learners. In the next step, we combine the output of these models according to their individual forecasting success. We use another deep learning model to determine the success of the individual models. Our proposed model is a flexible ensemble prediction model that can be updated based on traffic data. To evaluate the performance of the proposed model, we use a publicly available dataset. Experimental results show that the developed DEM model has a mean square error of 0.06 and a mean absolute error of 0.15 for single-step prediction; it shows that achieves a mean square error of 0.25 and a mean absolute error of 0.32 for multi-step prediction. We compared our proposed model with many models in different categories; individual deep learning models (i.e., LSTM, CNN, GRU), selected traditional machine learning models (i.e., linear regression, decision tree regression, k-nearest-neighbors regression) and other ensemble models such as random-forest regression. These results also support the claim that ensemble learning models perform better than individual models.

A novel deep learning intelligent clustered hybrid models for wind speed and power forecasting

A high-precision automatic diagnosis method of maize developmental stage based on ensemble deep learning with IoT devices

Ovarian cancer is one of the leading causes of death worldwide among the female population. Early diagnosis is crucial for patient treatment. In this work, our main objective is to accurately detect and classify ovarian cancer. To achieve this, two datasets are considered: CT scan images of patients with cancer and those without, and biomarker (clinical parameters) data from all patients. We propose an ensemble deep neural network model and an ensemble machine learning model for the automatic binary classification of ovarian CT scan images and biomarker data. The proposed model incorporates four convolutional neural network models: VGG16, ResNet 152, Inception V3, and DenseNet 101, with transformers applied for feature extraction. These extracted features are fed into our proposed ensemble multi-layer perceptron model for classification. Preprocessing and CNN tuning techniques such as hyperparameter optimization, data augmentation, and fine-tuning are utilized during model training. Our ensemble model outperforms single classifiers and machine learning algorithms, achieving a mean accuracy of 98.96%, a precision of 97.44%, and an F1-score of 98.7%. We compared these results with those obtained using features extracted by the UNet model, followed by classification with our ensemble model. The transformer demonstrated superior performance in feature extraction over the UNet, with a mean Dice score and mean Jaccard score of 0.98 and 0.97, respectively, and standard deviations of 0.04 and 0.06 for benign tumors and 0.99 and 0.98 with standard deviations of 0.01 for malignant tumors. For the biomarker data, the combination of five machine learning models—KNN, logistic regression, SVM, decision tree, and random forest—resulted in an improved accuracy of 92.8% compared to single classifiers.

Radiologists often face challenges in differentiating benign from malignant sacral bone lesions due to their similar imaging characteristics. This study aimed to develop an ensemble deep learning (DL) model that can preoperatively distinguish between benign and malignant sacral tumors using noncontrast computed tomography images. Preoperative sacral CT scans from 569 patients with confirmed sacral lesions were analyzed. Data from Center 1 were utilized in model development and internal test via fivefold cross-validation, and those from Centers 2 and 3 were employed in external test. Various ensemble models combining human-readable interpretation and DL were developed. The diagnostic performance of the models and radiologists was assessed using metrics such as precision, recall, accuracy, area under the curve (AUC), F1 score, and confusion matrix. Furthermore, the clinical benefits derived from radiologists' interpretations and supported by the DL model were evaluated. The ensemble model, which integrates 3D-DenseNet121 with human interpretation, exhibited the most robust performance. The ensemble model demonstrated high performance on the internal and external test sets and achieved AUCs of 0.9139 and 0.8713, F1 scores of 0.9054 and 0.8571, precision of 0.9041 and 0.8824, recall of 0.9136 and 0.8333, and accuracy of 0.8630 and 0.8182, respectively. Across the external test cohort, all radiologists experienced improvements in AUC, accuracy, sensitivity, and specificity. Notably, junior radiologists demonstrated significant improvements compared with senior radiologists. The potential clinical application of the DL model lies in its capacity to considerably enhance the diagnostic efficiency of radiologists. This study presents the first ensemble deep learning model integrating 3D-DenseNet121 with radiologists' interpretation for preoperative differentiation of sacral tumors on noncontrast CT that improved diagnostic performance across all experience levels, particularly for junior radiologists. First artificial intelligence-radiologist ensemble for noncontrast computed tomography (NCCT)-based sacral tumor classification. Boosts all radiologists' performance, with the greatest gains for juniors, potentially reducing referrals. Enables reliable NCCT diagnosis, overcoming contrast/magnetic resonance imaging dependency in musculoskeletal oncology.

An ensemble of deep models is commonly used to provide more robust and accurate performance for medical image classification. A drawback of the most common ensemble aggregation operators is that they give the same importance to all models in the ensemble. As a consequence, they cannot identify weak models that may negatively influence the ensemble performance. In this work, we propose a new method based on the Dirichlet distribution and Mahalanobis distance to learn dynamic weights to an ensemble of deep learning models. Through this method, it is possible to reduce the influence of weak models for each new sample evaluated by the ensemble and perform online ensemble pruning. We evaluate this method for an ensemble of six well-known deep models applied to four medical imaging datasets. The experiments show that our method achieves the best balanced accuracy for 2 out of 4 datasets and increases the confidence of the ensemble predictions.

The era of the internet has transformed the way people share their thoughts and viewpoints. It is now achieved mostly through blog entries, product review blogs, social networks, and so on. We get immersive media through online networks, where users notify and affect others through the internet. In this research, positive and negative sentiments are used to do the document-level sentiment analysis using deep and traditional ensemble models. In this study, we attempt to evaluate the performances of recent deep learning ensemble models and traditional ensemble models for obtaining the highest accuracy for binary sentiment classification. Three traditional ensemble models (i.e., Voting Ensemble, Bagging Ensemble, and Boosting Ensemble) and three deep learning ensemble layout models (i.e., 7 Layer Convolutional Neural Network (7-L CNN) + Gated Recurrent Unit (GRU), 7-L CNN + GRU + Globe Embedding, and 7-L CNN + Long Short-Term memory (LSTM) + Attention Layer) have been applied in two different datasets to perform the sentiment classification. The deep learning ensemble models perform better than the traditional ensemble models in most cases. In both of the datasets, the deep learning ensemble models namely 7-L CNN + GRU + Globe and 7-L CNN + LSTM + Attention Layer achieve the highest accuracy by securing 94.19% and 96.37% respectively for the product and restaurant review dataset.

A key element in solving real-life data science problems is selecting the types of models to use. Tree ensemble models (such as XGBoost) are usually recommended for classification and regression problems with tabular data. However, several deep learning models for tabular data have recently been proposed, claiming to outperform XGBoost for some use cases. This paper explores whether these deep models should be a recommended option for tabular data by rigorously comparing the new deep models to XGBoost on various datasets. In addition to systematically comparing their performance, we consider the tuning and computation they require. Our study shows that XGBoost outperforms these deep models across the datasets, including the datasets used in the papers that proposed the deep models. We also demonstrate that XGBoost requires much less tuning. On the positive side, we show that an ensemble of deep models and XGBoost performs better on these datasets than XGBoost alone.