An Empirical Evaluation of a Novel Ensemble Deep Neural Network Model and Explainable AI for Accurate Segmentation and Classification of Ovarian Tumors Using CT Images

A novel acoustic scene classification model using the late fusion of convolutional neural networks and different ensemble classifiers

<p>Cotton plant (Gossypium herbaceum), is one of the significant fiber crop grown worldwide. However, the crop is quite prone to leaves diseases, for which deep learning (DL) techniques can be utilized for early disease prediction and prevent stakeholders from losing the harvest. The objective of this paper is to develop a novel ensemble based deep convolutional neural network (DCNN) model developed on two base pretrained models named: VGG16 and InceptionV3 for early detection of cotton leaves diseases. The proposed ensemble model trained on cotton leaves dataset reports higher training and testing prediction accuracies as compared to the base pretrained models. Given that, deep learning architectures have hyper-parameters, this paper presents exhaustive experimental evaluations on ensemble model to tune hyper-parameters named learning rate, optimizer and no of epochs. The suggested hyper-parameter settings can be directly utilized while employing the ensemble model for cotton plant leaves disease detection and prediction. With suggested hyper-parameters settings of learning rate 0.0001, 20 epochs and stochastic gradient descent (SGD) optimizer, ensemble model reported training and testing accuracies of 98% and 95% respectively, which was higher than the training and testing accuracies of VGG16 and InceptionV3 pretrained DCNN models.</p>

The most common causes of spine fractures, or vertebral column fractures (VCF), are traumas like falls, injuries from sports, or accidents. CT scans are affordable and effective at detecting VCF types in an accurate manner. VCF type identification in cervical, thoracic, and lumbar (C3-L5) regions is limited and sensitive to inter-observer variability. To solve this problem, this work introduces an autonomous approach for identifying VCF type by developing a novel ensemble model of Vision Transformers (ViT) and best-performing deep learning (DL) models. It assists orthopaedicians in easy and early identification of VCF types. The performance of numerous fine-tuned DL architectures, including VGG16, ResNet50, and DenseNet121, was investigated, and an ensemble classification model was developed to identify the best-performing combination of DL models. A ViT model is also trained to identify VCF. Later, the best-performing DL models and ViT were fused by weighted average technique for type identification. To overcome data limitations, an extended Deep Convolutional Generative Adversarial Network (DCGAN) and Progressive Growing Generative Adversarial Network (PGGAN) were developed. The VGG16-ResNet50-ViT ensemble model outperformed all ensemble models and got an accuracy of 89.98%. Extended DCGAN and PGGAN augmentation increased the accuracy of type identification to 90.28% and 93.68%, respectively. This demonstrates efficacy of PGGANs in augmenting VCF images. The study emphasizes the distinctive contributions of the ResNet50, VGG16, and ViT models in feature extraction, generalization, and global shape-based pattern capturing in VCF type identification. CT scans collected from a tertiary care hospital are used to validate these models.

Brain tumor diagnosis at an early stage can improve the chances of successful treatment and better patient outcomes. In the biomedical industry, non-invasive diagnostic procedures, such as magnetic resonance imaging (MRI), can be used to diagnose brain tumors. Deep learning, a type of artificial intelligence, can analyze MRI images in a matter of seconds, reducing the time it takes for diagnosis and potentially improving patient outcomes. Furthermore, an ensemble model can help increase the accuracy of classification by combining the strengths of multiple models and compensating for their individual weaknesses. Therefore, in this research, a weighted average ensemble deep learning model is proposed for the classification of brain tumors. For the weighted ensemble classification model, three different feature spaces are taken from the transfer learning VGG19 model, Convolution Neural Network (CNN) model without augmentation, and CNN model with augmentation. These three feature spaces are ensembled with the best combination of weights, i.e., weight1, weight2, and weight3 by using grid search. The dataset used for simulation is taken from The Cancer Genome Atlas (TCGA), having a lower-grade glioma collection with 3929 MRI images of 110 patients. The ensemble model helps reduce overfitting by combining multiple models that have learned different aspects of the data. The proposed ensemble model outperforms the three individual models for detecting brain tumors in terms of accuracy, precision, and F1-score. Therefore, the proposed model can act as a second opinion tool for radiologists to diagnose the tumor from MRI images of the brain.

Machine learning models are employed widely in the classification, diagnosis, and prediction of various health disorders. This paper does a comparison study of the popular traditional data classification models with ensemble and hybrid classification models developed and applied in diagnosis and prediction of one of the most frequently occurring endocrinal illness, viz Thyroid disorders. According to a survey conducted in India in 2016, disorders of thyroid gland are increased in an exponential rate in Indian subcontinent. Approximately 1 in every 10 adults suffer from thyroid disorders like hypothyroidism, a condition of hormonal imbalance in which the thyroid produces too little hormone for proper functioning of human body. This condition is more common in women of child-bearing age. Some agents that unfavourably affect the thyroid performance are stress, trauma, infection etc. Various machine learning algorithms popularly applied are Regression models, Bayesian classifier, k Nearest Neighbours, Decision Tree induction, Support Vector Machine, and they are employed in Thyroid disorder prediction researches and tools. Nowadays, ensemble models have replaced simple classification techniques in this area. This paper provides an exclusive comparative study of standard traditional classification models with novel Ensemble models used in the classification and prediction of thyroid disorders.

Transformer fault diagnosis technology based on AdaBoost enhanced transferred convolutional neural network

Skin lesion computational diagnosis of dermoscopic images: Ensemble models based on input feature manipulation

We designed 5 convolutional neural network (CNN) models and ensemble models to differentiate malignant and benign thyroid nodules on CT, and compared the diagnostic performance of CNN models with that of radiologists. We retrospectively included CT images of 880 patients with 986 thyroid nodules confirmed by surgical pathology between July 2017 and December 2019. Two radiologists retrospectively diagnosed benign and malignant thyroid nodules on CT images in a test set. Five CNNs (ResNet50, DenseNet121, DenseNet169, SE-ResNeXt50, and Xception) were trained-validated and tested using 788 and 198 thyroid nodule CT images, respectively. Then, we selected the 3 models with the best diagnostic performance on the test set for the model ensemble. We then compared the diagnostic performance of 2 radiologists with 5 CNN models and the integrated model. Of the 986 thyroid nodules, 541 were malignant, and 445 were benign. The area under the curves (AUCs) for diagnosing thyroid malignancy was 0.587-0.754 for 2 radiologists. The AUCs for diagnosing thyroid malignancy for the 5 CNN models and ensemble model was 0.901-0.947. There were significant differences in AUC between the radiologists' models and the CNN models (p < 0.05). The ensemble model had the highest AUC value. Five CNN models and an ensemble model performed better than radiologists in distinguishing malignant thyroid nodules from benign nodules on CT. The diagnostic performance of the ensemble model improved and showed good potential.

Impact of resampling methods and classification models on the imbalanced credit scoring problems

The surfaces of airless planetary bodies exposed to the interplanetary environment experience space weathering (SW). Over time, the SW process distorts certain reflectance spectral features such as spectral slope, albedo, or absorption band depths and widths. Extensive laboratory experiments studied the evolution of these spectral parameters under simulated SW conditions. However, despite these efforts, the precise relationship between SW duration and the alteration of reflectance spectra remains not fully understood. Various experiments give different, often non-linear evolution rates (Loeffler et al. 2009, Vernazza et al. 2009, Hapke et al. 2001). This research aims to develop and apply machine-learning models to estimate the surface exposure time of S-type asteroids as a function of space weathering agents and dose.The primary source of the training spectral data are published empirical space weathering simulations. We gathered 200 spectra from publications and mineral and meteorite databases: Re-Lab, and C-TAPE. This included materials such as olivine (85), pyroxene (47), mixtures of olivine and pyroxene (9), as well as chondritic meteorites (59). These materials reflect the composition often found in S-type asteroids. Then we calculated the exposure time at 1 AU for the reflectance spectra based on the laboratory simulation conditions (Sasaki et al. 2002, Schwenn 2000). The exposure time of the dataset ranges from fresh material to 109 years. In this study, we developed a Convolutional Neural Network (CNN) model (model 1) and two models using the ensemble learning technique. The ensemble learning technique combines the results of several models to improve the overall model prediction (Phyo et al. 2022). In the first ensemble model (model 2), we combined four decision tree algorithms: Gradient Boosting Tree Regressor (GBT), Decision Tree Regressor (DT), Extra Tree Regressor (ET), and K-neighbors Tree Regressor (KNT). In the second ensemble model (model 3), we combined a CNN with GBT, DT, ET, and KNT. Then we trained and evaluated these models using the stratified K-fold cross-validation method to assess the performance and generalization ability of a predictive model. The models were fed with reflectance spectra and SW conditions as independent variables, while exposure time was predicted as the dependent variable.Figure 1 illustrates a comparative analysis of actual values versus predicted values from the three models, with the standard deviation across 10 iterations. Notably, as we progress from model 1 to model 3, there is a noticeable enhancement in prediction accuracy, coupled with a reduction in standard deviation. In Figure 2, it is observed that 68% of the predicted values from model 1 estimate time with accuracy better than 2.5 times. However, with the utilization of ensemble model 2 and ensemble model 3, this metric sees a notable improvement, reaching 77% and 80% accuracy respectively. The combination of the CNN model with decision tree algorithms not only reduces absolute error but also improves standard deviation, thus increasing the reliability of predictions. Consequently, the ensembled model, featuring CNN in combination with decision tree algorithms, yields more dependable prediction compared to a standalone CNN model. The constraint of a small dataset size limits the CNN model's capacity to assemble and extrapolate the relationship between SW and surface exposure time from irradiated samples. Conversely, decision tree-based algorithms have shown better performance under such circumstances. It is worth noting that various tree-based algorithms exhibit varying degrees of proficiency across different segments of the dataset. However, their combination notably enhances the overall performance of the model. Furthermore, the combination of the CNN model with the tree-based ensemble model yields further improvements in results. The next step of this work would be to apply this model to asteroid spectra to determine their exposure time.Hapke, B. 2001 DOI 10.1029/2000JE001338Loeffler et al. 2009 DOI 10.1029/2008JE003249Phyo et al. 2022 DOI 10.3390/sym14010160Sasaki et al. 2002 DOI 10.1016/S0273-1177(02)00012-1Schwenn, R. 2000 DOI 10.1201/9781003220435Vernazza et al. 2009 DOI 10.1038/nature07956&amp;#160;&amp;#160;

Early diagnosis of plant diseases is of vital importance since they cause social, ecological, and economic losses. Therefore, it is highly complex and causes excessive workload and time loss. Within the scope of this article, nine tomato plant leaf diseases as well as healthy ones were classified using deep learning with new ensemble architectures. A total of 18.160 images were used for this process. In this study, in addition to the proposed two new convolutional neural networks (CNN) models, four other well-known CNN models (MobileNetV3Small, EfficientNetV2L, InceptionV3 and MobileNetV2) are used. A fine-tuning method is applied to the newly proposed CNNs models and then hyperparameter optimization is performed with the particle swarm optimization algorithm (PSO). Then, the weights of these architectures are optimized by the grid search method and triple and quintuple ensemble models are created and the datasets are classified with the help of the five-fold cross-validation. The experimental results demonstrate that the proposed ensemble models stand out with their fast training and testing time and superior classification performances with an accuracy of 99.60%. This research will help experts enable the early detection of plant diseases in a simple and quick manner and prevent the formation of new infections.

The kernel-based geometric learning model has been successfully applied in bevel gearbox fault diagnosis. However, due to its shallow architecture and problems with its sensitivity to noise and outliers, its generalization ability and robustness need to be further improved. Ensemble learning can improve the classification accuracy of sub-classifiers, but it is effective only when the sub-classifiers meet the requirements of difference and accuracy at the same time. However, as strong classifiers, geometric learning models are difficult to produce sub-classifiers with differences. To solve these problems, this study proposes a novel ensemble model, the ensemble convex hull (CH)-based (EnCH) classification model. CH has the advantages of clear geometric meaning and is easy to deform. This paper considers the clustering characteristics of the sample points in the feature space, or both distance and density, and performs differential shrinkage deformation on the original CH. For one thing, this can produce differential CHs to build differential sub-classifiers for the ensemble. Also, it can suppress the interference of noise and outliers to improve robustness. The results of our experiments on the fault dataset of a bevel gear box indicate that the EnCH classification model can improve the generalization of the geometric learning model and has excellent tolerance to noise and outliers.

LADA Diabetes is a complex disease, but often dismissed as a potential individual disease within its own right. A comprehensive understanding of previously unknown aspects of LADA diabetes has the potential to not only ascertain a greater comprehension of LADA but also can assist the classification of Type 1 and Type 2 diabetes, as LADA characterises the attributes of both Type 1 and Type 2 diabetes. This paper proposes a novel heterogeneous ensemble model comprising of Neural network with Feature Extraction, Neural network alongside Multilayer Perceptron with Multiple Layers with the intention of classifying LADA diabetes along with weighting the importance of conventional variables including family history, age, gender, BMI, cholesterol level, and waist size in the classification. These conventional variables are analysed based on the aforementioned three-algorithm ensemble stack, and the entire architecture is tuned for optimal classification performance. The proposed novel ensemble stack delivers a reliable prediction accuracy in the identification of case, control, and variable importance. Performance evaluation of the proposed ensemble model based on statistics such as ROC/AUC curve, precision and recall demonstrated a higher predictive accuracy of 92.00%, sensitivity of 91.77%, and specificity of 92.23% alongside a precision of 92.23%, recall at 91.79% and an F1 score of 92.02%, ultimately outperforming well-known classical classification models. Further analysis has determined waist as an important and influential variable in the classification process, whereby a 100% association of LADA diabetes with waist is exhibited.

Glioma grading is vital for therapeutic planning where the higher level of glioma is associated with high mortality. It is a challenging task as different glioma grades have mixed morphological characteristics of brain tumour. A computer-aided diagnosis (CAD) system based on three-dimensional textural grey level co-occurrence matrix (GLCM) and machine learning is proposed for glioma grading. The purpose of this paper is to assess the usefulness of the 3D textural analysis in establishing a malignancy prediction model for glioma grades. Furthermore, this paper aims to find the best classification model based on textural analysis for glioma grading. The classification system was evaluated using leave-one-out cross-validation technique. The experimental design includes feature extraction, feature selection, and finally the classification that includes single and ensemble classification models in a comparative study. Experimental results illustrate that single and ensemble classification models, can achieve efficient prediction performance based on 3D textural analysis and the classification accuracy result has significantly improved after using feature selection methods. In this paper, we compare the proficiency of applying different angles of 3D textural analysis and different classification models to determine the malignant level of glioma. The obtained sensitivity, accuracy and specificity are 100%, 96.6%, 90% respectively. The prediction system presents an effective approach to assess the malignancy level of glioma with a non-invasive, reproducible and accurate CAD system for glioma grading.

Renal cancer is one of the leading causes of cancer-related deaths worldwide, and early detection of renal cancer can significantly improve the patients' survival rate. However, the manual analysis of renal tissue in the current clinical practices is labor-intensive, prone to inter-pathologist variations and easy to miss the important cancer markers, especially in the early stage. In this work, we developed deep convolutional neural network (CNN) based heterogeneous ensemble models for automated analysis of renal histopathological images without detailed annotations. The proposed method would first segment the histopathological tissue into patches with different magnification factors, then classify the generated patches into normal and tumor tissues using the pre-trained CNNs and lastly perform the deep ensemble learning to determine the final classification. The heterogeneous ensemble models consisted of CNN models from five deep learning architectures, namely VGG, ResNet, DenseNet, MobileNet, and EfficientNet. These CNN models were fine-tuned and used as base learners, they exhibited different performances and had great diversity in histopathological image analysis. The CNN models with superior classification accuracy (Acc) were then selected to undergo ensemble learning for the final classification. The performance of the investigated ensemble approaches was evaluated against the state-of-the-art literature. The performance evaluation demonstrated the superiority of the proposed best performing ensembled model: five-CNN based weighted averaging model, with an Acc (99%), specificity (Sp) (98%), F1-score (F1) (99%) and area under the receiver operating characteristic (ROC) curve (98%) but slightly inferior recall (Re) (99%) compared to the literature. The outstanding robustness of the developed ensemble model with a superiorly high-performance scores in the evaluated metrics suggested its reliability as a diagnosis system for assisting the pathologists in analyzing the renal histopathological tissues. It is expected that the proposed ensemble deep CNN models can greatly improve the early detection of renal cancer by making the diagnosis process more efficient, and less misdetection and misdiagnosis; subsequently, leading to higher patients' survival rate.