Ensemble Deep Learning Method Research Articles

Ensemble Deep Learning for Cervix Image Selection toward Improving Reliability in Automated Cervical Precancer Screening.

Automated Visual Examination (AVE) is a deep learning algorithm that aims to improve the effectiveness of cervical precancer screening, particularly in low- and medium-resource regions. It was trained on data from a large longitudinal study conducted by the National Cancer Institute (NCI) and has been shown to accurately identify cervices with early stages of cervical neoplasia for clinical evaluation and treatment. The algorithm processes images of the uterine cervix taken with a digital camera and alerts the user if the woman is a candidate for further evaluation. This requires that the algorithm be presented with images of the cervix, which is the object of interest, of acceptable quality, i.e., in sharp focus, with good illumination, without shadows or other occlusions, and showing the entire squamo-columnar transformation zone. Our prior work has addressed some of these constraints to help discard images that do not meet these criteria. In this work, we present a novel algorithm that determines that the image contains the cervix to a sufficient extent. Non-cervix or other inadequate images could lead to suboptimal or wrong results. Manual removal of such images is labor intensive and time-consuming, particularly in working with large retrospective collections acquired with inadequate quality control. In this work, we present a novel ensemble deep learning method to identify cervix images and non-cervix images in a smartphone-acquired cervical image dataset. The ensemble method combined the assessment of three deep learning architectures, RetinaNet, Deep SVDD, and a customized CNN (Convolutional Neural Network), each using a different strategy to arrive at its decision, i.e., object detection, one-class classification, and binary classification. We examined the performance of each individual architecture and an ensemble of all three architectures. An average accuracy and F-1 score of 91.6% and 0.890, respectively, were achieved on a separate test dataset consisting of more than 30,000 smartphone-captured images.

A New Deep Learning Model Selection Method for Colorectal Cancer Classification

Deep learning is one of the most commonly used techniques in computer-aided diagnosis systems. Their exploitation for histopathological image analysis is important because of the complex morphology of whole slide images. However, the main limitation of these methods is the restricted number of available medical images, which can lead to an overfitting problem. Many studies have suggested the use of static ensemble learning methods to address this issue. This article aims to propose a new dynamic ensemble deep learning method. First, it generates a set of models based on the transfer learning strategy from deep neural networks. Then, the relevant subset of models is selected by the particle swarm optimization algorithm and combined by voting or averaging methods. The proposed approach was tested on a histopathological dataset for colorectal cancer classification, based on seven types of CNNs. The method has achieved accurate results (94.52%) by the Resnet121 model and the voting strategy, which provides important insights into the efficiency of dynamic ensembling in deep learning.

A Novel Ensemble Deep Learning Approach for Sleep-Wake Detection Using Heart Rate Variability and Acceleration

Sleep-wake detection is of great importance for the measurement of sleep quality. In this article, a novel ensemble deep learning framework is proposed to detect sleep-wake states based on heart rate variability (HRV) and acceleration. We firstly design a local feature based long short-term memory (LF-LSTM) network to encode temporal dependency and learn features from acceleration data with high sampling frequency. In the meantime, some handcrafted features are extracted from HRV which has a special data format. After that, we develop a unified framework to integrate these two types of features, i.e., the features extracted from HRV and the features learned by LF-LSTM from acceleration, to form a complete feature set. Finally, an efficient ensemble learning scheme is proposed to further boost the performance of sleep-wake classification. A real dataset has been collected to verify the effectiveness of the proposed approach. We also compare with some well-known benchmark approaches for sleep-wake detection. The results demonstrate that the proposed ensemble deep learning method outperforms all the benchmark approaches.

Ensembling handcrafted features with deep features: an analytical study for classification of routine colon cancer histopathological nuclei images

The use of Deep Learning (DL) based methods in medical histopathology images have been one of the most sought after solutions to classify, segment, and detect diseased biopsy samples. However, given the complex nature of medical datasets due to the presence of intra-class variability and heterogeneity, the use of complex DL models might not give the optimal performance up to the level which is suitable for assisting pathologists. Therefore, ensemble DL methods with the scope of including domain agnostic handcrafted Features (HC-F) inspired this work. We have, through experiments, tried to highlight that a single DL network (domain-specific or state of the art pre-trained models) cannot be directly used as the base model without proper analysis with the relevant dataset. We have used F1-measure, Precision, Recall, AUC, and Cross-Entropy Loss to analyse the performance of our approaches. We observed from the results that the DL features ensemble bring a marked improvement in the overall performance of the model, whereas, domain agnostic HC-F remains dormant on the performance of the DL models.

An enhanced selective ensemble deep learning method for rolling bearing fault diagnosis with beetle antennae search algorithm

Rolling bearing fault diagnosis is a meaningful yet challengeable task. To improve the performance of rolling bearing fault diagnosis, this paper proposes an enhanced selective ensemble deep learning method with beetle antennae search (BAS) algorithm. Firstly, multiple deep base models are constructed to automatically capture sensitive features from raw vibration signals. Secondly, to ensure the diversity of the base models, sparse autoencoder, denoising autoencoder and linear decoder are used to construct different deep autoencoders, respectively, and also Bootstrap is used to design distinctive training data subsets for each base model. Thirdly, an enhanced weighted voting (EWV) combination strategy with class-specific thresholds is proposed to implement selective ensemble learning. Finally, BAS algorithm is used to adaptively select the optimal class-specific thresholds. Experimental bearing data are used to verify the effectiveness of the proposed method. The results suggest that the proposed method can more accurately and robustly recognize different kind of faults than both the individual base models and other existing ensemble learning methods.

Skin Lesion Segmentation in Dermoscopic Images With Ensemble Deep Learning Methods

© 2013 IEEE. Early detection of skin cancer, particularly melanoma, is crucial to enable advanced treatment. Due to the rapid growth in the number of skin cancers, there is a growing need of computerised analysis for skin lesions. The state-of-the-art public available datasets for skin lesions are often accompanied with a very limited amount of segmentation ground truth labeling. Also, the available segmentation datasets consist of noisy expert annotations reflecting the fact that precise annotations to represent the boundary of skin lesions are laborious and expensive. The lesion boundary segmentation is vital to locate the lesion accurately in dermoscopic images and lesion diagnosis of different skin lesion types. In this work, we propose the fully automated deep learning ensemble methods to achieve high sensitivity and high specificity in lesion boundary segmentation. We trained the ensemble methods based on Mask R-CNN and DeeplabV3+ methods on ISIC-2017 segmentation training set and evaluate the performance of the ensemble networks on ISIC-2017 testing set and PH2 dataset. Our results showed that the proposed ensemble methods segmented the skin lesions with Sensitivity of 89.93% and Specificity of 97.94% for the ISIC-2017 testing set. The proposed ensemble method Ensemble-A outperformed FrCN, FCNs, U-Net, and SegNet in Sensitivity by 4.4%, 8.8%, 22.7%, and 9.8% respectively. Furthermore, the proposed ensemble method Ensemble-S achieved a specificity score of 97.98% for clinically benign cases, 97.30% for the melanoma cases, and 98.58% for the seborrhoeic keratosis cases on ISIC-2017 testing set, exhibiting better performance than FrCN, FCNs, U-Net, and SegNet.

Adverse drug events and medication relation extraction in electronic health records with ensemble deep learning methods.

ObjectiveIdentification of drugs, associated medication entities, and interactions among them are crucial to prevent unwanted effects of drug therapy, known as adverse drug events. This article describes our participation to the n2c2 shared-task in extracting relations between medication-related entities in electronic health records.Materials and MethodsWe proposed an ensemble approach for relation extraction and classification between drugs and medication-related entities. We incorporated state-of-the-art named-entity recognition (NER) models based on bidirectional long short-term memory (BiLSTM) networks and conditional random fields (CRF) for end-to-end extraction. We additionally developed separate models for intra- and inter-sentence relation extraction and combined them using an ensemble method. The intra-sentence models rely on bidirectional long short-term memory networks and attention mechanisms and are able to capture dependencies between multiple related pairs in the same sentence. For the inter-sentence relations, we adopted a neural architecture that utilizes the Transformer network to improve performance in longer sequences.ResultsOur team ranked third with a micro-averaged F1 score of 94.72% and 87.65% for relation and end-to-end relation extraction, respectively (Tracks 2 and 3). Our ensemble effectively takes advantages from our proposed models. Analysis of the reported results indicated that our proposed approach is more generalizable than the top-performing system, which employs additional training data- and corpus-driven processing techniques.ConclusionsWe proposed a relation extraction system to identify relations between drugs and medication-related entities. The proposed approach is independent of external syntactic tools. Analysis showed that by using latent Drug-Drug interactions we were able to significantly improve the performance of non–Drug-Drug pairs in EHRs.