Efficient Ensemble Research Articles

DaE2: Unmasking malicious URLs by leveraging diverse and efficient ensemble machine learning for online security

Over 5.44 billion people now use the Internet, making it a vital part of daily life, enabling communication, e-commerce, education, and more. However, this huge Internet connectivity also raises concerns about online privacy and security, particularly with the rise of malicious Uniform Resource Locators (URLs). Recently, conventional ensemble models have attracted attention due to their notable benefits of reducing the variance in models, enhancing predictive performance, improving prediction accuracy, and demonstrating high generalization potential. But, its application in addressing the challenge of malicious URLs is still an open problem. These URLs often hide behind static links in emails or web pages, posing a threat to individuals and organizations. Despite blacklisting services, many harmful sites evade detection due to inadequate scrutiny or recent creation. Hence, to improve URL detection, a Diverse and Efficient Ensemble (DaE2) machine learning algorithm was developed using four ensemble models, that is, AdaBoost, Bagging, Stacking, and Voting to classify URLs. After preprocessing, the experimental result shown that all models achieved over 80 % accuracy, with AdaBoost reaching 98.5 % and Stacking offering the fastest runtime. AdaBoost and Bagging also delivered strong performance, with F1 scores of 0.980 and 0.976, respectively.

An Efficient Ensemble Approach for Brain Tumors Classification Using Magnetic Resonance Imaging

Tumors in the brain can be life-threatening, making early and precise detection crucial for effective treatment and improved patient outcomes. Deep learning (DL) techniques have shown significant potential in automating the early diagnosis of brain tumors by analyzing magnetic resonance imaging (MRI), offering a more efficient and accurate approach to classification. Deep convolutional neural networks (DCNNs), which are a sub-field of DL, have the potential to analyze rapidly and accurately MRI data and, as such, assist human radiologists, facilitating quicker diagnoses and earlier treatment initiation. This study presents an ensemble of three high-performing DCNN models, i.e., DenseNet169, EfficientNetB0, and ResNet50, for accurate classification of brain tumors and non-tumor MRI samples. Our proposed ensemble model demonstrates significant improvements over various evaluation parameters compared to individual state-of-the-art (SOTA) DCNN models. We implemented ten SOTA DCNN models, i.e., EfficientNetB0, ResNet50, DenseNet169, DenseNet121, SqueezeNet, ResNet34, ResNet18, VGG16, VGG19, and LeNet5, and provided a detailed performance comparison. We evaluated these models using two learning rates (LRs) of 0.001 and 0.0001 and two batch sizes (BSs) of 64 and 128 and identified the optimal hyperparameters for each model. Our findings indicate that the ensemble approach outperforms individual models, having 92% accuracy, 90% precision, 92% recall, and an F1 score of 91% at a 64 BS and 0.0001 LR. This study not only highlights the superior performance of the ensemble technique but also offers a comprehensive comparison with the latest research.

Harnessing machine learning for enhanced quantitative financial strategies: Applications and case studies

Machine learning algorithms have long been an essential tool for quantitative finance, as they play an important role in the asset pricing, portfolio optimisation and financial risk management. This work discusses how to apply Random Forest, Support Vector Machine (SVM), and Neural Networks to further develop and enhance the quantitative investment research. With automatic feature selection, reconstruction of high-dimensional data and resist against over-fitting, Random Forest has been an efficient ensemble approach with remarkable prediction performance. For this reason, Random Forest is leveraged for the stock price prediction. SVM, particularly known for its robustness in high-dimensional classification task, has a non-linear optimisation ability that can be employed to capture the inherent non-linear dependencies in options prices. Neural Networks, as a learning approach capable of discovering complex relationships, is employed for the portfolio optimisation, with a promise for further improving the risk-adjusted portfolio return. The work is conducted through practical case studies to simulate how Random Forest, SVM and Neural Networks enable us to further improve the financial performance, and to illustrate its advantage, disadvantage and practical application in quantitative finance.

An Efficient Ensemble Framework for Human Gait Recognition Using CNN-LSTM With Extra Tree Classifier and Smartphone Sensors in Real-World Environment

An Efficient Ensemble Framework for Human Gait Recognition Using CNN-LSTM With Extra Tree Classifier and Smartphone Sensors in Real-World Environment

An efficient ensemble explainable AI (XAI) approach for morphed face detection

Numerous deep neural convolutional architectures have been proposed in literature for face Morphing Attack Detection (MADs) to prevent such attacks and lessen the risks associated with them. Although, deep learning models achieved optimal results in terms of performance, it is difficult to understand and analyze these networks since they are black box/opaque in nature. As a consequence, incorrect judgments may be made. There is, however, a dearth of literature that explains decision-making methods of black box deep learning models for biometric Presentation Attack Detection (PADs) or MADs that can aid the biometric community to have trust in deep learning-based biometric systems for identification and authentication in various security applications such as border control, criminal database establishment etc. In this work, we present a novel visual explanation approach named Ensemble XAI integrating Saliency maps, Class Activation Maps (CAM) and Gradient-CAM (Grad-CAM) to provide a more comprehensive visual explanation for a deep learning prognostic model (EfficientNet-B1) that we have employed to predict whether the input presented to a biometric authentication system is morphed or genuine. The experimentations have been performed on three publicly available datasets namely Face Research Lab London (FRLL) dataset, Wide Multi-Channel Presentation Attack (WMCA) dataset, and Makeup Induced Face Spoofing (MIFS) dataset. The experimental evaluations affirms that the resultant visual explanations highlight more fine-grained details of image features/areas focused by EfficientNet-B1 to reach decisions along with appropriate reasoning.

Encoding the Latent Posterior of Bayesian Neural Networks for Uncertainty Quantification.

Bayesian Neural Networks (BNNs) have long been considered an ideal, yet unscalable solution for improving the robustness and the predictive uncertainty of deep neural networks. While they could capture more accurately the posterior distribution of the network parameters, most BNN approaches are either limited to small networks or rely on constraining assumptions, e.g., parameter independence. These drawbacks have enabled prominence of simple, but computationally heavy approaches such as Deep Ensembles, whose training and testing costs increase linearly with the number of networks. In this work we aim for efficient deep BNNs amenable to complex computer vision architectures, e.g., ResNet-50 DeepLabv3+, and tasks, e.g., semantic segmentation and image classification, with fewer assumptions on the parameters. We achieve this by leveraging variational autoencoders (VAEs) to learn the interaction and the latent distribution of the parameters at each network layer. Our approach, called Latent-Posterior BNN (LP-BNN), is compatible with the recent BatchEnsemble method, leading to highly efficient (in terms of computation and memory during both training and testing) ensembles. LP-BNNs attain competitive results across multiple metrics in several challenging benchmarks for image classification, semantic segmentation, and out-of-distribution detection.

LiFoL: An Efficient Framework for Financial Distress Prediction in High-Dimensional Unbalanced Scenario

Corporate financial distress will significantly damage the company’s and its stakeholders’ interests and even lead to a global financial crisis. Therefore, finding an efficient method for financial distress prediction (FDP) to avoid greater losses is essential. Although there is a lot of research and progress in this field, the existing methods rarely consider the problems of high dimensionality and class imbalance, which will largely limit the models to achieve satisfactory performance. To alleviate these problems, this article first proposes a novel Lightspace-SMOTE upsampling method, which can reduce the feature dimensionality and increase the signal-to-noise ratio (SNR) of the original data and then upsample it to increase the number of minor class samples. In addition, this article proposes an efficient ensemble framework (LiFoL) that combines Lightspace-SMOTE, focal loss (FL), and LightGBM, which can not only focus more on minor class and the hard-to-class samples but also obtain better performance. At the same time, the feature importance provided by the model can provide strong support for model interpretability. Experimental results show that the Lightspace-SMOTE upsampling method can help the model achieve higher scores in area under ROC curve (AUC) and recall, especially in the case of longer prediction periods. Compared with current methods, LiFoL can achieve more than 10% improvement in AUC and more than 20% in recall.

Efficient Ensemble Broad Learning System Based on Dropout and DropConnect

Broad Learning System is an emerged efficient algorithm for training single hidden layer feedforward neural network (SLFN) with fast speed and good generalization ability. However, it is very difficult to determine the appropriate broad learning system structure and broad learning system may perform overfitting due to the dependence between nodes in processing fully connected network. In order to deal with these problems, efficient ensemble broad learning system based on Dropout and DropConnect are proposed in this paper. The proposed Dropout Ensemble Broad Learning System randomly discards hidden nodes to improve diversity between individuals and reduce the synergy between nodes to improve prediction stability. The DropConnect Ensemble Broad Learning System randomly drops connect weights to generate more complementary models by adding input attribute disturbance. The experimental results and statistical analysis on UCI data sets confirm that the proposed method can solve the problem of model overfitting caused by the strong dependence between the nodes of ensemble broad learning system and also show that the proposed approaches outperform the original BLS on prediction stability and classification accuracy.

A reliable diabetic retinopathy grading via transfer learning and ensemble learning with quadratic weighted kappa metric

The most common eye infection in people with diabetes is diabetic retinopathy (DR). It might cause blurred vision or even total blindness. Therefore, it is essential to promote early detection to prevent or alleviate the impact of DR. However, due to the possibility that symptoms may not be noticeable in the early stages of DR, it is difficult for doctors to identify them. Therefore, numerous predictive models based on machine learning (ML) and deep learning (DL) have been developed to determine all stages of DR. However, existing DR classification models cannot classify every DR stage or use a computationally heavy approach. Common metrics such as accuracy, F1 score, precision, recall, and AUC-ROC score are not reliable for assessing DR grading. This is because they do not account for two key factors: the severity of the discrepancy between the assigned and predicted grades and the ordered nature of the DR grading scale. This research proposes computationally efficient ensemble methods for the classification of DR. These methods leverage pre-trained model weights, reducing training time and resource requirements. In addition, data augmentation techniques are used to address data limitations, improve features, and improve generalization. This combination offers a promising approach for accurate and robust DR grading. In particular, we take advantage of transfer learning using models trained on DR data and employ CLAHE for image enhancement and Gaussian blur for noise reduction. We propose a three-layer classifier that incorporates dropout and ReLU activation. This design aims to minimize overfitting while effectively extracting features and assigning DR grades. We prioritize the Quadratic Weighted Kappa (QWK) metric due to its sensitivity to label discrepancies, which is crucial for an accurate diagnosis of DR. This combined approach achieves state-of-the-art QWK scores (0.901, 0.967 and 0.944) in the Eyepacs, Aptos, and Messidor datasets.

Advancing Barrett's Esophagus Segmentation: A Deep-Learning Ensemble Approach with Data Augmentation and Model Collaboration.

This approach provides a thorough investigation of Barrett's esophagus segmentation using deep-learning methods. This study explores various U-Net model variants with different backbone architectures, focusing on how the choice of backbone influences segmentation accuracy. By employing rigorous data augmentation techniques and ensemble strategies, the goal is to achieve precise and robust segmentation results. Key findings include the superiority of DenseNet backbones, the importance of tailored data augmentation, and the adaptability of training U-Net models from scratch. Ensemble methods are shown to enhance segmentation accuracy, and a grid search is used to fine-tune ensemble weights. A comprehensive comparison with the popular Deeplabv3+ architecture emphasizes the role of dataset characteristics. Insights into training saturation help optimize resource utilization, and efficient ensembles consistently achieve high mean intersection over union (IoU) scores, approaching 0.94. This research marks a significant advancement in Barrett's esophagus segmentation.

Efficient Ensemble via Rotation-Based Self- Supervised Learning Technique and Multi-Input Multi-Output Network

Efficient Ensemble via Rotation-Based Self- Supervised Learning Technique and Multi-Input Multi-Output Network

An Efficient Ensemble Algorithm for Boosting k-Nearest Neighbors Classification Performance via Feature Bagging

An Efficient Ensemble Algorithm for Boosting k-Nearest Neighbors Classification Performance via Feature Bagging

A Robust and Efficient Ensemble of Diversified Evolutionary Computing Algorithms for Accurate Robot Calibration

A Robust and Efficient Ensemble of Diversified Evolutionary Computing Algorithms for Accurate Robot Calibration

Composing Diverse Ensembles of Convolutional Neural Networks by Penalization

Ensemble-based systems are well known to have the capacity to outperform individual approaches if the ensemble members are sufficiently accurate and diverse. This paper investigates how an efficient ensemble of deep convolutional neural networks (CNNs) can be created by forcing them to adjust their parameters during the training process to increase diversity in their decisions. As a new theoretical approach to reach this aim, we join the member neural architectures via a fully connected layer and insert a new correlation penalty term in the loss function to obstruct their similar operation. With this complementary term, we implement the standard guideline of ensemble creation to increase the members’ diversity for CNNs in a more detailed and flexible way than similar existing techniques. As for applicability, we show that our approach can be efficiently used in various classification tasks. More specifically, we demonstrate its performance in challenging medical image analysis and natural image classification problems. Besides the theoretical considerations and foundations, our experimental findings suggest that the proposed technique is competitive. Namely, on the one hand, the classification rate of the ensemble trained in this way outperformed all the individual accuracies of the state-of-the-art member CNNs according to the standard error functions of these application domains. On the other hand, it is also validated that the ensemble members get more diverse and their accuracies are raised by adding the penalization term. Moreover, we performed a full comparative analysis, including other state-of-the-art ensemble-based approaches recommended for the same classification tasks. This comparative study also confirmed the superiority of our method, as it overcame the current solutions.

EMAML: Design of an Efficient Ensemble Model for Detection of Adversarial Attacks in Machine Learning Environments

EMAML: Design of an Efficient Ensemble Model for Detection of Adversarial Attacks in Machine Learning Environments

Highly efficient ensemble algorithms for computing the Stokes–Darcy equations

In this report we propose two highly efficient ensemble algorithms incorporating the gPAV and the rotational pressure correction methods for computing Stokes-Darcy flow ensembles. All variables are fully decoupled including the three components of the velocity, leading to smaller linear systems to be solved at each time step. Moreover, all ensemble members share the same constant coefficient matrix for which the fast block CG method can be applied for computing the ensemble at one pass at significantly reduced computational cost. We prove the two new ensemble algorithms are unconditionally stable with respect to the modified energy without any constraints on the uncertain parameters or the time steps. We will provide details of implementation and discuss how to efficiently solve the corresponding linear systems. Numerical examples are presented to show the efficiency and effectiveness of the algorithms.

A Consolidated Tree Structure Combining Multiple Regression Trees With Varying Depths, Resulting in an Efficient Ensemble Model

A Consolidated Tree Structure Combining Multiple Regression Trees With Varying Depths, Resulting in an Efficient Ensemble Model

An ensemble model to detect packet length covert channels

<span lang="EN-US">Covert channel techniques have enriched the way to commit dangerous and unwatched attacks. They exploit ways that are not intended to convey information; therefore, traditional security measures cannot detect them. One class of covert channels that difficult to detect, mitigate, or eliminate is packet length covert channels. This class of covert channels takes advantage of packet length variations to convey covert information. Numerous research articles reflect the useful use of machine learning (ML) classification approaches to discover covert channels. Therefore, this study presented an efficient ensemble classification model to detect such types of attacks. The ensemble model consists of five machine learning algorithms representing the base classifiers. The base classifiers include naive Bayes (NB), decision tree (DT), support vector machine (SVM), k-nearest neighbor (KNN), and random forest (RF). Whereas, the logistic regression (LR) classifier was employed to aggregate the outputs of the base classifiers and thus to generate the ensemble classifier output. The results showed a good performance of our proposed ensemble classifier. It beats all single classification algorithms, with a 99.3% accuracy rate and negligible classification errors.</span>

An Adaptive Grid Search Based Efficient Ensemble Model for Covid-19 Classification in Chest X-Ray Scans

Covid has resulted in millions of deaths worldwide, making it crucial to develop fast and safe diagnostic methods to control its spread. Chest X-Ray imaging can diagnose pulmonary diseases, including Covid. Most research studies have developed single convolution neural network models ignoring the advantage of combining different models. An ensemble model has higher predictive accuracy and reduces the generalization error of prediction. We employed an ensemble of Multi Deep Neural Networks models for Covid.19 classification in chest X-Ray scans using Multiclass classification (Covid, Pneumonia, and Normal). We improved the accuracy by identifying the best parameters using the sklean Grid search technique and implementing it with the Optimized Weight Average Ensemble Model, which allows multiple models to predict. Our ensemble model has achieved 95.26% accuracy in classifying the X-Ray images; it demonstrates potential in ensemble models for diagnosis using Radiography images.

Optimizing an efficient ensemble approach for high-quality de novo transcriptome assembly of Thymus daenensis

Non-erroneous and well-optimized transcriptome assembly is a crucial prerequisite for authentic downstream analyses. Each de novo assembler has its own algorithm-dependent pros and cons to handle the assembly issues and should be specifically tested for each dataset. Here, we examined efficiency of seven state-of-art assemblers on ~ 30 Gb data obtained from mRNA-sequencing of Thymus daenensis. In an ensemble workflow, combining the outputs of different assemblers associated with an additional redundancy-reducing step could generate an optimized outcome in terms of completeness, annotatability, and ORF richness. Based on the normalized scores of 16 benchmarking metrics, EvidentialGene, BinPacker, Trinity, rnaSPAdes, CAP3, IDBA-trans, and Velvet-Oases performed better, respectively. EvidentialGene, as the best assembler, totally produced 316,786 transcripts, of which 235,730 (74%) were predicted to have a unique protein hit (on uniref100), and also half of its transcripts contained an ORF. The total number of unique BLAST hits for EvidentialGene was approximately three times greater than that of the worst assembler (Velvet-Oases). EvidentialGene could even capture 17% and 7% more average BLAST hits than BinPacker and Trinity. Although BinPacker and CAP3 produced longer transcripts, the EvidentialGene showed a higher collinearity between transcript size and ORF length. Compared with the other programs, EvidentialGene yielded a higher number of optimal transcript sets, further full-length transcripts, and lower possible misassemblies. Our finding corroborates that in non-model species, relying on a single assembler may not give an entirely satisfactory result. Therefore, this study proposes an ensemble approach of accompanying EvidentialGene pipelines to acquire a superior assembly for T. daenensis.