Efficient Ensemble Research Articles

Modification of the Principal Component Analysis Method Based on Feature Rotation by Class Centroids

Feature engineering is a branch of science that provides tools to support, for example, the preparation of feature spaces for a pattern recognition task. The present work focuses on the problem of feature extraction. The proposed model is based on the mechanisms of PCA principal component analysis. It fills a gap in the implementation of feature extraction by looking for spaces that best discriminate between classes. This was realized by rotating the features according to the centroids of the classes. In addition, a measure of their consistency was determined which allows precise estimation of the number of features for a particular component. Four experiments were conducted in this study. The first two were done on synthetic datasets, while the next two were conducted on ten real datasets. The synthetic data allowed to determine the characteristics depending on the percentage of informative features, the number of input features, the level of imbalance and the number of output components in the extraction task. The obtained results showed that the developed solution allows for a more precise extraction, thus increasing the quality of classification. Moreover, it was shown that the method based on class centroids allows to construct efficient ensembles of classifiers.

Adaptive evolution strategy with ensemble of mutations for Reinforcement Learning

Evolving the weights of learning networks through evolutionary computation (neuroevolution) has proven scalable over a range of challenging Reinforcement Learning (RL) control tasks. However, similar to most black-box optimization problems, existing neuroevolution approaches require an additional adaptation process to effectively balance exploration and exploitation through the selection of sensitive hyper-parameters throughout the evolution process. Therefore, these methods are often plagued by the computation complexities of such adaptation processes which often rely on a number of sophisticatedly formulated strategy parameters. In this paper, Evolution Strategy (ES) with a simple yet efficient ensemble of mutation strategies is proposed. Specifically, two distinct mutation strategies coexist throughout the evolution process where each strategy is associated with its own population subset. Consequently, elites for generating a population of offspring are realized by co-evaluation of the combined population. Experiments on testbed of six (6) black-box optimization problems which are generated using a classical control problem and six (6) proven continuous RL agents demonstrate the efficiency of the proposed method in terms of faster convergence and scalability than the canonical ES. Furthermore, the proposed Adaptive Ensemble ES (AEES) shows an average of 5 - 10000x and 10 - 100x better sample complexity in low and high dimension problems, respectively than their associated base DRL agents.

An efficient genetic programming approach to design priority rules for resource-constrained project scheduling problem

In recent years, machine learning techniques, especially genetic programming (GP), have been a powerful approach for automated design of the priority rule-heuristics for the resource-constrained project scheduling problem (RCPSP). However, it requires intensive computing effort, carefully selected training data and appropriate assessment criteria. This research proposes a GP hyper-heuristic method with a duplicate removal technique to create new priority rules that outperform the traditional rules. The experiments have verified the efficiency of the proposed algorithm as compared to the standard GP approach. Furthermore, the impact of the training data selection and fitness evaluation have also been investigated. The results show that a compact training set can provide good output and existing evaluation methods are all usable for evolving efficient priority rules. The priority rules designed by the proposed approach are tested on extensive existing datasets and newly generated large projects with more than 1,000 activities. In order to achieve better performance on small-sized projects, we also develop a method to combine rules as efficient ensembles. Computational comparisons between GP-designed rules and traditional priority rules indicate the superiority and generalization capability of the proposed GP algorithm in solving the RCPSP.

Robust SAV-Ensemble algorithms for parametrized flow problems with energy stable open boundary conditions

In this report we propose fast and robust ensemble simulation algorithms for parametrized nonlinear flow problems with open boundaries. We synthesize the recent SAV idea, the ensemble timestepping and the energy stable open boundary conditions, and develop efficient ensemble algorithms that only require a single matrix assembly for all realizations and all time steps. Moreover, the proposed algorithms are unconditionally stable without any restricting assumptions on the variance of the viscosities or the time steps, and they always produce positive values for the auxiliary variables. The second order algorithm is numerically tested on three benchmark flow problems with open boundaries, and is shown to be both accurate and effective in preventing the backflow instabilities.

Artificial Intelligence-Based Solution in Personalized Computer-Aided Arthroscopy of Shoulder Prostheses

Background: Early recognition of prostheses before reoperation can reduce perioperative morbidity and mortality. Because of the intricacy of the shoulder biomechanics, accurate classification of implant models before surgery is fundamental for planning the correct medical procedure and setting apparatus for personalized medicine. Expert surgeons usually use X-ray images of prostheses to set the patient-specific apparatus. However, this subjective method is time-consuming and prone to errors. Method: As an alternative, artificial intelligence has played a vital role in orthopedic surgery and clinical decision-making for accurate prosthesis placement. In this study, three different deep learning-based frameworks are proposed to identify different types of shoulder implants in X-ray scans. We mainly propose an efficient ensemble network called the Inception Mobile Fully-Connected Convolutional Network (IMFC-Net), which is comprised of our two designed convolutional neural networks and a classifier. To evaluate the performance of the IMFC-Net and state-of-the-art models, experiments were performed with a public data set of 597 de-identified patients (597 shoulder implants). Moreover, to demonstrate the generalizability of IMFC-Net, experiments were performed with two augmentation techniques and without augmentation, in which our model ranked first, with a considerable difference from the comparison models. A gradient-weighted class activation map technique was also used to find distinct implant characteristics needed for IMFC-Net classification decisions. Results: The results confirmed that the proposed IMFC-Net model yielded an average accuracy of 89.09%, a precision rate of 89.54%, a recall rate of 86.57%, and an F1.score of 87.94%, which were higher than those of the comparison models. Conclusion: The proposed model is efficient and can minimize the revision complexities of implants.

Recognition and Tracking of Objects in a Clustered Remote Scene Environment

Object recognition and tracking are two of the most dynamic research sub-areas that belong to the field of Computer Vision. Computer vision is one of the most active research fields that lies at the intersection of deep learning and machine vision. This paper presents an efficient ensemble algorithm for the recognition and tracking of fixed shape moving objects while accommodating the shift and scale invariances that the object may encounter. The first part uses the Maximum Average Correlation Height (MACH) filter for object recognition and determines the bounding box coordinates. In case the correlation based MACH filter fails, the algorithms switches to a much reliable but computationally complex feature based object recognition technique i.e., affine scale invariant feature transform (ASIFT). ASIFT is used to accommodate object shift and scale object variations. ASIFT extracts certain features from the object of interest, providing invariance in up to six affine parameters, namely translation (two parameters), zoom, rotation and two camera axis orientations. However, in this paper, only the shift and scale invariances are used. The second part of the algorithm demonstrates the use of particle filters based Approximate Proximal Gradient (APG) technique to periodically update the coordinates of the object encapsulated in the bounding box. At the end, a comparison of the proposed algorithm with other state-of-the-art tracking algorithms has been presented, which demonstrates the effectiveness of the proposed algorithm with respect to the minimization of tracking errors.

An Efficient Ensemble Model for Various Scale Medical Data

Electronic Health Records (EHRs) are the digital form of patients’ medical reports or records. EHRs facilitate advanced analytics and aid in better decision-making for clinical data. Medical data are very complicated and using one classification algorithm to reach good results is difficult. For this reason, we use a combination of classification techniques to reach an efficient and accurate classification model. This model combination is called the Ensemble model. We need to predict new medical data with a high accuracy value in a small processing time. We propose a new ensemble model MDRL which is efficient with different datasets. The MDRL gives the highest accuracy value. It saves the processing time instead of processing four different algorithms sequentially; it executes the four algorithms in parallel. We implement five different algorithms on five variant datasets which are Heart Disease, Health General, Diabetes, Heart Attack, and Covid-19 Datasets. The four algorithms are Random Forest (RF), Decision Tree (DT), Logistic Regression (LR), and Multi-layer Perceptron (MLP). In addition to MDRL (our proposed ensemble model) which includes MLP, DT, RF, and LR together. From our experiments, we conclude that our ensemble model has the best accuracy value for most datasets. We reach that the combination of the Correlation Feature Selection (CFS) algorithm and our ensemble model is the best for giving the highest accuracy value. The accuracy values for our ensemble model based on CFS are 98.86, 97.96, 100, 99.33, and 99.37 for heart disease, health general, Covid-19, heart attack, and diabetes datasets respectively.

An efficient ensemble framework for outlier detection using bio-inspired algorithm

Outliers are unexpected observations present in data that has to be identified systematically to prevent from catastrophic effects. The detection of outliers plays a crucial role in many application domains, as they provide exciting and critical information about abnormal events. So, in this research work, the bio-inspired ensemble-based outlier detection (BIEOD) is employed to determine the outliers. This ensemble framework combines the sequential and parallel ensembles, thereby creating a hybrid model that utilises the advantages of both feature bagging and XGBoost. The chicken swarm optimisation (CSO) is used to increase the deviation between the outliers and inliers according to the chicken competition in optimising the outlier scores. Finally, the XGBoost predicts the outliers based on the optimised feature set. The obtained results on the benchmark datasets reveal that this framework offers significant advancement over the base learners and other existing ensemble methods of outlier detection.

An Efficient Ensemble Architecture for Privacy and Security of Electronic Medical Records

Electronic medical records, one of the sensitive data, are stored in public or private cloud service providers. Cloud systems provide security with firewall and intrusion detection systems, and these systems ensure privacy with access control and end-to-end encryption. However, while sending data to the cloud system, attackers can capture the data with the help of Man in the Middle attacks and vulnerabilities of the storage systems. In the middleware architecture proposed in this study, access control protocol, key distributor and end-to-end hybrid encryption which are based on user roles were innovatively used to overcome security issues in data transmission. In this system, writing and updating requests are encrypted asymmetrically, and reading requests were encrypted symmetrically. This solution distinguishes the proposed method from previous studies. According to this solution the operating performance of the system is increased. In addition, the attacker cannot see the actual data in a cyber-attacks because the sensitive data is distributed to the users with their private keys. This result shows that the access, write and update of electronic medical records are performed with the principles of security and privacy.

Retracted: Efficient ensemble to combat flash attacks

AbstractFlash event generates enormous traffic and the cloud service providers use sustaining techniques like scaling and content delivery network to up their services. One of the main bottlenecks that the cloud service providers still find difficult to tackle is flash attacks. Illegitimate users send craftily designed packets to land up inside the server for wreaking havoc. As deep learning autoencoder has the potential to detect malicious traffic it has been used in this research study to develop an ensemble. Convolutional neural network is efficacious in overcoming the issue of overfitting; deep autoencoder is proficient in extracting features through dimensionality reduction. In order to obtain both these advantages it was decided to develop an ensemble keeping denoising autoencoder as the core element. The process of addressing a flash attack requires first detecting the presence of bot in malicious traffic, second studying its nature by observing its behavioral manifestations. Detection of botnet was achieved by three ensembles, namely, DAE_CNN, DAE_MLP, and DAE_XGB. But capturing its external manifested behavior is challenging, because the bot signatures are always in a state of flux. The simulated empirical study yielded an appreciable outcome. Its accuracy rate was 99.9% for all the three models and the false positive rates were 0, 0.006, and 0.001, respectively.

COFE-Net: An ensemble strategy for Computer-Aided Detection for COVID-19

Biomedical images contain a large volume of sensor measurements, which can reveal the descriptors of the disease under investigation. Computer-based analysis of such measurements helps detect the disease, and thereby swiftly aid medical professionals to choose adequate therapy. In this paper, we propose a robust deep learning ensemble framework known as COVID Fuzzy Ensemble Network, or COFE-Net. This strategy is proposed for the task of COVID-19 screening from chest X-rays (CXR) and CT Scans, as a part of Computer-Aided Detection (CADe) for medical practitioners. We leverage the strategy of Transfer Learning for Convolutional Neural Networks (CNNs) widely adopted in recent literature, and further propose an efficient ensemble network for their combination. The principles of fuzzy logic have been leveraged to combine the measured decision scores generated by three state-of-the-art CNNs – Inception V3, Inception ResNet V2 and DenseNet 201 – through the Choquet fuzzy integral. Experimental results support the efficacy of our approach over empirical ensembling, as the fuzzy ensembling strategy for biomedical measurement consists of dynamic refactoring of the classifier ensemble weights on the fly, based upon the confidence scores for coalitions of inputs. This is the chief advantage of our biomedical measurement strategy over others as other methods do not adjust to the multiple generated measurements dynamically unlike ours.Impressive results on multiple datasets demonstrate the effectiveness of the proposed method. The source code of our proposed method is made available at: https://github.com/theavicaster/covid-cade-ensemble.

SAV decoupled ensemble algorithms for fast computation of Stokes–Darcy flow ensembles

Numerical modeling and simulation of complex systems is often subject to uncertainties in model parameters. Many popular uncertainty quantification (UQ) methods require repeated simulations of the underlying physical system with different samples of the uncertain model parameters. This poses great challenges to many practical engineering applications due to the high demand for computational resources. In this report we propose highly efficient ensemble simulation algorithms for fast computation of coupled flow ensembles. The proposed ensemble algorithms are based on two recently developed numerical approaches: scalar auxiliary variable (SAV) and ensemble timestepping. We introduce a new decoupling strategy using the SAV idea and incorporate the ensemble timestepping method to develop two decoupled ensemble schemes for the Stokes–Darcy system: SAV-BE-En and SAV-BDF2-En. The two ensemble algorithms are specially designed for UQ computations where a number of realizations of the underlying coupled PDE system are required for analyzing and interpreting flow statistics. Compared with traditional methods which solve for each realization independently, our proposed ensemble algorithms result in a common coefficient matrix for all realizations and efficient iterative solvers such as block CG or block GMRES can be used to solve for all realizations simultaneously reducing both computer storage and overall simulation time. We prove that both ensemble algorithms are long time stable without any time step conditions. We also provide a comprehensive error analysis for the fully discrete SAV-BE-En algorithm, and present a few illustrative numerical examples to demonstrate the efficiency and effectiveness of the algorithms.

Efficient ensemble generation for uncertain correlated parameters in atmospheric chemical models: a case study for biogenic emissions from EURAD-IM version 5

Abstract. Atmospheric chemical forecasts heavily rely on various model parameters, which are often insufficiently known, such as emission rates and deposition velocities. However, a reliable estimation of resulting uncertainties with an ensemble of forecasts is impaired by the high dimensionality of the system. This study presents a novel approach, which substitutes the problem into a low-dimensional subspace spanned by the leading uncertainties. It is based on the idea that the forecast model acts as a dynamical system inducing multivariate correlations of model uncertainties. This enables an efficient perturbation of high-dimensional model parameters according to their leading coupled uncertainties. The specific algorithm presented in this study is designed for parameters that depend on local environmental conditions and consists of three major steps: (1) an efficient assessment of various sources of model uncertainties spanned by independent sensitivities, (2) an efficient extraction of leading coupled uncertainties using eigenmode decomposition, and (3) an efficient generation of perturbations for high-dimensional parameter fields by the Karhunen–Loéve expansion. Due to their perceived simulation challenge, the method has been applied to biogenic emissions of five trace gases, considering state-dependent sensitivities to local atmospheric and terrestrial conditions. Rapidly decreasing eigenvalues state that highly correlated uncertainties of regional biogenic emissions can be represented by a low number of dominant components. Depending on the required level of detail, leading parameter uncertainties with dimensions of 𝒪(106) can be represented by a low number of about 10 ensemble members. This demonstrates the suitability of the algorithm for efficient ensemble generation for high-dimensional atmospheric chemical parameters.

Efficient ensemble for image-based identification of Pneumonia utilizing deep CNN and SGD with warm restarts

Childhood pneumonia, the leading cause of children mortality globally, is most commonly diagnosed based on the radiographic data, which requires radiologic interpretation of X-ray images. With recent advancements in the field of deep learning, the convolutional neural networks (CNN) have proven to be able to achieve great performance in medical image segmentation, analysis and classification tasks. However, developing and training methods utilizing CNN is still a complex and time-consuming process with several open issues — generalization, demand for large datasets, and high time complexity. The optimization objective in the training of CNN models has multiple minima, which do not necessarily generalize well and may result in poor performance. Ensemble methods are commonly employed to address the generalization issue, but they require a group of diverse models and are generally even more time-consuming. To address the issues of generalization, dataset size and time complexity, we developed an ensemble method based on stochastic gradient descent with warm restarts (SGDRE) that exploits the generalization capabilities of ensemble methods and SGD with warm restarts mechanism, which is adopted to obtain a diverse group of classifiers, necessary for ensemble, in one single training process, spending the same or less training time than a single CNN classification model. The SGDRE method has been trained on publicly available pediatric chest X-ray images dataset and evaluated using 10-fold cross-validation approach. The experimental results show a significant improvement of SGDRE over the two compared baseline methods. With an achieved test accuracy of 96.26% and AUC of 95.15%, the proposed method proved to be a very competitive classification method.

SemRec – An efficient ensemble recommender with sentiment based clustering for social media text corpus

SummaryThe frequent user interactions happening in the form of textual contents like reviews, ratings, tags, blogs, testimonials, and so forth transformed the social media platform into a contextualized and personalized data warehouse focusing its users' unique likes and dislikes. The huge volume of social media content makes it difficult for the end users to consume relevant information by themselves. The need of a tool to deal with such scenario leads to the development of recommendation systems. This work proposes an ensemble multi‐stage recommender system with sentiment based clustering to deal with social media text corpus where each stage performing unique functionalities of information retrieval, natural language processing, user segmentation, prediction, and recommendation generation. The proposed system leverages a hybrid approach of content‐based, collaborative, and demographic filtering techniques to predict and recommend contents, products, or services according to user interests. The experimental results gathered using standard datasets are promising and found more efficient than the traditional approaches.

An Efficient Ensemble Binarized Deep Neural Network on Chip with Perception-Control Integrated.

Lightweight UAVs equipped with deep learning models have become a trend, which can be deployed for automatic navigation in a wide range of civilian and military missions. However, real-time applications usually need to process a large amount of image data, which leads to a very large computational complexity and storage consumption, and restricts its deployment on resource-constrained embedded edge devices. To reduce the computing requirements and storage occupancy of the neural network model, we proposed the ensemble binarized DroNet (EBDN) model, which implemented the reconstructed DroNet with the binarized and ensemble learning method, so that the model size of DroNet was effectively compressed, and ensemble learning method was used to overcome the defect of the poor performance of the low-precision network. Compared to the original DroNet, EBDN saves more than 7 times of memory footprint with similar model accuracy. Meanwhile, we also proposed a novel and high-efficiency hardware architecture to realize the EBDN on the chip (EBDNoC) system, which perfectly realizes the mapping of an algorithm model to hardware architecture. Compared to other solutions, the proposed architecture achieves about 10.21 GOP/s/kLUTs resource efficiency and 208.1 GOP/s/W energy efficiency, while also providing a good trade-off between model performance and resource utilization.

An Efficient ensemble of Brain Tumour Segmentation and Classification using Machine Learning and Deep Learning based Inception Networks

In recent times, Brain Tumor (BT) has become a common phenomenon affecting almost all age group of people. Identification of this deadly disease using computer tomography, magnetic resonance imaging are very popular now-a-days. Developing a Computer Aided Design (CAD) tool for diagnosis and classification of BT has become vital. This paper focuses on designing a tool for diagnosis and classification of BT using Deep Learning (DL) models, which involves a series of steps via acquiring (CT) image, pre-processing, segmenting and classifying to identify the type of tumor using SIFT with DL based Inception network model. The proposed model uses fuzzy C means algorithm for segmenting area of interest from the BT image acquired. Techniques like Gaussian Naïve Bayes (GNB) and logistic regression (LR) are used for classification processes. To ascertain all the techniques for its efficiency a benchmark dataset was used. The simulation outcome ensured that the performance of the proposed method with maximum sensitivity of 100%, specificity of 97.41% and accuracy of 97.96%.

Classification with ensembles and case study on functional magnetic resonance imaging

The ensemble is a technique that strategically combines basic models to achieve better accuracy rates. Diversity, combination methods, and selection topology are the main factors determining ensemble performance. Consequently, it is a challenging task to design an efficient ensemble scheme. Even though numerous paradigms have been proposed to classify ensemble schemes, there is still much room for improvement. This paper proposes a general framework for creating ensembles in the context of classification. Specifically, the ensemble framework consists of four stages: objectives, data preparing, model training, and model testing. It is comprehensive to design diverse ensembles. The proposed ensemble approach can be used for a wide variety of machine learning tasks. We validate our approach on real-world datasets. The experimental results show the efficiency of the proposed approach.

Rapid Identification of Inhibitors and Prediction of Ligand Selectivity for Multiple Proteins: Application to Protein Kinases.

Rapid identification of inhibitors for a family of proteins and prediction of ligand specificity are highly desirable for structure-based drug design. However, sequentially docking ligands into each protein target with conventional single-target docking methods is too computationally expensive to achieve these two goals, especially when the number of the targets is large. In this work, we use an efficient ensemble docking algorithm for simultaneous docking of ligands against multiple protein targets. We use protein kinases, a family of proteins that are highly important for many cellular processes and for rational drug design, as an example to demonstrate the feasibility of investigating ligand selectivity with this algorithm. Specifically, 14 human protein kinases were selected. First, native docking calculations were performed to test the ability of our energy scoring function to reproduce the experimentally determined structures of the ligand-protein kinase complexes. Next, cross-docking calculations were conducted using our ensemble docking algorithm to study ligand selectivity, based on the assumption that the native target of an inhibitor should have a more negative (i.e., favorable) energy score than the non-native targets. Staurosporine and Gleevec were studied as examples of nonselective and selective binding, respectively. Virtual ligand screening was also performed against five protein kinases that have at least seven known inhibitors. Our quantitative analysis of the results showed that the ensemble algorithm can be effective on screening for inhibitors and investigating their selectivities for multiple target proteins.

A Computationally Efficient Ensemble Filtering Scheme for Quantitative Volcanic Ash Forecasts

AbstractA method of assimilating satellite observations in quantitative ensemble forecasting models of airborne volcanic ash is presented in this study. The method employs many trial dispersion model simulations that are generated by both deterministic and random perturbations of the source term and use of an ensemble of numerical weather prediction model fields. An ensemble filter is then applied to the trial simulations, which are either selected or rejected by the filter based on their degree of agreement with observations within a specified time window. The observations may be in the form of quantitative satellite retrieved mass load fields or qualitative ash detection fields, which means that useful results can be obtained even when retrievals are not available in real time provided that the ash boundaries can be identified. The filtering process is repeated several times with different random realizations of the source term to reduce sampling error and minimize filter degeneracy, a phenomenon that plagues all ensemble filter models. The selected members are then propagated forward in time beyond the observational time window to form the forecast ensemble. We show, using several eruption case studies, that forecast ensembles constructed in this way are generally superior in skill to reference forecasts that do not assimilate observations.