Ensemble Classification Method Research Articles

An Intelligent Ensemble Classification Method For Spam Diagnosis in Social Networks

In recent years, the destructive behavior of social networks spammers has seriously threatened the information security of ordinary users. To reduce this threat, many researchers have extracted the behavioral characteristics of spam and obtained good results based on machine learning algorithms to identify them. However, most of these studies use a single classification technique that often works differently for different spam data. In this paper, an intelligent ensemble classification method for social networks spam detection is introduced. The proposed heterogeneous ensemble learning framework is based on stack generalization and uses an evolutionary algorithm to improve the modeling process and reduce complexity. In particular, particle swarm optimization has been used as an evolutionary algorithm to optimize model parameters to reduce model complexity. These parameters include a subset of effective features and a subset of the most appropriate single classification techniques. The SPAM E-mail dataset used in this article contains the correct and effective features in spam prediction. Experimental results show that the proposed algorithm effectively improves the detection rate of spam and performs better than the methods used.

Network intrusion detection based on ensemble classification and feature selection method for cloud computing

AbstractCloud computing security is the most critical factor for providers, cloud users, and organizations. The various novel approaches apply host‐based or network‐based methods to increase cloud security performance and detection rate. However, due to the virtual and distributed environment of the cloud, conventional network intrusion detection systems (NIDS) have been unreliable in handling these security attacks. Therefore, we design a methodology that incorporates feature selection and classification using ensemble techniques to provide efficient and accurate intrusion detection to address these problems. This proposed model combines the three most effective feature selection techniques (gain‐ratio, chi‐squared, and information gain) to offer a qualifying result and four top classifiers (SVM, LR, NB, and DT) using enhanced weighted majority voting. Moreover, we proposed an experimental technique using a new dataset called Honeypot. All experiments utilized three datasets: Honeypots, Kyoto, and NSL: KDD. In addition, the results of this experimental study were compared with other approaches and performed the statistical significance analysis. Finally, the results reveal that the proposed intrusion detection based on the Honeypot dataset was better and more efficient than other methods because we have an accuracy of 98.29%, FAR of 0.012%, DR of 97.9%, and AUC = 0.9921.

Texture analysis based graph approach for automatic detection of neonatal seizure from multi-channel EEG signals

Seizure detection is a particularly difficult task for neurologists to correctly identify the Electroencephalography (EEG)-based neonatal seizures in a visual manner. There is a strong demand to recognize the seizures in more automatic manner. Developing an expert seizure detection system with an acceptable performance level can partly fill this research gap. This paper proposes a new framework for the automated detection of neonatal seizures based on the Morse Wavelet approach that is coupled with a local binary pattern algorithm, and a graph-based community detection algorithm. An ensemble classifier method is designed to detect neonatal seizures prevalent in EEG signals. Our findings show that only 59 of the texture features can exhibit the abnormal increase in an EEG amplitude and the spikes notable during a seizure. The present results demonstrate that the proposed seizure detection model is more accurate for the detection of seizures compared with some of the traditional approaches.

Wetland classification using parcel-level ensemble algorithm based on Gaofen-6 multispectral imagery and Sentinel-1 dataset

Wetland ecosystems have experienced dramatic challenges in the past few decades due to global climate change and human activities. Wetland maps are essential tools for conservation and management of terrestrial ecosystems. The objective of this study was to obtain an accurate wetland map using a parcel-level ensemble method based on Sentinel-1 SAR time series and segmentations generated from GF-6 MPS images. The Dongting Lake wetlands in China, which has a heterogeneous landscape, was deliberately chosen as a challenging case study. First, both VV and VH polarization backscatters (σ0 VV, σ0 VH) were generated from time series Sentinel-1 SAR. Speckle noise inherent in SAR data can affect the performance on image segmentation by resulting in over-segmentation or under-segmentation, and that will degrade the accuracies of wetland classification. Subsequently, optical data (Gaofen-6), which is effectively capable of delineating meaningful parcels, were applied to get the segmentations in this study. Finally, the ensemble method was utilized to extract the wetland information at parcel-level. The overall accuracy and Kappa coefficient of the object-based ensemble method are 90.58% and 0.88, which are 4.25% and 0.04 higher than that of the pixel-based method, respectively. Moreover, the object-based ensemble method for classification in high heterogeneity areas is superior and can greatly improve the performance compared with single classifiers.

A Meta-heuristic Algorithm for the Minimal High-Quality Feature Extraction of Online Reviews

Feature extraction and selection are critical in sentiment analysis (SA) to extract and select only the appropriate features by removing those deemed redundant. As such, the successful implementation of this process leads to better classification accuracy. Inevitably, selecting high-quality minimal features can be challenging given the inherent complication in dealing with over-fitting issues. Most of the current studies used a heuristic method to perform the classification process that will result in selecting and examining only a single feature subset, while ignoring the other subsets that might give better results. This study explored the effect of using the meta-heuristic method together with the ensemble classification method in the sentiment classification of online reviews. Adding to that point, the extraction and selection of relevant features used feature ranking, hyper-parameter optimization, crossover, and mutation, while the classification process utilized the ensemble classifier. The proposed method was tested on the polarity movie review dataset v2.0 and product review dataset (books, electronics, kitchen, and music). The test results indicated that the proposed method significantly improved the classification results by 94%, which far exceeded the existing method. Therefore, the proposed feature extraction and selection method can help in improving the performance of SA in online reviews and, at the same time, reduce the number of extracted features.

Breast Cancer Classification on Multiparametric MRI – Increased Performance of Boosting Ensemble Methods

Introduction: This study aims to assess the utility of Boosting ensemble classification methods for increasing the diagnostic performance of multiparametric Magnetic Resonance Imaging (mpMRI) radiomic models, in differentiating benign and malignant breast lesions. Methods: The dataset includes mpMR images of 140 female patients with mass-like breast lesions (70 benign and 70 malignant), consisting of Dynamic Contrast Enhanced (DCE) and T2-weighted sequences, and the Apparent Diffusion Coefficient (ADC) calculated from the Diffusion Weighted Imaging (DWI) sequence. Tumor masks were manually defined in all consecutive slices of the respective MRI volumes and 3D radiomic features were extracted with the Pyradiomics package. Feature dimensionality reduction was based on statistical tests and the Boruta wrapper. Hierarchical Clustering on Spearman's rank correlation coefficients between features and Random Forest classification for obtaining feature importance, were implemented for selecting the final feature subset. Adaptive Boosting (AdaBoost), Gradient Boosting (GB), Extreme Gradient Boosting (XGBoost) and Light Gradient Boosting Machine (LightGBM) classifiers, were trained and tested with bootstrap validation in differentiating breast lesions. A Support Vector Machine (SVM) classifier was also exploited for comparison. The Receiver Operator Characteristic (ROC) curves and DeLong's test were utilized to evaluate the classification performances. Results: The final feature subset consisted of 5 features derived from the lesion shape and the first order histogram of DCE and ADC images volumes. XGboost and LGBM achieved statistically significantly higher average classification performances [AUC = 0.95 and 0.94 respectively], followed by Adaboost [AUC = 0.90], GB [AUC = 0.89] and SVM [AUC = 0.88]. Conclusion: Overall, the integration of Ensemble Learning methods within mpMRI radiomic analysis can improve the performance of computer-assisted diagnosis of breast cancer lesions.

Ml-Kfhe: Multi-Label Ensemble Classification Algorithm Exploiting Sensor Fusion Properties of the Kalman Filter

Despite the success of ensemble classification methods in multi-class classification problems, ensemble methods based on approaches other than bagging have not been widely explored for multi-label classification problems. The Kalman Filter-based Heuristic Ensemble (KFHE) is a recent ensemble method that exploits the sensor fusion properties of the Kalman filter to combine several classifier models, and that has been shown to be very effective. This work proposes a multi-label version of KFHE, ML-KFHE, demonstrating the effectiveness of the KFHE method on multi-label datasets. Two variants are introduced based on the underlying component classifier algorithm, ML-KFHE-HOMER, and ML-KFHE-CC which uses HOMER and Classifier Chain (CC) as the underlying multi-label algorithms respectively. ML-KFHE-HOMER and ML-KFHE-CC sequentially trains multiple HOMER and CC multi-label classifiers and aggregates their outputs using the sensor fusion properties of the Kalman filter. Experiments and detailed analysis performed on thirteen multi-label datasets and eight other algorithms, including state-of-the-art ensemble methods, show that for both versions, the ML-KFHE framework improves the ensembling process significantly with respect to bagging based combinations of HOMER and CC, thus demonstrating the effectiveness of ML-KFHE. Also, the ML-KFHE-HOMER variant was found to perform consistently and significantly better than existing multi-label methods including existing approaches based on ensembles.

Phishing Websites Detection by Using Optimized Stacking Ensemble Model

Phishing attacks are security attacks that do not affect only individuals’ or organizations’ websites but may affect Internet of Things (IoT) devices and networks. IoT environment is an exposed environment for such attacks. Attackers may use thingbots software for the dispersal of hidden junk emails that are not noticed by users. Machine and deep learning and other methods were used to design detection methods for these attacks. However, there is still a need to enhance detection accuracy. Optimization of an ensemble classification method for phishing website (PW) detection is proposed in this study. A Genetic Algorithm (GA) was used for the proposed method optimization by tuning several ensemble Machine Learning (ML) methods parameters, including Random Forest (RF), AdaBoost (AB), XGBoost (XGB), Bagging (BA), GradientBoost (GB), and LightGBM (LGBM). These were accomplished by ranking the optimized classifiers to pick out the best classifiers as a base for the proposed method. A PW dataset that is made up of 4898 PWs and 6157 legitimate websites (LWs) was used for this study's experiments. As a result, detection accuracy was enhanced and reached 97.16 percent.

Distance classifier ensemble based on intra-class and inter-class scatter

Distance classifier ensemble method based on Intra-class and Inter-class Scatter is proposed in this paper. By Bootstrap technology, the training samples are sampled repeatedly to generate several subsample set, define Intra-class and Inter-class Scatter matrix with subsample set, train subsample set with scatter matrix, generate individual classifier. In the classifier ensemble, the results are integrated with the relative majority voting method. Experiment is tested on UCI standard database, the experimental results show that the proposed ensemble method based on Intra-class and Inter-class Scatter for distance classifier is effective, and it is superior to other methods in classification performance.

Using Ensemble and TOPSIS with AHP for Classification and Selection of Web Services

Due to the increasing number of Web Services with the same functionality, selecting a Web Service that best serves the needs of the Web Client has become a tremendously challenging task. Present approaches use non-functional parameters of the Web Services but they do not consider any preprocessing of the set of functionally Similar Web Services. The lack of preprocessing results in increased use of computational resources due to unnecessary processing of Web Services that have a very low to no chance of satisfying the consumer’s requirements. In this paper, we propose an Ensemble classification method for preprocessing and a Web Service Selection method based on the Quality of Service (QoS) parameters. Once the most eligible Web Services are enumerated through classification, they are ranked using the Technique of Order Preference by Similarity to Ideal Solution (TOPSIS) method with Analytic Hierarchy Process (AHP) used for weight calculation. A prototype of the method is developed, and experiments are conducted on a real-world Web Services dataset. Results demonstrate the feasibility of the proposed method.

Electronic Recommendation Based on Customer Review Using Text Mining

In machine learning, high-dimensional data classification is one of the significant challenges. For dimensionality reduction, the traditional classifiers enhance the variety of classifiers. The conventional method has some restrictions; information loss is caused during dimensionality reduction, minimizing accuracy. The selection of the sample is vulnerable to redundant features and noise. The proposed method, Hybrid Dimensionality Reduction Forest (HDRF) with Random Forest (RF) ensemble classifier and kappa measure, were used to overcome those restrictions. Initially, the Kappa measure is used for pruning, and the higher degree is selected from the forest. For partitioning the features, a tree-based selection method is used. Principal Component Analysis (PCA) is used for feature extraction, noise reduction, and dimensionality reduction. The proposed method removes the weak classifiers and eliminates redundancy. Also, it reduces the unselected structures and the fundamental structures into a new system. The evaluation results on 25 high-dimensional data, the proposed method outperforms with Random Forest ensemble classifier methods and provides enhanced results obtained on 21 out of 25.

Melanoma Classification from Dermoscopy Images Using Ensemble of Convolutional Neural Networks

Human skin is the most exposed part of the human body that needs constant protection and care from heat, light, dust, and direct exposure to other harmful radiation, such as UV rays. Skin cancer is one of the dangerous diseases found in humans. Melanoma is a form of skin cancer that begins in the cells (melanocytes) that control the pigment in human skin. Early detection and diagnosis of skin cancer, such as melanoma, is necessary to reduce the death rate due to skin cancer. In this paper, the classification of acral lentiginous melanoma, a type of melanoma with benign nevi, is being carried out. The proposed stacked ensemble method for melanoma classification uses different pre-trained models, such as Xception, Inceptionv3, InceptionResNet-V2, DenseNet121, and DenseNet201, by employing the concept of transfer learning and fine-tuning. The selection of pre-trained CNN architectures for transfer learning is based on models having the highest top-1 and top-5 accuracies on ImageNet. A novel stacked ensemble-based framework is presented to improve the generalizability and increase robustness by fusing fine-tuned pre-trained CNN models for acral lentiginous melanoma classification. The performance of the proposed method is evaluated by experimenting on a Figshare benchmark dataset. The impact of applying different augmentation techniques has also been analyzed through extensive experimentations. The results confirm that the proposed method outperforms state-of-the-art techniques and achieves an accuracy of 97.93%.

Wi-CAS: A Contactless Method for Continuous Indoor Human Activity Sensing Using Wi-Fi Devices.

With the new coronavirus raging around the world, home isolation has become an effective way to interrupt the spread of the virus. Effective monitoring of people in home isolation has also become a pressing issue. However, the large number of isolated people and the privatized isolated spaces pose challenges for traditional sensing techniques. Ubiquitous Wi-Fi offers new ideas for sensing people indoors. Advantages such as low cost, wide deployment, and high privacy make indoor human activity sensing technology based on Wi-Fi signals increasingly used. Therefore, this paper proposes a contactless indoor person continuous activity sensing method based on Wi-Fi signal Wi-CAS. The method allows for the sensing of continuous movements of home isolated persons. Wi-CAS designs an ensemble classification method based on Hierarchical Clustering (HEC) for the classification of different actions, which effectively improves the action classification accuracy while reducing the processing time. We have conducted extensive experimental evaluations in real home environments. By recording the activities of different people throughout the day, Wi-CAS is very sensitive to unusual activities of people and also has a combined activity recognition rate of 94.3%. The experimental results show that our proposed method provides a low-cost and highly robust solution for supervising the activities of home isolates.

Machine learning-assisted environmental surveillance of Legionella: A retrospective observational study in Friuli-Venezia Giulia region of Italy in the period 2002–2019

Legionella spp. are considered an important cause of potentially preventable morbidity and mortality, making environmental surveillance a crucial component of risk assessment plans. In this work, historical data of about 24,000 water samples collected during a series of environmental surveys in the Italian region of Friuli-Venezia Giulia in the period 2002–2019 were analysed using machine learning methods. The objective was to determine the most relevant factors linked to the Legionella spread in the considered territory, and to compare such findings to those already presented in the literature. Specifically, retrospective unsupervised spatio-temporal analyses on geocoded data were carried out to identify potential clusters with an excess of cases in the higher-risk categories. In addition, tree-based ensemble classification methods were employed to try to predict the samples serogroup and contamination levels based on climatic and geographical factors, highlighting relationships between the values of such predictors and the presence of specific serogroups. Results based on cross-validation show that it is possible to identify spatially restricted zones with unusual contamination based on data collected during routine surveillance, and that the serogroup appears to be influenced by humidity, temperature and latitude, although the interaction with environmental factors is complex and still not entirely clarified. Despite that, our analyses highlighted several relationships between environmental factors and the presence of Legionella. Most importantly, there is an indication that some Legionella strains are more tolerant to specific microclimates, suggesting that further work should be conducted in this sense when more data will become available.

An Ensemble Classification Method for High-Dimensional Data Using Neighborhood Rough Set

Mining useful knowledge from high-dimensional data is a hot research topic. Efficient and effective sample classification and feature selection are challenging tasks due to high dimensionality and small sample size of microarray data. Feature selection is necessary in the process of constructing the model to reduce time and space consumption. Therefore, a feature selection model based on prior knowledge and rough set is proposed. Pathway knowledge is used to select feature subsets, and rough set based on intersection neighborhood is then used to select important feature in each subset, since it can select features without redundancy and deals with numerical features directly. In order to improve the diversity among base classifiers and the efficiency of classification, it is necessary to select part of base classifiers. Classifiers are grouped into several clusters by k-means clustering using the proposed combination distance of Kappa-based diversity and accuracy. The base classifier with the best classification performance in each cluster will be selected to generate the final ensemble model. Experimental results on three Arabidopsis thaliana stress response datasets showed that the proposed method achieved better classification performance than existing ensemble models.

Basic study of epileptic seizure detection using a single-channel frontal EEG and a pre-trained ResNet.

Epilepsy is a neurological disorder that causes sudden seizures due to abnormal excitation of neurons in the brain. Approximately 30 % of patients cannot control their seizures using medication. In addition, since seizures can occur anywhere and at any time, caregivers must always be with the patient. Various researchers have developed seizure detection methods using multichannel EEG to improve the quality of life of patients and caregivers. However, the large size of the measurement device impedes transportation. We believe that a portable measurement device with a small number of channels is suitable for detecting seizures in daily life. Therefore, we need a system that can detect seizures using a small number of channels. The purpose of this research is to develop a seizure detection algorithm using a single-channel frontal EEG and to confirm its basic performance. We used EEG signals from a single electrode position (Fp1-F7, Fp2-F8), which is a bipolar derivation of the frontal region. We segmented the EEG using a 2 s sliding window with 50 % overlap and converted the segments into images. After preprocessing, we fine-tuned ResNet18, pre-trained on ImageNet, and developed an ensemble classification method. In the experiments with 10 epileptic patients (3 - 19 years old) registered in the CHB-MIT scalp EEG database, the results showed that the average sensitivity was 88.73 %, the average specificity was 98.98 %, and the average detection latency time was 7.39 s. In conclusion, the developed algorithm was validated as sufficiently accurate to detect epileptic seizures.Clinical Relevance- This establishes an image recognition algorithm that can detect epileptic seizures using a single- channel frontal EEG.

Fusing CNNs and statistical indicators to improve image classification

Convolutional Neural Networks have dominated the field of computer vision for the last ten years, exhibiting extremely powerful feature extraction capabilities and outstanding classification performance. The main strategy to prolong this trend in the state-of-the-art literature relies on further upscaling networks in size. However, costs increase rapidly while performance improvements may be marginal. Our main hypothesis is that adding additional sources of information can help to increase performance and that this approach is more cost-effective than building bigger networks, which involve higher training time, larger parametrisation space and higher computational resources requirements. In this paper, an ensemble method for accurate image classification is proposed, fusing automatically detected features through a Convolutional Neural Network and a set of manually defined statistical indicators. Through a combination of the predictions of a CNN and a secondary classifier trained on statistical features, a better classification performance can be achieved cheaply. We test five different CNN architectures and multiple learning algorithms in a diverse number of datasets to validate our proposal. According to the results, the inclusion of additional indicators and an ensemble classification approach help to increase the performance in all datasets. Both code and datasets are publicly available via GitHub at: https://github.com/jahuerta92/cnn-prob-ensemble.

A Two-Round Weight Voting Strategy-Based Ensemble Learning Method for Sea Ice Classification of Sentinel-1 Imagery

Sea ice information in the Arctic region is essential for climatic change monitoring and ship navigation. Although many sea ice classification methods have been put forward, the accuracy and usability of classification systems can still be improved. In this paper, a two-round weight voting strategy-based ensemble learning method is proposed for refining sea ice classification. The proposed method includes three main steps. (1) The preferable features of sea ice are constituted by polarization features (HH, HV, HH/HV) and the top six GLCM-derived texture features via a random forest. (2) The initial classification maps can then be generated by an ensemble learning method, which includes six base classifiers (NB, DT, KNN, LR, ANN, and SVM). The tuned voting weights by a genetic algorithm are employed to obtain the category score matrix and, further, the first coarse classification result. (3) Some pixels may be misclassified due to their corresponding numerically close score value. By introducing an experiential score threshold, each pixel is identified as a fuzzy or an explicit pixel. The fuzzy pixels can then be further rectified based on the local similarity of the neighboring explicit pixels, thereby yielding the final precise classification result. The proposed method was examined on 18 Sentinel-1 EW images, which were captured in the Northeast Passage from November 2019 to April 2020. The experiments show that the proposed method can effectively maintain the edge profile of sea ice and restrain noise from SAR. It is superior to the current mainstream ensemble learning algorithms with the overall accuracy reaching 97%. The main contribution of this study is proposing a superior weight voting strategy in the ensemble learning method for sea ice classification of Sentinel-1 imagery, which is of great significance for guiding secure ship navigation and ice hazard forecasting in winter.

ScHiCStackL: a stacking ensemble learning-based method for single-cell Hi-C classification using cell embedding.

Single-cell Hi-C data are a common data source for studying the differences in the three-dimensional structure of cell chromosomes. The development of single-cell Hi-C technology makes it possible to obtain batches of single-cell Hi-C data. How to quickly and effectively discriminate cell types has become one hot research field. However, the existing computational methods to predict cell types based on Hi-C data are found to be low in accuracy. Therefore, we propose a high accuracy cell classification algorithm, called scHiCStackL, based on single-cell Hi-C data. In our work, we first improve the existing data preprocessing method for single-cell Hi-C data, which allows the generated cell embedding better to represent cells. Then, we construct a two-layer stacking ensemble model for classifying cells. Experimental results show that the cell embedding generated by our data preprocessing method increases by 0.23, 1.22, 1.46 and 1.61$\%$ comparing with the cell embedding generated by the previously published method scHiCluster, in terms of the Acc, MCC, F1 and Precision confidence intervals, respectively, on the task of classifying human cells in the ML1 and ML3 datasets. When using the two-layer stacking ensemble framework with the cell embedding, scHiCStackL improves by 13.33, 19, 19.27 and 14.5 over the scHiCluster, in terms of the Acc, ARI, NMI and F1 confidence intervals, respectively. In summary, scHiCStackL achieves superior performance in predicting cell types using the single-cell Hi-C data. The webserver and source code of scHiCStackL are freely available at http://hww.sdu.edu.cn:8002/scHiCStackL/ and https://github.com/HaoWuLab-Bioinformatics/scHiCStackL, respectively.

Ensemble Method for Mobile Malware Detection using N-Gram Sequences of System Calls

Mobile device has become an essential tool among the community across the globe and has turned into a necessity in daily life. An extensive usage of mobile devices for everyday life tasks such as online banking, online shopping and exchanging e-mails has enable mobile devices to become data storage for users. The data stored in these mobile devices can contain sensitive and critical information to the users. Hence, making mobile devices as the prime target for cybercriminal. To date, Android based mobile devices is one of the mobile devices that are dominating the phone market. Moreover, the ease of use and open-source feature has made Android based mobile devices popular. However, the widely used Android mobile devices has encourage malware author to write malicious application. In a short duration of time mobile malware has rapidly evolve and have the capability to bypass signature detection approach which requires a constant signature update to detect mobile malware. To overcome this drawback an anomaly detection approach can be used to mitigate this issue. Yet, using a single classifier in an anomaly detection approach will not improve the classification detection performance. Based on this reason, this research formulates an ensemble classification method of different n-gram system call sequence features to improve the accuracy of mobile malware detection. This research proposes n-number of classifier models for each different n-gram sequence call feature. The probability output of each classifier is then combined to produce a better classification performance which is better compared to a single classifier.