High Number Of Features Research Articles

Characterizing the Contribution of Dependent Features in XAI Methods.

Explainable Artificial Intelligence (XAI) provides tools to help understanding how AI models work and reach a particular decision or outcome. It helps to increase the interpretability of models and makes them more trustworthy and transparent. In this context, many XAI methods have been proposed to make black-box and complex models more digestible from a human perspective. However, one of the main issues that XAI methods have to face especially when dealing with a high number of features is the presence of multicollinearity, which casts shadows on the robustness of the XAI outcomes, such as the ranking of informative features. Most of the current XAI methods either do not consider the collinearity or assume the features are independent which, in general, is not necessarily true. Here, we propose a simple, yet useful, proxy that modifies the outcome of any XAI feature ranking method allowing to account for the dependency among the features, and to reveal their impact on the outcome. The proposed method was applied to SHAP, as an example of XAI method which assume that the features are independent. For this purpose, several models were exploited for a well-known classification task (males versus females) using nine cardiac phenotypes extracted from cardiac magnetic resonance imaging as features. Principal component analysis and biological plausibility were employed to validate the proposed method. Our results showed that the proposed proxy could lead to a more robust list of informative features compared to the original SHAP in presence of collinearity.

Artificial Intelligence Workload Allocation Method for Vehicular Edge Computing

Real-time applications such as smart transportation systems require minimum response time to increase performance. Incorporating edge computing, processing units near end devices, achieving fast response time. The collaboration between edge servers and cloud servers is beneficial in achieving the lowest response time by using edge servers and high computational resources by using cloud servers. The workload allocation between edge–cloud servers is challenging, especially in a highly dynamic system with multiple factors varying over time. In this paper, the workload allocation decisions among the edge servers and cloud are considered for autonomous vehicle systems. The autonomous vehicle system generates multiple tasks belonging to different AI applications running on the vehicles. The proposed method considers allocating the tasks to edge or cloud servers. The cloud servers can be reached through a cellular network or a wireless network. The proposed method is based on designing a neural network model and using a high number of features that contribute to the decision-making process. A huge dataset has also been generated for the implementation. The EdgeCloudSim is used as a simulator for implementation. The competitor's methods considered for the comparison are random, simple moving average (SMA) based, multi-armed bandit (MAB) theory-based, game theory-based, and machine learning-based workload allocation methods. The result shows an improvement in the average Quality of Experience (QoE), ranging from 8.33% to 28.57%, while the average failure rate achieved enhancement up to 50%.

Analysis of Variance Combined with Optimized Gradient Boosting Machines for Enhanced Load Recognition in Home Energy Management Systems.

Load recognition remains not comprehensively explored in Home Energy Management Systems (HEMSs). There are gaps in current approaches to load recognition, such as enhancing appliance identification and increasing the overall performance of the load-recognition system through more robust models. To address this issue, we propose a novel approach based on the Analysis of Variance (ANOVA) F-test combined with SelectKBest and gradient-boosting machines (GBMs) for load recognition. The proposed approach improves the feature selection and consequently aids inter-class separability. Further, we optimized GBM models, such as the histogram-based gradient-boosting machine (HistGBM), light gradient-boosting machine (LightGBM), and XGBoost (extreme gradient boosting), to create a more reliable load-recognition system. Our findings reveal that the ANOVA-GBM approach achieves greater efficiency in training time, even when compared to Principal Component Analysis (PCA) and a higher number of features. ANOVA-XGBoost is approximately 4.31 times faster than PCA-XGBoost, ANOVA-LightGBM is about 5.15 times faster than PCA-LightGBM, and ANOVA-HistGBM is 2.27 times faster than PCA-HistGBM. The general performance results expose the impact on the overall performance of the load-recognition system. Some of the key results show that the ANOVA-LightGBM pair reached 96.42% accuracy, 96.27% F1, and a Kappa index of 0.9404; the ANOVA-HistGBM combination achieved 96.64% accuracy, 96.48% F1, and a Kappa index of 0.9434; and the ANOVA-XGBoost pair attained 96.75% accuracy, 96.64% F1, and a Kappa index of 0.9452; such findings overcome rival methods from the literature. In addition, the accuracy gain of the proposed approach is prominent when compared straight to its competitors. The higher accuracy gains were 13.09, 13.31, and 13.42 percentage points (pp) for the pairs ANOVA-LightGBM, ANOVA-HistGBM, and ANOVA-XGBoost, respectively. These significant improvements highlight the effectiveness and refinement of the proposed approach.

NEW STRATEGIES FOR IMPROVING NETWORK SECURITY AGAINST CYBER ATTACK BASED ON INTELLIGENT ALGORITHMS

Gradually, since the number of linked computer systems that use networks linked to the Internet is raised the information that is delivered through those systems becomes more vulnerable to cyber threats. This article presents proposed algorithms based on Machine Learning (ML) that ensure early detection of cyber threats that cause network breaking through the use of the Correlation Ranking Filter feature selection method. These proposed algorithms were applied to the Multi-Step Cyber-Attack Dataset (MSCAD) which consists of 66 features. The proposed strategy will apply machine learning algorithms like Adaptive Boosting-Deep Learning (AdaBoost-Deep Learning) or (ABDL), Multi-Layer Perceptron (MLP), Bayesian Networks Model (BNM), and Random Forest (RF), the feature would be decreased to high valuable of 46 features were included with a threshold of 0.1 or higher. The accuracy would be increased when the no. of features decreased to 46 with a threshold of ≥ 0.1 with the ABDL algorithm producing an accuracy of 99.7076%. The obtained results showed that the proposed algorithm delivered a suitable accuracy of 99.6791% with the ABDL algorithm even with a higher number of features.

A Multi-Model Framework to Explore ADHD Diagnosis from Neuroimaging Data

Attention Deficit Hyperactivity Disorder (ADHD) is a frequent neurodevelopmental disorder in children that is commonly diagnosed subjectively. The objective detection of ADHD based on neuroimaging data has been a complex problem with low ranges of accuracy, possibly due to (among others) complex diagnostic processes, the high number of features considered and imperfect measurements in data collection. Hence, reliable neuroimaging biomarkers for detecting ADHD have been elusive. To address this problem we consider a recently proposed multi-model selection method called Sparse Wrapper AlGorithm (SWAG), which is a greedy algorithm that combines screening and wrapper approaches to create a set of low-dimensional models with good predictive power. While preserving the previous levels of accuracy, SWAG provides a measure of importance of brain regions for identifying ADHD. Our approach also provides a set of equally-performing and simple models which highlight the main feature combinations to be analyzed and the interactions between them. Taking advantage of the network of models resulting from this approach, we confirm the relevance of the frontal and temporal lobes as well as highlight how the different regions interact to detect the presence of ADHD. In particular, these results are fairly consistent across different learning mechanisms employed within the SWAG (i.e. logistic regression, linear and radial-kernel support vector machines) thereby providing population-level insights, as well as delivering feature combinations that are smaller and often perform better than those that would be used if employing their original versions directly.

Adaptive cooperative coevolutionary differential evolution for parallel feature selection in high-dimensional datasets

In many fields, it is a common practice to collect large amounts of data characterized by a high number of features. These datasets are at the core of modern applications of supervised machine learning, where the goal is to create an automatic classifier for newly presented data. However, it is well known that the presence of irrelevant features in a dataset can make the learning phase harder and, most importantly, can lead to suboptimal classifiers. Consequently, it is becoming increasingly important to be able to select the right subset of features. Traditionally, optimization metaheuristics have been used with success in the task of feature selection. However, many of the approaches presented in the literature are not applicable to datasets with thousands of features because of the poor scalability of optimization algorithms. In this article, we address the problem using a cooperative coevolutionary approach based on differential evolution. In the proposed algorithm, parallelized for execution on shared-memory architectures, a suitable strategy for reducing the dimensionality of the search space and adjusting the population size during the optimization results in significant performance improvements. A numerical investigation on some high-dimensional and medium-dimensional datasets shows that, in most cases, the proposed approach can achieve higher classification performance than other state-of-the-art methods.

An integrated clustering and BERT framework for improved topic modeling.

Topic modelling is a machine learning technique that is extensively used in Natural Language Processing (NLP) applications to infer topics within unstructured textual data. Latent Dirichlet Allocation (LDA) is one of the most used topic modeling techniques that can automatically detect topics from a huge collection of text documents. However, the LDA-based topic models alone do not always provide promising results. Clustering is one of the effective unsupervised machine learning algorithms that are extensively used in applications including extracting information from unstructured textual data and topic modeling. A hybrid model of Bidirectional Encoder Representations from Transformers (BERT) and Latent Dirichlet Allocation (LDA) in topic modeling with clustering based on dimensionality reduction have been studied in detail. As the clustering algorithms are computationally complex, the complexity increases with the higher number of features, the PCA, t-SNE and UMAP based dimensionality reduction methods are also performed. Finally, a unified clustering-based framework using BERT and LDA is proposed as part of this study for mining a set of meaningful topics from the massive text corpora. The experiments are conducted to demonstrate the effectiveness of the cluster-informed topic modeling framework using BERT and LDA by simulating user input on benchmark datasets. The experimental results show that clustering with dimensionality reduction would help infer more coherent topics and hence this unified clustering and BERT-LDA based approach can be effectively utilized for building topic modeling applications.

A gradient-based approach for adversarial attack on deep learning-based network intrusion detection systems

Intrusion detection systems are an essential part of any cybersecurity architecture. These systems are critical in defending networks against a variety of security threats. In recent years, deep neural networks have proved their performance and efficiency in various machine learning tasks, including intrusion detection. However, it is shown that deep learning models are highly vulnerable to adversarial attacks. This paper proposes a new approach for performing an adversarial attack against deep learning-based malicious network activity classification. We use the Jacobian Saliency Map to find the best group of features, with different features and perturbation magnitude, to generate adversarial examples. We evaluate our method on three CIC-IDS2017, CIC-IDS2018, and CIC-DDoS2019 datasets. Our experiments show that our proposed method can achieve better performance while using fewer features in adversarial sample generation than other attacks that depend on a higher number of features. Our technique can generate adversarial samples for more than 18% of samples in CIC-IDS2017, 15% of samples in CIC-IDS2018, and 14% of samples in CIC-DDoS2019, using only three features and 0.1 as the perturbation magnitude. We do a deeper analysis of the attack based on its parameters, distance metrics, and the target model performance. Also, an evaluation model with three criteria, including success rates of the best feature sets, average confidence of the adversarial class, and adversarial samples transferability, is used in our analysis.

Machine learning-based feature selection to search stable microbial biomarkers: application to inflammatory bowel disease.

Biomarker discovery exploiting feature importance of machine learning has risen recently in the microbiome landscape with its high predictive performance in several disease states. To have a concrete selection among a high number of features, recursive feature elimination (RFE) has been widely used in the bioinformatics field. However, machine learning-based RFE has factors that decrease the stability of feature selection. In this article, we suggested methods to improve stability while sustaining performance. We exploited the abundance matrices of the gut microbiome (283 taxa at species level and 220 at genus level) to classify between patients with inflammatory bowel disease (IBD) and healthy control (1,569 samples). We found that applying an already published data transformation before RFE improves feature stability significantly. Moreover, we performed an in-depth evaluation of different variants of the data transformation and identify those that demonstrate better improvement in stability while not sacrificing classification performance. To ensure a robust comparison, we evaluated stability using various similarity metrics, distances, the common number of features, and the ability to filter out noise features. We were able to confirm that the mapping by the Bray-Curtis similarity matrix before RFE consistently improves the stability while maintaining good performance. Multilayer perceptron algorithm exhibited the highest performance among 8 different machine learning algorithms when a large number of features (a few hundred) were considered based on the best performance across 100 bootstrapped internal test sets. Conversely, when utilizing only a limited number of biomarkers as a trade-off between optimal performance and method generalizability, the random forest algorithm demonstrated the best performance. Using the optimal pipeline we developed, we identified 14 biomarkers for IBD at the species level and analyzed their roles using Shapley additive explanations. Taken together, our work not only showed how to improve biomarker discovery in the metataxonomic field without sacrificing classification performance but also provided useful insights for future comparative studies.

Kurtosis-Based Feature Selection Method using Symmetric Uncertainty to Predict the Air Quality Index

Feature selection is vital in data pre-processing in machine learning, and it is prominent in datasets with many features. Feature selection analyses the relevant, irrelevant, and redundant features in the dataset. Feature selection removes the irrelevant features, which improves both the accuracy and prediction performance. The significant advantages of reducing the number of features from the dataset are reducing the training time, reducing overfitting, decreasing the curse of dimensionality, and simplifying the prediction model. The filter feature selection techniques can handle the issues with the high number of features, and this paper uses the symmetric uncertainty coefficient to verify the relevance of the independent features. In this paper, a new feature selection method named as kurtosis-based feature selection has been proposed to select the relevant features which affect the air pollution. Kurtosis-based feature selection is compared with seven filter feature selection techniques on air pollution dataset and validated the performance of the proposed algorithm. It has been observed that the kurtosis-based feature selection extracts only PM2.5 as the key feature and has been compared to the accuracy of the five existing methods. The experimental results illustrate that the kurtosis-based feature selection algorithm reduces the original feature set up to 91.66\%, but the existing filter feature selection techniques reduce the feature set to only 50\%.

Early detection of Alzheimer's disease using single nucleotide polymorphisms analysis based on gradient boosting tree

Alzheimer's disease (AD) is a degenerative disorder that attacks nerve cells in the brain. AD leads to memory loss and cognitive & intellectual impairments that can influence social activities and decision-making. The most common type of human genetic variation is single nucleotide polymorphisms (SNPs). SNPs are beneficial markers of complex gene-disease. Many common and serious diseases, such as AD, have associated SNPs. Detection of SNP biomarkers linked with AD could help in the early prediction and diagnosis of this disease. The main objective of this paper is to predict and diagnose AD based on SNPs biomarkers with high classification accuracy in the early stages. One of the most concerning problems is the high number of features. Thus, the paper proposes a comprehensive framework for early AD detection and detecting the most significant genes based on SNPs analysis. Usage of machine learning (ML) techniques to identify new biomarkers of AD is also suggested. In the proposed system, two feature selection techniques are separately checked: the information gain filter and Boruta wrapper. The two feature selection techniques were used to select the most significant genes related to AD in this system. Filter methods measure the relevance of features by their correlation with dependent variables, while wrapper methods measure the usefulness of a subset of features by training a model on it. Gradient boosting tree (GBT) has been applied on all AD genetic data of neuroimaging initiative phase 1 (ADNI-1) and Whole-Genome Sequencing (WGS) datasets by using two feature selection techniques. In the whole-genome approach ADNI-1, results revealed that the GBT learning algorithm scored an overall accuracy of 99.06% in the case of using Boruta feature selection. Using information gain feature selection, the proposed system achieved an average accuracy of 94.87%. The results show that the proposed system is preferable for the early detection of AD. Also, the results revealed that the Boruta wrapper feature selection is superior to the information gain filter technique.

Various Soft Computing Based Techniques for Developing Intrusion Detection Management System

Protecting networks and data requires an effective Intrusion Detection System (IDS). Contextual knowledge processing may be used to identify attacks that are specific to certain applications and networks, which is becoming more common due to the fast advancement of network technology. A hybrid intrusion detection system may help overcome this problem (IDS). Packet flooding is a common tactic employed in DoS attacks, which aims to overload the victim's infrastructure. It is now possible to interrupt networks of virtually any size with these kinds of assaults. Testing a high-performance hybrid IDS is hampered by having to deal with massive amounts of data with many characteristics. In order to identify dangerous patterns, a high number of features might make it harder, leading to a protracted training and testing procedure, an increased resource demand, and a lower detection rate. As a result, minimizing and deleting irrelevant features from the benchmark dataset is needed. Preprocessing removes out-of-range values, unlikely data combinations, and missing values from the dataset. Feature Selection (FS) methods are used to reduce the dimensionality of a dataset by deleting irrelevant and redundant attributes, thus improving classifier accuracy. To solve this problem, the AODV protocol, in conjunction with soft computing techniques, is very useful.

Development and Application of an LC-MS/MS Untargeted Exposomics Method with a Separated Pooled Quality Control Strategy.

Pooled quality controls (QCs) are usually implemented within untargeted methods to improve the quality of datasets by removing features either not detected or not reproducible. However, this approach can be limiting in exposomics studies conducted on groups of exposed and nonexposed subjects, as compounds present at low levels only in exposed subjects can be diluted and thus not detected in the pooled QC. The aim of this work is to develop and apply an untargeted workflow for human biomonitoring in urine samples, implementing a novel separated approach for preparing pooled quality controls. An LC-MS/MS workflow was developed and applied to a case study of smoking and non-smoking subjects. Three different pooled quality controls were prepared: mixing an aliquot from every sample (QC-T), only from non-smokers (QC-NS), and only from smokers (QC-S). The feature tables were filtered using QC-T (T-feature list), QC-S, and QC-NS, separately. The last two feature lists were merged (SNS-feature list). A higher number of features was obtained with the SNS-feature list than the T-feature list, resulting in identification of a higher number of biologically significant compounds. The separated pooled QC strategy implemented can improve the nontargeted human biomonitoring for groups of exposed and nonexposed subjects.

Assessing Feature Importance for Short-Term Prediction of Electricity Demand in Medium-Voltage Loads

The design of new monitoring systems for intelligent distribution networks often requires both real-time measurements and pseudomeasurements to be processed. The former are obtained from smart meters, phasor measurement units and smart electronic devices, whereas the latter are predicted using appropriate algorithms—with the typical objective of forecasting the behaviour of power loads and generators. However, depending on the technique used for data encoding, the attempt at making predictions over a period of several days may trigger problems related to the high number of features. To contrast this issue, feature importance analysis becomes a tool of primary importance. This article is aimed at illustrating a technique devised to investigate the importance of features on data deemed relevant for predicting the next hour demand of aggregated, medium-voltage electrical loads. The same technique allows us to inspect the hidden layers of multilayer perceptrons entrusted with making the predictions, since, ultimately, the content of any hidden layer can be seen as an alternative encoding of the input data. The possibility of inspecting hidden layers can give wide support to researchers in a number of relevant tasks, including the appraisal of the generalisation capability reached by a multilayer perceptron and the identification of neurons not relevant for the prediction task.

Incremental Learning Framework for Mining Big Data Stream

At this current time, data stream classification plays a key role in big data analytics due to its enormous growth. Most of the existing classification methods used ensemble learning, which is trustworthy but these methods are not effective to face the issues of learning from imbalanced big data, it also supposes that all data are pre-classified. Another weakness of current methods is that it takes a long evaluation time when the target data stream contains a high number of features. The main objective of this research is to develop a new method for incremental learning based on the proposed ant lion fuzzy-generative adversarial network model. The proposed model is implemented in spark architecture. For each data stream, the class output is computed at slave nodes by training a generative adversarial network with the back propagation error based on fuzzy bound computation. This method overcomes the limitations of existing methods as it can classify data streams that are slightly or completely unlabeled data and providing high scalability and efficiency. The results show that the proposed model outperforms state-of-the-art performance in terms of accuracy (0.861) precision (0.9328) and minimal MSE (0.0416).

High Performance Classification of Cancer Types with Gene Microarray Datasets: Hybrid Approach

Currently the approach of biological meaningfulness detection from gene microarray datasets obtained with microarray technology is used effectively in many areas such as disease diagnosis and differentiation of cancer types. However, since datasets obtained with this technology measure gene expression profiles collectively, the number of features in the dataset can be quite high. The small number of samples in gene microarray datasets, the high number of features and where the data is noisy significantly complicates the preparation process of these datasets. In order for machine learning models to successfully classify, the number of features that represent the size of the dataset should be reduced. In the proposed method, gene microarray data is taken as input and Information Gain, Fisher Correlation Scoring, ReliefF and, Chi-Square methods are applied separately for feature selection. After this stage, a sub-dataset containing the new genes is obtained and a pool of genes for Genetic Algorithm is created according to this dataset. Bayes classifier is trained using the sub-dataset created with the genes of the most successful chromosome. Thus, the classification process of cancer data is successfully completed. The model proposed in this study was applied to datasets that are frequently used in the literature and high success rates were obtained in classification. As a result; acceptable feature selection methods and the hybrid method based on Genetic Algorithm generally provided the most appropriate results on the all test data.

Audio based depression detection using Convolutional Autoencoder

Depression is a serious and common psychological disorder that requires early diagnosis and treatment. In severe episodes the condition may result in suicidal thoughts. Recently, the need for building an effective audio-based Automatic Depression Detection (ADD) system has sparked the interest of the research community. To date, most of the reported approaches to recognize depression rely on hand-crafted feature extraction for audio data representation. They combine wide variety of audio-related features to improve the classification performance. However, combining many hand-crafted features including relevant and less-relevant can enlarge the feature space which can lead to high-dimensionality issues as not all the features would carry significant information regarding depression. Having high number of features can make the pattern recognition more difficult and increase the risk of overfitting. To overcome these limitations, an audio-based framework of depression detection which includes an adaptation of a deep learning (DL) technique is proposed to automatically extract the highly relevant and compact feature set. This proposed framework uses an end-to-end Convolutional Neural Network-based Autoencoder (CNN AE) technique to learn the highly relevant and discriminative features from raw sequential audio data, and hence to detect depressed people more accurately. In addition, to address the sample imbalance problem we use a cluster-based sampling technique which highly reduces the risk of bias towards the major class (non-depressed). To evaluate the performance and effectiveness of the proposed pipeline, we perform the experiments on Distress Analysis Interview Corpus-Wizard of Oz (DAIC-WOZ) dataset and compare them with the hand-crafted feature extraction methods and other outstanding studies in this domain. The results show that proposed method outperforms other well-known audio-based ADD models with at least 7% improvement in F-measure for classifying depression.

Disease Single Nucleotide Polymorphism Selection using Hybrid Feature Selection Technique

According to recent studies the Single Nucleotide Polymorphism (SNPs) plays very important role as genetic marker in various complex diseases. Lots of machine learning techniques are already applied on SNPs data to distinguish between affected and healthy individuals. The major problem with the SNPs dataset is high number of features and small number of samples which are referred as ‘large p’ and ‘small s’ problem. In this paper we proposed a hybrid feature selection method for selecting an optimal subset of SNPs and from that we select the significant SNPs, which act as marker for disease. The method is a hybrid technique based on combination of filter and wrapper method, the (mRMR) Minimum Redundancy Maximum Relevancy and Particle Swarm Optimization for Gene Selection with Support Vector machine (PGOGS-SVM) respectively. The proposed mRMR+PSOGS-SVM approach has been applied to mental retardation SNP dataset taken from NCBI-GEO website. The method has achieved high classification accuracy up to 88% and outperformed all other compared feature selection techniques.

FGAAM: A fast and resizable genetic algorithm with aggressive mutation for feature selection

The paper introduces a modified version of a genetic algorithm with aggressive mutation (GAAM) called fGAAM (fast GAAM) that significantly decreases the time needed to find feature subsets of a satisfactory classification accuracy. To demonstrate the time gains provided by fGAAM both algorithms were tested on eight datasets containing different number of features, classes, and examples. The fGAAM was also compared with four reference methods: the Holland GA with and without penalty term, Culling GA, and NSGA II. Results: (i) The fGAAM processing time was about 35% shorter than that of the original GAAM. (ii) The fGAAM was also 20 times quicker than two Holland GAs and 50 times quicker than NSGA II. (iii) For datasets of different number of features, classes, and examples, another number of individuals, stored for further processing, provided the highest acceleration. On average, the best results were obtained when individuals from the last 10 populations were stored (time acceleration: 36.39%) or when the number of individuals to be stored was calculated by the algorithm itself (time acceleration: 35.74%). (iv) The fGAAM was able to process all datasets used in the study, even those that, because of their high number of features, could not be processed by the two Holland GAs and NSGA II.

Comparative Analysis of Intrusion Detection Attack Based on Machine Learning Classifiers

In current day information transmitted from one place to another by using network communication technology. Due to such transmission of information, networking system required a high security environment. The main strategy to secure this environment is to correctly identify the packet and detect if the packet contains a malicious and any illegal activity happened in network environments. To accomplish this, we use intrusion detection system (IDS). Intrusion detection is a security technology that design detects and automatically alert or notify to a responsible person. However, creating an efficient Intrusion Detection System face a number of challenges. These challenges are false detection and the data contain high number of features. Currently many researchers use machine learning techniques to overcome the limitation of intrusion detection and increase the efficiency of intrusion detection for correctly identify the packet either the packet is normal or malicious. Many machine-learning techniques use in intrusion detection. However, the question is which machine learning classifiers has been potentially to address intrusion detection issue in network security environment. Choosing the appropriate machine learning techniques required to improve the accuracy of intrusion detection system. In this work, three machine learning classifiers are analyzed. Support vector Machine, Naïve Bayes Classifier and K-Nearest Neighbor classifiers. These algorithms tested using NSL KDD dataset by using the combination of Chi square and Extra Tree feature selection method and Python used to implement, analyze and evaluate the classifiers. Experimental result show that K-Nearest Neighbor classifiers outperform the method in categorizing the packet either is normal or malicious.