Entropy-based Feature Selection Research Articles

Attack’s Feature Selection-Based Network Intrusion Detection System Using Fuzzy Control Language

Network intrusion detection system has wide range of disputes due to lack of security over the networks. The network intrusion detection system must be reliable to detect the emerging threats over the networks and perform effectively, efficiently to manage large amount of traffic. This paper proposes entropy-based feature selection to select the important features, layered fuzzy control language to generate fuzzy rules, and layered classifier to detect various network attacks namely neptune, smurf, back, and mailbomb. Layered classifier has improved the performances and reduces the computational time. KDD Dataset which consists of three components, namely “Corrected Dataset,” “10 % Dataset,” and “Full Dataset,” are employed to evaluate the performances of the proposed system. The experiments are carried out using an open source java library called jFuzzyLogic and the results show considerable improvement in the detection rate, reduce the false positive rate, significant improvement in recall value and reduces the computational time.

A multi-step outlier-based anomaly detection approach to network-wide traffic

Outlier detection is of considerable interest in fields such as physical sciences, medical diagnosis, surveillance detection, fraud detection and network anomaly detection. The data mining and network management research communities are interested in improving existing score-based network traffic anomaly detection techniques because of ample scopes to increase performance. In this paper, we present a multi-step outlier-based approach for detection of anomalies in network-wide traffic. We identify a subset of relevant traffic features and use it during clustering and anomaly detection. To support outlier-based network anomaly identification, we use the following modules: a mutual information and generalized entropy based feature selection technique to select a relevant non-redundant subset of features, a tree-based clustering technique to generate a set of reference points and an outlier score function to rank incoming network traffic to identify anomalies. We also design a fast distributed feature extraction and data preparation framework to extract features from raw network-wide traffic. We evaluate our approach in terms of detection rate, false positive rate, precision, recall and F-measure using several high dimensional synthetic and real-world datasets and find the performance superior in comparison to competing algorithms.

An experimental comparison of feature selection methods on two-class biomedical datasets

Feature selection is a significant part of many machine learning applications dealing with small-sample and high-dimensional data. Choosing the most important features is an essential step for knowledge discovery in many areas of biomedical informatics. The increased popularity of feature selection methods and their frequent utilisation raise challenging new questions about the interpretability and stability of feature selection techniques. In this study, we compared the behaviour of ten state-of-the-art filter methods for feature selection in terms of their stability, similarity, and influence on prediction performance. All of the experiments were conducted on eight two-class datasets from biomedical areas. While entropy-based feature selection appears to be the most stable, the feature selection techniques yielding the highest prediction performance are minimum redundance maximum relevance method and feature selection based on Bhattacharyya distance. In general, univariate feature selection techniques perform similarly to or even better than more complex multivariate feature selection techniques with high-dimensional datasets. However, with more complex and smaller datasets multivariate methods slightly outperform univariate techniques.

An ANN Approach for Network Intrusion Detection using Entropy based Feature Selection

With the increase in Internet users the number of malicious users are also growing day-by-day posing a serious problem in distinguishing between normal and abnormal behavior of users in the network. This has led to the research area of intrusion detection which essentially analyzes the network traffic and tries to determine normal and abnormal patterns of behavior.In this paper, we have analyzed the standard NSL-KDD intrusion dataset using some neural network based techniques for predicting possible intrusions. Four most effective classification methods, namely, Radial Basis Function Network, SelfOrganizing Map, Sequential Minimal Optimization, and Projective Adaptive Resonance Theory have been applied. In order to enhance the performance of the classifiers, three entropy based feature selection methods have been applied as preprocessing of data. Performances of different combinations of classifiers and attribute reduction methods have also been compared.

Hybrid methodologies for electricity load forecasting: Entropy-based feature selection with machine learning and soft computing techniques

Scientific community is currently doing a great effort of research in the area of Smart Grids because energy production, distribution, and consumption play a critical role in the sustainability of the planet. The main challenge lies in intelligently integrating the actions of all users connected to the grid. In this context, electricity load forecasting methodologies is a key component for demand-side management. This research compares the accuracy of different Machine Learning methodologies for the hourly energy forecasting in buildings. The main goal of this work is to demonstrate the performance of these models and their scalability for different consumption profiles. We propose a hybrid methodology that combines feature selection based on entropies with soft computing and machine learning approaches, i.e. Fuzzy Inductive Reasoning, Random Forest and Neural Networks. They are also compared with a traditional statistical technique ARIMA (AutoRegressive Integrated Moving Average). In addition, in contrast to the general approaches where offline modelling takes considerable time, the approaches discussed in this work generate fast and reliable models, with low computational costs. These approaches could be embedded, for instance, in a second generation of smart meters, where they could generate on-site electricity forecasting of the next hours, or even trade the excess of energy.

Rough Set and Entropy based Feature Selection for Online Forums Hotspot Detection

exponential growth of web arouses much attention on public opinion. The rapid progress of online forums, micro blogs and new reports are having large volume of public opinion information. These are proving to be extremely valuable resources in helping to anticipate, detect and forecast societal events. But most of the online data is unstructured or semi structured and that is difficult to decipher automatically. Therefore, it is very much essential to analyze in time and understands the trends of their opinion correctly. The hotspot detection is one of the promising research areas in web mining and it helps to make appropriate decision in timely manner. Feature selection is an essential component in text categorization to identify the relevant features and reduces the dimensionality of data to gain the improved higher accuracy. The proposed system integrates rough set approach with entropy for detecting the online forums hotspot. The experimental results demonstrate that the proposed hybrid feature selection method outperforms with Naive Bayes and Support Vector Machine based hotspot detection models.

A relative decision entropy-based feature selection approach

Rough set theory has been proven to be an effective tool for feature selection. To avoid the exponential computation in exhaustive methods, many heuristic feature selection algorithms have been proposed in rough sets. However, these algorithms still suffer from high computational cost. In this paper, we propose a novel heuristic feature selection algorithm (called FSMRDE) in rough sets. To measure the significance of features in FSMRDE, we propose a new model of relative decision entropy, which is an extension of Shannon׳s information entropy in rough sets. Moreover, to test the effectiveness of FSMRDE, we apply it to intrusion detection and other application domains. Experimental results show that by using the relative decision entropy-based feature significance as heuristic information, FSMRDE is efficient for feature selection. In particular, FSMRDE is able to achieve good scalability for large data sets.

Information Theory Based Feature Selection for Multi-Relational Naïve Bayesian Classifier

Today data’s are stored in relation structures. In usual approach to mine these data, we often use to join several relations to form a single relation using foreign key links, which is known as flatten. Flatten may cause troubles such as time consuming, data redundancy and statistical skew on data. Hence, the critical issues arise that how to mine data directly on numerous relations. The solution of the given issue is the approach called multi-relational data mining (MRDM). Other issues are irrelevant or redundant attributes in a relation may not make contribution to classification accuracy. Thus, feature selection is an essential data pre-processing step in multi-relational data mining. By filtering out irrelevant or redundant features from relations for data mining, we improve classification accuracy, achieve good time performance, and improve comprehensibility of the models. We had proposed the entropy based feature selection method for Multi-relational Naive Bayesian Classifier. We have use method InfoDist and Pearson’s Correlation parameters, which will be used to filter out irrelevant and redundant features from the multi-relational database and will enhance classification accuracy. We analyzed our algorithm over PKDD financial dataset and achieved the better accuracy compare to the existing features selection methods.

A threshold fuzzy entropy based feature selection for medical database classification

Feature selection is one of the most common and critical tasks in database classification. It reduces the computational cost by removing insignificant features. Consequently, this makes the diagnosis process accurate and comprehensible. This paper presents the measurement of feature relevance based on fuzzy entropy, tested with a Radial Basis Function Network classifier for a medical database classification. Three feature selection strategies are devised to obtain the valuable subset of relevant features. Five benchmarked datasets, which are available in the UCI Machine Learning Repository, have been used in this work. The classification accuracy shows that the proposed method is capable of producing good results with fewer features than the original datasets.

Entropy-based feature selection for improved 3D facial expression recognition

Facial expressions contain most of the information on human face which is essential for human–computer interaction. Development of robust algorithms for automatic recognition of facial expressions with high recognition rates has been a challenge for the last 10 years. In this paper, we propose a novel feature selection procedure which recognizes basic facial expressions with high recognition rates by utilizing three-Dimensional (3D) geometrical facial feature positions. The paper presents a system of classifying expressions in one of the six basic emotional categories which are anger, disgust, fear, happiness, sadness, and surprise. The paper contributes on feature selections for each expression independently and achieves high recognition rates with the proposed geometric facial features selected for each expression. The novel feature selection procedure is entropy based, and it is employed independently for each of the six basic expressions. The system’s performance is evaluated using the 3D facial expression database, BU-3DFE. Experimental results show that the proposed method outperforms the latest methods reported in the literature.

Spoken Language Recognition With Prosodic Features

Speech prosody is believed to carry much language-specific information that can be used for spoken language recognition (SLR). In the past, the use of prosodic features for SLR has been studied sporadically and the reported performances were considered unsatisfactory. In this paper, we exploit a wide range of prosodic attributes for large-scale SLR tasks. These attributes describe the multifaceted variations of F0, intensity and duration in different spoken languages. Prosodic attributes are modeled by the bag of n-grams approach with support vector machine (SVM) as in the conventional phonotactic SLR systems. Experimental results on OGI and NIST-LRE tasks showed that the use of proposed attributes gives significantly better SLR performance than those previously reported. The full feature set includes 87 prosodic attributes and redundancy among attributes may exist. Attributes are broken down into particular bigrams called bins. Four entropy-based feature selection metrics with different selection criteria are derived. Attributes can be selected by individual bins, or by attributes as batches of bins. It can also be done in a language-dependent or language-independent manner. By comparing different selection sizes and criteria, an optimal attribute subset comprising 5,000 bins is found by using a bin-level language-independent criterion. Feature selection reduces model size by 2.5 times and shortens the runtime by 6 times. The optimal subset of bins gives the lowest EER of 20.18% on NIST-LRE 2007 SLR task in a prosodic attribute model (PAM) system which exclusively modeled prosodic attributes. In a phonotactic-prosodic fusion SLR system, the detection cost, C <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">avg</sub> is 2.09%. The relative detection cost reduction is 23%.

Pathways to decoding the clinical potential of stress response FOXO-interaction networks for Huntington's disease: of gene prioritization and context dependence

The FOXO family of transcription factors is central to the regulation of organismal longevity and cellular survival. Several studies have indicated that FOXO factors lie at the center of a complex network of upstream pathways, cofactors and downstream targets (FOXO-interaction networks), which may have developmental and post-developmental roles in the regulation of chronic-stress response in normal and diseased cells. Noticeably, FOXO factors are important for the regulation of proteotoxicity and neuron survival in several models of neurodegenerative disease, suggesting that FOXO-interaction networks may have therapeutic potential. However, the status of FOXO-interaction networks in neurodegenerative disease remains largely unknown. Systems modeling is anticipated to provide a comprehensive assessment of this question. In particular, interrogating the context-dependent variability of FOXO-interaction networks could predict the clinical potential of cellular-stress response genes and aging regulators for tackling brain and peripheral pathology in neurodegenerative disease. Using published transcriptomic data obtained from murine models of Huntington's disease (HD) and post-mortem brains, blood samples and induced-pluripotent-stem cells from HD carriers as a case example, this review briefly highlights how the biological status and clinical potential of FOXO-interaction networks for HD may be decoded by developing network and entropy based feature selection across heterogeneous datasets.

Heath monitoring of a glass transfer robot in the mass production line of liquid crystal display using abnormal operating sounds based on wavelet packet transform and artificial neural network

This paper presents the fault detect method of a moving transfer robot in the mass production line of liquid crystal display (LCD) manufacturers based on the wavelet packet transform (WPT) for feature extraction and the artificial neural network (ANN) for fault classification. Most of fault detection methods in a mechanical system have been researched based on the vibration signal. Unlike the existing methodologies, this study aims to minimize the uncertainty of a field engineer's decision making process for determining whether a fault is present or not based on the human auditory perception by developing a fault diagnosis system that uses the abnormal operating sound radiated from a moving transfer robot as a source signal. Abnormal operating sound radiated from a moving transfer robot has been used for this work instead of other source signals such as vibration, acoustic emission, electrical signal, etc. Its advantage as a source signal makes it possible to monitor the status of multiple faults by using only a microphone despite a relatively low sensitivity. In the application of ANN, since it is important to minimize the error of trained ANN in terms of the accuracy of fault diagnosis logic, in the paper, the number of input and target data samples was increased through a regeneration process based on statistical properties, and then the uncorrelated nodes in the input vector were also removed to improve the orthogonality of the input vector based on the entropy based feature selection method. Consequently, it can be concluded that the abnormal operating sound is sufficiently useful as a source signal for the fault diagnosis of mechanical components as well as other source signals.

Neural Network Classifier with Entropy Based Feature Selection on Breast Cancer Diagnosis

The aim of this research is to combine the feature selection (FS) and optimization algorithms as the optimal tool to improve the learning performance like predictive accuracy of the Wisconsin Breast Cancer Dataset classification. An ensemble of the reduced data patterns based on FS was used to train a neural network (NN) using the Levenberg-Marquardt (LM) and the Particle Swarm Optimization (PSO) algorithms to devise the appropriate NN training weighting parameters, and then construct an effective Neural Network classifier to improve the Wisconsin Breast Cancers' classification accuracy and efficiency. Experimental results show that the accuracy and AROC improved emphatically, and the best performance in accuracy and AROC are 98.83% and 0.9971, respectively.

Adaptive mixture observation models for multiple object tracking

Multiple object tracking (MOT) poses many difficulties to conventional well-studied single object tracking (SOT) algorithms, such as severe expansion of configuration space, high complexity of motion conditions, and visual ambiguities among nearby targets, among which the visual ambiguity problem is the central challenge. In this paper, we address this problem by embedding adaptive mixture observation models (AMOM) into a mixture tracker which is implemented in Particle Filter framework. In AMOM, the extracted multiple features for appearance description are combined according to their discriminative power between ambiguity prone objects, where the discriminability of features are evaluated by online entropy-based feature selection techniques. The induction of AMOM can help to surmount the incapability of conventional mixture tracker in handling object occlusions, and meanwhile retain its merits of flexibility and high efficiency. The final experiments show significant improvement in MOT scenarios compared with other methods.

Contingency ranking for voltage collapse using parallel self-organizing hierarchical neural network

On-line monitoring of the power system voltage security has become a vital factor for electric utilities. This paper proposes a voltage contingency ranking approach based on parallel self-organizing hierarchical neural network (PSHNN). Loadability margin to voltage collapse following a contingency has been used to rank the contingencies. PSHNN is a multi-stage neural network where the stages operate in parallel rather than in series during testing. The number of ANNs required is drastically reduced by adopting a clustering technique to group contingencies of similar severity into one cluster. Entropy based feature selection has been employed to reduce the dimensionality of the ANN. Once trained, the proposed ANN model is capable of ranking the voltage contingencies under varying load conditions, on line. The effectiveness of the proposed method has been demonstrated by applying it for contingency ranking of IEEE 30-bus system and a practical 75-bus Indian system.