Proposed Feature Selection Approach Research Articles

Binary BAT algorithm and RBFN based hybrid credit scoring model

Credit scoring is a process of calculating the risk associated with an applicant on the basis of applicant’s credentials such as social status, financial status, etc. and it plays a vital role to improve cash flow for financial industry. However, the credit scoring dataset may have a large number of irrelevant or redundant features which leads to poorer classification performances and higher complexity. So, by removing redundant and irrelevant features may overcome the problem with huge number of features. This work emphasized on the role of feature selection and proposed a hybrid model by combining feature selection by utilizing Binary BAT optimization technique with a novel fitness function and aggregated with for Radial Basis Function Neural Network (RBFN) for credit score classification. Further, proposed feature selection approach is aggregated with Support Vector Machine (SVM) & Random Forest (RF), and other optimization approaches namely: Hybrid Particle Swarm Optimization and Gravitational Search Algorithm (PSOGSA), Hybrid Particle Swarm Optimization and Genetic Algorithm (PSOGA), Improved Krill Herd (IKH), Improved Cuckoo Search (ICS), Firefly Algorithm (FF) and Differential Evolution (DE) are also applied for comparative analysis.

CFS: An effective statistical texture descriptor

<p class="IETAbstractText">Coarseness is an effective description for texture analysis and many approaches capture this property of texture in different methods. In this research, we propose a new texture descriptor (CFS) that works based on coarseness and the fineness similarity score. For tuning of its configuration, we collect coarse and fine textures based on human visual perception at first. After that, we relabel the "coarse" and "fine" categories of the gathered textures during the configuration of the operator in a proposed framework. We concatenated the features of two pyramid representations with the CFS information of the original texture (PCFS). In addition, we combine the PCFS information with our last proposed feature selection approach to improve the efficiency of the CFS. We evaluated the proposed feature extraction method with classification of one of the well-known data set, Outex. Experimental results depict satisfactory performance of the CFS with very low-dimensional feature length.</p>

Feature Selection for Opinion Mining Using Shuffled Frog Leaping Algorithm

An essential part of our information-collecting behavior has always been to find out what other people think. With the growing availability and popularity of opinion-rich resources such as online review sites and personal blogs, new opportunities and challenges arise as people now can, and do, actively use information technologies to seek out and understand the opinions of others. Hence, Sentiment Analysis research has increased tremendously in recent times. Sentiment analysis deals with the methods that automatically process the text contents and extract the opinion of the users. In this work, biomedical opinions are extracted from twitter which contains many features needed to classify the opinions. However, such datasets contain many irrelevant or weak correlation features which influence the predictive accuracy of classification. Without a feature selection algorithm, it is difficult for the existing classification techniques to accurately identify patterns in the features. The purpose of feature selection is to not only identify a feature subset from original set of features but also to reduce the computation overhead in data mining .In the proposed feature selection approach, Shuffled Frog Leaping Algorithm (SFLA) algorithm optimizes the process of feature selection and yields the best optimal feature subset which increases the predictive accuracy of the classifier. SFLA is used as a feature selector and generates the feature subset and Naive Bayes, SVM and K-nn classification used to evaluate the feature subset produced. Experimental results show that the Naive Bayes classification produces better accuracy when the selected features from shuffled frog leaping algorithm are used.

A feature selection approach based on sensitivity of RBFNNs

Abstract Feature selection is an important issue in pattern recognition and machine learning, which aims at selecting relevant features from a set of candidates. Obviously, the establishment of proper criteria to evaluate the relevance of features is pivotal to the selection. In this paper, a criterion is proposed to assess the relevance of individual input features based on Radial Basis Function Neural Networks (RBFNNs) involved in solving classification and regression problems. The criterion takes a quantified output sensitivity of RBFNNs to input variation as a measure, which is defined as a mathematical expectation and can in a statistic sense reflect the effect of an RBFNN's input variation on its output. The basic idea is that a well-trained RBFNN can capture relevant features of the problem it deals with and thus become more sensitive to the variation of those input features that make more contributions to the RBFNN's behavior. Since the sensitivity is difficult to exactly compute, a numerical integral technique is employed to approximately compute the sensitivity. Experimental results on several artificial and real datasets show that our proposed feature selection approach works well.

Risk stratification of 2D ultrasound-based breast lesions using hybrid feature selection in machine learning paradigm

Abstract Selection of relevant and appropriate features to characterize breast patterns is of paramount importance in breast tissue representation and classification in machine learning paradigm. Feature selection based on single evaluation criterion has shown limited capability in breast tumor detection and classification due to their biases towards single criterion. In this paper, a new hybrid feature selection scheme is used to determine most relevant features for classification of benign and malignant tumors in breast ultrasound images. The proposed approach uses ten different evaluation criteria to decide the relevance of a particular feature. The existing feature selection techniques are also reviewed. A new database of 178 breast ultrasound images consisting of 88 benign and 90 malignant cases are used in experiments. The performance of the proposed approach is compared with that of existing feature selection techniques using back-propagation artificial neural network (BPANN) and support vector machine (SVM) based classifiers. The results demonstrate that proposed feature selection approach outperformed traditional methods achieving significantly higher classification accuracy of 96.6% and 94.4% with BPANN and SVM classifiers respectively.

Nature Inspired Feature Selection Approach for Effective Intrusion Detection

Objectives: To reduce the dimensionality of network traffic dataset by selecting the relevant and irredundant features for accurate and quick intrusion detection. To achieve the target, we proposed a new feature selection approach based on the nature. Methods/Statistical Analysis: The proposed Modified CuttleFish Algorithm (MCFA) approach plays a crucial role in intrusion detection by selecting appropriate subset of most relevant features from huge amount of dataset. Griewank fitness function is used to calculate the fitness of the modified cuttlefish algorithm. Naive bayes classifier is employed at the generated subset of features from benchmark KDD 99 dataset in WEKA data mining tool Compare the results of proposed approach with the existing approaches of WEKA and Improved Cuttlefish Algorithm (ICFA), with different performance metrics. Findings: As per the outcomes are obtained by the WEKA Experimenter with the 9 feature selection approaches on KDD 99 10% training dataset, it has been observed that the Consistency Subset feature selection approach with Greedy stepwise search method gives higher accurate results than other approaches and from the literature survey has been found that the ICFA performs well, but still there is problem of low True positive (TP) rate and False negative (FN) rate. These problems are addressed by the proposed feature selection approach which outer perform best from others in accuracy rate (91.79%), True positive rate (0.947), false positive rate (0.025) and ROC area (0.9982) with the minimum amount of time at 19 relevant subset of features instead 41 features. As the consequence, the proposed approach is novel from existing approaches to increase the intrusion detection rate and discard the redundant and irrelevant subset of features. Application/Improvements: MCFA has improved intrusion detection rate by increasing the TP rate and decreasing the FP rate, so MCFA can be used for the real time applications of intrusion detection system. Keywords: Accuracy, Dimensionality, Feature Selection Approach, FP Rate, Intrusion Detection, Modified Cuttlefish Algorithm, TP Rate

Frames Beyond Words

Methods to automatically analyze media content are advancing significantly. Among others, it has become increasingly popular to analyze the framing of news articles by means of statistical procedures. In this article, we investigate the conceptual validity of news frames that are inferred by a combination of k-means cluster analysis and automatic sentiment analysis. Furthermore, we test a way of improving statistical frame analysis such that revealed clusters of articles reflect the framing concept more closely. We do so by only using words from an article’s title and lead and by excluding named entities and words with a certain part of speech from the analysis. To validate revealed frames, we manually analyze samples of articles from the extracted clusters. Findings of our tests indicate that when following the proposed feature selection approach, the resulting clusters more accurately discriminate between articles with a different framing. We discuss the methodological and theoretical implications of our findings.

Hybrid BFO and PSO Swarm Intelligence Approach for Biometric Feature Optimization

Nature-inspired novel swarm intelligence algorithms have gained more proliferation due to a variety of applications and uses in optimization of complex problems and selection of discriminatory sets of features to classify huge datasets during the past few decades. Feature selection is an efficient and useful pre-processing technique for solving classification problems in computer vision, data mining and pattern recognition. The major challenges of solving the feature selection problems lay in swarm intelligence algorithms which are capable of handling the vast number of feature sets from involved databases. In biometric based recognition systems, face recognition is a non-intrusive approach to identify individuals based on their discriminatory sets of facial feature vectors. In this paper, the authors tend to propose a unique novel hybrid based on Bacterial Foraging Optimization (BFO) and Particle swarm optimization (PSO) approach for the selection of best facial feature vectors that enhance the identification accuracy of the individual recognition because concerned facial info will contain useless and redundant face expression. The proposed hybrid approach mitigates irrelevant facial features in the feature space and selects the relevant set of features from the facial feature space. The proposed feature selection approach presents promising experimental results with respect to the number of facial feature subsets. The identification accuracies are superior to other approaches from the literature.

Optimal Feature Subset Selection using Ant Colony Optimization

Background/Objectives: Data mining is the process of extracting large volumes of raw data from hidden knowledge. The health care industry requires the use of data mining techniques as it generates huge and complex volumes of data. The applications of data mining techniques to medical data extract patterns which are useful for diagnosis, prognoses and treatment of diseases. This extraction of patterns allows doctors and hospitals to be more effective and more efficient. The huge volume of data is the barrier in the detection of patterns. Feature selection techniques mainly used in data preprocessing for data mining. Methods/Statistical Analysis: Classification task leads to reduction of the dimensionality of feature space, feature selection process is used for selecting large set of features. The Ant Colony Optimization based feature selection method is applied on cancer datasets. Findings: This research work proposes about feature selection mechanism based on Ant Colony Optimization. In an ACO algorithm, the activities of ants have significance for solving different combinatorial optimization problem which selects most relevant features. Through several iterations filter based method finds the optimal feature subset. Based on the similarity between features the feature relevance will be computed, that shows to the minimization of the redundancy. To validate the proposed feature selection method Support Vector Machine classification is applied. The accuracy of classification for whole feature set and the reduced feature subset are compared. The improved accuracy proves that the proposed feature selection approach has selected informative feature of the cancer datasets. Applications/Improvements: The possibilities of using PSO algorithm is applied for finding the best features in future. Other algorithms are also considered for further implementation.

Efficient levels of spatial pyramid representation for local binary patterns

Local binary patterns (LBPs) are a well‐known operator that shows the ability for rotation and scale invariant texture classification. A recent extension of this operator is the pyramid transform domain approach on LBPs (PLBP). Obtaining more accuracy by using more pyramid representations is an important result of PLBP, which increases not only feature dimensionality, but also classification computational time (CT). This study illustrates that more pyramid image representations will not improve the performance of the PLBP. We evaluate efficient levels of representation for the PLBP descriptor. In addition, the authors propose some feature selection approaches, such as the multi‐level and multi‐resolution (ML + MR) approach and the ML, MR and multi‐band (ML + MR + MB) approach and discuss their efficiency and CT. Experimental results show that the proposed feature selection approaches improve the accuracy of texture classification with fewer pyramid image representations. In addition, replacing the Chi‐2 similarity measurement with Czekannowski improves the accuracy of texture classification.

A Novel Feature Selection Approach Based on FODPSO and SVM

A novel feature selection approach is proposed to address the curse of dimensionality and reduce the redundancy of hyperspectral data. The proposed approach is based on a new binary optimization method inspired by fractional-order Darwinian particle swarm optimization (FODPSO). The overall accuracy (OA) of a support vector machine (SVM) classifier on validation samples is used as fitness values in order to evaluate the informativity of different groups of bands. In order to show the capability of the proposed method, two different applications are considered. In the first application, the proposed feature selection approach is directly carried out on the input hyperspectral data. The most informative bands selected from this step are classified by the SVM. In the second application, the main shortcoming of using attribute profiles (APs) for spectral-spatial classification is addressed. In this case, a stacked vector of the input data and an AP with all widely used attributes are created. Then, the proposed feature selection approach automatically chooses the most informative features from the stacked vector. Experimental results successfully confirm that the proposed feature selection technique works better in terms of classification accuracies and CPU processing time than other studied methods without requiring the number of desired features to be set a priori by users.

Multisensor-based real-time quality monitoring by means of feature extraction, selection and modeling for Al alloy in arc welding

Multisensory data fusion-based online welding quality monitoring has gained increasing attention in intelligent welding process. This paper mainly focuses on the automatic detection of typical welding defect for Al alloy in gas tungsten arc welding (GTAW) by means of analzing arc spectrum, sound and voltage signal. Based on the developed algorithms in time and frequency domain, 41 feature parameters were successively extracted from these signals to characterize the welding process and seam quality. Then, the proposed feature selection approach, i.e., hybrid fisher-based filter and wrapper was successfully utilized to evaluate the sensitivity of each feature and reduce the feature dimensions. Finally, the optimal feature subset with 19 features was selected to obtain the highest accuracy, i.e., 94.72% using established classification model. This study provides a guideline for feature extraction, selection and dynamic modeling based on heterogeneous multisensory data to achieve a reliable online defect detection system in arc welding.

Classical and swarm-based approaches for feature selection in spam filtering

Feature selection is a significant stage in classification and data mining systems. As a preprocessing step to machine learning, it is very effective in reducing dimensionality, removing irrelevant data, increasing learning accuracy, and improving result comprehensibility. Swarm intelligence, dealing with natural and artificial systems, composed of many individuals that coordinate using decentralised control and self-organisation, represents an interesting recent trend for feature selection. In this article, we propose several algorithms for feature selection based on classical feature ranking methods, in addition to different variants of ant colony optimisation and binary particle swarm optimisation for e-mail classification. This work presents a comparative study about the performance of these approaches when they are applied in conjunction with Naive Bayes classifiers. Our experimental results show that the proposed feature selection approaches can achieve significant predictive error rates in spam filtering; furthermore, the number of selected features is significantly decreased. These results were confirmed with our experiments on other public databases.

Joint Embedding Learning and Sparse Regression: A Framework for Unsupervised Feature Selection

Feature selection has aroused considerable research interests during the last few decades. Traditional learning-based feature selection methods separate embedding learning and feature ranking. In this paper, we propose a novel unsupervised feature selection framework, termed as the joint embedding learning and sparse regression (JELSR), in which the embedding learning and sparse regression are jointly performed. Specifically, the proposed JELSR joins embedding learning with sparse regression to perform feature selection. To show the effectiveness of the proposed framework, we also provide a method using the weight via local linear approximation and adding the l2,1 -norm regularization, and design an effective algorithm to solve the corresponding optimization problem. Furthermore, we also conduct some insightful discussion on the proposed feature selection approach, including the convergence analysis, computational complexity, and parameter determination. In all, the proposed framework not only provides a new perspective to view traditional methods but also evokes some other deep researches for feature selection. Compared with traditional unsupervised feature selection methods, our approach could integrate the merits of embedding learning and sparse regression. Promising experimental results on different kinds of data sets, including image, voice data and biological data, have validated the effectiveness of our proposed algorithm.

Optimal feature selection for classification of the power quality events using wavelet transform and least squares support vector machines

In this paper, a new optimal feature selection based power quality event recognition system is proposed for the classification of power quality events. While Apriori algorithm is capable of processing categorical data, an effective feature vector, which represents distinctive features of digital power quality event data, has been obtained by means of the proposed k-means based Apriori algorithm feature selection approach. The proposed k-means based Apriori algorithm feature selection approach is presented with a power quality event recognition system. In the power quality event recognition system, normalization and segmentation processes have been applied to three-phase event voltage signals. Using 9-level multiresolution analysis, wavelet transform coefficients of the event signals have been obtained. By applying nine different feature extraction processes to these coefficients, a 90 dimensional feature vector belonging to three-phase event voltage signals has been extracted. Optimal feature vector has been obtained by applying the k-means based Apriori algorithm feature selection approach to the obtained feature vector, which has been applied as the last step to the input of the least squares support vector machine classifier and recognition performance results have been obtained. Real power quality event data have been used to evaluate the performance of the proposed feature selection approach and power quality event recognition system. According to the results, the proposed k-means based Apriori algorithm feature selection approach and power quality event recognition system are efficient, reliable and applicable and classify three-phase event types with a high degree of accuracy.

Unsupervised Band Selection for Hyperspectral Imagery Classification Without Manual Band Removal

The rich information available in hyperspectral imagery has provided significant opportunities for material classification and identification. Due to the problem of the “curse of dimensionality” (called Hughes phenomenon) posed by the high number of spectral channels along with small amounts of labeled training samples, dimensionality reduction is a necessary preprocessing step for hyperspectral data. Generally, in order to improve the classification accuracy, noise bands generated by various sources (primarily the sensor and the atmosphere) are often manually removed in advance. However, the removal of these bands may discard some important discriminative information, eventually degrading the classification accuracy. In this paper, we propose a new strategy to automatically select bands without manual band removal. Firstly, wavelet shrinkage is applied to denoise the spatial images of the whole data cube. Then affinity propagation, which is a recently proposed feature selection approach, is used to choose representative bands from the noise-reduced data. Experimental results on three real hyperspectral data collected by two different sensors demonstrate that the bands selected by our approach on the whole data (containing noise bands) could achieve higher overall classification accuracies than those by other state-of-the-art feature selection techniques on the manual-band-removal (MBR) data, even better than the bands identified by the proposed approach on the MBR data, indicating that the removed “noise” bands are valuable for hyperspectral classification, which should not be eliminated.

Multi-objective feature selection by using NSGA-II for customer churn prediction in telecommunications

This paper proposes a new multiobjective feature selection approach for churn prediction in telecommunication service field, based on the optimisation approach NSGA-II. The basic idea of this approach is to modify the approach NSGA-II to select local feature subsets of various sizes, and then to use the method of searching nondominated solutions to select the global nondominated feature subsets. Finally, the method FBSM which yields the fitness thresholds is proposed to choose the global solutions with the lowest ranks as the final solutions. In order to evaluate the proposed approach, experiments were carried out and the experimental results show that the proposed feature selection approach is efficient for churn prediction with multiobjectives.

Feature group optimisation for machinery fault diagnosis based on fuzzy measures

With the development of modern multi-sensor based data acquisition technology often used with advanced signal processing techniques, more and more features are being extracted for the purposes of fault diagnostics and prognostics of machinery integrity. Applying multiple features can enhance the condition monitoring capability and improve the fault diagnosis accuracy. However, an excessive number of features also increases the complexity of the data analysis task and often increases the time associated with the analysis process. A method of bringing some efficiency into this process is to choose the most sensitive feature subset instead. Fuzzy measures are helpful in this regard and have the ability to represent the importance and interactions among different criteria. Based on fuzzy measure theory, a feature selection approach for machinery fault diagnosis is presented in this paper. A heuristic least mean square algorithm is adopted to identify the fuzzy measures using training data set. Shapley values with respect to the fuzzy measures are applied as importance indexes to help choose the most sensitive features from a set of features. Interaction indexes with respect to the fuzzy measures are then employed to remove the redundant features. Vibration signals from a rolling element bearing test rig are used to validate the method. The results show that the proposed feature selection approach based on fuzzy measures is effective for fault diagnosis.