Filter-based Feature Selection Algorithm Research Articles

Bio inspired Boolean artificial bee colony based feature selection algorithm for sentiment classification

Over the past few years, sentiment analysis has become most predominant aspect to be extracted in the modern digital era. Due to digitization all over the world, the number of existing digital documents increasing exponentially day by day. Analyzing such documents to extract the exact meaning itself becomes a tedious task. And analyzing those documents in sentimental point of view has increased the modality of difficultness to another level. In this research work, an effective bio-inspired filter-based feature selection algorithm is proposed to effectively handle the documents from sentiment point of view. A popular bio inspired algorithm called Artificial Bee Colony algorithm is modified using Boolean operators to effectively handle feature selection problem in classifying the documents from sentimental aspect. A total of 9 different documents instances are taken for evaluating the performance of proposed model. A total of 5 different filter-based measures are used to quantify the working model of the algorithm. The outputs are analyzed to state-of-the-art algorithms, and the comparative results demonstrate a substantial improvement over the existing models.

Machine Learning Enabled Photoacoustic Spectroscopy for Noninvasive Assessment of Breast Tumor Progression In Vivo: A Preclinical Study.

Breast cancer is a dreaded disease affecting women the most in cancer-related deaths over other cancers. However, early diagnosis of the disease can help increase survival rates. The existing breast cancer diagnosis tools do not support the early diagnosis of the disease. Therefore, there is a great need to develop early diagnostic tools for this cancer. Photoacoustic spectroscopy (PAS), being very sensitive to biochemical changes, can be relied upon for its application in detecting breast tumors in vivo. With this motivation, in the current study, an aseptic chamber integrated photoacoustic (PA) probe was designed and developed to monitor breast tumor progression in vivo, established in nude mice. The device served the dual purpose of transporting tumor-bearing animals to the laboratory from the animal house and performing PA experiments in the same chamber, maintaining sterility. In the current study, breast tumor was induced in the nude mice by MCF-7 cells injection and the corresponding PA spectra at different time points (day 0, 5, 10, 15, and 20) of tumor progression in vivo in the same animals. The recorded photoacoustic spectra were subsequently preprocessed, wavelet-transformed, and subjected to filter-based feature selection algorithm. The selected top 20 features, by minimum redundancy maximum relevance (mRMR) algorithm, were then used to build an input feature matrix for machine learning (ML)-based classification of the data. The performance of classification models demonstrated 100% specificity, whereas the sensitivity of 95, 100, 92.5, and 85% for the time points, day 5, 10, 15, and 20, respectively. These results suggest the potential of PA signal-based classification of breast tumor progression in a preclinical model. The PA signal contains information on the biochemical changes associated with disease progression, emphasizing its translational strength toward early disease diagnosis.

Applying data mining techniques to predict vitamin D deficiency in diabetic patients.

Vitamin D is among the vitamins necessary for both adults' and children's health. It plays a significant role in calcium absorption, the immune system, cell proliferation and differentiation, bone protection, skeletal health, rickets, muscle health, heart health, disease pathogenesis and severity, glucose metabolism, glucose intolerance, varying insulin secretion, and diabetes. Because the 25-hydroxyvitamin D (25OHD) test, which is used to measure vitamin D is expensive and may not be covered in healthcare benefits in many countries, this study aims to predict vitamin D deficiency in diabetic patients. The prediction method is based on data mining techniques combined with feature selection by using historical electronic health records. The results were compared with a filter-based feature selection algorithm, namely relief-F. Non-valuable features were eliminated effectively with the relief-F feature selection method without any performance loss in classification. The performances of the methods were evaluated using classification accuracy (ACC), sensitivity, specificity, F1-score, precision, kappa results, and receiver operating characteristic (ROC) curves. The analyses have been conducted on a vitamin D dataset of diabetic patients and the results show that the highest classification accuracy of 97.044% was obtained for the support vector machines (SVM) model using radial kernel that contains 18 features.

A Hybrid GCN and Filter-Based Framework for Channel and Feature Selection: An fNIRS-BCI Study

In this study, a channel and feature selection methodology is devised for brain-computer interface (BCI) applications using functional near-infrared spectroscopy (fNIRS). A graph convolutional network (GCN) is employed to select the appropriate and correlated fNIRS channels. Furthermore, in the feature extraction phase, the performance of two filter-based feature selection algorithms, (i) the minimum redundancy maximum relevance (mRMR) and (ii) ReliefF, is investigated. The five most commonly used temporal statistical features (i.e., mean, slope, maximum, skewness, and kurtosis) are used, whereas the conventional support vector machine (SVM) is utilized as a classifier for training and testing. The proposed methodology is validated using an available online dataset of motor imagery (left- and right-hand), mental arithmetic, and baseline tasks. First, the efficacy of the proposed methodology is shown for two-class BCI applications (i.e., left- vs. right-hand motor imagery and mental arithmetic vs. baseline). Second, the proposed framework is applied to four-class BCI applications (i.e., left- vs. right-hand motor imagery vs. mental arithmetic vs. baseline). The results show that the number of appropriate channels and features was significantly reduced, resulting in a significant increase in classification accuracy for both two-class and four-class BCI applications, respectively. Furthermore, both mRMR (i.e., 87.8% for motor imagery, 87.1% for mental arithmetic, and 78.7% for four-class) and ReliefF (i.e., 90.7% for motor imagery, 93.7% for mental arithmetic, and 81.6% for four-class) yielded high average classification accuracy p < 0.05 . However, the results of the ReliefF algorithm are more stable and significant.

Firefly-Based feature selection algorithm method for air pollution analysis for Zonguldak region in Turkey

Air pollution in cities is a serious environmental issue. In Turkey, the air quality index values of the measurement stations are calculated according to European Union standards. There are many kinds of measurement parameters (features) and 6 different kinds of air quality classes according to measurement stations in Turkey. Non-valuable features can be eliminated effectively with feature selection methods without any performance loss in classification. This study aims to investigate, analyze and implement a feature selection method using the FireFly Optimization Algorithm (FOA) approach. In the study, data from measurement stations for the Zonguldak region, which is known as the most polluted region in Turkey, are obtained and analyzed. Along with the acquired data, new features have been added such as day type day slots and the Covid19 feature since it is thought that curfew restrictions have an impact on air quality. The results were compared with a filter-based feature selection algorithm namely ReliefF. Experimental results show that FOA based feature selection method outperforms the ReliefF method at classification using the Random Forest classifier for air pollution even if with a fewer number of features. The Macro averaged F-score of the data set is increased from 0.685 to 0.988 using the FOA-based feature selection method.

Ensemble machine learning approach for electronic nose signal processing

Electronic nose (e-nose) systems have been reported to be used in many areas as rapid, low-cost, and non-invasive instruments. Especially in meat production and processing, e-nose system is a powerful tool to process volatile compounds as a unique ‘fingerprint’. The ability of the pattern recognition algorithm to analyze e-nose signals is the key to the success of the e-nose system in many applications. On the other hand, ensemble methods have been reported for favorable performances in various data sets. This research proposes an ensemble learning approach for e-nose signal processing, especially in beef quality assessment. Ensemble methods are not only used for learning algorithms but also sensor array optimization. For sensor array optimization, three filter-based feature selection algorithms (FSAs) are used to build ensemble FSA such as reliefF, chi-square, and gini index. Ensemble FSA is developed to deal with different or unstable outputs of a single FSA on homogeneous e-nose data sets in beef quality monitoring. Moreover, ensemble learning algorithms are employed to deal with multi-class classification and regression tasks. Random forest and Adaboost are used that represent bagging and boosting algorithms, respectively. The results are also compared with support vector machine and decision tree as single learners. According to the experimental results, our ensemble approach has good performance and generalization in e-nose signal processing. Optimized sensor combination based on filter-based FSA shows stable results both in classification and regression tasks. Furthermore, Adaboost as a boosting algorithm produces the best prediction even though using a smaller number of sensors.

Comparative analysis of proficiencies of various textures and geometric features in breast mass classification using k-nearest neighbor

This paper introduces a comparative analysis of the proficiencies of various textures and geometric features in the diagnosis of breast masses on mammograms. An improved machine learning-based framework was developed for this study. The proposed system was tested using 106 full field digital mammography images from the INbreast dataset, containing a total of 115 breast mass lesions. The proficiencies of individual and various combinations of computed textures and geometric features were investigated by evaluating their contributions towards attaining higher classification accuracies. Four state-of-the-art filter-based feature selection algorithms (Relief-F, Pearson correlation coefficient, neighborhood component analysis, and term variance) were employed to select the top 20 most discriminative features. The Relief-F algorithm outperformed other feature selection algorithms in terms of classification results by reporting 85.2% accuracy, 82.0% sensitivity, and 88.0% specificity. A set of nine most discriminative features were then selected, out of the earlier mentioned 20 features obtained using Relief-F, as a result of further simulations. The classification performances of six state-of-the-art machine learning classifiers, namely k-nearest neighbor (k-NN), support vector machine, decision tree, Naive Bayes, random forest, and ensemble tree, were investigated, and the obtained results revealed that the best classification results (accuracy = 90.4%, sensitivity = 92.0%, specificity = 88.0%) were obtained for the k-NN classifier with the number of neighbors having k = 5 and squared inverse distance weight. The key findings include the identification of the nine most discriminative features, that is, FD26 (Fourier Descriptor), Euler number, solidity, mean, FD14, FD13, periodicity, skewness, and contrast out of a pool of 125 texture and geometric features. The proposed results revealed that the selected nine features can be used for the classification of breast masses in mammograms.

Benchmarking performance of machine and deep learning-based methodologies for Urdu text document classification

In order to provide benchmark performance for Urdu text document classification, the contribution of this paper is manifold. First, it provides a publicly available benchmark dataset manually tagged against 6 classes. Second, it investigates the performance impact of traditional machine learning-based Urdu text document classification methodologies by embedding 10 filter-based feature selection algorithms which have been widely used for other languages. Third, for the very first time, it assesses the performance of various deep learning-based methodologies for Urdu text document classification. In this regard, for experimentation, we adapt 10 deep learning classification methodologies which have produced best performance figures for English text classification. Fourth, it also investigates the performance impact of transfer learning by utilizing Bidirectional Encoder Representations from Transformers approach for Urdu language. Fifth, it evaluates the integrity of a hybrid approach which combines traditional machine learning-based feature engineering and deep learning-based automated feature engineering. Experimental results show that feature selection approach named as normalized difference measure along with support vector machine outshines state-of-the-art performance on two closed source benchmark datasets CLE Urdu Digest 1000k, and CLE Urdu Digest 1Million with a significant margin of 32% and 13%, respectively. Across all three datasets, normalized difference measure outperforms other filter-based feature selection algorithms as it significantly uplifts the performance of all adopted machine learning, deep learning, and hybrid approaches. The source code and presented dataset are available at Github repository https://github.com/minixain/Urdu-Text-Classification .

An ensemble method based on rotation calibrated least squares support vector machine for multi-source data classification

ABSTRACT This paper proposed an extended rotation-based ensemble method for the classification of a multi-source optical-radar data. The proposed method was actually inspired by the rotation-based support vector machine ensemble (RoSVM) with several fundamental refinements. In the first modification, a least squares support vector machine was used rather than the support vector machine due to its higher speed. The second modification was to apply a Platt calibrated version instead of a classical non-probabilistic version in order to consider more suitable probabilities for the classes. In the third modification, a filter-based feature selection algorithm was used rather than a wrapper algorithm in order to further speed up the proposed method. In the final modification, instead of a classical majority voting, an objective majority voting, which has better performance and less ambiguity, was employed for fusing the single classifiers. Accordingly, the proposed method was entitled rotation calibrated least squares support vector machine (RoCLSSVM). Then, it was compared to other SVM-based versions and also the RoSVM. The results implied higher accuracy, efficiency and diversity of the RoCLSSVM than the RoSVM for the classification of the data set of this paper. Furthermore, the RoCLSSVM had lower sensitivity to the training size than the RoSVM.

A novel health indicator developed using filter-based feature selection algorithm for the identification of rotor defects

In this work, a novel health indicator is developed for the identification of rotor defects. The indicator is developed by extracting features from vibration data acquired from horizontal and vertical directions of rotors. A total of 38 features were initially extracted from time-domain signal, frequency-domain signal, and time–frequency representation. Out of many features, six most important features were selected using filter-based feature selection process. Thereafter, important features were fused together using manifold learning to develop health indicator. The developed indicator is used to identify misalignments (angular misalignment and parallel misalignment), rub, and unbalance. The major benefit of the proposed method is that it not only indicates the presence of defect in the rotor but also indicates the severity of defect. The experimental study presented in this article justifies that the proposed method is sensitive to the increasing levels of horizontal and angular misalignment and unbalance. The developed indicator is sensitive enough to indicate the presence of rub.

Hybrid fast unsupervised feature selection for high-dimensional data

The emergence of ``curse of dimensionality” issue as a result of high reduces datasets deteriorates the capability of learning algorithms, and also requires high memory and computational costs. Selection of features by discarding redundant and irrelevant features functions as a crucial machine learning technique aimed at reducing the dimensionality of these datasets, which improves the performance of the learning algorithm. Feature selection has been extensively applied in many application areas relevant to expert and intelligent systems, such as data mining and machine learning. Although many algorithms have been developed so far, they are still unsatisfying confronting high-dimensional data. This paper presented a new hybrid filter-based feature selection algorithm based on acombination of clustering and the modified Binary Ant System (BAS), called FSCBAS, to overcome the search space and high-dimensional data processing challenges efficiently. This model provided both global and local search capabilities between and within clusters. In the proposed method, inspired by genetic algorithm and simulated annealing, a damped mutation strategy was introduced that avoided falling into local optima, and a new redundancy reduction policy adopted to estimate the correlation between the selected features further improved the algorithm. The proposed method can be applied in many expert system applications such as microarray data processing, text classification and image processing in high-dimensional data to handle the high dimensionality of the feature space and improve classification performance simultaneously. The performance of the proposed algorithm was compared to that of state-of-the-art feature selection algorithms using different classifiers on real-world datasets. The experimental results confirmed that the proposed method reduced computational complexity significantly, and achieved better performance than the other feature selection methods.

A Deep Learning Method With Filter Based Feature Engineering for Wireless Intrusion Detection System

In recent years, the increased use of wireless networks for the transmission of large volumes of information has generated a myriad of security threats and privacy concerns; consequently, there has been the development of a number of preventive and protective measures including intrusion detection systems (IDS). Intrusion detection mechanisms play a pivotal role in securing computer and network systems; however, for various IDS, the performance remains a major issue. Moreover, the accuracy of existing methodologies for IDS using machine learning is heavily affected when the feature space grows. In this paper, we propose a IDS based on deep learning using feed forward deep neural networks (FFDNNs) coupled with a filter-based feature selection algorithm. The FFDNN-IDS is evaluated using the well-known NSL-knowledge discovery and data mining (NSL-KDD) dataset and it is compared to the following existing machine learning methods: support vectors machines, decision tree, K-Nearest Neighbor, and Naive Bayes. The experimental results prove that the FFDNN-IDS achieves an increase in accuracy in comparison to other methods.

Selection of Optimal Object Features in Object-Based Image Analysis Using Filter-Based Algorithms

With the increase in spatial resolution of recent sensors, object-based image analysis (OBIA) has gained importance for producing detailed land use maps. One of the main advantages of OBIA is that a variety of spectral, spatial and textural features can be extracted for the segmented image objects that are later utilized in classification. However, using a large number of features not only increases the required computational time, but also requires a large number of ground samples, which is unavailable in most cases. For these reasons, feature selection (FS) has become an important research topic for OBIA based classification studies. In this study, three filter-based FS algorithms namely, Chi square, information gain and ReliefF were applied to determine the most effective object features that ensure high separability among landscape features. For this purpose, importance degree (i.e. ranks) of 110 input object features were firstly estimated by the algorithms, and correlation-based merit function was then applied to determine optimum feature subset size. Multi-resolution segmentation algorithm was applied for segmenting a WorldView-2 image. Support vector machine, random forest and nearest neighbour classifiers were all utilized to classify segmented image objects using the selected object features. Results revealed that the FS algorithms were effective for selecting the most relevant features. Also, the classifiers produced the highest performances with 24 out of 110 features selected by the information gain (IG) algorithm. Particularly, the support vector machine classifier produced the highest overall accuracy (92.00%) with 24 selected features determined by the IG algorithm. A significant improvement of about 4% was achieved by applying FS procedures that was found statistically significant in terms of Wilcoxon signed-ranks test.

Ensemble swarm behaviour based feature selection and support vector machine classifier for chronic kidney disease prediction

Kidney Disease and kidney failure is the one of the complicated and challenging health issues regarding human health. Without having any symptoms few diseases are detected in later stages which results in dialysis. Advanced excavating technologies can always give various possibilities to deal with the situation by determining important realations and associations in drilling down health related data. The prediction accuracy of classification algorithms depends upon appropriate Feature Selection (FS) algorithms decrease the number of features from collection of data. FS is the procedure of choosing the most relevant features, removing irrelevant features. To identify the Chronic Kidney Disease (CKD), Hybrid Wrapper and Filter based FS (HWFFS) algorithm is proposed to reduce the dimension of CKD dataset. Filter based FS algorithm is performed based on the three major functions: Information Gain (IG), Correlation Based Feature Selection (CFS) and Consistency Based Subset Evaluation (CS) algorithms respectively. Wrapper based FS algorithm is performed based on the Enhanced Immune Clonal Selection (EICS) algorithm to choose most important features from the CKD dataset. The results from these FS algorithms are combined with new HWFFS algorithm using classification threshold value. Finally Support Vector Machine (SVM) based prediction algorithm be proposed in order to predict CKD and being evaluated on the MATLAB platform. The results demonstrated with the purpose of the SVM classifier by using HWFFS algorithm provides higher prediction rate in the diagnosis of CKD when compared to other classification algorithms.

Ensemble of feature selection methods: A hesitant fuzzy sets approach

Recently, there has been a great attention to develop feature selection methods on the microarray high dimensional datasets. In this paper, an innovative method based on Maximum Relevancy and Minimum Redundancy (MRMR) approach by using Hesitant Fuzzy Sets (HFSs) is proposed to deal with feature subset selection; the method is called MRMR-HFS. MRMR-HFS is a novel filter-based feature selection algorithm that selects features by ensemble of ranking algorithms (as the measure of feature-class relevancy that must be maximized) and similarity measures (as the measure of feature-feature redundancy that must be minimized). The combination of ranking algorithms and similarity measures are done by using the fundamental concepts of information energies of HFSs. The proposed method has been inspired from Correlation based Feature Selection (CFS) within the sequential forward search in order to present a robust feature selection tool to solve high dimensional problems. To evaluate the effectiveness of the MRMR-HFS, several experimental results are carried out on nine well-known microarray high dimensional datasets. The obtained results are compared with those of other similar state-of-the-art algorithms including Correlation-based Feature Selection (CFS), Fast Correlation-based Filter (FCBF), Intract (INT), and Maximum Relevancy Minimum Redundancy (MRMR). The outcomes of comparison carried out via some non-parametric statistical tests confirm that the MRMR-HFS is effective for feature subset selection in high dimensional datasets in terms of accuracy, sensitivity, specificity, G-mean, and number of selected features.

Building an Intrusion Detection System Using a Filter-Based Feature Selection Algorithm

Redundant and irrelevant features in data have caused a long-term problem in network traffic classification. These features not only slow down the process of classification but also prevent a classifier from making accurate decisions, especially when coping with big data. In this paper, we propose a mutual information based algorithm that analytically selects the optimal feature for classification. This mutual information based feature selection algorithm can handle linearly and nonlinearly dependent data features. Its effectiveness is evaluated in the cases of network intrusion detection. An Intrusion Detection System (IDS), named Least Square Support Vector Machine based IDS (LSSVM-IDS), is built using the features selected by our proposed feature selection algorithm. The performance of LSSVM-IDS is evaluated using three intrusion detection evaluation datasets, namely KDD Cup 99, NSL-KDD and Kyoto 2006+ dataset. The evaluation results show that our feature selection algorithm contributes more critical features for LSSVM-IDS to achieve better accuracy and lower computational cost compared with the state-of-the-art methods.

Multiple-cause discovery combined with structure learning for high-dimensional discrete data and application to stock prediction

Causal discovery in observational data is crucial to a variety of scientific and business research. Although many causal discovery algorithms have been proposed in recent decades, none of them is effective enough in dealing with high-dimensional discrete data. The main challenge is the complex interactions among large volume of variables, leading to numerous spurious causalities found. In this work, we propose a novel multiple-cause discovery method combined with structure learning (McDSL) to eliminate the spurious causalities. The method is carried out in two phases. In the first phase, conditional independence test is used to distinguish direct causal candidates from the indirect ones. In the second phase, causal direction of multi-cause structure is carefully determined with a hybrid causal discovery method. Validation experiments on synthetic data showed that McDSL is reliable in discovering multi-cause structures and eliminating indirect causes. We then applied this algorithm in discovering multiple causes of stock return based on 13-year historical financial data of the Shanghai Stock Exchanges of China, and established a stock prediction model. Experimental results showed that the McDSL discovered causes revealed changes of key risk factors of the stock market over 13 years, which indicated investors should change their investment strategy over time. Moreover, the causes discovered by McDSL have better performance in predicting stock return than that of other common filter-based feature selection algorithms.

MFSPFA: An Enhanced Filter based Feature Selection Algorithm

Selection is the process of selecting the momentous feature subset from the original ones. This technique is frequently used as a preprocessing technique in data mining. In this study, a new feature selection algorithm is proposed and is called Modified Fisher Score Principal Feature Analysis (MFSPFA). The new algorithm is developed by combining the proposed Modified Fisher Score (MFS) and Principal Feature Analysis (PFA). The proposed algorithm is tested on publicly available datasets. The experimental results show that, the proposed algorithm is able to reduce the futile features and improves the classification accuracy.

A Filter Based Feature Selection Algorithm Using Null Space of Covariance Matrix for DNA Microarray Gene Expression Data

We propose a new filter based feature selection algorithm for classification based on DNA microarray gene expression data. It utilizes null space of covariance matrix for feature selection. The algorithm can perform bulk reduction of features (genes) while maintaining the quality information in the reduced subset of features for discriminative purpose. Thus, it can be used as a pre-processing step for other feature selection algorithms. The algorithm does not assume statistical independency among the features. The algorithm shows promising classification accuracy when compared with other existing techniques on several DNA microarray gene expression datasets.